Notes on software engineering, systems thinking, papers, books and other things I find interesting

Misc: open doors, concrete problems

Below, a few recent highlights. From Hamming, “You and Your Research” (June 6, 1995). The example I’ve given you already is working with the door closed or open. If you work the door closed, you won’t be interrupted and you get your work done. You work your door open, people come by and stop and chat and so on and so on. But I’ve noticed very clearly, at Bell Laboratories, those who work with door shut may be working just as hard ten years later but they don’t know what to work on. They are not connected with reality. Those who have the door open may very well know what’s important. Now I cannot prove to you whether the open door causes the open mind or whether the open mind causes the open door. I suspect it – I can only establish the correlation, and it was quite spectacular. Almost always the guy with the door closed were often very well able, very gifted, but they seem to work always on slightly the wrong problem. So you’ll have to get wide feeling for what is going on and the supreme example of this closure is the Institute for Advanced Study at Princeton. They take in people who’ve done something great. They give them luxury, a beautiful office a beautiful restaurant to dine in, a wonderful grounds and everything else like that. Adequate salary to live on. No cares, no worries, no nothing, you’re freed for life on anything at all. What happens? The bulk of ’em continue working on the problem they made that made them famous. They keep on elaborating and so on. Well, they’ve already made it famous. It doesn’t have to be added to. They got the thing going. Rarely do they change. Now, von Neumann was different. He was the Institute and he did go out in reality and turn up in Washington in other places. He traveled widely and was receptive of new ideas. But the bulk of the people got appointed the Institute for Advanced Study don’t keep the door open on life as it were. And they don’t do anything comparable to what they had done before. They are very able people, but the Institute in my opinion sterilized them a great extent. So what do you think are the ideal working conditions are not.

Misc: string bends, Minsky's censors

String bending and visual navigation on the guitar Just realized that I want to try to think about the “destination” note of the string bend when playing it, rather than the starting note, which I usually do. What do I mean about “visualizing the destination” of the bend? Well, pretty much what I mean by visualizing any fret on the neck. But because I bend to the destination note and not fret it directly, I tend to focus on where my finger is at, instead of where the note is at.

All models are wrong: reflections on becoming a systems scientist [2002]

A few highlights from reading All models are wrong: reflections on becoming a systems scientist; John D. Sterman. This is the paper form of a lecture held in 2002 by John D. Sterman, upon receiving the Jay Forrester award. What is System Dynamics? It can be seen through multiple lens, as: Science: the modeling process assumes hypothesizing, building a model, running an experiment - the simulation of the system through that model, and potentially invalidating the model. Engineering: the building of the model itself, building the simulation is engineering practice. Applied Mathematics: SD is grounded in Control Theory and Nonlinear Dynamics. Stocks & flows compile to differential equations which can be simulated on a computer. Social Science: modeling social systems, explaining social phenomena. Philosophy: it is a world view. Seeing feedbacks everywhere. A paradigm, Sterman says. Consulting: organizations want to achieve goals, stay competitive, etc.. Theory of Action: models are used to inform policy changes. They offer theoretical grounding for intervention through policy. Concepts Narrow Model Boundaries. “almost nothing is exogenous”; “invisible fences in the mind”, “infinite sources and infinite absorption capacity in sinks”, “a narrow boundary has cut critical feedback loops” I think this is one of the main points of the paper. Awarness of model boundaries. This means being aware of what you leave out of the model, how much is assumed exogenous in the sense that it’s not sourced by the system nor is it affected by the system. How much is part of “the environment” in which the system exists. How much is ignored. There’s an example in the paper that depicts Model Narrowness - the economics of Petroleum Resource Exploitation. The first narrow model presented is that of a economics-only view point which ignores geological aspects of resource exploration & exploitation, and considers resources as “exogenous”, meaning they have no beginning and no end, and are not affected by the system’s operation. Such a model fails to incorporate critical feedback loops, which makes it’s predictions unreliable, and the policies (or actions) derived from that worldview, ineffective. Non-Locality of cause & effect. In complex systems, cause and effect may reach large time & space distance! Intuitively we’re used to understanding locality, but that intuition is not as effective for grasping complex dynamics spanning wider space/time intervals. Event-oriented worldviews. Shift in feedback loop dominance. I love this one because it makes you ask “what is the dominant influence over this variable” and “how can this influence change over time”. Say we’re interested in a particular quantity, that’s measured in a system which is governed by multiple feedback loops. Each feedback acts with a certain power over the quantity we’re interested in. Over time, feedback dynamics can change: a reinforcing loop may be diminished by a stronger negative feedback, causing the quantity to start falling. Policy Resistence. When you have a certain understanding of a system, a given mental model in your head. You picture an intervention to improve some outcome. Policy resistence happens when, despite you implementing that change, that new policy, that new structure, the system over time re-adapts and renders your solution counterproductive. “Today’s solutions become tomorrow’s problems”. Some quick examples: increasing taxes intention: accrue more tax revenue actual system response: tax revenue income falls, as entities migrate to other jurisdictions. building more roads intention: more fluid traffic, less traffic jams actual system response: heavy traffic is maintained, since using the car becomes more attractive, more people use cars, and the problem compounds Time Delayed feedback. TBD, there’s a whole chapter in Business Dynamics studying the effects of delays in systems with feedbacks. Quotes “One [student] once told me I was a model professor. I thought this was high praise until I realized that a model is a small imitation of the real thing.” “People frequently talk about unexpected surprises and side effects as if they were a feature of reality.” “Most people, regardless of their background, are comfortable with their current philosophy”. “As system thinkers, we must always strive to break down the false barriers that divide us.” “Today’s solutions become tomorrow’s problems. The result is Policy Resistence, the tendency for interventions to be defeated by the response of the system to the intervention itself. [..] At the root of this phenomenon lies the narrow, event-based, reductionist world view that most people live by”. (the remaining 2 paragraphs are beautiful, but I should stop quoting here.) “We are unaware of the majority of the feedback effects of our actions. Instead, we see most of our experience as a kind of weather: something that happens to us but over which we have no control. Failure to recognize the feedbacks in which we are embedded, the way in which we shape the situation in which we find ourselves, leads to policy resistence in which we persistently react to the symptoms of difficulty, intervening in low-leverage points and triggering delayed and distant but powerful feedbacks.” “School teaches us that every subject is different and knowledge is fragmented. The invisible lines in the mind are the boundaries of our mental models.” “We are continuously pressured by our clients, our students, colleagues and our own egos to slip out of the role of the questioner and learner into the role of expert and teacher. Doing so often fails, by generating defensiveness and resistance. The phrase ‘getting client buy-in’ should be banned from our lexicon. Selling a product is antithetical to the process of inquiry.” “There is no learning without feedback, without knowledge of the results of our actions.” “Developing the capacity to see the world through multiple lenses and to respect differences cannot become an excuse for indecision, for a retreat to impotent scholasticism. We have to act. We must make the best decisions we can despite the inevitable limitations of our knowledge and models, then take personal responsibility for them. Mastering this tension is an exceptionally difficult discipline, but one essential for effective systems thinking and learning.” “It’s by asking these ‘Why?’ questions that we gain insight into how we are both shaped by and shape the world, where we can act most effectively, where we can make a difference and what we are striving for.”

DeepDive: Declarative Knowledge Base Construction [2017]

I stumbled upon this paper while contemplating whether it’s possible (and valuable) to build a network representation of technology - eg. which tech requires which lower-level tech, which tech depends on which natural phenomena, how tech shifts over time, what feedback loops are present, … An ill-defined task that remains just a draft, a fun thought experiment; but let’s switch to the paper now. Link: DeepDive: declarative knowledge base construction The paper The author(s) are part of Hazy Research, the group that recently created FlashAttention - an algorithm to speed up training, increase context window of transformers (and something that I need to read more about). Check out their blog, it’s full of good stuff.

The Strategy of Conflict, Thomas Schelling

WIP notes from reading The Strategy of Conflict by Thomas C. Schelling. Why this book? I wanted to see some real-world use of game theory in the writing of a classic. Thomas Crombie Schelling (April 14, 1921 – December 13, 2016) was an American economist and professor of foreign policy, national security, nuclear strategy, and arms control at the School of Public Policy at University of Maryland, College Park. Theory development “We live in an age of exponential growth in knowledge, and it is increasingly futile to teach only polished theorems and proofs. We must abandon the guided tour through the art gallery of mathematics, and instead teach how to create the mathematics we need. In my opinion, there is no long-term practical alternative.” - Richard Hamming

442 words

Misc: digital fabrication, mRNA hacks

This is absolute Misc stuff, just a bucket of paragraphs. Digital Fabrication A few days ago I listened to the Neil Gershenfeld episode of Lex’s podcast. It’s extremely inspiring! Highly recommend. A few highlights (please excuse the lack of details): Digital Fabrication, where “digital” is understood not as “binary” computer stuff, but by it’s deeper meaning. The concept is rooted in Vannevar Bush/Claude Shannon/Von Neumann Analog computer age and the transition to Digital as a way to get a grip on accumulated error. Actually, better just read his awesome article in Foreign Affairs here: “How to Make Almost Anything - The Digital Fabrication Revolution”. Multi-scale fabrication The stories of quantum compute 99 of 100 students: “i really admire your work, would love to join the lab”. 1 of 100: “here’s where you’re wrong, here’s an alternative” Ready Fire Aim - love the line as well as the stories behind it. There is no trash in nature - a quote which I heard not long ago in a John Baez talk on Math of 21st century. many more… As a result, I ordered his book https://designingreality.org/.

Interpolation

A Chronology of Interpolation: From Ancient Astronomy to Modern Signal and Image Processing. Printed this paper a few weeks ago while I was reading about KZG1, 2, 3 and left it aside. It’s aged enough, time to actually read it. The problem of constructing a continuously defined function from given discrete data is unavoidable whenever one wishes to manipulate the data in a way that requires information not included explicitly in the data. The relatively easiest and in many applications often most desired approach to solve the problem is interpolation, where an approximating function is constructed in such a way as to agree perfectly with the usually unknown original function at the given measurement points.

130 words

Mind your Instaparse recursion order

Sketching a parser for graphviz/dot. Really slow on a ~300 statement input. Got about a 10x improvement by using Left instead of Right-recursion.

520 words

Debugging Clojure using the REPL

A summary of techniques to debug Clojure code via REPL (aka “the interactive console”, the “read-eval-print loop”) and Cider debugger.

The Humble Programmer by Dijkstra (1972)

The original article by Dijkstra can be found here (pdf). These notes are mainly to summarize my read of the article for future ref.