Promise Theory—What Is It?

During the past 20 years, there has been a growing sense of inadequacy about the “command and control” model for managing IT systems. Years in front of the television with a remote control have left us hard pressed to think of any other way of making machines work for us. But, the truth is that point-and-click, imperative scripting and remote execution do not scale very well when you are trying to govern the behavior of a large number of things.

IT installations grow to massive size in data centers, and the idea of remote command and control, by an external manager, struggles to keep pace, because it is an essentially manual human-centric activity. Thankfully, a simple way out of this dilemma was proposed in 2005 and has acquired a growing band of disciples in computing and networking. This involves the harnessing of autonomous distributed agents.

Imagine that we want to implement a software design or enact a business process. For cultural reasons, our first thought is to want to issue a list of instructions to all the parts involved to carry out some kind of an algorithm, but that might not be the best choice at scale. An algorithm is just a story we tell ourselves about how we envisage the process in our minds. Often we try to tell the same story to a computer or to a team of people, in the form of a flow chart or scripted language. This is one way of describing, building or processing the design. The algorithm might involve a lot of detail and take the form of step-by-step instructions, but other approaches use parallelism or non-linear stories to lay out the job more efficiently. In either case, the instructor assumes that all of the parts involved simply will do exactly as they are instructed. Alas, this is not necessarily true. Remote parts of a system might be unable to comply with these instructions, or even be unwilling to comply, depending on their nature and their circumstances.

This command and control model is called an obligation model in computer science. It has many problems. One of those problems is that it separates intent from implementation, creating uncertainty of outcome. The place where the instructions are determined is not the place where they must be carried out, so the instructor does not have the information about conditions where the local work needs to take place. If two different instructors start issuing conflicting instructions to the same set of parts, there would start a conflict, which no part of the system would be able to resolve, because none of the parts would have the information in one place: intent is not local to the point of action. The instructors might not even be aware of each other to resolve the conflict.

Figure 1. Obligation Model vs. Promise Theory

Luckily, there is a complementary approach to looking at design that fixes these deficiencies, not in terms of obligations, but in terms of promises.

In a promise-based design, each part behaves only according to the promises it makes to others. Instead of instructions from without, we have behavior promised from within. Since the promises are made by “self” (human self or machine self), it means that the decision is always made with knowledge of the same circumstances under which implementation will take place. Moreover, if two promises conflict with one another, the agent has complete information about those circumstances and conflicting intentions to be able to resolve them without having to ask for external help.

A promise-oriented view is somewhat like a service view. Instead of trying to remote-control things with strings and levers, one makes use of an ecosystem of promised services that advertise intent and offer a basic level of certainty about how they will behave. Promises are about expectation management, and knowing the services and their properties that will help us to compose a working system. It doesn't matter here how we get components in a system to make the kinds of promises we need—that is a separate problem.

Electronics are built in this way, as is plumbing and other commoditized construction methods. You buy components (from a suitable supplier) that promise certain properties (resistance, capacitance, voltage-current relationships), and you combine them based on those expectations into a circuit that keeps a greater promise (like being a radio transmitter or a computer).

To offer an example of a promise-oriented language, think of HTML and cascading style sheets on the Web. One can break up a Web page into labeled and tagged objects like titles and paragraphs, tables and images and so on. Each of these may be tagged with a specific identity, simply a type (like title, paragraph, link and so on). In a Promise Theory model, these regions could be viewed as “autonomous agents”, which through CSS, make promises about what color, font and spacing they will use to render their content initially and over time. The HTML file has regions of the form:

<h1>Title...</h1>
<p>Text....</p>

Although these are regions of a text file, this is not substantially different from files on a disk. It is just a collection of containers. The CSS promises look like this:

h1.main {color: red; font-size: 12px; }
p.main  {text-align:justify;}

That is, a header h1 in section “main” would be red and use 12-point text. Compare this to files. The configuration tool CFEngine for building and maintaining computers, allows us to keep promises about what attributes system resources will have. For example, instead of colors and fonts, files can promise to have certain permissions, content and names. One would write:

files:
 debian::
   "/etc/passwd"
      perms => mo("root", "644");

The language looks superficially different, but it basically is the same kind of declarative association between patterns of objects and what they promise. The promise a region makes itself is the one it will render for all time, until the promise has changed. So this is not a fire-and-forget push-button command, but rather the description of a state to be continuously maintained. In CFEngine style, we could write this alternative form of the HTML style sheet:

html:
 main::
  "h1"
    font_size => "12px",
        color => "red";
  "p"
    text_align => "justify";

From Promise Theory, we see that these patterns are basically the same; thus, one could say that CFEngine is a kind of “style sheet for servers”, in this sense.

Promise Theory deals with how to think, in this way, about a much wider range of problems. It was proposed (by myself) in 2005 as a way to formalize how the UNIX configuration engine CFEngine intuitively addressed the problem of managing distributed infrastructure. Such formal models are important in computer science to prove correctness. It has since been developed by myself and Jan Bergstra, and it is being adopted by an increasing number of others.

This complementary non-command way of thinking seems unnatural to us in the context of infrastructure, but more usual in the context of Web pages. Its strengths are that it applies equally to human and machine systems, and that it enforces a discipline of documenting the information necessary and sufficient to implement a design as promises. It is literally actionable (executable) documentation. One can easily convert to a complementary promise view by changing:

"YOU MUST (NOT)..." ---> "I PROMISE TO (NOT) ..."

A side effect of documenting the necessary and sufficient conditions for a purpose is that one sees all of the possible failure modes of a design enumerated as promises: “what if that promise is not kept?” Autonomy guarantees no hidden assumptions.

The main challenge faced in this view is how to see the desired effect emerge from these promises. What story do we tell about the system? In imperative programming languages, the linear story is the code itself. However, in a promise language, the human story is only implicit in a set of enabling promises. We have to tell the story of what happened differently. For some, this is a difficult transition to make, in the noble tradition of Prolog and Lisp and other functional languages. However, at scale and complexity, human stories are already so hard to tell that the promise approach becomes necessary.

The way we view the world in Promise Theory is as a collection of agents or nested containers of things that can keep promises. These containers could be:

The reference software with this kind of thinking is still CFEngine (version 3), as it was designed explicitly in this model, though several software systems work in an implicitly promise-oriented fashion. Promise thinking permeates the world of networking too, where autonomy is built into the modus operandi of the devices. Other examples where promises are made to users include interfaces and APIs, microservices or Service Oriented Architecture (SOA).

Promises turn design and configuration into a form of knowledge management, by shifting the attention away from what changes (or which algorithms to enact), onto what interfaces exist between components and what promises they keep and why. The service-oriented style of programming, made famous by Amazon and Netflix, uses this approach for scalability (not only machine scalability but for scaling human ownership). It is hailed as a cloud-friendly approach to designing systems, but Promise Theory also tells us that it is the route to very large scale. Applications have to be extensible by cooperation (sometimes called horizontal scaling through parallelism rather than vertical scaling through brute force). Databases like Cassandra illustrate how to deal with the issues of scale, redundancy and relativity.

Perhaps interestingly, biology has selected redundant services as its model for scaling tissue-based organisms. This offers a strong clue that we are on the right track. Avoiding strong dependencies is a way to avoid bottlenecks, so this shows the route to scalability.

Autonomy and standalone thinking seem to fly in the face of what we normally learn about programming—that is, to share resources, but this is not necessarily true. Security and scalability both thrive under autonomy, and complexity melts away when the dependencies between parts are removed and all control is from within. The near future world of mobile and embedded devices will not tolerate a remote-control model of management, and the sprawl of the cloud's server back end will force us to decentralize 3-D-printed micro-data centers, like the electricity substations of today. In this world, I think we can expect to see more of the autonomous thinking pervading society.