The first thing that one notices
about a programming language is its syntax. When we think of COBOL we see lots
of capital letters and verbs like "PERFORM". Lisp is associated with
lots of parentheses. I have my personal memories of Algol 60 with
its wonderful typography with boldface delimiters such as begin and
end
way before there was boldface on computers! We often hear about graphical programming
languages
also. These languages all came with various claims about their
applicability. For example, COBOL was designed for business applications and
Lisp for artificial intelligence. How is the syntax of the language connected
with its power or applicability? Intentional Software has some very specific
views on this.

Let’s look at equivalent forms in different languages. For
example, consider the conditional statements:

IF (A .LT. 10) …
if A < 10 then …
test a ls 10 ifso …
if (A < 10) …

It is clear that the syntactic differences are completely
superficial, and it is difficult to see why people would have battled so hard
for or against them. Indeed it is disappointing to consider that this list
represents more than 40 years of programming language evolution from Fortran
through C#.

By the way, the 3rd line is written in the
historically important but otherwise little known language BCPL from Bell Labs.
It was followed by the language B which in turn was replaced by Kernighan and
Richie’s C. The joke of the time was to guess whether the successor of C would
be called “D” or “P”? Check out this overview of the family tree of programming
languages.

Now, if we compare the Fortran IV statements:

    IF (A – 10) 2, 2, 1
1   …
    GO TO 3
2    …
3  CONTINUE

with Fortran 77:

IF (A .LT. 10)
   …
ELSE
   …

we see more than just a syntactic difference. The early so
called “arithmetic if” statement awkwardly aped the way hardware expressed the
comparison: whether the control should be transferred to label 1 when the
difference (a-10) was negative, or to label 2 if the difference was zero or
positive. In a later version of Fortran (as in all current languages) we see
the structured construct with the comparison operator and with an ELSE clause, which expresses the same intention
much more directly. In fact, if we were asked what the first segment “meant” we
would probably say something very close to the second, assuming we could
correctly follow all the details and paths.

The structured form is much shorter, yet it contains the
same information. How is that possible? It is because the “arithmetic if” and
“goto” statements have more degrees of freedom (in other words, more parameters)
than we need; namely, the three label references after the IF and the one label
reference in the GOTO. These could
have easily represented a loop or maybe up to three different choices, whether
on purpose or, perhaps, by mistake. The earlier form is longer because it has
to represent the programmer’s intention in the larger parameter space. In
contrast, the structured form has fewer parameters – its “degrees of freedom”
property is more like the problem at hand, to wit, do something if a is less
than 10 and otherwise do something else. Therefore the structured form is
shorter. It cannot do a loop, nor can it do more than two things, which means
that many opportunities for mistakes are eliminated. There are other forms for
doing those things explicitly if that is intended.

We could say that the “arithmetic if” and the “goto”
statement in FORTRAN IV are less intentional, and that the structured “if” and
“else” in FORTRAN 77 are more intentional. We also note that intentionality has
to do with how the degrees of freedom in the problem compare with the degrees
of freedom in the notation. We know that we cannot express something with fewer
degrees of freedom than what is in the problem itself. But they can be equal:
that is the intentional optimum. As the degrees of freedom property in the
notation becomes greater than in the problem, we can call the notation less
intentional and more implementation-like. The difference can be very large in
the case of assembly code, or even machine code. In fact the number of bits
used to represent the assembly code for our “if” statement might even be sufficient
also to represent a recognizable icon of President Lincoln (756 bits)!
Clearly we would be using more bits than the problem dictates.

What is wrong with more degrees of freedom? Nothing if that
is what the problem needs: for example, if we are trying to create an iconic
directory of the Presidents. But if the problem is to choose between two
possibilities (as in “if..then..else”), or indeed for any problem, if we are
given more degrees of freedom than we need, we incur costs of navigating in the
larger space when we seek our solution and we incur costs of bugs. The bug
costs are due to the real possibility that we get lost in the large space. For
a given problem, the more parameters needed to express the solution, the
greater the chances that one of them will be incorrect.

On the other hand, we also need to recognize that even
excess degrees of freedom can be useful in some cases. In the early days of
computing, use of assembly code was normal practice. If we know very little
about the solution and it is expensive to change the language of the solution,
then we had better ensure that every possible solution will be accessible
within the same language and thereby benefit from the degrees of freedom. But
if we know quite a bit about the solution, then we can reduce risks of
incurring costs by shifting to new expressions with fewer degrees of freedom
where the structure of the solution can more closely approach the structure of
the problem.

So just as “arithmetic if”s were replaced with
“if..then..else”, the process of elimination of excess degrees of freedom has continued.
For example, instead of the common loop pattern:

startloop:
    if (!a) goto endloop;
    …
    if (x) goto endloop;
    …
    goto startloop;
endloop:

we can currently write:

while (a) {
    …
    if (x) break;
    …
}

One problem with the former loop pattern was that the
programmer was required to come up with reasonably unique, but still meaningful
names for the labels. These are really contradictory requirements. For example
“endloop” would have not worked very well in any real system because it would
not be unique. Using “L123” would be unique but probably not as meaningful.

The “while” loop and “break” statement cured those
contradictions and eliminated unnecessary degrees of freedom. But why should we
stop improvements at the current “while”? We can ask next: “What are we doing
with the while and its contents?” Chances are that there is some answer like: we
are waiting for something, we are searching for something, we are enumerating
over some set. In each case, we could imagine some construct that would do
exactly what we wanted if we had the right degrees of freedom. For example we
now see “foreach” in languages like Perl, C# and Java, where the freedom (and burden)
to separately identify the loop limits while iterating over a collection has
been removed. “Foreach” is more intentional than a traditional “for” loop.
Although it adds complexity to a language by introducing some new syntax, it
more directly gets to the intended behavior of iterating through a collection.

Advocates of object-oriented languages point out that optimal
constructs are already within reach – just define a class and its methods the
way you want them. But this works well only in cases where a class is an exact
representation of an underlying intent. What about a “while” loop, which is not
a class? Certainly a class-based enumerator could be built with some effort,
but then the programmer would have to introduce a name for the method that
represents the body and depending on the implementation other names would have
to be invented as well. So building a class-based “while” loop enumerator out
of some more primitive components would still be moving toward the past, raising
the old issues again.

Instead, we should move further toward the problem. Once we
have introduced the search primitive that the “while” implemented, we should
continue asking questions: “What are we doing with this search?” and create a
construct for the intention behind the search. To be able to continue creating
new constructs, we have to look beyond classes or even aspects.

We could help this process of creating new constructs move
along by making notation and semantics independent. We should be able to simply
list the degrees of freedom – the parameters – that are required for a
construct. The parameters themselves could be any other constructs with no a priori
limitations: expressions, statement lists, declarations, or types. And notations will
have to accommodate this flexibility.

Share →

5 Responses to Notations and Programming Languages

  1. Shane Clifford says:

    I have had experience with the importance of matching the “degrees of freedom” to the problem and the effect it has on success that I would like to share. Reading Charles’ post has led me to a conclusion that I had never reached before. At a former employer, I had the opportunity to lead a team developing a pair of domain specific IDEs. Both of these tools used structured code and both used diagrams based on UML for the visual representation of the systems. However, as I will discuss further, one of them was very successful and the other only marginally so.
    Our first editor was very domain specific. It was intended to implement an “abstraction-layer”—mapping transactions from a low-level communications protocol into a higher-level protocol that made development of the rest of our distributed systems easier and more generic. We used UML activity diagrams as the visual representation of the transactions, but they were not free-form. Every activity diagram contained three swim-lanes (one for the client, one for the abstraction layer, and one for the server that supported the low-level protocol.) Developers could only construct the transactions using predefined “chunks” of activity diagram (I’ll call them super-activities) that represented possible transaction behavior. The super-activities automatically positioned themselves in the correct swim-lanes and their transitions could only be connected to other activities in a valid swim-lane. Each of the super-activities could then be configured to some extent through a dialog to perform its correct role in the transaction. All of this information was stored in XML and interpreted by a server at runtime.
    This editor, after some initial deployment hiccups, was quite successful. The target developers learned to use it very quickly and it filled its niche quite nicely. I hope that my description above was good enough to illustrate that the environment had “degrees of freedom” very closely matched to its domain. Therefore, people who were experts in that domain readily adopted it. In fact, the runtime server was eventually replaced with a third-party product, but a new code generator was written for our editor to produce configuration for the new system so that developers could use our editor rather than the tools shipped with the purchased software.
    Our next editor was intended to be much more powerful. Its target domain involved developing the clients of the servers that the first editor produced. These systems were intended to control business process and required state-full, active objects—a system with more apparent complexity than the transaction processing servers. For this editor, we chose to employ more concepts from generative programming and introduced “domain engineering” and “application engineering” roles. The visual representation of these systems was UML class diagrams, hierarchical state diagrams, and feature diagrams (see Generative Programming, by Krzysztof Czarnecki and Ulrich W. Eisenecker, for an overview of feature diagrams, as well as domain and application engineering.) The “Domain Engineers” had complete freedom to design classes of objects for the system family that had very complex hierarchical state machines. It was up to them to model the attributes and behavior of each of the objects in their domain. “Application Engineers” would then pick and choose classes and features to meet the requirements of their specific customer and generate a specific system instance to deploy. This editor also stored the system description in XML, but it included a code generation step that produced a C++ server rather than interpreting the XML at runtime.
    This editor met with mixed success. People adapted to the “Application Engineering” role very easily and we were able to fill that role with people that were skilled more as requirements analysts than as expert coders. Unfortunately, the learning curve for becoming a “Domain Engineer” in our environment was very steep. The freedom to create a system using a complex hierarchical state diagram meant that people in the role had to have a VERY good understanding of how those diagrams worked. Plus there was added complexity of doing additional configuration of each of the features. A developer that understood was able to rapidly develop using our IDE. Unfortunately, we were only able to train a handful of engineers outside of our original development team to fill the “Domain Engineer” role. I have been referring to the roles in quotes, by the way, because I am not convinced that our implementation was close enough to the spirit of what is intended by those roles.
    It has always bothered me that the second editor was not as well received as the first (my team and I definitely considered it to be the superior product; we actually even considered dropping the first one because the second was quite capable of generating the same systems.) I never had a good feel for why it had worked out that way, but I think Charles’ post has cleared some of that up for me. The first editor directly addressed its problem space with appropriate degrees of freedom. Our second attempt overshot the mark and moved too far into the general purpose realm. Rather than producing a tool that directly addressed the problem domain, we had simply created a visual programming tool with the same learning curve as any other general purpose language. Nice—even fun to use, but without enough benefit to sway a developer to embrace it rather than something they already knew.

  2. Johannes Link says:

    There’s one thing about the “degrees of freedom” that has not been discussed: In most if not all cases a new level of abstraction (a DSL, a new PL, MDA or whatever) actually has too few degrees of freedom even for the intended range of problems. Consider Java as a PL which abstracts away manual memory handling. This is a good thing in 99.9% of the cases, but once in a while I’d like to have more degrees of freedom as for whent to claim and release memory from/to the OS. In those cases I have to struggle VERY hard to get what I need. Maybe I even have to call in someone who knows about the abstraction layer beneath, i.e. the details of Java’s GC algorithms and configuration options.
    My personal view is that there exists a threshold for every abstraction (what percentage of all targeted problems can be solved using it) below which the abstraction brings more burden than relief. The concrete value of the threshold depends on many things; some are domain-specific, some tool-specific and some differ for the individual user or team. However, we must be aware that such a threshold exists when choosing a development approach. My feeling is that current approaches like MDA or AOP haven’t reached that threshold yet for most people (and are not even close). The need to debug and find bugs within a certain abstraction layer pushes the threshold even further towards the 100% mark.

  3. Charles Simonyi says:

    Johannes’ post is an important reminder of the problem of “too few degrees of freedom” that I did not address adequately. With most computer languages one can always “manufacture” degrees of freedom, so we have seen Lisp programming implemented in Fortran arrays and the like but as Johannes points out this is always a struggle and not competitive with a natural implementation in the right language. So the ability to adjust the degrees of freedom using the appropriate notation, appropriate naming and the desired run-time implementation is important in both directions: more toward the domain and more toward the implementation. These roughly correspond to the less or more degrees of freedom, or higher or lower level of programming abstractions.

  4. Aspect limits second time

    Although this blog was opened six months ago, I’ve discovered its existence only recently. Interestingly, Charles’ contribution little formalizes an idea I mentioned six months ago.

  5. Pranab Das says:

    Hi,
    I wrote to one of the Amigos about a simple scheme of AOP. If we use token, line #, or pattern matching, we can instrument source codes with extra functionality, which can then be used in lieu of aspect, albeit only after compilation with the same language compiler (no specific AOP compiler is needed). The instrumented code need not modify the original source. The scheme is outwardly comparable with how AOP works at this time. This can be dynamically extended to runtime objects, as well.
    This way, with some additional hurdles, AOP can provide a solution to the problem of creating interface anywhere and also it is possible to recurse through cross-cutting concerns to any level.
    Shall be grateful to have an opinion from you. Also, please ask me if I am not clear about my intentions. The topic, is ‘N-dimensional programming to replace current AOP’.
    Regards,
    Pranab