Information distribution aspects of design methodology Parnas, 1971
We’re continuing with Liskov’s listthis week, and today’s paper is another classic from David Parnas in which you can see some of the same thinking as in ‘On the criteria….’ Parnas talks about the modules of a system (for contemporary feel, we could call them ‘microservices’ once more), how we end up with inadvertent tangling / coupling between microservices, and what we can do to prevent that. One of his key recommendations is that information about a microservice be carefully controlled – releasing information about how it works internally outside of the team that are working on it is not just unhelpful, it is positively harmful. Writing in 1971, this information communication was primarily through documentation, and so this is what Parnas discusses. I wonder if in 2016 he would make the claim “making the source code for a microservice available outside of the team working on it is harmful” ??? That would certainly be a statement likely to cause robust debate!
Update: David Parnas himself posted a comment answering my question. Wow! Here’s his reply:
You wrote, “I wonder if in 2016 he would make the claim “making the source code for a microservice available outside of the team working on it is harmful” ??? That would certainly be a statement likely to cause robust debate!” The answer is YES. Papers that I wrote subsequently talk about the need to design and then document interfaces and then give other developers the interface documentation instead of the code. In fact, you can find that in the 1973 papers. The difference is that I have much better techniques for documentation today than I did forty years ago.
David Lorge Parnas
A system has structure: the set of modules (microservices) in the system, and the connections between them.
Many assume that the “connections” are control transfer points, passed parameters, and shared data… Such a definition of “connection” is a highly dangerous oversimplification which results in misleading structure descriptions. The connections between microservices are the assumptions which the microservices make about each other.
Consider a change that needs to be made. “What changes can be made to one microservice without involving change to other services?”
We may make only those changes which do not violate the assumptions made by other microservices about the service being changed.
Making design decisions
During the design and development of a system we make decisions which eliminate some possibilities for system structure. This evolves over time. Three common considerations during the decision process are:
- Making sure the system delivers a great experience for its users
- Ensuring the system can be delivered as quickly as possible
- Making sure the system can easily support future changes
Each of these considerations suggests an optimal partial order for decision making, but those orderings are usually inconsistent!
- Delivering a great experience suggests using an outside-in approach whereby the external characteristics are decided first (rather than letting them be unnoticed implications of decisions about other aspects of system structure).
- Delivering the system as quickly as possible requires dividing it early into separate microservices:
Competitive pressures may require the use of large groups to produce a system in a sharply limited period of time. Additional developers speed up a project significantly only after the project has been divided into sub-projects in such a way that separate groups can work with little interaction (i.e. spending significantly less time in inter-group decisions than in intra-group decisions).
The desire to make the division early in the project lifecycle so that the team can ‘get on with it’ encourages a splitting along familiar lines and in agreement with the organisational structure (Conway).
Time pressures encourage groups to make the split before the externals are defined. Consequently, we find some adverse effect on the usability of the product. Haste also makes poor internal structure more likely.
- When it comes to changeability, Parnas makes an interesting observation: the earlier a decision was made, the more difficult it is likely to be to change it because other parts of the system grow assumptions about it.
These considerations suggest that the early decisions should be those which are the least likely to change; i.e. those based on “universal” truths or reasoning which takes into account little about a particular environment… the possibility of change suggests using the most general information first.
Since the thing that often changes the most is the external characteristics, starting there may make the system harder to change.
Good programmers fight against the system
The crux of Parnas’ argument hinges on this assertion:
A good programmer makes use of the available information given him or her.
Sometimes those uses are obvious: calling a subroutine in another module, or reusing a reference table. Sometimes they are less obvious, for example exploiting knowledge that a list is searched or sorted in a certain order.
Such uses of information have been so costly that we observe a strange reaction. The industry has started to encourage bad programming…. Derogatory names such as “kludger,” “hacker,” and “bit twiddler” are used for the sort of fellow who writes terribly clever programs which cause trouble later on. They are subtly but effectively discouraged by being assigned to work on small independent projects such as application routines (the Siberia of the software world) or hardware diagnostic routines (the coal mines). In both situations the programmer has little opportunity to make use of information about other modules.
(I wonder what the modern Siberia and coal mines would be?)…
Those that remain (the non-bit-twiddlers) are usually poor programmers. While a few refrain from using information because they know it will cause trouble, most refrain because they are not clever enough to notice that the information can be used. Such people also miss opportunities to use facts which should be used. Poor programs result.
A programmer can disastrously increase the connectivity of the system structure by using information he or she possesses about other services.
We must deliberately control information distribution
If you buy Parnas’ argument that if you make information available, good programmers can’t help but make use of it, whether that is in the overall good of the system or not, then an obvious solution presents itself: don’t make information available!
We can avoid many of the problems discussed here by rejecting the notion that design information (or source code?) should be accessible to everyone. Instead we should allow the designers, those who specify the structure, to control the distribution of design information as it is developed.
Say your system depends on an external service, it’s likely that all you know about that service is the documentation for its REST API, or the client SDK built on top. But when you depend on an internal service, you typically know much more…
We should not expect a programmer to decide not to use a piece of information, rather he should not possess information that he should not use.
The last word
I consider the internal restriction of information within development groups to be of far more importance than its restriction from users or competitors. Much of the information in a system document would only harm a competitor if they had it. (They might use it!).
And the modern ‘system document’ is generally the source code, written in a suitably high-level language.