Meta: Literate Programming

Back in the Twentieth Century, Donald Knuth published a book, and provided several tools, supporting "literate programming". This involved using two sets of preprocessors allowing a single file to generate organized documentation for code and the code itself. The blocks of code would appear as inset blocks in the text of the documentation. The text itself was to be organized discursively, explaining intent and details of execution. Knuth used this to write the TeX program and its close relatives.

This is quite distinct from tools like doxygen, which take comments from code written as code and generates documentation from them, though doxygen and Javadoc and their relatives are descendants of the Literate Programming movement.

Another developer ported the tools to C/C++ (Knuth had used Pascal), and I used them for a while in the 1990s.

They had every virtue save one. Writing discursively and analytically about code as you write the code really does assist in getting better code, not just in the sense of bug-free code (though it assists with that, too) but in the sense of structure, choice of algorithms, and code clarity. The documentation produced provided a far better reference than the typical output of doxygen. Outputs were superior, period. I would also guess that overall, the input process was shorter as well, once one got used to the tools, because changes which would normally occur on multiple iterations were more likely to be rolled into the first pass.

What was the drawback? What happens when one leaves. Knowledge of TeX was not really widespread, and some knowledge was needed to format the text effectively; familiarity with the tools was likely only in a small number of developers. So one became more productive oneself, but without an organizational mandate, training, and building at least an interest into hiring criteria that code would end up largely unmaintainable. In addition, writing extended amounts of fluent discursive and analytical text is likely to be just fine if you are a Master of Arts who is fluent in English; in a programming environment where a majority of developers have English as a second language and even those who were born to it are not what one might call highly literate, and are likely to dislike doing documentation, the documentation generated is likely to be poor and the benefit from the feedback loop in writing is likely to be minimal.

Instead, we're stuck with the second or third best case, at best: documentation dredged out of class comments made after the fact and meant to be short summaries rather than real discussions of what choices were made and why.

Even this sort of project, where discussion is generated after the fact, but organized in a discursive way, delivers only part of the benefit. It does, in my experience, lead to considerable refactoring because of the extra attention given to the code. Some of those refactorings make it into discussion, many are made silently before it. But the overall discussion is still summary, made after the fact, and larger structural changes tend to be excluded. (One asks oneself "Could I have done that better" (or, occasionally, "what the hell did I think I was doing?") rather than "was this the best thing to do at all?")

I have kept workplace blogs focussed on concrete software practices on work-based social media at two places of work which in principle encouraged that sort of thing. I was the only person to do so. This is not going to become a general practice, either.

Literate programming, and even the literate discussion of programming, has a good deal to contribute to code quality and maintainability, if it could be managed, and in a way orthogonal to the application of SOLID principles. But I'm not sure how it can do so organizationally outside, perhaps, the most rigorous and rarefied of professional programming environments.