> Eventually the pain of working within a single massive source file became overwhelming, and I decided to write my own literate programming tool that could consume a filesystem hierarchy
I wonder if that’s where they went wrong.
Perhaps when your code is too big to fit in a file, it is too big to describe serially at all. And it’s time to split it in two separate pieces, two modules with a formal interface between.
The idea that you could have a nonlinear narrative without any formalism dictating the relationships between paths is perhaps asking too much of the narrative form.
Perhaps that’s the whole point of a module system: creating formalisms so that you have a chance at holding in your head the relationship between one series of procedures and another.
(This is not entirely academic. I have about 100 richly connected single file modules on NPM)
> Perhaps when your code is too big to fit in a file, it is too big to describe serially at all. And it’s time to split it in two separate pieces, two modules with a formal interface between.
I'm not sure these two issues are related. The "tangled" PDF output of haskell-dbus 0.9 was split into chapters, each chapter being roughly a single Haskell module (with a specified API). And books are by nature serial, yet they can express enormously complex ideas -- think of a compilers textbook.
This was all six years ago so my memory's a bit fuzzy, but I remember feeling lost in my own code when it was all in one file. I couldn't do things like "split window and go to top" to find the imports/exports of the current module, and searches for a symbol name return far noisier results when they're all implicitly project-global.
It might be possible to fix these with better tooling, like an editor that could parse literate source and show the rendered output. But then you'd be treating the "source" as a sort of opaque input and editing at the level of "compiled" output, which is behavior I associate more with reverse-engineering than typical software development.
> And books are by nature serial, yet they can express enormously complex ideas -- think of a compilers textbook.
I'd argue that, in many cases, well-organized books have split their material up into chunks with formal interfaces between them. A textbook's table of contents and introduction create a high-level structure, cross-references allow like material to be grouped with like material to make a coherent picture of each subsystem or topic, and chapters can be read out of order or even skipped entirely by some readers. Textbooks for mathematics or computer science are the most obvious examples, but I find the same level of organization and ability to jump around even in popular science books and philosophical discourses. Even the simplistic "five paragraph essay" that is taught in elementary school can be interpreted as an implementation of a formal interface.
All of that is true of a single source file too -- editors can generate tables of content and cross-references. The experience of using a well-organized paper encyclopedia and Wikipedia are fundamentally different. The point of "serial" is that if I want to read a book, I have to pick an origin point and then start scanning linearly. There's no search bar in a textbook.
As a side note: I find the literate programming pattern is more appropriate for combined demo and tests. I am not sure implementation really belongs in the same source location.
I wonder if that’s where they went wrong.
Perhaps when your code is too big to fit in a file, it is too big to describe serially at all. And it’s time to split it in two separate pieces, two modules with a formal interface between.
The idea that you could have a nonlinear narrative without any formalism dictating the relationships between paths is perhaps asking too much of the narrative form.
Perhaps that’s the whole point of a module system: creating formalisms so that you have a chance at holding in your head the relationship between one series of procedures and another.
(This is not entirely academic. I have about 100 richly connected single file modules on NPM)