The Essence of Programming (2021)

by jwdunneon 8/17/2022, 9:46 PMwith 64 comments

by harry8on 8/18/2022, 7:00 AM

The essence of programming is designing a way to transform data. You ask yourself, "what is the nature input data and how it arrives" "what is the nature of the desired output data."

When I started learning programming from books, the internet and later formal study this wasn't stressed nearly enough. A program transforms data. So much else starts falling into place about all the various abstraction techniques useful to perform that kind of transformation (procedural, oo, functional etc) and technologies used to assist, (testing, debuggers, etc) once you keep that thought front and central in your mind. At least I feel it really did it for me.

Reading code is the same. Where are the places the data coming in? Where is it going out? Sometimes this is a better place to start than "main"

I'm sure that's really obvious to many but I'm not sure it is to the author in a way that is being successfully communicated:

”programming is a tool to solve problems that you have in the domain of computers”

Taking a baseball bat to a computer, on some level can meet the suggested definition of programming, but I don't like it that it does. I also don't much care for the use of "problems" there either.

The above is especially opinionated because I'm particularly interested if people disagree!

edit: I'm definitely still learning programming here.

by avereveardon 8/18/2022, 3:15 AM

Ugh. Programming is not craft. It becomes craft when journeyman that haven't yet matured into being able to understand the whole system see the application of holistic approach with success and think they've reached some sort of enlightenment.

Approaching the issue of solving problems with incomplete information, they end up with circular conclusion, like the essence of programming being operating the computer.

At some point some realize that programming is modeling a domain and it's transformations into a format that is computable and then engineering starts to happen.

by gorgoileron 8/17/2022, 10:47 PM

This seems on the right track. On the notion of ”programming is a tool to solve problems that you have in the domain of computers” though:

Writing a computer program serves two separate purposes: to communicate a set of functions to the machine, but also to communicate the purpose of these functions to a human. The former is for execution and the latter is for code review and, later, debugging.

Any old clown can get a computer to do something approximating a solution to the problem at hand. If it has bugs then the author — or more likely some other poor sap down the line — will be on the hook to first understand the intent, and then make the code match the intent.

In a professional shop you will have code review where this “debugging” step happens first. Your reviewer will read your code to infer your intent and upon agreeing with that, check your style and composition for inconsistencies or lack of clarity.

To that extent, a large codebase is akin to a mathematical proof where each new module and function represents a lemma on which the final proof depends. No serious scientist would present their thesis without a progression of isolated and carefully laid out lemmas and corollaries, and no serious programmer throws up a thousand line function with 15 arguments, global state, and — if in a dynamic language like Python — inconsistent return types.

Your code is a proof to your reviewer that you are on the right track. The fact that it happens to execute without bugs and produces the correct output is far less important than the code being readable, comprehensible, and consistent as part of the larger system.

by kazinatoron 8/18/2022, 12:05 AM

I don't think programmers, let alone beginners, worry that much about the best way to solve a problem. Usually they are struggling with any way to solve it at all.

You may get the impression from reading some online Q&A sites that everyone wants "the best" solution because of the way a lot of questions get asked. But in fact, phrases like "a good way" or "the best way" are more often than not purely rhetorical devices whose purpose is to appear competent.

The person is trying to ask the question in a way which suggests that he or she does know how to solve the problem in ways that are not so great, and is just looking for a better way. In fact, the reason they are asking is that they don't know even a bad way to solve the problem. They may have a partial solution, which has hit some roadblock or whatever.

People in this type of field sometimes have a hard time admitting they don't know how to do something. Particularly if they are not such beginners and have a track record of solving problems on their own, which has become part of their self-image.

by eline44on 8/17/2022, 11:36 PM

The author said that he found that most beginners spend exorbitant amounts of time finding the best way - I don't have the same experiences; It goes mostly like this: Imagine wanting to play a game of chess but not knowing the rules - of course, you won't be playing much chess, maybe you will be moving some pieces, but that would not be the chess, you don't know what moves/games are possible - it will feel daunting and impossibly difficult; now if you ever watched chess master play the game, it feels so super easy, mostly because he has the idea/intuition about the game; he doesn't have to think about every single possible move, only a portion of them that he knows (through a pattern matching - he played tens of thousands of games, and probably more) that are good; now replace 'game' with programming; beginners are handicapped by abstraction and complexity that is today's world of programming, and not the by the 'perfectionism'. Once they understand the scope of the problem they are trying to solve, and the possible ways they can go around solving it, the implementation is just trivial, even for beginners. For example, take junior web devs for example. How can you not expect them to waste time finding the 'right choice' (not even the best one), when there are hundreds of new web frameworks, css toolkits, html preprocessors etc...

Also, about the 'best way' - given the 'right' set of constrains, there is the best way to do almost anything. Unfortunately, most of the times, there is almost no constrains.

by ozimon 8/18/2022, 10:39 AM

I just got my revelation on this when I got to writing scripts to manage servers.

Since I am developer I was always starting from position where I need "reusable and flexible" script. This approach got me stranded in doing stuff like updating TLS certificates by clicking in IIS settings - because I did not want even to start if I did not have "perfect script".

At some point I went "OK I am just going to do script for this single server" - well script is reusable because it is simple and for another server I just change variables inside of the script. It saves me a lot of time already.

Lots of business line software does not need flexibility but developers tend to have need to build a framework because somehow they feel like they are not real developers.

by ncmncmon 8/18/2022, 10:15 AM

Any claimed "essence of programming" has as much validity as an "essence of art". It is each generation's responsibility to overturn their spiritual parents' essence and invent a new one.

Of course nothing is lost, and we are all richer for the adoption of new ways of seeing and thinking.

The experience of the '90s and the then insistence on primacy of "paradigms" should stand as a red warning to us all. Programming is nothing if not a creative endeavor, with as much scope for original thought and insight as any other. If you are not learning new "essences" on a regular basis, you are wasting your life.

by edfletcher_t137on 8/18/2022, 12:35 AM

"Data dominates. If you've chosen the right data structures and organized things well, the algorithms will almost always be self-evident. Data structures, not algorithms, are central to programming." - Rob Pike, "Rules of Programming"

The essence of programming is data.