Favorite line "... the software development process is exploratory by nature."
I always say the customer/client does not have requirements, they have problems. You will not discover all the requirements until you start solving some of the problems and providing solutions. Only then will they say "oh but...." and drop more requirements on you that they didn't think of up front.
Back to that quote. It's not that software development is exploratory in itself. It's that the development is intertwined with an exploration of the problem being solved.
As a lead developer building a platform dealing with the intricacies of union agreements and labor restrictions, this summarizes exactly the thought process that my team has gone through in the last year.
We started with a simple problem that plagues HR departments in every conceivable industry with unions, finding substitute personnel and erroneously assumed that it was a simple fix. Over the past year and a half we have accumulated a great deal of knowledge after interacting with as many people as possible and have finally released a version that meets our original criteria (and much more). It was obviously not a simple fix.
If I have one thing to tell anyone who is looking for business ideas to try out their new programming skills on, I strongly suggest taking the time to learn as much as possible about the people to whom you want to provide a solution, then recruiting one of them to help you build it, lest you become another project that solves a non-issue beautifully.
"Being able to program is not the problem. Understanding the problem is the problem" - one of my lecturers in University.
Agree 100%. In fact, this is how I built my career -- I knew the tech ok, but realized that I was never going to have the type of influence I needed unless I got into the product side as well.
In the consulting world, we call this job "enterprise architecture". It does, in fact, pay very well: it requires someone with both a sharp business mind and comprehensive technical skills, and those are very difficult to find in one person. I personally am more of a "jack of all trades" type; but you can be a successful architect by focusing on specific technologies as well.
I honestly find that it's easier to take someone who's a hacker type, and teach them the business. You look at the business itself as a large, complex system and model your application development around that. But you also have to be a good enough technologist yourself so that you can tell your dev team when their designs don't match up to the business problem (this is a common problem when requirements are not clearly communicated).
A good architect is the person who understands both the business context and the technology implementation. You don't have to be in-the-weeds building the product, but often you do have to build quick POCs to prove out an approach before handing off the designs to development - so being able to code is a necessity IMO.
> The nature of the beast is that software requirements rarely change
Put like this, it's hard to agree with the above statement.
Following the spirit of the article, I assume the author means that the problem domain is pretty stable. But I've been in this for more than 20 years, and I know that requirements always change. Not only our understanding, but also the customer/user's understanding of their needs and priorities change.
(Edit: typo)
Yeah programmers / developers / software engineers act as an interface for other people for computers. It's not surprising that these qualities of that 'interface' affects price:
- intuitiveness i.e how easy is it to communicate with this person (language fluency, etc...)
- quality i.e. how well do this person understand not only the requirements but also the actual goals
- 'latency' i.e. how convenient and how fast can you communicate with this person (time zone, can you both see facial expressions, hear changes in voice, etc...)
This is a post which touches on a subset of "hard" parts while downplaying other hard parts. I agree that a developer with a deep domain knowledge is precious. In fact, I have seen quite a few developers rising up the hierarchy despite being mediocre on technical skills. The code written by them is big ball of mud but they are good at communicating with stakeholders and understand the business well(And also good at drawing boxes) . This does not mean that there is no cost to it. It is just hidden from others. The memory leak issue, the several bugs introduced due to multiple if-elses encoding the business rules, swallowing exceptions or failing to set right properties for client timeouts. These can exist despite the "deep context" and most of the time junior developers are to be blamed for this because the so-called architect is busy all day in meeting with management.
Also the experience may vary. In my admittedly not so long career(less than a decade) I have seen teams where business rules are the major source of complexity while there are other teams which have less business rules (example the databases/data warehouse/build systems team). Admittedly there are less teams of second type in the world so the general perception is that the hardest part is communication and understanding of business context.
Coming to domain knowledge, even the Mainframe and COBOL chaps make a lot of money while smart open-source contributors freelancing don't. Money is not the only way to judge the hardest problem about software development.
It is exactly this which makes interviewing so flawed in the field.
Solving well defined algorithm puzzles has zero bearing on the skills described by the author.
If you found the article interesting or insightful, I would encourage you strongly to take a look at the book "Peopleware" by Lister and DeMarco. The first edition was published in 1978; the edition I have is from the early 90s, and I think there are newer versions since (at the usual astronomical cost of books that get sold as required reading for college courses), but you can get whatever is cheap for a used copy. Most of it hasn't changed much.
That, ironically, is part of the authors' point: software engineering hasn't changed that much. They were saying that in the late 70s, looking back on the past two decades (all the way back to the late 50s, when software was typically written in assembler or machine language and had to be rewritten if you bought a new computer!), but it hasn't become much less true. The technology changes, sure, but the failure of software development projects has never been mostly the result of technology. It's always been on the human side -- failure to understand the requirements, failure to meet them, failure to estimate the effort appropriately, failure to work with the customer, failure of the customer to understand what they were paying for, etc. There's a long list of things that go wrong, and I suspect anyone who has been in software for a while has seen many of them.
It was eye-opening for me, the first time I read it, to realize that people had been dealing with the same issues I was dealing with, for longer than I'd been alive. (And a bit depressing, too, that we seemingly haven't gotten much better.) Languages and project-management methodologies come and go, and the tech skills and understanding are certainly necessary, but they are not sufficient. The business knowledge and human factors seem to be the difference, or at least the largest controllable variable that leads to a difference, in a successful or failed outcome.
> Sadly, offices are perhaps the closest modern equivalent to a "village" that we have left.
Ha! We need a "tribal leadership" discussion forking off this point.
Why does some independent contractor make bank while others are fighting for pennies? Because it's not about "what you know but who you know."
The author could have just left the "Remote" part out. This piece is really about hiring independents for gigs vs. building and educating your own team.
Once you are experienced enough you'll realize two things, that all programming languages let you solve problems and that programming in itself is not the problem, but rather applying it to solve business requirements is the true problem to solve.
This is an insanely useful article to read before a major project kick off. Kudos!
Here's the problem:
- Here's a pool of knowledge about software development: hardware, operating systems, memory, disks, file formats, databases, networks, protocols, languages, debuggers, design patterns, security, accessibility, UI/UX, distributed systems, paradigms, typical algorithms & data structures, and CS problems
- There's a pool of knowledge about whatever industry you get into as a developer: user demands, existing workflows, existing infrastructure, previous decisions, legal regulation & compliance, physical laws, profitability, and practical limits.
Your software development skills should reach a point where you don't write "Bad Code" -- anything that's wrong like loading a entire database table that eats memory when you can read individual rows, storing passwords in cleartext, or not doing anything for accessibility (this is not design pattern, space/tab debates). These have been done hundreds of times by new and 'experienced' people.
It takes time to get to this point. More time than anyone likes to admit because the pool of knowledge grows and shrinks daily, but has undoubtedly had a net expansion since computers were a thing.
It takes time to get deep knowledge about whatever industry you get into. This is different for every industry. There's a practical minimum that you need to work on solutions or do maintenance on software within this industry. This is to avoid "Bad Code" which will hurt you, other people, or your business.
You can gain industry knowledge by just being given problems and being shown. This is probably how most of us know our industries from the get-go. A minority of us came from those industries and transitioned to programming later, so we already had a base level of knowledge of our problems.
If I've got the definition of Deep Context right from this article, it means to get to that point, you have to spend a good amount of time within the industry. It's not something you can gain completely by reading out of a book.
If you're to gain deep context within an industry, you have to devote some time away from software. You can't do both at the same instant (but certainly within a day). When you study an industry, there's an opportunity cost to not learning something new about software and vice versa.
When you add more requirements to a single job, it increases the time we have to spend before we're employable. Not every industry changes as fast as software does, but some certainly do, possibly catalyzed by software.
If you increase the time requirements, it's going to reduce the available pool of engineers as long as all of the engineers are honest and don't apply for jobs or remote contracts until they're ready.
If you don't want the time requirements to increase, you have pay the opportunity costs from one of your pools of knowledge.
So really, we need a much better "good enough" for employing developers and career development, including teaching software and industry knowledge. Because eventually the time requirements are going to become steeper and steeper. It can't go up forever.
This article sounds a lot like a big argument for Domain Driven Design without naming it.
Is there a way to read this article without logging in?
The hardest thing about software development is having people in charge of you that know jack shit about software.
I thought the hard thing in software development was naming things and off by one errors?
I agree completely. Rarely the hard problems in coding are thinking of a clever algorithm, or solving a particularly nasty scaling problem. Usually I've struggled the most with understanding and building what the customer needs. A spec can be interpreted many different ways and will always have to be polished after its completed. Someone who doesn't know what the customer expects will almost certainly create an implementation that has these micro problems that make the product miss it's mark.
It makes me think that we're training the wrong people in college by making CS a very difficult, math heavy field which often causes the more human skilled people to drop out. Programming doesn't have to be anymore math heavy than building a house yet we force undergrads to implement algorithms on paper? The amount of wasted potential talent due to college is staggering.