I mostly blog about more or less abstract topics like Agile development processes, design patterns, and TDD strategies because it's what I'm mostly interested in at the moment. I definitely think these topics are important, but it's often doing the basic things poorly that make a system unreliable and hard to modify and support. There are some underlying activities that we have to do to make a system function well, and some of these things aren't really described very well by the Agile canon.
Case in point from a conversation I had with a colleague at lunch today, the legacy systems that my team centers around share a database that lacks some logical referential integrity checks and uniqueness constraints. We were groaning that the database schema allows a motley collection of orphan records and duplicate data that is making our lives harder. That's the kind of activity that I would consider so intrinsic to building software that I wouldn't even think to talk about it, but the effect of the missing database constraints has caused us some difficulties. I wouldn't say that all of us need to be experts, but I think every single enterprise developer needs to have some level of comfort with referential database theory.
Here are some other basic things that I think fall through the cracks a bit if you're not paying attention:
- Database indexes. Too much, too little, or just the wrong indexing will drag your code down
- The ACID rules and transactional boundaries. I think the mass majority of systems I've seen do a bad job of governing their transactional boundaries. I'm not talking about "look those jokers didn't use the perfect transaction isolation level there," I mean zero transactions at all. When things blow up and go wrong, will your code and database end up in an invalid state? Will you lose data? Drop messages? It's not just important for database development either, you also need to consider the interaction with any kind of external system or messaging infrastructure as well.
- Database archiving. Databases can get awfully slow when table sizes get bigger. I watched a batch processing system implemented by a very large and prominent consulting company (here comes the script kiddies!) become almost useless in its 6th week of production due to severe performance problems. The culprit? A very poor database design from the software vendor was compounded by the lack of a database archiving strategy. I got to lead a team that built a replacement a couple months later. Our system archived records the second that the workflow was completed. Our throughput was 2 orders of magnitude higher.
- File system archiving. Weird, nasty things happen when servers run out of disk space.
- One of the things I love about .Net over the "Set objEverything = Nothing" nonsense in VB6 is garbage collection, but it isn't magic. Gotta make sure that unmanaged resources are cleaned up.
- Database connection discipline. One of the many reasons I think we all need to stop writing so much ADO.Net code by hand is to eliminate the possibility of screwing up database connection usage. I've been told this isn't as big an issue with the newer Oracle and Sql Server versions as it was 5 years ago, but I wouldn't bet the farm on the database being able to handle a couple thousand orphaned database connections.
- This is probably rare, but I had to fight with an XP zealot PM over this one time. If you're going to use some sort of offline pessimistic lock, you need to have some sort of mechanism to clear out stale locks. Things will go wrong.
- Adequate tracing and auditing - that is easily accessible and useful to the production support folks. In Agile process speak - remember that the production support organization is an important stakeholder in your project. In my case we're the support as well, so I think we'll care a bit about this one. On the other hand I detest code with too much tracing because I think it obfuscates the code and generates too much noise to be useful.
- Coding standard. Consistency is the hobgoblin of little minds but the hallmark of a smoothly running software team. This is called out very specifically by XP as a core practice, but I think my company has done a poor job of developing and following a coding standard in the past.
I've seen Agile teams screw themselves by taking a too literal approach to the XP ideals by putting on the blinders and only focusing on the functional requirements from the customer. There are additional technical requirements that could be termed a "cost of doing business" expense for building a system. You have to fulfil these requirements to make the system viable. It's our responsibility as developers and architects to communicate these issues to the project manager to insure these purely technical stories are adequately accomodated by the iteration and release plans. Now, what to do when the PM doesn't listen to you? Um, run away?