Subscribe: Lee Harkness's Weblog
Added By: Feedage Forager Feedage Grade B rated
Language: English
data  dave  design patterns  design  engineer  engineering  good  pattern  patterns  problem  requirements  software  system 
Rate this Feed
Rate this feedRate this feedRate this feedRate this feedRate this feed
Rate this feed 1 starRate this feed 2 starRate this feed 3 starRate this feed 4 starRate this feed 5 star

Comments (0)

Feed Details and Statistics Feed Statistics
Preview: Lee Harkness's Weblog

Lee Harkness's Weblog

Lee Harkness's Scattered tech-related thoughts

Last Build Date: Fri, 8 Oct 2010 14:29:15 -0400

Copyright: Copyright 2010

Slight revision in thinking

Tue, 23 Mar 2004 00:06:29 -0500

Went to another NoFluffJustStuff symposium this weekend. Got to chat with Dave Thomas (amongst others). I asked Dave in particular what his thoughts were with respect to the following observation/question:

Other engineering disciplines would not tolerate the failure rates we have (he started shaking his head at this point, but very politely did not interrupt, though I could see that he very much wanted to) so what could we do to get our discipline up to snuff? Of course, I am of the opinion that Design Patterns are key to this, I just wanted the pragmatic programmer's input. Later on I asked him rather more pointedly about the role Design Patterns could play in our industry.

Dave basically said (and I am paraphrasing here, these are not Dave's words, but more what I took away from Dave's words) that we need to stop pretending like we are engineers. Civil engineering has been around for thousands of years, our industry has been around for, what, 50 on the outside? We need to stop trying to apply engineering priniciples to our field and apply the apprecntice/journeyman/master model. After all, how do we get better at *anything*? We practice. So, we aren't doing engineering here, we need to practice our craft more, and we need to stop making the same mistakes over and over again. AS for the role of design patterns, Dave sees them as I do, I think, as encapsulated expertise, as maxims - not recipes. But, and here's the point that makes sense to me, but I will have to think on more, you cannot introduce Design Patterns into the mix too early. If a person does not have sufficient experience (insert Dreyfus Model discussion) to understand the role of Design Patterns in the big picture, then that person will see them as recipes to be followed, not as maxims to be mulled over and kept in mind to guide design work.

So I go home, and mull this over. And I think about what Dave said with respect to us not being engineers. I take his point, and in retrospect I don't know what else I would expect the co-author of "The Pragmatic Programmer" to say. Thinking of that made me think of another book I started to read on the subject "Software Craftsmanship" by Pete McBeen. I decided to dig that up and re-check it out. It basically says the same thing Dave was saying, and you know the funny part? The foreword to that book was written by Dave Thomas. Funny that.

Data Driven Design

Mon, 22 Mar 2004 23:36:29 -0500

Yes, I know the first bit is common knowledge, just bear with me.

The main difference, to my mind, between software engineering and traditional engineering is the fluidity of the requirements. With traditional engineering the steps seem to be:

* Requirerments Analysis * Design * Build

End of story. Once you get past the design phase requirements are, sometimes literally sometimes figuratively, set in stone. With software engineering, the requirements are always changing. Even after the build phase is complete, the requirements continue to change because, in the words of Dave Thomas, the world is changing.

How are we supposed to cope with this? How do you build a system that can be changed in ways you cannot forsee? My friend George Fehrenbach, and also the aforementioned Dave Thomas, as well as lots of folk who write on this subject, have thrown out Data Driven Design as a way to do that.

To me, that makes a lot of sense. It doesn't really solve the problem though. I think it's possible the problem is intractable, so looking for a solution might be pointless. I think designing data driven software mitigates this problem, and that's a good thing.

A data-driven design is one that removes dependencies on a specific implementation of business rules from the system. The specifics of the business rules that the system implements are encapsulated in some data store, and used to configure the system. In effect, each system becomes a highly-specialized framework, using dynamic configuration as much as possible. You'll note that the problem, that we can't forsee how our system will need to be changed, is only mitigated, not removed. We will still have to try and predict what specifics will need to be configurable through data, and if we guess wrong we will have to change the underlying structure to incorporate those changes. But, presumably, if we design carefully, any such changes will be large, far reaching ones, and thus infrequent. The design insulates the system from the smaller, more frequent changes it will see.

In my opinion, combining a highly data-driven approach with the idea of a highly specialized, dynamically extensible framework should lead to more robust software systems.

Design Patterns - Motivation (Part II)

Wed, 11 Feb 2004 18:36:26 -0500

The answer my friend, is Abstract Data Types. Some smart folks looked around and thought "Hey, it seems like I am always making lists of things. It doesn't matter what things I am dealing with, or what language I am writing in, I am always making lists of things that I want to add to, delete from, and iterate over. I could talk about this "List" idea as an abstract concept, defined only by the operations it supports". Once this realization was made, the next step could be taken. Abstract Data Types allowed designers to talk about their design at a higher level than lines of code, without ever talking about implementation. This was a Good Thing. And to get there we used the same trick, abstracting operation from data, but in a slightly different way. In fact, the Standard Template Library in C++ did exactly what we did before, abstracted away the operation from the data and parameterized the data type. Abstract Data Types are good, but they don't give us the catalog of known good bridge designs that we were after. They give us the equivalent of standardized materials and fasteners. Engineering certainly benefits from standardized materials and fasteners, but what do we need to do to take the next step? As you may have guessed, I believe that the next step is the use of Design Patterns. At this point, we need to define what we mean by a Design Pattern. I will use the one that the GoF used, which is "A pattern is a proven [abstract] solution to a problem in a context". The word "proven" is important as it implies that this way of doing things has been peer-reviewed and has passed muster in all situations that it's been analyzed. The "solution to a problem" bit is important because it keeps us on point, and reminds us that this stuff isn't all theory, we are trying to get work done. The "in a context" part is important because the pattern doesn't exist in a vacuum. One of the benefits to using standard patterns is that of rapid comprehension, when you say "I am using Pattern A" your audience makes some assumptions about how and why you are using Pattern A. Those assumptions figure prominently in the rapid communication of your architecture and intent, and they are also the context that this pattern exists in. To illustrate, imagine that you have a message on your answering machine from a friend that says "come over to my place and bring a screwdriver". There are a set of assumptions you might make as to what your friend needs help with. You might bring over a goodly amount of reference material involving the torsional analysis of the screwdriver. Upon arrival at your friend's place you find him holding a paint can. Now, let's assume that you are not very bright. Your friend might have to explain to you what he wants you to do with the screwdriver, and why. Now, you *can*, of course, pry up the lid of the paint can with the screwdriver, but you lose a couple things when you do - all the analysis of the screwdriver that's been done is lose, since it assumes you will be turning something, and the ability to rapidly communicate the job to be done is lost, for the same reason. You are using the tool outside its context. So it is with Design Patterns that we realize the goal of having a set of "known good" designs. Using these designs in your software architecture gives you a couple main benefits, as shown in the above "screwdriver" story, namely: *The ability to quickly communicate design and intent *The ability to take advantage of time-tested, peer-reviewed techniques. Patterns aren't the answer to all of Software Engineering's woes. We still have to work hard to write bug-free, easily maintainable code. The current hype (that appears to wax and wane) that surrounds Design Patterns could lead to conditions that dilute or even eliminate one of its main benefits. If you end up calling *everything* a pattern, which is easy to do since the definition must, of necessity, be broad and vague, then the term loses its meaning. I could start refer[...]

Design Patterns - Motivation

Thu, 16 Oct 2003 00:02:12 -0400

To talk about the motivation for and role of design patterns in software engineering, let's take a look at how a "traditional" engineer and a software engineer might do their work. A community determines a need for a bridge. The community retains the services of a civil engineer. The engineer will confer with the community to get some requirements such as the expected average and maximum load to be supported and maybe some fiscal limitations, as well as the inteded location for the bridge. The engineer might then go to the intended location for the bridge and gather some requirements from there such as length and depth of the gap to span and the characteristics of the substrate to anchor to. Then the engineer might consult a catalog of known good bridge designs and evaluate each one with respect to the set of requirements, eventually using some judgement to determine a good bridge design. At this point, it's all implementation. Compare this to what a Software Engineer might do. A client determines a need for a software system. The client retains the services of a software engineer. The engineer will confer with the client to get some requirements such as the functional requirements and expected load as well as the intended environment for the system. There will be additional non-functional requirements such as latency and fiscal issues. The engineer then goes off and - what? Designs the system. Typically there is only personal experience and maybe anecdotal evidence to assist in determining the best design. There is no catalog of known good designs. There are methodologies that try to assist you in making sure you've thought of all the typical things. But by and large it's a black art at this point. And this is a Bad Thing. It's a bad thing because software is controlling more and more critical systems. We have software assisting in delivery of radiation doses into patients, the movement of planes and trains and rockets, the timing of traffic signals. We can't afford sloppy engineering. Can you imagine if bridges would randomly collapse and, when asked about this phenomenon the engineer would respond "yeah, I dunno, sometimes they just do that". We wouldn't tolerate that, but for some reason we think it's ok that software behaves that way. We can't afford to let this behahvior continue - ask the Italian Space Agency. Due to a numeric type overflow the Ariane 5 blew up 30 some odd seconds after launch. Because of a bad design in the software system a rocket, a satellite, and for a while the reputation of the ISA went up in smoke. There are many such examples. So, how do we fix this? To answer that, let's look at what we've done so far. I would like to present a brief history of software engineering from the dawn of time until now, hitting highlights carefully selected to make my point. In the beginning we flipped switches and watched light bulbs, and that was good. But the novelty soon wore off, and even after we came up with clever ways to persist our switch settings onto paper or magnetic media we realized that interacting with the computer at this level was highly inefficient. At some point some smart folk said "Hey, I am always keying in the same set of 32 1's and 0's to add numbers - well, almost thhe same - the only thing that changes is the numbers to add. Why don't I write 1's and 0's to take English-looking words (like "ADD") representing operations and the data they operate on as input and output the appropriate 1's and 0's. And voila, assembler language was born. This was a Good Thing. It was good for two reasons. Error prevention and detection and moving closer to the problem space. If you have to key in 32 1's and 0's to add two numbers, the odds of you flipping a bit somewhere are fairly good. And if you do, the odds of you noticing are not good. Errors are easy to make and hard to spot. If you get to write "ADD 1, 1" the odds of you misspelling "ADD" or keying in the wrong number are[...]

Thoughts on Patterns

Sun, 1 Jun 2003 19:07:12 -0400

As seen by a low-level peon.

Back in the day we flipped switches to tell the computer what to do, and that was good. The fact that it worked at all was quite enough. Then the number of switches we could flip became too much for our little brains, and we looked for ways to reduce the load. We noticed recurring patterns of switches, and gave them names, parameterized them and we were able to reduce the complexity and move on. This was a good move. So good that we repeated it to go from assemblers to high level languages, procedures, and then objects. But what we didn't realize at first is that with each step to a higher level of abstraction we moved a little further away from the machine and closer to the problem domain, thus giving up a little expressiveness. We didn't notice because the things we could still express were pretty well common to most all the problems we were interested in solving. The "named sets of switch settings", like "print" or "readFile", were generic enough to be suitable for almost all problem domains. But, when we tried to apply this technique a fourth time, we noticed that step would take us too far into a given problem domain. To me, this is why CASE tools and 4GL languages (except maybe things like SQL) never really took off. Because they do one thing pretty well and easily, but once you try and apply them to a broad range of problems they fall down. They just aren't expressive or flexible enough.

To make this next step we need a way to gain another level of abstraction without losing expressiveness. I see Design Patterns as a step in the right direction. A design pattern clearly states a way to solve a problem, but doesn't address how to go about doing this, thus we get a named chunk of experience, but without losing expressiveness. To me, design patterns are the logical "next step" after third generation languages, ADTs and objects, in the evolution of software development techniques.

And I think making this next step is of extreme importance. If you consider Software Engineering a "real" engineering discipline, consider that we don't have any best practices (or rather, we are just starting to) thus we start from scratch everytime we build something. If we care to communicate expertise we have to start from scratch in that discussion as well; describing objects and interfaces and their relationships. Design patterns give us encapsulated expertise with names.

I realize, as pointed out to me today by Maciej Zawadzki, that with the sudden explosion of ad-hoc "patterns" without much regard to the value (or even correctness) or appropriateness of the thing offered up as a pattern that the value of design patterns is diluted, as it screws us out of both benefits (expertise and nomenclature). But with a little judicious "pruning" we should be able to catalog best practices and retain the benefit, enabling our discipline to gain what disciplines have enjoyed for years, something solid to build on. Till then, we are starting from scratch everytime.