Best practices in brownfield development

The wikipedia definition for the term 'brownfield is this : ''In the United States city planning jargon, brownfield land (or simply a brownfield) is land previously used for industrial purposes or certain commercial uses. The land may be contaminated by low concentrations of hazardous waste or pollution, and has the potential to be reused once it is cleaned up. Land that is more severely contaminated and has high concentrations of hazardous waste or pollution, such as a Superfund site, does not fall under the brownfield classification. Mothballed brownfields are properties which the owners are not willing to transfer or put to productive reuse.[2]' .

I couldn't agreed more with this definition, especially about the hazardous waste and pollution parts. Brownfield development in software industry is very common, rarely a developer will develop a system from scratch without dealing with legacy code and applications.

In my opinion brownfield development is hard, and understudied. Every book on software development teaches how to develop a green field project, e.g. a project developed from scratch. Which is good. But no one is taught the essential skills and practices of brownfield development.

And that hurts. Very much.

What are the problems associated with brownfield development? They are many and varied and depend on both technical and human reasons:

• Lack of understanding on the underlying reasons why a technical solution was chooses to solve a particular problem.


• Lack of any legacy technical documentation and tests.


• Developers think they know the 'only right way' to develop software.


• Fear and uncertainty of breaking existing, working functionality.

When working on brownfield project I've found the following to be true and very helpful and productive:

Do not try to re-engineering the project when implementing new functionality, when the time comes that will be the only thing you will be doing. That way your will not bog down on useless, defect prone work.

• Do it as the other guy, use the same existing infrastructure and follow the same development principles. That way you will not add competing solutions and will achieve a homogeneous environment.



• Write a short, concise documentation for each feature you develop. That way everybody will know what you have done and why.



• Document key technical solution and practices in the project. Some problems and questions keep reappearing over and over. That way you will save everyone the time and effort of figuring them by them selves.



• Write smoke tests (functional or unit) for the core functionality of the project. That way you will always know if the core works or not.

The goal here is to not increase needlessly the complexity of the system, to save time and effort and to quickly gain the core knowledge the other guys had. If the client wanted you to re-engineer their system they would have hired you for that specific goal. It is often better to follow existing practices and procedures than to try to reinvent the wheel and fall in a bottomless pit of defects keeping popping up and deadlines never met.

Another day sprint

I'm concluding another day and a half development sprint. I started yesterday at five a.m. with a trip to Zagreb which lasted till six p.m. , filled with back to back blood grinding meetings. I've returned to Labin at eleven p.m. and started working on a deadline pressed project with three other developer. We've succesfully deployed the newest application version fifteen minutes ago. Another hour, and I'm crashing and burning into the nearest bed.

What lessons I've learned:

1. Writing at least functional, or just smoke, tests solves much development problems


2. Task development is half of well done feature


3. Regular standup meetings for tracking progresss and quick reactions to changes in the project situation


4. Wokrin in a virtual enviroment saves time and money, and increases the speed a new developer can join an undergoing project


5. Power naps help a lot

What is the motivation for doing a day and a half development sprint? For me is the challenge and the opportunity to be a hero, to save the day. To complete something hard and critical, which not many people can do. For others, I don't know exactly, but I recon is the same as me. I feel downright special right now.

Well I am special. so are my colleagues, we are a special team which completed something special. Nobody can take that away from us.

Sharing your knowledge

Why are developers, and people in general , so prone to entropy? Most of the software developers I know are really quick to throw aways good engineering practices in favor of writing more code. And use the same code writing exercise as an excuse of not doing those practices in the first place.
I have to much work to do! See, how many bugs I have to fix! What do I get from writing all these down, nobody will read that! Testing is stupid, I will be the only one doing it , everyone else doesn't do it!
Oh , how many such sentences I've heard from everyone I talked to. And the energy I spent correcting each and different attitude. Ah! And then to see them fall back to their evil no engineering ways, practically makes me wonder why do I try.
OK, I try , because I really want to teach others what I know and think that should be a practice for all experienced developers, only this way our profession will grow and evolve. I do not gain anything by not sharing and teaching others.
What are the benefits of sharing (for example with your team mates):

• You can shuffle boring and uninteresting work to them, while you focus on more challenging stuff



• You can discuss problems with someone who understands the topic , and possibly made some learning on their own



• You learn by teaching



• People around you will write code with greater quality

Teaching others , or sharing knowledge , has its drawbacks :

• Time spent teaching others is not time spent on improving you


• Teaching others (in my experience) often diminishes your knowledge in the eyes of others , since for them is so easily gained


• Teaching is a strenuous and continuous process with unknown results (if you want to see big bangs for you bucks quickly, better be ready for disappointment)

And the greatest truth of all: you can't force people to really learn and implement that they are not interested in. Especially if you do not a rifle on their heads.

Back to work

'm nervous, really nervous. For the last two weeks I was at home, resting and relaxing on my hard earned holiday. These were the first totally relaxed two weeks of holiday in , I think, more then five years. And now, today, I'm going back to work. And I'm so nervous. Really.

For the past two weeks I haven't studied anything related to the field of Software Engineering. Instead I spent my time with my son Sven and my wife Tijana, I read fantasy novels (Steven Erikson “The Bonehunters”) and prepared a 4E D&D quest for my wife and our two friends. In short I was a lazy bug in all. Now is time to go to work and enter the grinding machine of stress, constant learning, long hours and impossible deadlines (no deadline is impossible, it is just impossible what you want to do in that time span with the resources you have).

How do I cope with this emotion? Well I woke early, I restarted my morning pilates exercises, opened a DNR TV video for Windows Workflow Foundation. During lunch break I will read “Software Engineering” 8th edition, and will begin a new practicing advanced ASP.NET 3.5 in hopes of one day continue with WCF, WPF and F#.

Will I succeed? Will I be my old self and continue my work as usual? Or will I fall victim to laziness and the eternal question: Which is best, fighter or wizard?

Second Opencoffe Istra meeting

The second Istrian Open Coffee meeting will be held on Saturday 15th at 18:00 hours at the “Cvajner” caffe on the ancient roman square of Pula, on the tip and shorelines of Istra, the greatest peninsula of world. Come and join your fellow developers with excellent coffee, free Internet wireless connection and exhilarating talk about everything web development, web design, business and marketing on the web. Everything goes, as long is remotely related to the web.

To find out more please join the Opencoffee Istra group. There is no shortage of people willing to help a fellow web enthusiast.

A special call is made to all web designers or front end engineers out there. On the last meeting the tip of the scale went to the developer, business side of things. The present web designers felt a bit alone with their tags, photoshops and animated gifs (or should I say flashy thingis). If you are a web front-end enthusiast your are more then welcome on out next gathering.

Do not forget, the next meeting will be held on the 15th of August, at the “Cvajner” caffee in Pula on the ancient roman forum .

Document first , document later

There is a great debate in my head, what is best – to document first , or to document latter. The wanna be engineer in my head would scream “Document first!”, but my practical side, the one that has been into the trenches, that has seen things, horrible things would say “Document later!”. How to reconcile those two opposing views?

Documents can be generally be placed into two broad categories, those that are:

• Generally written before development


• Generally written afterwards development

Why do I say Generally? Because in software development nothing is set in stone, and everything is fluid, thus documents cannot be said to have been written always before head or afterwards a piece of software is developed( class, module, library etc.) . A set of documents could be generally written before head, like requirements or the high level architecture, and some are generally written afterwards like in some cases detailed API technical specification.

What makes a document fall into one of the two general categories. Two things :

• Type of document


• Current state of the software life cycle

The Type of the document generally dictates when a software piece will be written. Like, requirements or the functional specification or something like that which is generally needed before a piece of software is written. And some which cannot be (or can, but albeit very hardly depending on the skill of the developer and the knowledge of the system), like detailed technical specification which is not ever accurate when written first and is ever (hopefully) updated once a particular piece of software is written.

The other most contributing factor is the current state of the software life cycle a team find its self in. When developing a green field software it is generally best to do the documentation first ( I'm not discussing waterfall here, you could develop module after module, and before each module write the technical documentation for it before going into development), but when a team finds it self knee deep in a brownfield development project, which generally doesn't have any documentation what so ever, which isn't even commented properly so you cannot generate an API spec automatically which is remotely useful to anyone, when you need to fix unknown defects, when you need to implement a piece of functionality and don't know how to place it inside the project and how it is to be connected to everything else, writing documentation first could be and exercise in futility since you don't know enough about the system. This will change gradually during brownfield development when you will reach a state where writing design documentation first would be a better choice in terms of quality of work and writing speed place into the construction of the document.

So, what to say to may two different parts? It depends. And that is the only answer I can give them, which places my little self once again on the verge of a nervous breakdown due to over thinking so simple things. But documentation is important, some cannot see that, but those cocky bastards will soon find them selves trying to remember how they did that, or where somebody placed this and how is generally this implemented and so on, and will loose valuable money debugging the thing out of its life in order to find something which would take five flimsy minutes to read. Who is stupid now, ha?!

Database development blues

In most line of business application there are two major types of artifacts involved : the database and the application. Sometime there are two types of developers, but mostly that is not the case. The same pure smuck that built an entity inside the application will usually write the database and UI code for it.
Well generally I don't think thats wrong. I strongly belive in cross functional teams where each developer can work with equal skill on each application tier. When you work in the outsourcing industry as I do, you quickly came to appreciate those team members who seem to be able to do evertyhing: database modeling, stored procedures, domain objects, html and css layouts, documentation, testing and so on.
The major problem with the approach of one developer doing both the application and the database work is that that developer approaches the database work in the same way as he approaches the development of a typical application. But those two types of artifacts are not the same, they behave differently during each phase of the product life cyle.
The application code is fairly easy to modify, extend and delete where the database code is always resistant to change, and it grows more resistant as the product and its database grows. Why is that?
The database is composed of three different types of artifacts:
Data structure
Programmed logic
Data
Data structure is the logical representation of the types of data stored in the database and their respectiove relations. We call them tables.
The programmed logic is what we say to the database it should do with the data that goes in and out of a table. We call this : stored procedures, functions, triggers, constraints etc. To most developers this is the closest thing to real development they will find in a database project.
The data, well, there is not much to say about that. This is the meat and bones of what a database is to most people. The reason d'etre of a database. Users data, application or system data stored for eternity in a database for all to see and use. Most databases store data about them selves in special databases and tables. Figure that.
Well the thing is with databases that the data stored inside them is the most valuable thing in a line of business project. A developer can frag the entire application code and it wouldn't be even as worse if he fragged the data stored in a database. So, losing or modifying data in a database is not a good thing.
As a database (and its accompanied application code) is being developed the volume of useful, production data inside it grows. Each change to the database structure becomes much hardere, since you cannot loose data (thats why most people try to nail the database structure before starting the development process) , and it gets even worse when the database goes live to live free in the wild of an enterprise IT infrastructure. If you frag the database there you can kiss your ass, head and small fingers good bye because angry people in business suits will descend apon your small little you like vultures on a helpless white rabbit with its back pawns broken. Its that bad.
So working with the database is hard and risky.
The underlying fact with the database is that its always changing. New data is added inside it, changed, deleted. The structure is change, new tables are being added each day. Even the programmed logic changes from time to time. It is always changing regardless of it being in development or in production. While the nice and good application code changes quite rapidly and easily during development but its damn impossible to change when its compiled in an executable and shipped to remote production server in the prairie.
So what to do when you have a line of business product to ship on a rapid development track with database and application work done in parallel by the same people. Well , its simple.
Stop making changes in the database manually. Write scripts.
Repeatable, stable , data saving, scripts.
Then , respect the database. It is not your run of mill .NET application. It requires different development practices.
In database work you done have the write, compile, test life cycle of application development. Once you loose data it is gone, once you mess thing its hard to go back.
Think of a new development process for working with the database. It will repay you ten fold in virgins.