08 September 2010

How much granularity is enough?

I worked sometime ago on a small/medium sized ERP project. Once again it was a ASP.NET web application.
In order to write a feature the following steps must had to be taken:

  1. Write a stored procedure, or two

  2. For each procedure write a method in the related data service in the data access layer of the application

  3. For each method in the underlying data access service write one or more methods in one or more repository objects in the logic layer.

  4. Write one or more entities, value objects and aggregates related to the procedure written.

  5. For each function write a method in the appropriate object service on top of the logic layer.

  6. On the front end write a service facade for collecting errors and results from the logic service, once again is a one to one mapping (I used the ObjectDataSource control extensively in this project and we had a centralized message collecting system).

  7. Write the functionality on the screen.


When looked from a theoretical aspect each step made sense. A clear separation of concerns, each object doing its small thing etc. But during the implementation I just often found my self wondering Why am I doing this? Each object has only a small variation from the object bellow. Isn't this to much granular?.
It sure made writing tests easier, since the logic in each object wasn't very complicated so the corresponding tests were easier to write and were written much faster, especially on upper layers where I made extensive use of the Moq mocking framework.

07 September 2010

Just unit test?

I just realized something today. Something deep, and profound and totally astonishing . Well, for me anyway. I realized that maybe, just maybe doing unit tests on project is enough. On most projects I have an influence on I strive to have functional and integration tests besides regular unit test. I just feel is best to cover each layer of the application in one fell swoop with functional tests or acceptance test (it depends really how you do them) with a good coverage of unit test for each object separately.
Today I've done a lot of unit tests. I worked late, and I've done a lot of coding. For class I've done I wrote unit test. And with those tests I covered a lot of functionality of the application. When I reached the user interface there wasn't much to test, everything worked fine since I got the majority of the defects with unit tests on the lower layers.
It seems strange to me since we do tests in complete isolation from each other, so no outside dependencies between tests are allowed. Everything is mocked. But with a real fine granularity of objects for a specific functionality it is possible in my opinion to just have unit tests and cover the majority of the expected functionality of the application.

03 September 2010

How to cranck unit tests under pressure, and not feeling dirty at the same time

It's called doing the right thing.
Well thats what I tell my self it is. When I read "Extreme programming Explained" by Kent Beck I read about a story of his about a project where he introduced extreme programming as a development process. The project went well until the deadline got really close and wished he could just throw XP from the window, hack the project together and completed the project in time to collect a big, fat, juicy bonus. He addmitted he panicked, and then calmed him self down when a colleague told him that nothing really changed, that they should plan for the last two weeks of the project as it were an ordinary iteration, and execute it as such with unit testing and everything.
Now I was in a pinch some time ago. Not something unusual, per self, it seems there is not enough time to ever do a project. The project was then involved was in development for some time, and the deadline of the clients patience got really closer as each week passed.
It was a good project, I've architected it and technically oversaw the development for much of its life time as I was in parallel involved in other enterprises. It got a stable architecture, which followed closely my plan, it got unit tests, inversion of control, mocking ,functional tests and a set of Selenium based acceptance tests.
I event wrote a set of QUnit (a javascript unit testing framework) test for the Javascript classes we used on the project.
The problem was that the we got a week left and still a lot of stuff to do if wanted to do it right, before the final client presentation. Like writing tests, logging code etc.
The pressure started to mount, just finish those damned screens everybody yelled (not really but my minds plays such beautiful tricks on me). Just get it done, write those pesky tests later.
The problem was I was feeling really good about the projects, the tests were written quickly and were really useful to ferret defects and I really didn't want to abandon all those three hundred plus tests to entropy and mocking disregard.
What I decided to do is to change my unit testing strategy:


  • Test only logic heavy code. If a method has an if, switch or any other control structure I would test it in respect to the general expected behavior which could result from those control structures.
  • Change focus from data to processes. I left trying to set replicas of real world data, and instead used generic values like "1" for ints and "Any" for strings. This left me with more time and didn't stress my mind to try to figure out real, representative , data sets.
  • Tell your self is good to copy paste (for now). I had tests which were five to seven lines long each. They all had 30%-40% of the same code base. What I originally do I refactor out the common code in a private method and reduce my overall tests size. The downside of this it takes time do do it. Time which I really didn't have. So I copy pasted.
  • In a object stack which covers a specific functionality (e.g. GetAllAccounts) do not retest the same functionality on each step, but instead focus really closely to the task of each object in the stack. For example I would focus really hard on querying and inserting data from the permanent storage in the Data Layer of the application, and focus on data transformation and repository logic in the middle tier of the application.


It kinda worked. I cranked a lot of more code this way and found a lot of defects. I wouldn't suggest it as a unit testing strategy for everyday situations but it kinda works when under pressure.

01 September 2010

Oh, my learning muse

I didn't write in four months now. It didn't seem to matter, nothing did which was related to my work. What happened to me?
Well, I've been pushing my self for the last three years very hard. Learning, working and applying my self in order to grow, learn and advance as a software engineer. And then I crashed, emotionally and physically. I become cranky , tired and frustrated.
I started then working from home, so I didn't have to drive for two hours a day on really low grade roads. I saw more of my family, and I was more rested.
One thing led to another, I started to shift my priorities to my son and wife. I started spending more time with them, going out to play by the sea, watching movies reading books, going out. Just living my life.
What was not right, was the nagging feeling that something was not right. A sick rotten feeling that I was leaving out something vital and important to me. My learning and education. My personal growth as a professional.
It got to the point that I felt my brain rot from not use. And I didn't like it. So I started learning again, "Patterns of Enterprise Application Architecture" by Martin Fowler, and "WPF Unleashed 4" by Adam Nathan. The former as my regular reading and the latter as my computer time in the evening and weekends reading.
I feel more right now. I guess. I don't spend as much time at the computer cranking code and learning new stuff as I would like.
Every time I can sit at my machine and learn , it seems it is ten in the evening and I'm to tired, my head hurts and I just want to unwind. It is really hard to keep up.

12 April 2010

5th WebUG meeting report

On the 8th of April 2010 I held a talk on the topic of "Software Engineering" at the 5th Web user group meeting which was a SQL/DEV user group meeting (since Windays 2010 are coming soon). The talk was held in the Microsoft offices on the top of business building in Zagreb in the Dante conference room.
The presentation went well. I was really nervous beforehand. I thought that there is for sure someone in the audience who will spot an error and point it out. Well no one did , or if he did he didn't point it out loud.
There were some technical difficulties at the beginning. The network was hard to get and the camera crew didn't come so the talk wasn't recorded. Bummer.
Everything else went just fine. The talk was well received by the audience, people came to me afterwards to congratulate me or to talk about some finer point in private.
The hosts were gracious enough to accommodate my special dietary habits so they ordered a vegetarian lasagna for me during the networking part of the event. Everybody else got "Ĩevape" which went away pretty fast.
All in all it was a nice event. I meet some good people and I hope to have a chance to present again on a future WebUG meeting.
Here are some photos taken from the talk:


03 April 2010

WebUG Presentation: Software Engineering

Next week on the 8th of April 2010 I will present on the monthly Web User Group meeting in Zagreb, at the Microsoft Croatia offices on the topic of Software Engineering.
Software Engineering is something very dear to me. It is also a very hard topic to condense in an one hour presentation.
For those who want to attend you can find more here http://www.webug.com.hr/.

18 March 2010

Key principles of software architecture

Software architecture is the first and most important design step of software development project where are defined and indentified the key components of a product encompassing the logical structure of the product being built, the technologies used and how they are used and the definition of the key processes handled by the system both business and technological.

Software architecture is software design , but not all software design is software architecture. Architecture implies a high level overview of things. It set the boundaries and key guidelines of the development effort without going into a detailed specification of each implemented use case on the technological or business level.

Every developed system has an architecture. Most software architectures are incidental, matters of accident and small decisions made by different people at different times without accord or shared vision.

Software architecture is both easy and hard. It is easy because it doesn't get into every detail of the system being built, that is for the detailed design effort it is hard because it requires an encompassing vision of each key element of a system and the capability to set it in clear, concise and understandable writing.

As the product changes the architecture will change. Decisions are being made by each key stroke, by each library reference and each guideline not met - no matter how small or temporary.

The software architecture document must be therefore written in a short and concise way with pictures describing each key section of it in order to facilitate rapid understanding and ease of maintenance.

There are muliple way of doing software architecture. A good , and free book published by Microsofts Patterns and Practices covers all aspects of it, the Microsoft Application Architecture Guide, 2nd Edition. I would recommend it to everyone.


A dummy high-level model for a HRM system


When I approach and design a high level architecture document I generally cover the following sections:


  • Project overview

  • Project goals

  • High level architecture diagram

  • Technology platform

  • Technology platform - process and techniques used for specific problems

  • Key processes covered by the product

  • Other



Project overview. A short overview of the project, usually a project sales pitch describing what is the project, who is the client and what the project is really about. I do not go into great details since there are other documents which are generally built to elaborate on the topic further.

Project goals. Or the project high level requirements are the handful of key things the project must be able to accomplish. This is the basis on which the architecture is being designed on. The architecture must be able to fully satisfy these , generally around five, requirements in order to be successful.

High level architecture diagram. The highlevel architecture diagram is the corner stone of the document. This is actually the picture or the sets of pictures most architectural discussions are done between engineers and which will set the broad strokes of the project. The key goal of the high level diagram is to set the relationships and communication channels between each compoment of a system. It can be done by using a block diagram with squares and arrows indicating componets (applications, modules, databases and etc. ) or it can be done by using UML's component diagrams.

Technology platform. A list of technologies used for the product organized in logical sections describing the goals and reasons for the usage of the technology. It is important to correlate the chosen technologies interdependencies, e.g. all the technologies must work well together and if not the risks must be identified, documented and review by all the architectural key stakeholders of the project.

Technology platform - process and techniques used for specific problems. Description of key usages of each technology. Most modern technologies used for the development of modern systems and products can be used in several ways, buy applying several techniques at ones disposal. For example one can rely heavily on the ObjectDataSource object in ASP.NET for all grid operations or could choose to manually bind data or just to build a normal HTML table with data into it. This important for a data heavy application where data will be often presented in a tabular fashion and this will be the main way a user will interact with the system.

Key processes covered by the product. This correlates directly with the high level requirements set for the product, e.g. the project goals and other critical issues which can applied to the entire product and are critical for its proper functioning. Here we could specify how the system reporting solution will work, how the system will interact with other key systems ( for example with an old dos accounting application critical to the client who can only write to the file system plain text files), how the authentication and authorization system will work , how will the caching solution work etc. Apart from the high level diagram this is the most critical part of the architecture and more so for the client. Often to support a specific process implementation an architectural spike will be developed to prove and disprove a point to the key architectural stakeholders.

Other. Architecture is for the most part a technical discipline done for the product development team. It is purposefully ten thousand feet up from the detailed specification of the product in other to cover, restrict and keep up with the full product life cycle. Each product is a separate beast with its issues and pitfalls, risks and challenges and there a myriad characteristic who could be added to the software architecture for a specific project.