01 January 2014

The emotional health of a software system

 

The eternal truth to software development is its inability to produce any meaningful systems without a coordinated effort from a diverse team of individuals, lots of disparate activities ranging from design to development and quality assurance and finally with the same effort applied to the maintenance and operations of a running system. From the deployment of patches and extensions, to the cleanup and data hygiene of databases to the scaling of the system and hardware and software maintenance efforts.

As much time and effort is spent in the art of software development, often not much effort is spent during the operations phase of a software project.

A rule I’ve often observed in many many projects I’ve been part off is the linear degradation in the effort of maintaining all the scaffolding and activities which are not directly related to coding of a software project. Thus enabled entropy degrades the product and seeps into its operations lifecycle resulting in emotional and business drawbacks for both the organization and the stakeholders included in the project.

Should we accept this? Should we accept a system of development with diminishing software engineering efforts , should we accept the fallibility of people to care and work at top quartile engineering capacity?

I’ve seen that any significant software system, shop and development environment needs guardians in the form of an engineering educated project manager and a strong cadre of technical leaders which have the authority and will to enact and drive and force a strong commitment to all software engineering scaffolding practices which combat the intrinsic entropy found in all human endeavors, especially software development with its heavy intellectual burden, high abstract concepts and emotional stresses.

A common image of software developers is of introverted emotionally challenged people. Truth be told I’ve worked with quite a few of them in my time. Thus a surprising aspect of our live is the emotional toll software development takes from each of us which is directly linked to the quality of work we do and the state of the system.

Human emotions, feelings and states directly transfer to the work we do. The pain of working with  highly abstract concepts, what in truth software development is, is the primary source of all the bad decisions, all the accumulated technical depts., all unsolved problems and defects a system has.

In order to ensure a proper functioning of a system and a team special need needs to be put in and placed under constant guardianship in order to focus on emotional and mental health of the team working under such adverse, but normal conditions.

08 December 2013

Code archaeology

 

 

8446583177_d575bdac50

There is no love, no tenderness, no emotion, no trust and no God in a piece of code. You can search, you can dig and you can wail at the injustice of it all but the only thing you are going to find are mistakes, abandoned paths, and lines upon lines of code written by people who are not here and all that they have felt, experienced during the creation of the code you are reading is forever lost, forgotten, not important.

But it is important. Recently I’ve done a lot of code archaeology, reading and analyzing solutions (some which are more then seven years old) discovering how such system are built and how are they organized with the goal of identifying pieces of code which can be transferred into the new ports of those system which are going to be built.

By reading that code I’ve found a story. A story of ideas not reached, of pressures and short cuts taken, of principles and practices which were a standard of the time that code come came from but now I know them only from my books and my own early work.

I was lucky and got to talk to the people who worked on those systems, and got to ask my questions. Why such a decision was made, why this and why not that. Is this used and why not. And I got a story of humans. People working and feeling their work end expressing them selves trough code.

The end result of each code archaeology activity was a new architecture document document the found architecture, all the technical decisions and patterns found within without judgment, written just like it is.

While doing it I found the following principles helpful:

  • Do not judge
  • Identify design pattern and name them
  • Identity individual sub architectures and follow them separately
  • Sketch

Do not judge is in my opinion the most important rule, principle and state of mind while doing code archaeology. The basic principle of the activity is to see and read what is built. The design patterns, architecture and coding style may well be something which is archaic, non modern and not pretty but it may not have been like that when it was built A web from application written in Visual Basic for .NET 1.0 is a child of its times and it is not reasonable to apply todays software development criteria to such an old system. Secondly I never met a developer who when faced for the first time with a system written and developed by someone else who didn’t start cursing and wailing about the poor quality and decisions made by the previous developers.

We developers are an opionanted bunch who would rather spend days and week arguing about what about the proper placing of business classes and their proper naming rather of accepting the current situation and getting some work done. This is actually good since is brings us to question everything and to advance as a industry as a whole

But when doing code archaeology, when reading and analyzing someone else's work this attitude will block us from seeing and understanding the software we are working with and finding it inner beauty.

The most important contribution to the modern practice of software development was the definition and introduction of Design patterns as a vocabulary and definition language which transcends all languages and software types. Thus when doing software archaeology it is most important to identify patterns of code use, name them accordingly and document their use. This practice makes the activity of analyzing and communicating the results of a software code archeology that much easier.

Any non trivial software project is actually composed of many individual sub architectures which together bring the solution to bear thus as an individual developer sees his work trough the integration of a group of classes thus a software architect sees a software system as an interaction between multiple sub architectures.

Not only it makes easy to document a complex software solution, it is the only easy way to analyze it. Modern software is complex. It has multiple moving parts which interact with each other trough time and space. A human mind can fathom only a limited set of the reality it is immersed into it at any given moment of time. Thus the only logical solution to comprehensively analyze any software system is to follow to exclusion of everything else each sub architecture boundary within is self separately from every other architecture.

Then, and only then, after each sub architecture was analyzed to its full extent. The next step is to analyze the interaction between various architecture, the same as we watch the interaction of several classes.

In the end software archeology is about understanding, documenting and the future. As archaeologist catalogue, analyze and document their findings for other to use, thus a software archeologists end product is document describing her finds. The best tool at the disposal of software archaeologist is the ability to visualize graphically old the processes, integrations and logical modules of all the key ideas being documented.

23 October 2013

Living somewhere else

 

Living in a different country it’s a strange occurrence. Feeling of fear, insecurity and danger are more accented, more real, more tangible. It becomes easier when some of the basic living necessities are handle. Finding friend, having a place to stay, a place to eat and a place to drink bear.

Meeting someone new also helps.

5f7b1ca23bf511e3ac4722000a9f393d_8

27 September 2013

On war and peace

4028380279_625a1ce97a_n

I’ve heard of MakerBot and the cheap 3D printing movement a few years ago by listening to a FLOSS weekly episode by driving on my daily commute. For me it was far away, costly and not something to think about. But listening to a couple of Dot Net Rocks shows and watching an Elementary episode I came to realize 3D printing may well starting to affect our lives, and it in the next five years it will greatly influence how we live. And not always in positive ways.

The internet was always full of dangerous things, the anarchist cookbook was one of the early examples. People could download it and learn how to make bombs. Want to hack a pay phone, you can find it on the internet. Learn how to pick a lock or steal a car. Find it on the internet. Learn how to torture someone. Internet.

Learn how to make gunpowder. Well on Wiki how you have picture by picture instructions even my six year old can follow. And once again, the internet.

Of course the majority of the people don't do that. Most people are not interested in those stuff. For me I’ve learned how to make a poncho from fleece, how to fix a broken sink and how to roll a cigarette.

I’ve read this article on endgadget. And I was scared. Home made guns. Readily available to everybody, without supervision and control.

I picture kids going to school with weapons, with sharp plastic knives and maces. With crossbows and other stuff they’ve printed on their home printer. Metal detectors would not detect them, well they are made of plastic.

The world just got scarier.

I and the majority of my generation have survived a war. We have witnessed first hand what war and weapons do to people and their lives. Thus I’m really glad I leave in a country where the majority of the population is legally disarmed and the right to bear arms is not in place.

I believe that firearms and lethal weapons have no place in modern society. What would be the use for them? We are not being invaded, or have to defend our selves by murdering war bands or have to survive a zombie apocalypse.

There are dangerous people out there. deranged, sad people with issues. But they are far an few between and you and me know where those people can be found and any sane person knows what to do to avoid them.

When I first heard of mainstream availability of 3d printing I thought “Oh, wow, how cool! Now I can make toys for my boy and Warhammer 40k figurines for me”. Predator Annihilator here a come. A 2000 pt Ultramarine army right at my fingertips. Special terrain, bunkers and all the things we made from scrap before just waiting to be printed and be used.

If I was thinking of making a weapon with a 3d printing I wouldn’t create a gun. A crossbow and darts would actually be better for me or short sword, mace or morning star. And lets go LARPing.

But seriously I wouldn’t do any of those things. I would think I have a weapon, but to use a weapon is to kill a person. I do not need nor whish to kill a person. I don’t really relish going to prison and do not really think about a scenario where I would need to constantly armed.

The place I currently live is filled with rolling hills and valleys which once were diving lines between great empires. In ages past the area around me was filled with fortresses where armies clashed, and maundering parties were common. Byzantines versus the Lombards, the Franchs, the Romans, the French , the Turks , the Germans and the Italians, the Americans. I live in the most peaceful battlefield in the world.

I wish it to remain so. I wish people would not make plastic guns at home. I know people in general can not be trusted with other people lives and safety, that’s why we have so many checks and balances.

I drank coffee today and watched clouds roll bellow me, following the Mirna valley hugging the land and covering bright lights for houses scattered across hills and forests. An artist discussed her new exhibition and people from all over came to see my land. In peace.

22 September 2013

On time

 

image

Time is everything to a software developer. It bounds every aspect of his life. The time to come to work, the time of work and the time after work. But more to the point the time it takes to complete work – concentration – rest cycle, the number of times such a cycle can fit in a work day. The time lost talking to each other, eating, going to the bathroom, phoning, compiling, publishing, reading, searching waiting.

Most do not notice such a thing as time spent per activity, nor the time ill used or lost between meaningful work. Some do, and thus try to leverage it, learn it and thus they try to learn about them selves and their relation to the dimension of time

I’m such a person and I’ve learned a fair share of things in my time as a developer. I’ve learned that time is a feeling and that not all time can be meaningfully spent. That you are bound to have two to three work-concentration-rest cycles a day and that is that. I’ve learned that you can leverage the time between to better your self, watch a presentation or read an article during compile times or listen to podcasts on your way to work.

Such activities seem trivial when taken by them selves but focus on that, those five minutes spent listening to a talk on infoq are five minutes that you are wiser and potentially know more then before, albeit it was five minutes worth and if those minutes pile up during a month or a year how much time did you spent bettering your self instead of reading comic books, facebooks, twitters or other useless information's?

But other than that time is crucial for the activity of software development. The mastering of time by individual contributors which need to give estimates on their work and technical leads who need to coordinate the continuous integration of different modules in order to utilize the full potential of the team and incur no loss in output and productivity to product managers who need to be responsible for the deadline, or at least know what will be realistically produced by when.

When doing the work of a technical lead in charge of leading a team of developers time synchronization is of the up most importance. Imagine for a second a team of four developer plus out valiant imaginary leader. They are building a distributed system which will constructed of two main parts.

The first part is a a server component exposing some kind of fanciful services over the network. That server component is built with the goal of serving many clients build by potentially different organizations in different technologies.

The second part is a client application built to use the services exposed by the server component.

Our valiant leader thinks for a seconds and break the team in two parts. Two developers will work on the server component and two developers will work the client component.

By sheer luck the specification for the server interface is completed and hopefully will not change during development (one can hope at least). The client team is crack A-team of mixed UI design and Software development specialist, who know all there is about human machine interaction, how to draw fancy stuff and all there is about design patterns and software development.

Since they must build the client without the server component to test on they’ve leveraged their design pattern knowledge (with the help of our valiant and heroic technical leader) and they we pushed the server interaction code into their lower most layer and mocked up the input and outputs. Thus the super uber client is build. But alas the server component hit a glitch, it seems more people were needed so while the client is fully built but not tested on a real server the server component is build and tested only up to 30%.

Jeez what now?

Described above is problem of resource allocation and timing. What could be done different? Well a lot of things, the client work could have been started later or the team should not be broken into two separate sections instead each developer or pair could have been given a cross section of client and server functionality thus the client and server parts would be build, tested and rolled at the same time.

In end time is the king that marks each cycle in the life of a software developer. Knowing its rules and limits, its ebbs and cycles it is crucial for a developer to be successful in her career.

20 September 2013

On the life

 

As software engineers our life are subject to forces which are not entirely in our control. The chief are health, family and politics. I will start from the latter and go the former, discussing how a software engineer can mitigate some of their ill effects on her life and career.

Politics is the art of the achievable, of getting things done. I’ve listened to an audio book version of the ‘Prince"’ by my namesake Nicollo Machiavelli. While some may disagree with its context or rebuff it in its entirety on moral grounds alone, I generally find its context true and suitable to any endeavor where you have a hierarchy of people working in environment with limited resources, or any environment where you can find humans. The truth is that in the base of the human nature lies the instinct of self preservation, which is often achieved and perceived to originate from ones advancement above her station in respect to the domination and control her peers. When working in any organization there will always be personal goals, issues, treats originating from people working around you, above you and bellow you.

‘I just want to do my work, and be done”, some may say. And you are obviously right in that life philosophy, but given the fact that out there are people who will benefit by inflicting or just by you incurring a loss of some kind. Thus a wise person should strive to position him self in a prepared state, watchful about what is going on around him, about the goals and positions of the people in his circle. He should align him self with the people who have ambitions in alignment with his own, make useful and not a threat to people above his station and respected and not an obvious impediment to people bellow his station. A wise person would also be aware that each state is only temporary and thus not permanent and that person will always need to be on the lookout for the next situation and have always a backup plan when all things start to fall apart.

In short going to work nine to five and forgetting about work afterward is not a really good strategy any more.

The second ill which may befall a software engineer is family. I’m a family man, my family is my sanctuary, my church and holy ground. I short my family is my life. But often the work we do puts us across the needs and wants of our family members. Our spouses, our fathers and mothers, out girlfriends. This is the truth.

If you read any book about people, or software processes or any topic related the the culture of software engineering you are going to pass by references to “Death march “ projects, long hours and weekends spent working, broken projects and divorces which resulted from those activities. The pressures and the time, of the time, we spend working maniacally in order to reach our deadlines reflect negatively on the needs and goals of those around us. No amount of money or bonuses is going to mitigate that. Maybe you live on a farm and regardless of your work you must spend time working on the family estate (new a guy who had a problem like that) or your wife is fed up being all alone with the children while you do whatever you do. And then meeting your business commitments and your family commitments is going to get really though, and your morale and your performance is going to drop and your private life is going, well down under.

What can a savvy engineer do than? First, you can’t please everybody. You have a limited resource, your time, and two opposing forces. What I’ve found out is that you need to be open at the beginning to both parties about what you can do and what you can’t do. And then be prepared to take blows. Or marry a wife who will not mind you being around so much. Or find a nine to five job.

The third and most important is your health. Your mental and physical health is the most important assent you are having in your career as a software developer. In order to achieve the level of performance and excellence you are able and required to produce you must be mentally and physically fit. Thus you must always leave room for physical exercise and strive to conserve a positive outlook to every situation and prospect. I can’t remember how many time those two thins helped me, and I can remember oh so well how many time one or the other failed me.

Experience is never taught, only learned by doing. Thus read and watch what others have done and experienced. A good documentary I watched five years go showed a year of frenzied development of a team of engineers at Netscape developing the first version the Firefox open source browser. That is a good example of what kind of life we lead.

But all is not dark, and all is not done, for we are human and we make mistake but to be human is to be able to learn and change, to adapt and improve thus we will make our lives better trough knowledge, spiritual advancement and physical toil.

2013-07-26 19.50.14

In the beginning there was the command line

 

In the begging there was the command line, and it was good. Then came the user interface, the mouse and expediency of ease of use. The word was “Don’t let me think” and thus the web was begotten.

And the cometh forth a beast of many heads, or many releases which swarmed into one version, and Google Chrome rose into the world and brought the command line back into the fold.

At first it was the ease of search, ease of address typing, moving and blurring the difference of an url and a query. Now, it was the turn of the information and the knowledge Google brought about your needs, your work, your life. And thus the time of search cometh to the world and the word was good and the masses were fed by the wine of information and food of quick results.

And the devil of engineering said “It shall not be enough, what was before shall come again, the circle of technology and innovation will turn again bringing what was before into the new and bright” and developers brought search to their applications and then it was not enough to have search but commands typed after the url came as orders of the divine and users could use the shell as the address bar and the web page as the console.

Thus in the end there was the command line. And is it was cometh again. Technology turns, what was old becomes anew, rediscovered, dreamed again.

I' am student of technology, servant of engineering and topologist of software. During my travels trough countless projects, books, presentations, frameworks, languages, databases spanning decades now I remember things which once were during the first books I’ve read at the end of the eighties and the nineties and the stuff I read and do now, and often I see old ideas and technologies rediscovered and re-implemented in new spaces.

I’ve read a book about the basics of computer and operating system architectures, how hardware and memory functions and how thing communicate and stood amazed how the same solutions and patterns are copied trough the architectures I’ve done and those I’ve dreamed.

I came to see this as truth about our loved profession. Everything which once was will come again, thus making the study of old systems, old patterns crucial for the future of our advancement. The critical thing which needs to be adopted as practice in our companies and universities and training grounds, and which is known to all old hands who survived several decades of shifting technologies and trend is that regardless that a technology is not used and more, or is redundant that there are core lessons and practices which will prove useful in the problem spaces of now and tomorrow. 

2013-07-07 00.28.31