Tuesday, December 22, 2009

Rock Star vs Rock Solid

Lately I've been interviewing a fair bit; checking out my options (Insert "should I stay or should I go now" Kylie lyrics) ....

I've seen a few places asking for "rock star programmers" or the like. I've been wondering do they really know what they are getting themselves into, or asking for. Lets look at two different approaches, as there seems to be in life and just about any situation out there.

The "Rock Star" programmer
Firstly I'm not knocking people who have awesome abilities (actually quite the opposite) or rock music, as along with funk it's by far my favourite.

Here we are trying to describe the individuals who identify themselves as rock stars, a key point being that they do.

There are a few areas of their behaviour  that are predominate. The overwhelming self belief that they can do anything, instantly and it will be perfect (a.k.a conceited), hey who needs testing or documentation ....... combine that with the attitude of their technical approach is without question as they already know everything, leaves the whole situation inflexible.

The only change that can happen here is more bravado, it's a catch-22 for the individual as the position is on a very high pedestal.

This makes the individuals hard to approach for other "lesser" team members, and fun to manage in a project context, owing to the ivory tower residence.

When you have someone who can actually produce to a high level and constantly (apart from watching them for burnout) you have to think what is going to happen when they leave, are they documenting and sticking to the company standards, what is the actual cost of what they are doing, and how many people will it take to decipher the work after them, will it even be usable ?

I know chaos theory works, I've looked at it. Could I patch it and write an API upgrade for it ...... or would I want to, let alone ask someone else to learn it ......... likely not.

Most "rock stars" in my experience also have been quite unapproachable (by juniors trying to learn or management trying to manage or lead), very sporadic in how they work and prone to leaving mid project as they've found something else with more flashing lights or money.

There can be good sides though ......... possible getting the latest CMS coded in 2 days flat, or deciphering a radio link crypt by ear and coding it directly into erlang can pay dividends.

On a final note ......... if you end up with someone who proclaims themselves to be a rock star and can't code their way out of a paper bag, you really need to sort out your interviewing process !!

The "Rock Solid" Programmer

Could start and end with humility, though might as well go over some other points .....

By rock solid I'm talking about someone who's spent their years in the trenches working away, gathering knowledge and skills, distilling that knowledge in practice and sharing what they can, always being interested in new things and others. Though for the sake of learning, not just having the most expensive skill as seasoned engineers learn principles, and then languages as tools.

They usually come across as dark horses, meek or mild, quietly confident. Always asking questions of others, to learn about them, the situation, the project, what needs to be done. Using the experience to look into the future, supporting others and basically being as subtle as possible.

A joy to manage as they typically do 99% of the job for you, you only have to ask them to share their ideas about the project as a whole, and things they are concerned about, and you're up to date in no time!

Methodically document and adhere to process, likely improving it as they go, without mentioning how great they think they are every second sentence ........

The Real World

As with most things it's little from column A and a little from column B, though in this case a hell of a lot more from Solid opposed to Star, if any.

Leave it to the marketing department, not engineering; and if you're being like that in marketing you're covering up for something you're not actually doing, or you'd just be transparent to begin with.

I've had the luxury of working with individuals from both sides of the coin to learn from, and know that allowing work to speak for itself over bravado and actually being humble, is the thing to do.

Knowing what you know in the IT world is likely already out of date, and to always be open to new learning is for me the way forward.

"I know that I know nothing"
~ Socrates


Saturday, December 12, 2009

Cooks and coders .......

C&C for some would be command and conquer, and rightly so it's a great game.

Though here I'm talking about Cooks & Coders. It would seem that there is a high correlation between people who enjoy building technical systems and preparing meals ....... and I think I've found a few commonalities as I've been a professional chef and a coder, and have been doing both for a while now !

Either process is a form of managing ingredients/components, getting the right things in the right place at the right time and in the right condition.

The Menu a.k.a. Design and Requirements

Now some things go well together, like web browsers and xhtml, gravy and steak, PHP and web servers, pasta and white sauce, etc. Other things do not, javascript and MySQL, ice cream and lobster (well maybe not .......), cobol and anything (ok ok I know it's good at things, just not near me).

So there are obvious compliments in both areas, some things work well in the situations they are either suited to or actually made for, simple enough. Any decent engineer or chef will know when it's not right and your pushing a square peg into a round hole, or ice cream into a lobster.

A recipe / white paper will give you the basis of what is needed in most areas, how to cook yorkshires for a sunday roast or how to set up LAMP and do some hello world web services calls using REST or the like.

Getting the correct ingredients is vital, second only to the quality of them. A bad cut of rotten meat is going to be about as much use as badly written alpha code in a production enviroment. A unprofessional or inept C&C might give it a go, ignorant about food poisoning or more likely just don't care or take pride in the end result.

As an aside, the care is another common trait of good C&Cs, the knowledge that the end result is a reflection of the care and thought that went into the creation. Personally I get a similar pleasure from cooking a full Sunday roast (as I'm just about to today !) as I do from seeing a completed and working chunk of code, strange but true.

You'll find the inexperienced, inept or ignorant trying to put in as many flavors, hacks, seasoning, Drupal modules as they can in an attempt to impress with pointless confusion, or cover up their own mess, burnt steak, opposed to focusing on what I think is one of the most basic traits of a good C&C :

"You know you've done it well when there is nothing left to take out, opposed to the second school of though who thinks they are done when there is nothing left to put in"

You can spot the second lot a mile off, messy kitchens, very messy code, hacks all over the place, no comments, bloat ware, no idea about what's on the menu.

Knowledge of ingredients

Some C&Cs really know their ingredients, for example an ex colleague Scott knows more about PHP that anyone else I know (and probably just about all the people I don't know, being a PHP guru for Facebook  likely helps ......) so when it comes to approaching a task having that knowledge will manifest itself by the use of the ingredients.

An example being I was messing about with some parsing oblivious to the DOMDocument class, a question to Scott and the whole issue was dealt with in 4 lines of simple code. Much the same and one cook trying to make a prawn cocktail sauce from 50 ingredients and another just mixing mayo and tomato ketchup.

Know your stuff and keep it simple with the correct things in the correct place.


Not only in humour, but in cooking and coding it's key. Bring a project in on a (realistic !) deadline is near identical in both areas. You have a set bunch of ingredients that will take a certain time to sort out, they have to be prepared and mixed in a certain order as well as all being done at just the right time so they can go out to the restaurant on the plate, or onto the production server in one go.

Experience, training, awareness and being mindful is a shared requirement of C&Cs. "Ah yes I need to get the onions going as they will take ages to soften properly before using them" = "Best I get Bob working on the database schema as we'll need that sorted before moving on".


Sitting down a sorting out a menu, shopping list, requirements, design then staffing your kitchen / development team is key.

It's my belief (which has been proven to me over and over) that it's the people that make a good product or menu, the procedure, method, buzz word bingo play a minor and usually distracting role. Good C&Cs know what to do and what needs to be done, give them the vision, support and get out of the way.

I can attest to that from being a kitchen and people either start messing with your soup or stand right in your way and ask if there is anything they can do (most of the honest answers like "No I don't think you can actually do anything" or "leaving would help lots" typically have to be kept in ones head, unfortunately).

Plenty of books and information out there on how teams gel and the importance of leaving ego out and focusing on the common goal and helping, learning and sharing with each other so I'll not go over that. The only point I'd touch on there is when you are in a decent team and you get to the point of flow it's wonderful, nothing can stop you the creativity is racked up to 11/10, the light bulb comes on above your head with such intensity that people think your a light house.

It's very easy to see and empathise with people who are trying to work and constantly get interrupted by thoses who are inept and naive of such things, or how creative people work.

Peopleware: Productive Projects and Teams (Second Edition), is an awesome read on this.


General Manager = CEO/CTO

If you don't have one of these that has worked in a kitchen or who will find a capable head chef then get out of the way, your are in for a LOT of trouble, avoid at all costs I say.

Head chef = Technical lead

Has to be strong, protect the menu and team from outside influences. Must be recognised by the company in that role, and supported by senior management. The good ones will have worked their way into the position from many kitchens/dev teams working in various languages/cooking styles. The bad ones typically BS their way in with a few buzz words then google'd "Oh yes we're going to use SCRUM and Ruby on Rails to build your browser based text parser", it's any coders job to point this out to the senior management. They will likely get upset and blame you for not helping or sorting it out, the good ones will fire them, learn and move on.

Attempt to micro manage and frustrate the good ones at your peril.

Chefs = Coders

Pretty much what the blog is about, get good ones, give them a decent kitchen (dev toys) quality ingredients (languages, IDEs, licenses for proprietary things when needed) and get the hell out of their way !

Matradee = Customer liaison

Now this is the bridge between the head chef and customer(s). Trusted to give the feed back to the chef and deal with tricky orders or upset customers, also knows enough about food/technology to speak about the menu/product to customers.

Maturity & a level head is vital, promising four roast boar and a salad buffet in 13 seconds to a customer is not an order you'd want to give to a head chef ........"Oh yes, control satellites AND translate Japanese into Russian in real time, yes that will only take an extra 2 hours on the project" ...... danger danger Will Robinson.

Waiters = Support Staff

Make sure they know what is on the menu, how to answer the common RTFM questions and how to ask chefs questions.


I remember working with a pastry chef once, she didn't care for doing much else though she could; Choux Pastry was her magic. Much the same as getting in a LDAP expert who just designs and sets up a directory will all ACLs as easy as falling out of bed.

Project Management

Basically ............. it's identical, it's all about teams of people.

So basically it's an expression of creativity and enjoyment in the end result and proof to why some autism is a good thing !!


Burnout and how to deal with it

Unfortunately I'm not talking about the driving game, I'm talking about the all too common and serious condition experienced by many creative professionals who find themselves in situations that promote it.

Who either don't get out, don't protect against it or work for an organisation that is ignorant (or even worse don't care) about it.

[UPDATE : When you're done there is  Part two, preventing burnout in the first place]

What is it, how does it feel

Well there are two sides to it, firstly there is what the individual experiences and second there is what others see (and how they commonly mistake it, or write it off as something else).

For the individual it is wholly depressing, life robbing, exhausting and  frustrating.

According to a Dutch study GPs have one of the highest rates of burnout, though I dare say software engineers are a close second ! Maslach who studied this created a Burnout Inventory that uses three measurements to identify it
  1. Exhaustion
  2. Cynicism
  3. Inefficacy
Personally I think exhaustion is the main indicator, as cynicism is too culturally sensitive. Being from a country where I think we have a balanced view, encountering what some call the "up beat American attitude" of "Yay everything is perfect or it's your fault", disagreeing with it can be call cynicism (or an opportunity for pill pusher to sell you something !).

I don't believe it is and it appears others are coming around to the idea of balance and shared responsibility.

Also to put inefficacy in context, I would describe what I think is meant as a "Reduction in average capacity", meaning someone can be inefficacy due to ineptitude, inexperience, lack of motivation, lack of skills, etc long before they are burnt out.

So a list of symptoms / phases created by Herbert Freudenberger and Gail North, which I've lifted wholly from wikipedia [cite],  as I agree with them, and also to illustrate the experience of the individual, are :
  • A compulsion to prove oneself
  • Working harder
  • Neglecting one's own needs
  • Displacement of conflicts (the person does not realize the root cause of the distress)
  • Revision of values (friends or hobbies are completely dismissed)
  • Denial of emerging problems (cynicism and aggression become apparent)
  • Withdrawal (reducing social contacts to a minimum, becoming walled off; alcohol or other substance abuse may occur)
  • Behavioural changes become obvious to others
  • Inner emptiness
  • Depression
  • Burnout syndrome

"Burnout syndrome" I think is a cyclical point here, either that or I haven't found what Freudenberger and North meant by it yet!

They are mostly self explanatory, and I can attest to them first hand though two major experiences of it in my life (so far, and hopefully never again).

A summary would be that you feel very alone, depressed, and judged. To compound it the more you try to get out of it (by doing more of the same) the more you experience it, much as a clever chap once said :

~~ Albert Einstein

We will come to what we can do about it in a bit.

How does it look, and what happens

Now for people on the "outside" of the experience, i.e. colleagues, close friends, partners, pets etc. It cant manifest itself in all manner of ways, very dependant on the personality and coping mechanisms of the individual. When pushed to the limit and beyond how do you behave ?

From personal experience, I know I freeze, withdraw and attempt to rationalise what is going on.

Which within itself is a double edged sword, I know I'm good at finding fault or noise (it feels autistic) which is great when dealing with systems (technical or social) as you can easily see the bit that is the root cause of the issue, it just is what it is.

Much the same way that a friend and colleague explained to me when writing regular expressions. When I asked how did he do it or how does it know it will work "Because it will, I looked at what you asked and that's what I saw".

So it's hard for people who aren't checking in or listening to see what is going on with me, till I'm pushed over the edge by more of the root cause, or the usual bug bears of engineers, constantly changing requirements, design by committee, poor environment, inept team member(s) who just make more work for you, etc.

At which point, I just don't get out of bed, depression is in full swing by this point (combine that with a hobby in making beer and the temptation to self medicate is quite strong..... another social reason that men are treated for alcoholism to a much greater degree, it's not socially (in most places) acceptable for us to ask for help, and there are little or no social support systems in place for men, though I digress ......).

I've seen others lash out against management who don't have a clue at what is going on, or at colleagues, a lot of brilliant but mismanaged (self and organisationally) people just resign and go else where, seeking a better environment.

I've heard the behaviour of the individuals being described as "volatile" (typically by the people(s) who are pushing them to do more work), angry, depressed, disconnected etc.

It's unfortunately rare for people who haven't had actually counselling / psychology training (as I have) to ask questions that would uncover what is going on.

One of the dangers of being in this situation is the behaviour will be written off as what ever it suits the senior management to assign it to, the usual include :
  • They have "personal" issues, they then focus on the symptoms of burnout to support their argument opposed to looking for root cause(s)
  • That they keep complaining about person X (root cause ineptness) is just a personality clash. Always compounded when person X is very good at distraction or manipulation
  • They are incompetent in their job and overwhelmed (disregarding past achievements i.e. proof)
  • They don't accept the "corporate culture" of "getting things done at any cost", i.e. defending a coding sweat shop
    An aside the last point is an interesting one as many "managers" see their role as "achieving maximum productivity" by hook or crook, any study on X vs Y management will give you and overview of that.

    Now ....... I've been over clocking CPUs for years I know how hot and unstable things get when you just keep telling them to go faster, and how utterly pointless it is, unless the first thing you put in place is the care, in the CPUs case water cooling and LOTS of monitoring.

    In a persons case, proper HR procedures, realistic time lines (ideally none) with progression and experienced project management, etc.

    Unfortunately the case it's usually the other way around, "managers" will push an individual/team to breaking point and then set that as the point they "expect" for ever more.

    There is a very lonely hot place, in a very hell for individuals like that.

    What to do when in it

    So we have looked at what it is like for the individual, what it is like for the organisation how the internal and external behaviour manifests itself. We've also explored some of the (negative) management assumptions and behaviours in response to it.

    Later we'll look at not allowing things to get there in the first place, though for now what to do in the heat of the moment.

    For the individual it is imperative to think of number 1, if you find yourself in a situation like this, issues (and likely people) way beyond your control have dumped a crap pie in your lap.

    If at any point you have a CEO or senior management sit you down to grill you about everything you did wrong in their eyes, leave immediately ........ seriously get out.

    You're in a situation where you are sacrificing yourself and things in your life for the sake of a organisation that obviously doesn't care about you, so return the favour and just stop, and take care of yourself and ignore them.

    Unless you're a single contractor or the CEO, It's not your fault.

    This is how I see responsibility and support in an organisation :

    If you are "above" someone's role in your role, the it is your job and responsibility to check in on them, possibly manage them, and very importantly make sure they are are fit for their job (and that includes making sure they are actually capable opposed to believing everything they say). So in a word facilitation.

    Also the responsibility for things flows up, it's the double edged sword in chain of command (it's why they get paid more ........) . Something goes wrong that lightning blot shoots up till it hits the person at the top, if they haven't thought about having the correct people and processes in place, then constant electrocution is their deserved reward.

    So with that in mind .............. it's not your fault.

    If you are in a large enough organisation, then go straight to HR and lay it all out for them.

    If you are in a smaller organisation, which is more common then you're going to have to find someone internally that you trust. In the absence of that you might be feeling very alone, though there will typically be some help available professionally within your community.

    Also your benefits (if you have some) might cover enough counselling to get things straight in your head that you can get back to a mental state from where you can deal with the issue (and it's not the project it's the organisation and the lack of process and care etc).

    Along with :
    • Get some exercise to help with general wellness (always do this !), as well as being properly tired for sleep
    • Take a break
    • Hang out with friends
    • Ensure you communicate with your partner about why you feel bad (so they don't think it's them)
    • Give the cat extra treats
    • Remove yourself from things that are associated with work (i.e. sitting in front of a PC all day)
    • Get help/advice from unbiased sources (very important, people at work have their own agendas)
    So it is quite individual for the person. One of the main things I learnt training as a counsellor is "self care" (why the hell they taught it at the end I have no idea ...... ), if you don't take care of number 1 you can't do anything else well, so it's unprofessional to do so.

    The analogy I usually use for this one is the lifeguard (I'm a SCUBA Rescue Diver), no lifeguard will put themselves in harm while saving a stranded swimmer as then you have two people in trouble.

    If you are doing a rescue dive near a pier and you have the unconscious person, if a wave is going to thrown you into the pier, you put them in between you and the pier so you can carry on and get them out (i.e. one person hurt and one OK, opposed to two out of action).

    In counselling it as unethical and very unprofessional to work when you are in a compromised state, as you will not be able to do your job and focus on the client, so just don't do it.

    Much the same if you are paid to perform as a Software Engineer if you can't do it for X reason, then don't. Likely X is something to do with the management of the situation or a lack of support so address that, opposed to blaming your behaviour in reaction to the situation.

    How to avoid it

    If you want a happy productive team, I would say at all costs. As with most things awareness and professionalism is the main key. Listening to individuals who are working at the coal face and in the fire line of production.

    An great example being that when they complain about other employees work and behaviour, explore it, don't fail over to the easy position of favouritism and write it off to a "personality clash" as you obviously have one individual not functioning (the one who is vocalising the issue) and very likely there is something wrong with the one they are pointing to.

    Ignore this at your peril. Unfortunately most do, or allow their reactions to be directed by personal bias, i.e. attacking the individual who is trying to explain the issue.

    In situations like this there is a Tessler coil of a bolt building up.

    Professional and unbiased personnel management is key, though usually absent unfortunately. Recent surveys have indicated that around 50% of individuals that left, or want to leave their employment, is due to poor management.

    The future

    Looks like we know what is wrong and when, though are too busy trying to grasp the outcome to see that it is the manner in which we do things that dictates the outcome, it's really that simple.

    "The product WILL be done by this date AND will be great, or I'll mount on the pressure till it's done, and then blame you when it goes wrong"

    .......... err no, lets try ............

    "The creation of the product will be approached in this manner, which will be the state it ends up in, we'll check in regularly along the way with the people, and then the project, as its the people that ARE the project. We will know when it's done".

    When I (use to) worked for vBulletin helping to build version 3 (and ImpEx) and being involved in bits after that, the product was a reflection of the people involved, the passion, the happiness and camaraderie. The product reflected that at the time, from the quality produced by the tech team all the way over to the responsiveness and unrelenting calmness of the support team, even in the face of some very challenging situations.

    That shined though and it came from the people.

    Quality is an approach and an experience, akin to respect, not something you demand or pay lip service to, something you exude, earn and are mindful of.

    I know having been though all these things makes me a better leader, I just have to ensure I don't forget what is important:

    Take care of number 1, so you can take care of those you are responsible for, and they will take care of the project and we all get what we want !

    (Also remember, this is an internet blog, I am sharing my experiences and ideas. You should always seek the help of professionals where necessary).



    Thursday, December 10, 2009

    Data migration : Part 1

    Having written and looked after ImpEx for vBulletin I've dealt with a few matters regarding users and their identification, along with their data, and moving it about. I thought I'd share some thoughts that I've had on the matter as most people seem to balk at the idea of data migration due it it's "messiness", it dosen't have to be like that.

    First an overview

    There are just over 120 source systems for ImpEx now and growing.

    They are split into three tiers, sorting themselves into these three groups on a criteria of most commonly used, most requested (by customers) and frequency of change.

    For example the most common and used importer is top of Tier 1, it gets a lot of use & attention from moi. Tier 2 ones are on tick over and Tier 3 are in the grave yard, the most common reason being that the version supported no longer is in active development, there for the importer will never change.

    The moving target has stopped moving, much like playing Unreal Tournament and getting a head shot every time on a target ........ oh the days.

    Whats in a name

    ImpEx's primary requirement is to protect the data model of vBulletin, i.e. the integrity of the database. All manner of things could be imported and done during an import, passwords being the most common hot topic of conversation. And yes I know how to do it and how it can be done, it's not a case that I don't know, it's that something else takes precedence !!

    Which is integrity, there should be no change to the schema or functionality of a standard install after an import, just a bunch more data that could of been created by vBulletin itself (with the exception of logs and the import ids a.k.a. foreign keys).

    Breaking the name down ImpEx is Import Export. It's not strictly a part of vBulletin, though it can run in the admincp and take the config details from vBulletin's config.php I typically run it standalone just against a default vBulletin database schema. The ImpEx Target refers that that schema, i.e. "what version of vBulletin are we Exporting this data from ImpEx into". The Source is the other half of the equation.

    The ImpEx washing machine

    "Stick it in one side, clean it up, pump it out the other. ImpEx, gets your posts whiter than white"

    "Cleanup imports", an import done to upgrade from a board with hacks or from one version of vBulletin to another (or even the same version) in an attempt to 'start again' aren't really the primary purpose of ImpEx, but can be a last ditch option. Though frowned upon and not officially supported.

    All the bits

    So we have the system, of which there is a core (ImpEx itself) and all the supported source systems, these are the source modules. Now to the Ex bit, the target modules.

    ImpEx only officially supports the latest stable release (3.8.4 at time of writing) though each time a new major release is available, there is a new target module (or upgraded existing one) released (so yes I have one in the wings being tuned up for vB 4).

    A simple way of maintaining an offering to the ongoing customers who upgrade at their own pace.

    So what does it do

    How data is sourced, linked, referred, captured, verified and written will be the topic of Part 2.