Sunday, May 6, 2012

When and why should you refactor code?

One of the tenets of the agile methodology is the concept of "re-factoring", or the "disciplined technique for restructuring an existing body of code, altering its internal structure without changing its external behavior".  I consider an attitude that favors re-factoring to be a very important trait of a good software engineer, and vital to the maintenance of quality software.  But, not always.

The "Engineering Tax"

In my team, we call re-factoring the "engineering tax".  That is, we won't always offer up all of our coding bandwidth strictly for the quickest route to the release of new features.  Sometimes we have to take a step back and restructure things for the longer term.  As engineers, we know the code best and it's up to us to recognize these needs when they arise.  We have to tell product managers that it will take longer this way, which means we better have a good pitch as to why it's necessary.

It can be a hard sell, because product managers want features that generate revenue.   If you're spending time on something whose value is not user-facing, they probably don't care about it.   Some of the arguments I make to sell an extended development cycle due to re-factoring are as follows:
  • If we change the underlying data-structures, it will enable a simpler UI.  Simpler is always better.
  • When you want "the next feature" in this area of the product, we'll be able to build it faster.  In fact, we've already thought about the "next feature" you've been considering, and we've realized that this restructuring enables it.  So time now has a direct payback against time later.
  • Our QA for this feature will be shorter, because with less complex code that more closely resembles the business logic, we will make fewer mistakes and there will be fewer bugs.
  • If we're being asked to build something that is "like something else we have, but a little different," we need to combine the code for the two features and share it between them.  This will prevent two parallel implementations going forward, and cut down long-term regression risks.
  • We can build unit tests for the new code, where there may not have been any for the old code.  More QA savings and regression risk reductions.
  • Perhaps our re-factoring plan includes a performance improvement...wouldn't that be nice?
In short, you're trying to sell a lower overall long-term cost due to re-factoring.  It will be difficult to prove ahead of time, especially to a product manager without an engineering background, but hopefully over time you'll be able to look back on some success stories from re-factoring and some failures from when it wasn't done.

When NOT to re-factor?

But like anything, the attitude for re-factoring can be taken too far.  There are right times and wrong times...right reasons and wrong reasons.  Here are some thoughts on when you should re-consider whether a re-factoring project is a good idea, at least right now.

Have you taken the time to thoroughly understand the existing design?
Some engineers have a default attitude that if it's existing code written by someone else, it's therefore bad code and needs to be re-written.  Often times, they're really just facing style differences.  There's more than one way to write good code and you owe it to your forbears to understand their intentions before you throw their code out the window.  In fact, if you decide to do this new thing in a different way, you could be harming overall code quality by introducing yet another "style" into the code and thereby confusing developers that come after you.  The worst travesty on earth is an "improvement" that makes things worse!

What are the product stresses at the moment?
Even as an engineer with a strong re-factoring ethic, you need to respect the point of view of product management.  They may be facing a revenue-impacting deadline that is just around the corner.  They may have made promises to customers.  If you know you can deliver the feature without the engineering tax, and your product manager is making a strong case for it, you need to listen.  You need to consider time pressures in your decision.

Is the re-factoring huge?
If you're wanting to charge an engineering tax of, say, 50%, meaning that half of your coding time is going towards re-writing with no end-user value, then it might not be worth it.  You better be taking a long hard look at the long-term benefits and making every attempt to quantify them so you can justify the cost.  In general, the higher the tax, the more justification is needed.  In my product, we've done a lot of re-factoring over the years, but we've also left a ton of ideas un-done, because the costs simply weren't warranted.

Is it close to a release?
Re-factoring can introduce risk in a code base.  It's more "new" code, and therefore it's more "untested" code.  If you're coming close to a release, you can't afford to introduce such risks, and you should consider making your changes in a code branch with a longer horizon before it ships.

Are you just being a perfectionist?
I've seen some developers who take the attitude to the extreme.   They re-write code they wrote yesterday, or next month they re-write thousands of lines they wrote last month.  I appreciate their high standards in their work, and as an engineer I can definitely relate to the nagging feeling that "I can still make this better".   But this has to have its limits.  Code needs to ship or it is useless to the world.  You need to reach a point where your new ideas go on the engineering backlog and you focus instead of stabilization and shipping.

In summary, the decision to re-factor or re-write code is a subjective one.  It can be difficult to quantify the costs and benefits, and many different engineers will have different opinions.  The factors above are some things to consider.

In my current case, I work on a very old application (10 years), and I consider re-factoring to be critical to its long-term health and code quality, so I generally support and encourage developers to re-factor.  But I also keep them in check with the questions and considerations above, and my vote is not always "yes" to the engineering tax.

Sunday, April 15, 2012

What about change ideas from others?

In my last post I talked about how to instigate process change, which is something managers should always be thinking about.  Now I'll take a look at it from another angle, which is: how do you handle ideas for change that come from others, especially those on your team.

You should consider yourself lucky if you have members of your team that think this way.  Nobody is likely to promote a change that they think will be detrimental.  They probably have good intentions and are trying to improve things.

The Scrum process teaches the "sprint retrospective", where at the end of each sprint, the team talks about how things went, and folks are encouraged to suggest how to make things better for the next go-around.  In my teams, some people are not very shy.  I've been a part of over 100 sprint retrospectives, and there's always someone with something to say in each one.

As a manager, I have a unique role in these conversations, in that my reactions to people's comments is very important.  The individual contributors with new ideas will look to me to implement them.  Some ideas will be great, out-of-the-box improvements (like: "let's automate this or that mundane task we do every day"), while others will just be new time sinks that aren't all that practical (maybe: "I don't like the architecture...we need to rewrite the app from the ground up").   Other times you'll just hear complaints in a retrospective, and you'll be the guy expected to come up with ways to make it better.  My favorite complaint (sarcastically speaking) is "people aren't communicating enough...".  Ok, so now what?

When you're in agreement...

The easy situation is when someone makes a suggestion and everyone, including you, agrees.  Ya, let's do that!  It's important, is valuable, and we can do it immediately.  The only problem you might have is whom should get the implementation task?  Well that's easy too.  I usually kick it back to the person with the idea.  If they really believe it, they should be the one to champion it.  Buy-in is already there.  Done.

When you're not in agreement....

Disagreement about a new idea might be your own, or it can come from others in the team.  But the handling for the manager is the same.  You should make sure my initial reaction is always positive.  I always try to start by voicing appreciation for the idea and the positive effects it could have, whether I like it or not.  Then its time to explore the idea further.

The most important question, and the guiding principle for this whole blog post is...

What problem are we trying to solve?  

As I said in my last post, I believe we should always be looking for ways to change or improve, but I also believe in "don't fix what's not broken."   Let's say, for example, someone suggests using a cool new app that does a virtual scrum board on an iPad, where you can drag sticky notes across the screen as you pass around the iPad in the daily standup.  Sounds cool, right?  Let's buy iPads and start doing this!  Or not.  What problem are we trying to solve?  Seriously, what real problem does your team have that this solves for you?  If you can't answer that clearly and easily, it's not worth making the change.

Once you know what problem you're trying to solve, now you have to examine the merits of the proposed solution.  Does it solve the problem long-term?  What is the cost and resources required to implement it?  What is the priority relative to other things going on?  What other ways are there to solve the same problem?

This might sound like a naysayer who's just looking for ways to say "no" to the idea.  That's not a very positive attitude, is it?  Well, no its not....and yes it is.  It depends.  Its all about the initial reaction being positive, and the subsequent considerations that lead you a decision.  If your initial reaction is always "no", you've started out all wrong, and you'll stifle future suggestions and damage team morale.  But, if you're ultimate answer is still "no", even after showing due consideration for the idea's merits,  then folks are likely to respect your decision much more.

Choose your battles

A final consideration for a manager when facing an idea that meets with disagreement is what you might call "choose your battles".  If someone has an idea you're not necessarily comfortable with, but their intentions are good and they seem really motivated, then you have to decide if nixing the idea is a good call, even after considering and discussing all its merits and pitfalls.

The point here is that someone had the idea, and they will feel good if it gets adopted.  You see flaws in the idea, and you would never suggest it yourself.  But you want people to feel good, so you should let them pursue their idea, right?  Sometimes.

This is where your own confidence comes into play.  If you're going to put your foot down, you darn well better know that you're right, because there will be a large morale cost to saying no if you're not able to convince others with reasoning.  If you're not sure of yourself, well, maybe you're wrong.  Maybe the idea will be a winner.  You'll also earn a lot of goodwill by letting go of your objections and empowering your employee.

In summary, the "happy place" is when your team is full of ideas, everyone loves them, they're all willing to implement them, and you have the resources available to do so.  I haven't discussed that much here.

But things are less rosy...when you have more ideas than resources, or ideas meet with disagreement, that's where your management skills have to kick in to help the team figure out when to say "yes", "no", or "maybe later".

How do you make change happen?

As a manager, part of my responsibility is to be an instigator for process change.  That's not to say I believe in change for change's sake.  Far from it actually.  I'm a firm believer in "don't fix what's not broken".  But at the same time I also believe there's always room for improvement, and if you're not going for it, you're stagnating.  If fact, you are even likely to regress and fall back into bad habits.

In this post I'll talk about change ideas that come from you, the manager.  In the next, I'll talk about ideas that come from those on your team.

Changes I Want to Make

Suppose I have a great idea for a process improvement.  As the manager, all I have to do is tell my team to do it, and its done, right?  Heck no!

The last thing I want is to be seen as a dictator who makes decisions unilaterally and imposes more work on people.  "More work" is exactly the negative connotation your idea will receive if it's pushed on people without their buy-in.

A better approach I like to use to socialize the idea over time.  First, the light bulb turns on in your head.  Then, spend a while thinking about it on your own.  Then, start looking for opportunities to sell it.  If your team does scrum, it might be in a sprint retrospective, where people are noting a problem that your idea solves.  Another great time is during one-on-one conversations, or over lunch, etc.  Over lunch is a particularly good time to share, because your listener is likely more relaxed and willing to hear you out.  They're not feeling rushed to move on through their work day.

As you do this, you'll get feedback to help you improve and develop the idea.  Perhaps that feedback will come from naysayers.  All the better.  You can use this feedback to improve it, and come back later to sell it again with their input taken into account.  Socializing your idea before mandating it can only help your cause.

The (in my view, acceptable) down side of this technique is that it requires patience.  It might take weeks, months, or even years!  Your idea might be some really big change that meets a lot of resistance, and there's no way you're going to make it happen overnight.

Let's say, for example, that you want to create a culture of Test Driven Development in a team that is accustomed to black box testing.   Your developers feel that TDD takes too much time, and your QA team feels threatened by it, or doesn't believe it produces a better result.  If you simply say "From now on we're doing 100% TDD" you'll surely be met with revolt.

Think baby steps.  Maybe start just by e-mailing online articles about your idea, so folks can see that it works elsewhere.  Come up with a smaller idea that's easier to sell and heads you in the right direction.  Make it happen, and then celebrate the results.  You could even start "doing it yourself" to set an example that others will follow.

Another technique is to build a base of allies gradually.  If you have one person on your side, try to get that person selling the idea for you.  If the naysayers hear it from more than one source, they're more likely to warm up to the idea.

And lastly, I hate to say it, but if you work through all these techniques with diligence and you're still not able to make your change happen, then maybe it shouldn't happen.  Simply put, if others don't buy in after patient and persistent encouragement, then maybe there's a flaw in the idea.  Putting your foot down and insisting on it will only make you unpopular, and them unhappy.

Saturday, March 31, 2012

Describe some of your role models for software management.















Anti-role models







And, some of them have....

Sunday, November 6, 2011

What's the most important responsibility for a software manager?






2. Keep your standards high...don't settle!

Invest the time


To that end, I always do the following...


Keep your standard's high



Sunday, April 24, 2011

How did you make the transition from software engineer to manager?

Like most career-minded folks, I've never been willing to say "I'm happy where I'm at, and I'm not interested in further career growth". With that in mind, I've had to constantly examine my limits, my talents, and my passions to figure out my direction.

My limits: I love Computer Science. I got my BS and MS degrees in Computer Science. But I don't identify myself as a "Computer Scientist." That, to me, is a much more intense title. I don't have the patience or the mental capacity to solve the hardest problems in my field. I find complex problems like search algorithms, frameworks, or massive scalability to be intellectually fascinating, and I love learning how they were solved, but I'm not the guy who's going to solve them myself. This is why I chose not to follow a purely technical ladder beyond the level of Senior Software Engineer, or go for a PhD.

My talents: What I bring to the table is a healthy mix of technical and architectural skills coupled with organization, communication, and interpersonal skills. I have a strong sense of the "keep it simple" design philosophy, and I'm unlike many engineers in that I see value in and am good at writing clear specs, diagrams, e-mails, plans, and other forms of engineering beyond code. As I like to say "Even a simple e-mail has a user experience that needs to be considered." Also, over time, I've gotten good at paying attention to more work going on around me, but at a higher level. This is a key function of a manager, but not so much a requirement for an engineer.

My passions: I a nutshell, my passion is for seeing ideas turn into applications in an elegant way. As an engineer, I got to do just that pretty much on my own, and it was rewarding. But I was often frustrated that the coding process just took too long. As a manager, I get to do it with leverage. I get to be involved in more projects coming to fruition. As long as I'm involved, I don't feel the need to be the actual coder. Quite often I'm heavily involved in conceptualizing requirements and creating high level design (like, for instance, the data model), but then leaving the coding to one of my engineers.

Anyhow, to answer the question, my transition to management happened in my current job, where I hired on as an engineer six years ago and gradually became more and more of a manager. Yes, there was a specific moment early on when my job title changed, but the mental transition was much more gradual, and in some sense is still not finished. With a team size that has grown from 2 to 9, my hand was somewhat forced, and I've had to learn to resist the inclination to just "do it myself." Its been a good learning experience.

At one point a couple years in, my boss said to me point blank "I value your management skills much more than I do your technical skills." That was a real eye opener, since at that point I still identified myself as an engineer, and a good one at that. It was hard not to take it as a put down.

But I've come to realize that good engineering skills are a required part of a bigger package that makes a good engineering manager. At this point in my career I'm cultivating the rest of the package.

Sunday, March 27, 2011

Tell me more about this marbles analogy...

Sure. And please note its not that I'm calling the folks on my team a bunch of marbles. Rather, the marbles are the issues a manager has to deal with every day. The point is, there's no focus on one single thing, like there is for an engineer. A manager has to be able to deal with a little bit of everything, and context switch almost constantly. One moment you're advising product management on the feasibility of a new feature, and the next you're deciding if a bug should be fixed or deferred, or being asked for advice on a topic you may know little about. Then, you've got recruiting to do, performance reviews to think about, and perhaps an action item or two you took in your last meeting. Sometimes you're reacting to a fire, and other times you're thinking proactively about change. For the manager, responsibilities are all over the map.

For the engineers that you manage, it should ideally be the opposite. Their job is to take on a coding project and get deep into the weeds with it. The more minutiae and boring (in their mind) meetings you can shield them from, the more the good engineers will thank you, and the more productive they will be. They don't want to be discussing the requirements...they want to be coding them.

If I were to title a different blog about the mindset of an engineer, it might be called "Getting the real work done." In fact, letting go of that mentality, that feeling of "I'm not being useful unless I'm coding," was one of my biggest struggles when I transitioned into management, but that's a story for another post.