“Efficiency is doing things right. Effectiveness is doing the right things.” – Peter Drucker

We live in a townhouse. If you don’t know what a townhouse is, it’s built high rather than wide, so our kitchen is on the middle floor. It’s my job to deal with the recycling, but rather than bring each can, tin, and paper box downstairs individually, I stick them in a bag and bring them down in one go.

I used to be an Engineer so I’m incredibly lazy, so I want to minimize the number of times I clomp down and up the stairs. However, if I leave it until the recycling bag is full you can imagine what happens. That’s right, I leave a trail of recyclables through the house like an eco-friendly Hansel and Gretel.

It’s more effective if empty my bag before it’s overflowing.

The same thing is true for your teams.

Why aren’t you writing code for eight hours a day?

With the economy the way it is, we’re all being pushed to do more with what we have. The years of being able to hire to take on extra demand are over, so our business partners are looking at how we spend our time more than ever.

You’ll be pushed to explain why your teams aren’t cutting code, testing, or delivering for a full eight hours every day.

You’ll be asked to provide timesheets, cut meetings, and push your teams more to get the work that directly delivers customer-value up to 100%. Ruthless efficiency may look good on paper, but is it effective?

  • What about collaboration? Are your teams working well together?
  • What about experimentation? Are your teams innovating and learning?
  • What about customer feedback? Are your teams building the right things?

Like the recycling, it’s more efficient to only make the trip when I’ve packed the bag full to overflowing, but it’s not effective because I have to tidy up afterwards.

Are your teams more efficient than effective?

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes. It is wrong to suppose that if you can’t measure it, you can’t manage it” – W. Edwards Demming

When I was a baby engineer, the VP of Engineering where I worked decided to measure build breaks as a proxy for quality. He was so excited by this, he then went wild with power and decreed that no developer can break the build more than once per quarter. So much quality, right?

You can guess what happened next – no developer committed to the code base more than once every four months, those changes were huge, and tracking down where the build was broken took months.

This is an example of Goodhart’s Law:

“Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”

Charles Goodhart

As soon as you start using metrics to manage people, those people will change their behaviour to game those metrics. It’s better to use measurements as a temperature check to examine your Engineering system then zooming into areas that aren’t working using qualitative techniques (a fancy way of saying TALK TO YOUR ENGINEERS).

What metrics are best?

In Working Backwards, Colin Bryar and Bill Carr at Amazon describe two types of measures: Output and Input metrics.

  • Output metrics: Measure how successful a process or activity is.
  • Input metrics: The activities that lead to changes in output metrics,

Most of us concentrate on output metrics, for example looking at defect rates, but forget that there is little we can do to directly improve that measure. To drive down defect rates, we might think about how much testing is automated and how often those tests run – this is your input metric.

Choose output metrics that align with the value you want to give to your customers and choose input metrics that you think will improve those. Always start with customer value.

Measure multiple dimensions

Don’t be like the VP – balance your metrics across multiple dimensions. Nicole Forsgren describes five dimensions for highly effective engineering teams:

  • Satisfaction and Wellbeing
  • Performance
  • Activity
  • Communication and Collaboration
  • Efficiency and Flow

You want your metrics to balance across two or three of these to avoid forcing your teams into strange behaviours.

Another example, if you’re measuring number of commits (activity) only, then it’s going to be really easy for teams to stop helping each other while trying to optimise getting code into mainline.

Balance is the key.

Being data-driven is really fashionable, especially as business push to do more with fewer people. Being only data-driven is not enough, remember your teams are people, remember your teams need to collaborate. In the end, engineering is a human endeavor.