I don't agree with most of the advice in this article but rather than complain let me suggest an alternative.<p>As a line manager, with software engineers reporting directly to you, you should be able to use your personal judgment to understand the productivity of your software engineers. Don't measure it with acronyms, with metrics like the number of commits, or by paying attention to how many hours a week people are working. Pay attention to whether people get things done, and are they getting big important things done, or only little nice-but-not-critical things. Make sure you communicate enough so that individual software engineers understand how you think and what you prioritize.<p>As a manager-of-managers, it is going to be very difficult for you to measure developer productivity. It's tempting to look at metrics like the number of code reviews a developer does. But these can at most be a sanity check, not the core metric to go for.<p>Instead, you can measure productivity of <i>teams</i>. Is the team getting things done, and are they big important things, or only little nice-but-not-critical things? Sometimes, a line manager will insist that everyone on their team is performing excellently, and yet you observe the team overall is not achieving very much. Probably one of the two of you is incorrect, and you should dig in to figure that out. The opposite also happens, where a manager states that everything is a disaster, but you observe that the team has actually delivered a lot.<p>The other thing you can do is to teach your line managers how to judge individual productivity. There's no silver bullet, it's just a natural outcome of having conversations about who is productive and who is not and how to tell and what to do about it, so be sure to have enough of those conversations.<p>None of this is easy to quantify, but the hard truth is, there is no natural mapping from numbers to developer productivity and it is usually a bad idea to try to quantify productivity. You are much better off using human language and intelligent thinking to evaluate productivity, rather than reductionist metrics.
Funny. I read a book a long time ago about developer productivity. They started with saying: "measuring productivity witk KLoCs is terribad". And later on: but we only have that, so let's use it anyway. "Stopped reading there". And here in 2022 it's exactly The Same Thing:<p>> Measuring developer outputs can be detrimental<p>And then:<p>> Design and coding: The number of design papers and specs, work items, pull requests, commits, and code reviews, as well as their volume or count.<p>All they do is add more and more metrics. But this has the exact same problem as with the infamous KLoCs measure: how do you interpret it? How do you know it is not gamed, to begin with?
Actually, now you have two problems: collecting and analyzing this mass of metrics can have a significant cost.
More lines of code = more liability. Also more commits, frequency of commits etc. does not equate to more productivity. In fact in some instances you actually are introducing more liability into a codebase doing that. The flaw with most of these metrics of "productivity" is that they inherently are saying that coding is analogous to a factory worker building something when in reality it is analogous to someone designing the things the factory worker has to assemble.<p>While I'm not a fan of subjectivity in ratings, the challenge is that it is very difficult, and I would argue, virtually impossible to do it objectively. So what happens instead is that when metrics are used to evaluate engineers, the smart ones figure out how to game them. Does that make them, or the team more productive? Nope. Can that have un-intended consequences that actually make the code less stable, and decrease productivity. Yup!<p>But if you're going to go with these measurements you might as well go big. Throw out anything related to Agile, require estimates that are accurate within 15 minutes and severely punish engineers for not getting estimates right. Might as well also add in heavy documentation requirements too. After all, this rigorous measurement etc. has all worked so well in the past <dripping sarcasm for this last paragraph>.
Measuring developer productivity is like observing quantum state, the act of measuring it generally fucks it all up.<p>Rather than task the developers with all manner of bureaucratic Agile bullshit like tracking hours, arguing about story points, submitting to kindergarten-style daily stand-ups, velocity tracking, retros, etc; I would suggest a different tact. How about measuring developer productivity by observing if they're building what you need at the rate you need it built. If not, then you need to figure out if you can afford to replace them with someone that can.
<i>> because productivity and contentment are linked, it's feasible that satisfaction can operate as a leading indicator of productivity; a drop in satisfaction and engagement could foreshadow impending burnout and lower output.</i><p>Great review of the hazards involved in quantifying developer productivity - the correlation above has been true everywhere I’ve ever worked.<p>If the company you’re working for:<p>- is not investing in improving the developer experience<p>- is not listening to developer complaints about slow, tedious, or error prone processes<p>- is perpetually pushing tech debt onto a backlog that only grows<p>Then chances are you work for a company whose leadership does not understand and value software engineering. They likely see it as a cost center, and they likely incentivize managers by rewarding initial delivery of projects, at the expense of maintainability and developer sanity.<p>I know I’m preaching to the choir, I just had to put it out there for all the young engineers. Don’t waste too much of your life and happiness trying to patch those sinking ships.
As many readers pointed out, team velocity and process bottlenecks are a much more valuable focal point than individual developer metrics. But for this, having the ability to observe what is happening and dig deeper into the data is critical, so that you can back iterations on your improvement efforts with data.<p>The research is also slowly laying the foundation of what are useful metrics to track and what excellence looks like for the industry. Unfortunately, those metrics are typically difficult to measure because the underlying data often spans multiple engineering systems: Lead Time, the poster child of DORA metrics, requires data from at least your source control and your CI/CD systems.<p>Btw, you might be interested in checking out Faros Community Edition: <a href="https://github.com/faros-ai/faros-community-edition" rel="nofollow">https://github.com/faros-ai/faros-community-edition</a> – an open-source engineering operations platform we’ve been building for this very purpose. Our goal is to bring visibility into engineering operations, and make it very easy to query and leverage data both within and across your systems. It’s container-based and built on top of Airbyte, Hasura, Metabase, dbt, and n8n.
> Each organization can set a wide range of metrics to follow every week, such as:<p>> Number of commits.<p>> Average commit size.<p>> Frequency of code reviews.<p>> Number of code reviews.<p>> Time to review.<p>> and so on...<p>No. This has been tried many times and companies think this is how you measure productivity, but it is not even sustainable. Developer productivity is not about moving the needle, it is about outcomes, and not outputs.<p>An outcome is finally merging an unsustainable PR that has sat for a month. It is not how many comments, reviews, meetings, or commits needed to get to the outcome.<p>The only people I know who want to implement these terrible measurements are the type of people who have ambitions as large as Mount Everest but die on the decline back down. The real goal is to be more like a f1 pit crew where you leave out the metrics and end up performing better than if you measured them.
There's a perhaps apocryphal story that, to avoid motivating programmers to pad out their comments, IBM decided to measure productivity not by number of lines of source code written, but rather by number of bytes of object code generated. And then when a new release of the PL/I compiler came out, management was quite pleased to learn that it had improved everyone's productivity significantly!
I really don't think you can measure developers by their productivity. the impact of productivity is predicated on design meeting requirements, the accuracy of requirements is predicated on stakeholders knowing what they need.<p>the only quality that matters is how effective the software is in its business function. how effective does it make stakeholders? how well does it capture engagement by users? the right question to ask changes in business context, but if you can't answer it, you might as well throw darts and flip coins. if you can measure the impact of their code before and after deployment you might have a chance, but it's probably hopeless.<p>as far as I can tell it boils down to a subjective and qualitative assessment of developer performance. you can also take the contrapositive: where would we be without this person? how long would we have taken to get there without them? what would we not have learned without this person?<p>I'm nervous about the implicit bias that comes with this kind of perspective, but I think it's the best we have for now.
There is no end to the search for a developer productivity metric, but it refuses to be found for reasons that are fairly obvious to technical people, but that doesn't stop people from trying – for decades. So now they've retreated to these vectors called "frameworks" that try to obscure with complexity the fact that they are not in any way able to "measure what matters" – in this case, the ratio of value output to value input – nor are in any way deserving of the term "metric". I contend that such non-measures are of absolutely no value to engineering managers; they're management theater and purely a distraction and a waste of time.<p>Let's leave aside for a moment that this piece begins with an impressively uninformed and circular definition – "Developer productivity, in general, refers to how productive a developer is during a specific time or based on any criteria." – and focus instead on the question of why does this stuff keep popping into existence; what's behind it?<p>As a tech exec who's researched and given several talks on this to large audiences of non-technical execs like CEOs and CFOs, I believe the root causes are an understandable and intense desire for "visibility" and exec accountability coupled with a set of false beliefs held by non-technical managers including "anything can be measured if you try hard enough" and "nothing can be managed unless it's measured" and the classic quantitative fallacy of "things that can be measured are more important than things that can't be". Besides, it's only fair that if the VPs of sales and marketing have to stand up and talk about funnel metrics and sales rep productivity (with real metrics like net new bookings divided by fully loaded sales rep cost) that the VP of engineering - an often enormous fraction of a SaaS company's budget – should be similarly held to account for some number, any number, we just need a number, so we can look for "trends" (actually, noise). It also seems to be driven by a push from HR for fairness in promotions and terminations, which is also totally understandable, yet misguided.<p>I have a wisecrack response to non-technical executives when discussing this which is "how do you measure your own productivity?" that helps them understand the absurdity of what they're trying to do and how common it is that no true measure of productivity exists. People really struggle to understand that some metrics, no matter how great it would be to have them, simply do not exist, and so we have this – measurement theater.<p>[edit: fixed typo]
How about starting with mean time to merge a code change? I get that there are other variables that contribute to productivity but things like satisfaction, collaboration etc are extremely difficult to measure well and just IMHO pretty tangential (disclaimer - I work on a dev prod team and my entirety of last year was spent building engineering metric dashboards and discussing what to measure, and it's not easy so I get it).
> Measuring developer outputs can be detrimental because there are not enough data points to understand if the unproductiveness was caused by the developer himself, or by <i>his surroundings/company</i>.<p>How about just doing your best as an organization and as a people manager to make your developers <i>happy</i> and <i>fulfilled</i>? That increases productivity and motivation to succeed more than anything, IMO. Give great pay raises regularly, give a ton of time off, get rid of people managers who are jerks, etc. If your company has goals, and your developers aren't producing code to meet the goals, your goals are probably too high, you have too few developers, or your developers aren't motivated to complete the goals because they are being treated like shit or don't agree with the goals.<p>Management always wants to think that they are right in every decision and the employees are the ones who are unproductive, but after decades of working for "the man" in about 10 different industries in different positions/careers, I have found the fault lies with management 80 to 90 percent of the time due to some leadership failure or combination of failures. The problem is poor leadership and lack of motivation, no doubt in my mind. I've also led large groups of people (in the military) and by far the best thing I could do for them was make their personal and work lives better by not getting in the way and by not acting like a dickhead. Adding metrics to things just caused more useless work for me. You can't force change in a system via metrics, the only place where measurement changes outcome is in quantum physics.<p>I hate to go on a "capitalism vs. communism" type rant, but the best places I have ever worked, with the best "productivity", have been flat orgs where the developers and other employees are included in the decision making and the management and execs are open and caring and don't try to put profits and the business above the personnel. When everyone shared the success or failure of the company on equal terms, we could all get things done that were unthinkable.
Features, Schedule, Cost<p>Well, schedule and cost at least have straightforward measurements.<p>The issue then is features. Or is that it?<p>The "pick two" model really is just the business view. Invariably you will also have:<p>- adherence to process (ideally process would be an overall enhancement to productivity, but it usually becomes a net-negative)<p>- maintenance costs (patching, libraries, language versions, database versions)<p>- infrastructure churn and upkeep<p>- random org shit: meetings, more meetings, training, certifications, HR, ticket walls, etc<p>- documentation. Is that important?<p>- ... are the requirements known? settled? at least ballparked?<p>As I see here, invariably measuring developer productivity is of course blaming the victim: WHY AREN'T YOU MORE PRODUCTIVE, and of course shrugging away the nigh-unlimited ways an org can hamstring or frustrate a developer.
If you are manager of developers and it isn't clear to you who the core developers are and what each member of the team contribute (or not) then you should be fired for incompetency. Talking to developers, and keeping an eye on who does what and knowing the skills of each developer, is the <i>minimum</i> I would expect from a manager. If you can't do that then stop being a manager.
For every productivity metrics don't forget to balance it with a quality one.<p>The reality is always in the middle.<p>Example: If you solve a problem quickly that leads to support ticket being open that's not good.
I’m getting a chuckle at the hubris in the comments so far.<p>Possibly the world expert at this point (Dr Nicole Forsgren) in this exact topic comes up with a framework based on the best of what she knows from years of studying this and refining her approach.<p>Random HN commenter: ahh just measure time to commit.<p>Random HN commenter: biases are cool, so just use personal judgement.
Good research leads to good design resulting in a small amount of code in the right places that either:<p>Fix a bug,
Add feature set
Do something new.<p>My epithet for one programmer was "He writes a lot of code"<p>Extra code makes it harder for the next guy to figure out what's going on, and has more space for bugs.<p>But that came from a "productive" developer and that code can tie down a dozen maintainers in dozens of customer sites.<p>The productivity is job creation for a bunch of folks whose main ambition is finding a job where they don't have to work with crap code.<p>I've done a number of projects where I got rid of multiples of code compared what I put in.<p>The best example was where I replaced a subroutine with a single character constant.
we(as an inudstry) measure it by claiming we do scrum but in actually just create pomp and circumstance and just do what we would do anyways. It gives us a number, we dont care if its accurate or effective