Why Is It So Hard to Measure Software Developer Productivity?
Measuring and increasing the productivity of software development teams has come up frequently in recent weeks. Given the startup financial environment there are tighter budgets and more financial scrutiny over big line items in the P&L. Your CFO is probably pushing to understand where all those R&D dollars are going. Why haven’t we delivered on that new feature that was presented in our yearly kick off as part of the Q1 roadmap? Can’t the team work faster?
Measuring and improving the productivity of developers is an ongoing challenge for technology leaders. CTOs, VPs and Directors of Engineering often struggle to communicate with their business counterparts, leaving the engineering team to operate in the shadows. Lenny Rachitsky tweeted about a recent paper from the team at DX titled “DevEx: What Actually Drives Productivity” that proposes a developer-centric approach known as Developer Experience. The article is relevant to the discussion as it proposes a thesis of looking at the lived experience of developers and the points of friction they encounter in their everyday work to provide valuable insights and opportunities.
The traditional measurement methods, such as evaluating output or task completion time, fall short in capturing the diverse and complex activities performed by developers. Some would say that every measurement is useless because all of them can be gamed. In my experience it all comes down to team culture. If you foster a culture of collaboration, safety and accountability you’re likely to not have to worry about anyone gaming the system. By aligning the team with your core purpose, you’ll be sure that everyone is pushing towards your common mission. What I like about the Developer Experience (DevEx), methodology is that it isn’t at odds with some of the other agile best practices that we endorse. The team should still go through agile rituals like grooming, planning, pokering, scrum and retros, but they can use DevEx as a way to balance some of the tougher high level questions of how to prioritize the ongoing work with bigger things that can increase the productivity of the team as a whole.
DevEx revolves around identifying the obstacles that developers encounter in their day-to-day work. It recognizes that factors like interruptions, unrealistic deadlines, and friction in development tools have a negative impact on their performance but they don’t show up in a sprint burndown. On the other hand, clear tasks, well-organized code, and strong CI/CD protocols contribute to performance and even to driving a positive attitude of productivity within the team.
The article presents a practical framework for comprehending DevEx, comprising three core dimensions: feedback loops, cognitive load, and flow state. These dimensions encompass the wide range of challenges experienced by developers. Feedback loops refer to the speed and quality of responses to developers’ actions, cognitive load relates to the mental processing required to complete tasks, and flow state signifies the state of focused and engaged productivity experienced by developers.
The folks at DX say that to effectively measure DevEx, organizations must capture developers’ perceptions as well as their workflows. For them surveys play a crucial role in capturing developers’ subjective experiences and can offer valuable insights into problem areas and emerging trends, however these seem to be hard to put in place in the real world. Compliance and lack of enthusiasm for filling them out from the software development team become a real problem.
The article emphasizes the significance of measuring key performance indicators (KPIs) and overall satisfaction to gauge the impact of DevEx improvements. KPIs such as productivity, satisfaction, engagement, and retention serve as guiding metrics, helping organizations prioritize their improvement efforts. By combining perceptual measures, workflow data, and KPIs, organizations can gain a comprehensive understanding of DevEx and make informed decisions to enhance developer productivity.
From my perspective, a good takeaway is to include a periodic survey in your process to understand the level of satisfaction of each team as it relates to their perceptions of Feedback Loops, Cognitive Load and Flow State. This can be very valuable and easily implemented taking some of the examples from the article:
- Satisfaction with automated test speed and output
- Satisfaction with time it takes to validate a local change
- Satisfaction with time it takes to deploy a change to production
- Perceived complexity of codebase
- Ease of debugging production systems
- Ease of understanding documentation
- Perceived ability to focus and avoid interruptions
- Satisfaction with clarity of task or project goals
- Perceived disruptiveness of being on-call
In conclusion, improving developer productivity necessitates adopting a developer-centric approach that prioritizes DevEx. By understanding the three dimensions of DevEx, capturing both perceptions and workflows, and tracking relevant KPIs, organizations can optimize their software delivery processes, foster innovation, and cultivate a positive work environment for developers.
0 Comments