How to optimize for speed and flow - insights from QCon London 2022
My colleague, Matthew Skelton (co-author of the book Team Topologies), was asked to curate a track on the final day of this year’s QCon London conference on the topic of ‘Optimizing for Speed and Flow’. I was fortunate to be able to attend and watch all the talks and panel discussion on this track (and live-tweet from the company account!). In this article, I have shared my key takeaways from the sessions in the track.
[ If you missed the in-person event in London, you can see versions of these talks at QCon Plus - 10 May 2022. ]
Key Takeaways
Systems thinking is key
In order to optimize for fast flow, organizations must take a holistic, systemic approach to how they do their work and how they create, deliver, and operate software. The organizations in which we work are sociotechnical systems, as are the software applications that those organizations create, and so we need to apply sociotechnical thinking to our work. We cannot focus on any single aspect or part of that system and ignore others.
We need humane ways of working
The ‘socio’ part of sociotechnical systems is about people and we must recognize that people are not simply cogs in a machine. Crucially, what is good for people is also good for business. Giving people space to grow their skills, time to reflect on their work, the ability to make decisions about what work they do and how they do it - all these things make people more productive and help improve flow.
Boundaries matter
We need to have a clearer picture and a better understanding of our software systems and the teams that create and operate them. Boundaries matter. With regards to the software, this means that we need to understand which parts of the system create value for the business, and which are ‘merely’ supporting functions. For the teams, it means understanding the need for each team to have end-to-end ownership of their part of the system and, importantly, understanding the idea of team cognitive load - that if a team is asked to be responsible for too many parts of the system, the health of the software and the team will suffer as a result.
There are lots of threads to pull on
There is no one ‘right’ way to go about optimizing for speed and flow: there are lots of threads to pull on. It’s about system architecture, teams, platforms, developer experience, observability; it’s about how we think about data, how we think about security, encouraging people to share their learning - the list goes on and on. This might seem daunting at times, but it’s also a good thing - it gives us lots of ways to approach the problem, it gives us a large ‘attack surface’. By being persistent, and consistently, but gently, pulling at these threads over time, we can make big improvements to flow.
Summaries of each talk in the track
I’ll now give summaries of each talk in the ‘Optimizing for Speed and Flow’ track.
Nick Tune: Sustaining Fast Flow with Socio-Technical Thinking
Nick Tune, an independent consultant and acknowledged guru in the field of Domain Driven Design (DDD) kicked off proceedings with a talk entitled ‘Sustaining Fast Flow with Socio-Technical Thinking’. His take on how to achieve sustainable fast flow was that it comes down to three things: incentivising your people, decoupling, and building an effective platform. These must all be underpinned by sociotechnical thinking.
He described the need to avoid what he called a “flow-destructive engineering culture” where there is a constant rush to always meet the next arbitrary deadline and people are viewed as fungible resources that can be moved from team to team, wherever the current deadline happens to loom closest.
He then moved on to talk about decoupling - the need to both understand the domain boundaries of your business and software systems, and the need to align your organizational and team structures to those boundaries: “Team organization has a big impact on flow. Finding the right team topology matters.”
The final part of Nick’s talk was about platforms and how having good developer experience is essential in optimizing for speed and flow. He described some examples of poor developer experience that he had seen in his work - developers’ laptops locked down and developers unable to install the tools they needed - and then went on to describe some of the attributes of a good, modern developer platform and the positive impact that has on flow.
This was a great talk, with a generalist feel to it that was just right to (gently) introduce the topic of ‘Optimizing for Speed and Flow’.
Truls Jørgensen & Audun Fauchald Strand: Optimizing for Fast Flow in Norway’s Largest Bureaucracy
Up next were Truls Jørgensen and Audun Fauchald Strand who are both principal engineers at NAV, which is the Norwegian Labour and Welfare Administration or, as Truls and Audun describe it, “Norway’s largest bureaucracy”. Their talk was the story of the changes which have taken place at NAV since 2016 as they have strived to achieve ‘alignment in the age of autonomy’.
They described the new approaches and practices which have been introduced at NAV that have resulted in greatly improved flow at the organization. As principal engineers they knew that putting good engineering principles on the company wiki isn’t enough, and instead saw their role as providing practical advice and guidance to the teams - should the teams want it - and recognised that each team was different, with its own unique context.
The new practices and approaches that were adopted at NAV fall into four categories and, for each one, Truls and Audun gave concrete examples of the bad ways of old and the new flow-enabling behavior which has replaced it. These were: data driven - examples are moving from point-to-point integration to publishing events, and from hard-to-find data to data mesh; culture - for example moving from centralized governance and closed source code to ‘be open and use open-source’; techniques - examples were moving away from feature branches to trunk-based development, and moving from centralized security to distributed, built-in security with Security Champions in development teams; and finally, sociotechnical: for example moving from centralized and coordinated interaction to Team APIs, and from financing projects to product teams.
This was an excellent talk: engaging and inspiring, with beautiful slides, and Truls and Audun were warm and genuinely funny speakers.
Jessica Kerr: Observability for Speed and Flow
Jessica Kerr, who blogs and tweets using the moniker of ‘Jessitron’, is a developer advocate at Honeycomb.io. Her talk was on the topic of observability and how that helps organizations looking to optimize for speed and flow.
Jessica began by describing how work flows through a system in different ways: requests through the software, changes made to the software, work through people, and changes in the people. The communication lines between people in an organization should match the requests made through the software; this is known as ‘social mirroring’ or Conway’s Law.
She then went on to explain how the part of the system that is the slowest to change has the most power because everything else has to bend around it, and that if the software cannot change, then your business cannot change. In order to be able to make the changes required, we need knowledge of our systems so we can work on them easily and quickly make decisions about what those changes should be.
She stressed that flow is an emergent property of the system and that focusing too closely on a single metric, such as number of features delivered, can actually block flow because having such a ‘laser focus’ prevents us from looking at other parts of the system.
I loved this talk. It was full of interesting ideas and covered a broad range of topics yet it was very easy to follow along and understand although live tweeting was somewhat problematic because it seemed like nearly every single thing that Jessica said was a ready-made tweet if only I could have typed fast enough!
Susanne Kaiser: Architecture for Flow with Wardley Mapping, DDD, and Team Topologies
The last of the talks was given by Susanne Kaiser, an independent tech consultant whose area of expertise is in combining Wardley Mapping, Domain Driven Design (DDD), and Team Topologies. Her talk was about how these three different perspectives and techniques provide a powerful toolset to design, build, and evolve adaptive systems for a fast flow of change.
Susanne began by explaining how Wardley Mapping is used to understand user needs and such things as the competitive landscape in which the business operates. It also shows how the existing system components meet those user needs, and whether those components are, or should be, being built in-house or be purchased as off-the-shelf products.
She then gave a brief overview of the main ideas of Team Topologies and described how there are four fundamental team types: stream-aligned, platform, complicated subsystem, and enabling, with three interaction modes: collaboration, facilitating, and x-as-a-service.
Then she turned her attention to DDD and explained how a business needs to distill the problem domain in which it operates to understand what its core domain, and supporting and generic subdomains are.
Susanne finished by outlining how the three are used together - with Wardley Mapping showing which domains should be custom-built and which served by off-the-shelf products; and Team Topologies principles being used to ensure that each domain is owned by only one team, that the teams are of the appropriate type.
I hardly need to say that I am grossly over-simplifying here. This was an incredibly dense and technical talk, crammed full of detail and challenging ideas. Susanne’s speaking style reminds me of some of my lecturers at university - she moves quickly and easily from challenging idea to challenging idea and expects her audience to just keep up with her. Challenging but rewarding.
The Panel
The panel discussion was on the topic of social practices, specifically those practices that are needed to foster a good engineering culture. Taking part were Nick Tune, who had already spoken earlier in the day; Emily Webber - independent consultant and author of the book on building successful Communities of Practice; Richard James - head of the Ways of Working Centre of Enablement at the Nationwide Building Society; and Victoria Morgan-Smith - Director of Delivery for the Engineering Enablement teams at the Financial Times and co-author (along with Matthew) of the book ‘Internal Tech Conferences’.
The conversation between the members of the panel, with some questions posed by Matthew and others by members of the audience, was interesting and covered topics including Communities of Practice, whether there is a difference between standards and standardization, ‘broken comb people’, and the need to give teams time and space to experiment and evolve better ways of working.
It was a great conversation, with plenty of insights and practical advice from the panelists, and 40 minutes disappeared all too quickly.
Summary
If I had to distill my takeaways from these talks into just one, it would be this: that flow is a property of the whole system; we cannot optimize for fast flow by focusing our attention on any one part of that system.
To see online versions of these talks, head to QCon Plus - 10 May 2022.
