At Skutopia, we wanted to get in-front of developer experience as we started to scale. After a vendor selection process we settled on using Swarmia as our platform. Swarmia allows us to have engineering surveys as well as DORA and SPACE metrics all in one place. This gives us less tooling and quick correlations between developer emotions and tangible metrics.
Why Swarmia?
Swarmia has a bunch of great features that we are finding to be really useful, streamlining productivity and giving engineers what they need to thrive. I'll dig into a couple of features that our engineers are loving.
The Swarmia Slack Bot
The Swarmia slack bot is well loved by our engineers. No longer do we rely on outdated looking webhooks coming directly from github bot. We have control at an individual on what we want the bot to tell us about. We can also set up team notifications that show us how we are tracking in terms of open and stale pull requests.
We can have full conversations with replies directly in the bot rather than logging into github, we get a notification if anything in our commit fails a check, or if anything wonky happens in a pipeline during deploy. The days of baby sitting a merge are gone.
The bot has had an impact reducing friction when it comes to pull requests, which is a great thing for engineers. The image below gives a glimpse of the notification config at an individual level.
Working Agreements
Swarmia describes working agreements as putting "teams in control of how they want to work together. Whether you’re looking to set numeric targets or build healthy habits, there’s a working agreement for that.".
The first few words are the key there. Working agreements are goals or limits as defined by the team themself. For example one of my teams decided they did not want to have anymore than six pull requests in progress at a time. This was because they wanted to focus on completing work, rather than having lots of work happening in parallel.
From the agreement we can see that they have not gone above five pull requests open. At one point this team was looking at ten to twelve open pull requests at a time. By tracking this metric we could then get warnings as we approached a breach.
If we end up with an exception we start to receive notifications from the slack bot allowing the team to take action and focus on closing some open pull requests off.
The Benefits of Metrics
The benefit of a platform like Swarmia is having all of the DORA and SPACE metrics and insights at hand.
I for one completely understand that metrics can be a negative thing when it comes to productivity. Metrics need to be understood at a deep level, and you almost certainly need to have an engineering background to have the context when it comes to engineering metrics.
In saying that, I find the team level metrics extremely helpful when it comes to focus areas. We need the metrics to be able to set targets for the team. It's one thing saying I feel like we have too many open pull requests, but by using metrics we can set a realistic target to improve that feeling.
When it comes to shipping code to production this is even more important. I don't think there is an engineer in the world that doesn't want to get their code out faster. Without knowing where we have bottlenecks in the ticket lifecycle, we are unable to put in place effective measures.
It's important to know whether it is pull requests sitting in ready to review for a number of hours, or whether a tooling choice means the build time is ten minutes instead of two minutes. We are given the ability to focus energy in the right places.
The Benefits of Surveys
Recently Swarmia have completed a private release of their survey feature, which is a feature we are really looking forward to. We have had some conversations with their product team and believe they will be a great asset for us.
The benefit of surveys is that we receive the emotional side of engineering. We get insight into what causes an engineer frustration, happiness or sadness. This again gives great insight where to focus energy on making improvements.
Surveys allow us to roll up the results to various levels. It maybe that a certain geographical location is having more of a challenge than another geography. This could be because of timezones and clarity of information for example.
We may also look to roll up by seniority. Are there challenges for junior engineers that are not the same for senior engineers. This could be along the lines of documentation where a senior engineer does not need documentation at at the same depth as someone learning their craft.
What We Are Looking to Improve
We are using Swarmia to improve anything and everything that we can within our software development lifecycle. The end to end visibility means that we are not limited in what we can improve.
The important thing is being able to identify what those things are. Having the metrics means that we can try different improvements, iterate, and receive feedback that tells us whether the change was good or bad. Surveys give the same feedback from an emotional perspective.
For us, the most important thing is to focus on improving what makes an engineers life harder or causes frustration. These are the pain points that we want to solve. This is a process that will carrying on with great frequency.
Wrapping Up
Skutopia uses Swarmia as our developer experience platform of choice. There are some great features that allow our engineers to be more productive, as well as helping to bring focus. We have both tangible metrics and emotional feedback into areas of the software development life cycle. This allows us to form the areas we need to improve, and gives feedback as to whether these changes are working.
Improving something today does not mean that it won't become a pain point again. It's almost a certainty that it will as tooling improves. Evolving when we need to is the key to improving our developer experience.