There’s been a lot of talk about Mobile DevOps lately; teams generally understand the benefits and are motivated to use it as a framework to improve their overall mobile practice. But actually measuring how your team is performing on the path to DevOps greatness can be extremely difficult to do accurately — or at all — and, as a result, it’s equally difficult to iterate and improve on your DevOps practices themselves.
The problem is that, even though there are well-regarded ways to quantify DevOps performance (e.g. DORA metrics), they require you to continuously crunch numbers on a lot of different data, spanning multiple tools and sources. Calculating things like failure rates, lead and cycle times, and time to recovery involves piecing together info from source control, your project management platform, the app stores, observability tools, and more. Doing this well calls for more than napkin math, and for metrics that are regularly refreshed.
Because Runway sits a level above the rest of your stack, we’re uniquely positioned to help here. Pulling together inputs from your entire toolchain and continuously crunching all the numbers for you, Runway will now surface a range of key Mobile DevOps metrics that allow your team to understand how you’re performing and help identify areas for improvement.
The entire development cycle is in scope: Runway analyzes items of work stretching as far back as first ticket creation and, crucially, integrations with the app stores allow for more granular and accurate insights on timing at the tail end of cycles as well.
Plus, certain metrics surfaced by Runway reflect the unique setup of your workflows and release process in the platform, giving you tailored insights into your team’s own way of working.Â
Read on for an overview of the different measures of Mobile DevOps performance that you can now keep tabs on with Runway!
How often do you ship bad releases?
Runway looks at multiple factors to determine whether a release went bad, like whether a hotfix was issued afterwards, if your team’s configured health metrics went “unhealthy” during rollout, or if you halted and never resumed a phased rollout.
If you do ship a bad release, how quickly do you get a fix out?
For any release considered “failed” (see previous metric), if your team decides to issue a hotfix, Runway will measure how long it takes to get that hotfix release out the door.
For a given item of work, how much time is spent each step of the way from inception to release?
Runway tracks work end-to-end – from first ticket, to code written, to code merged to trunk, to release. This gives your team a complete and granular understanding of how long it takes to get features and fixes out to users, and where bottlenecks exist.
Weekly? Biweekly? All over the place?
Runway will keep tabs on your release cadence and help ensure your team is shipping as often as you want to be.
How long does each part of your release process take?
To help you zoom in on and improve your team’s release process, Runway captures time spent every step of the way so you can pinpoint slowdowns and ship more efficiently.Â
How long are sign-offs and action items taking?
Runway can also help you track performance around the unique parts of your team’s process. For checklist and approvals items, Runway surfaces info on how long those take to get actioned by your team.
We’ve hinted at it in a previous update, but we’re super excited to finally announce the launch of Rollouts, a new part of the Runway platform that helps your team run rollouts with less context-switching and more confidence! 🚀
Rollouts allows your team to easily keep tabs on release health in just one place instead of many, alerts you whenever configured metrics become unhealthy, and safeguards release health by automatically halting rollouts based on thresholds you define. (You can automatically accelerate healthy rollouts, too!)
Read on for all the highlights, then hop into Runway to try Rollouts for yourself!
Rollouts integrates with the mix of different tools you use to measure app health – crash reporting, observability & product analytics, the app stores (for per-version user ratings) – to create a single source of truth and unified dashboard that gives your team a holistic and instantly understandable view of release health at a glance. No more context-switching, no more mental overhead, and no more bugging other folks on the team to dig up and interpret data for you.
Across all tools and every metric, you can configure granular thresholds that together define what “healthy” looks like for your team. Runway will surface a crystal clear view of how each release is tracking relative to that definition. With expectations around health codified this way, there’s no room for ambiguity or hand-waving when it comes to your team’s standards for quality of the product you’re shipping.
The moment any of your configured metrics become unhealthy during a rollout, Runway will alert your team, in specific channels of your choosing and with full context on which metrics are problematic. You can be confident that no bad trends will go unnoticed – and that they can be acted on more quickly.
Building on the custom thresholds you define, Runway can automatically halt unhealthy releases. With this automation enabled, if certain metrics of your choosing fall afoul of your acceptable thresholds during a rollout, Runway will automatically stop the rollout and alert your team. This prevents delays and human error, saving you from the negative consequences of a bad release making it out to even more users.
(Available on the Runway Pro plan and above. Existing Runway Growth customers are eligible for a free 90 day trial of this feature.)
If all of your metrics are looking good and your rollout has reached a minimum number of users you define, Runway can also accelerate things and release to all users immediately. Getting a good release out to all users more quickly can be just as impactful as halting a bad release!
(Available on the Runway Pro plan and above. Existing Runway Growth customers are eligible for a free 90 day trial of this feature.)
Many teams rely on Runway’s regression, approval, and checklist items to structure and track the many tasks and signoffs needed throughout dev and release cycles. Now, you can build an even clearer ownership model around these items by assigning them to individual owners.
You can assign items to any individual on your team, or to multiple team members at once – a great way to build shared ownership across distributed product- or feature-based squads. Separately, if you’ve assigned the Release Pilot role, that will now map to the specific person who’s the active Pilot for the release in question.
Assigning important tasks to folks is one thing – actually getting them to take care of those tasks is another. Runway’s new checklist item reminders help you avoid the usual chasing-down and cat-herding.
As your target submission and release dates approach, Runway will proactively send notifications into Slack reminding the owners of any pending tasks to complete them. If needed, you can also manually ping owners of specific items or trigger reminders for all incomplete items, right from Runway.
Since the beginning, one of the core aims of Runway has been to keep teams from ever needing to set foot in App Store Connect or Google Play Console.
Recent additions to Runway’s Metadata and Screenshots steps help make that possible in even more situations: you can now update your app’s screenshots, along with keywords on iOS and your app’s short description on Android.
We’ve heard from many of you that keeping users up to date in App Store Connect and Google Play Console as team members come and go can be a huge time sink. So, we’re excited to introduce Runway’s new user sync automation. With the automation enabled, we’ll sync your team members from Runway to App Store Connect and Google Play Console as needed: inviting new team members, with the appropriate roles assigned, and removing access when team members leave your org.
(Available on the Runway Pro plan and above)
We’re continuing to expand the types of stats and metrics you can keep tabs on with Runway, to help your team better understand how your app’s quality, performance, and process maturity are tracking from release to release.
One new addition on the iOS side is the ability to track how your app’s bundle size changes over time. And for both iOS and Android, see average user rating per release.Â
At the app and org levels, you’ll notice that insights on hotfixes are now surfaced, both total counts and average numbers of hotfixes your team is needing to ship per regular release.Â
If you’re using App Center to distribute beta or other pre-prod builds to testers, you can now hook that up to the beta testing step in Runway. Easily update release notes, distribute new builds to your preferred testing groups, and keep tabs on the latest available builds side-by-side with the rest of your release process in Runway.
It can be tricky, and time-consuming, to bounce between browser tabs and keep track of important markers of release health as you run your rollouts. Runway already pulls in a lot of the important signals, and now we’re giving them a real home! With the new Monitoring section in each release, we’re consolidating stability monitoring, user reviews, phased release status and, soon, product analytics – giving you one place to go to keep tabs on the health of rollouts.
We know that different team members collaborate on releases in different ways, and sometimes folks need to weigh in more efficiently. That’s why we’re introducing Runway slash commands in Slack. Your team can now do things like update the status of regression testing, mark a checklist item as completed, or update an Approvals item, just-in-time and all without leaving Slack. To see a list of available commands, just type `/runway help`. And, if there are other Runway actions you’d like us to add, let us know!
With so many moving pieces, it can be hard to keep track of everything that’s happening during release cycles – and even harder to make sense of things after the fact, if you need to retro or audit a release. To help your team understand exactly what is changing during releases, across all your tools and contributors, we’ve revamped event timelines in Runway. At release level, and also within individual steps, you now get a more complete and detailed picture of every single change and status update – plus, key upcoming lifecycle events and deadlines.  Â
For teams that apply late-arriving fixes to release branches, merging those changes back into your working branch (not to mention any other in-flight release branches) is an annoying manual process that wastes your time week-in, week-out. Now, Runway can automate all backmerges for you!    Â
You can now leverage Runway’s powerful functionality in concert with your team’s own internal tooling with the new Runway REST API. Using the API, you can create and update releases in Runway, update the status of checklist items, and more. To get started, create an API key from your organization’s settings page, then head to our API reference docs to learn more about the types of endpoints available.Â
Missing something your team needs? Let us know! We’ll be expanding the functionality of the API based on feedback.
(Available on the Runway Pro plan and above)
In the other direction, if you need Runway to call out to your stack when certain changes happen during release cycles, this can now be accomplished using outgoing webhooks. You can hook into events like release kickoff or rollout, release pilot changes, and checklist item status updates (plus many more!) to further customize your release workflow in response to changes happening within Runway and across your tools.
To get started using webhooks, add a new endpoint in the Webhooks section of your organization’s settings. You can choose which event types to subscribe to, as well as the specific apps you’re interested in receiving events for.
To see the full list of event types supported, and to explore the structure of the webhook payloads, check out our webhooks reference docs.
(Available on the Runway Pro plan and above)
We often hear that teams care about putting together beta tester notes for each release, but they waste time having to manually copy-paste the notes every time a new beta build lands.Now, Runway can automatically carry over your “What to Test” or “Release notes” from a previous beta build to the next ones.
If your team is one of the ones that always has to provide review attachments when you submit your iOS updates, you no longer need to waste time manually uploading those each and every release. Save your usual review attachment in Runway just once, and we’ll handle the rest automatically going forward.
