How Workday uses data for Canvas Design System

As we’re building Omlet, we’ve been talking to lots of design system teams about what they’re focused on and what challenges they’re facing. And from a data-driven perspective, we’re really curious to know: How are they tracking adoption? What design system metrics matter? Where do you start? How are they measuring success and what role does data play?

I had a great interview with Alan B. Smith, Design System Developer at Workday, where we talked about these things and more. His team builds and supports Canvas, a mature, 6+ year-old design system that’s a central resource for hundreds of designers and thousands of developers at Workday. More than 40 teams use Canvas and 90+ active repositories are pulling from it — whether for single products, entire business units (using a mono-repo), or various extended component libraries.

Most of the time when I’ve talked to design system teams, it’s more common that they’re in building mode and don’t have many resources. But Alan is working on a mature system with a well established team, within a large organization that‘s invested in their work. So when I heard that he and his team are using data for prioritization and decision-making for Canvas, that made me super excited to talk to him and learn more about the role data plays in their design system practice.

I’ll share the main highlights and takeaways from our conversation, including:

• His design system team structure and the Canvas design system
• What design system metrics matter to them and why
• The custom tool they built to track Canvas usage and adoption
• The design system goals and trends they’re paying attention to

Design system team structure and Canvas Design System

Workday’s design system team is actually a very well-invested, advanced team with lots of experience. Their team includes:

• 4 designers that are split between two teams in Europe and America.
• 2 design managers, one for each design team
• 7 engineers, one of which is a developer advocate who supports their internal community that promotes Canvas usage
• 1 PM to keep everything on track

They operate on a very open model where they hold office hours for anyone throughout the company to meet with them, maintain support channels to help capture feedback and troubleshoot issues with Canvas, and are a part of the regular design clinic with the larger design organization at Workday.

As for the Canvas design system itself, it’s the only one Workday uses internally. There are also no extended models or sub-systems — Workday’s acquisition companies may come in with their own design system, but they all typically transition over to or adapt Canvas if they are not a standalone product. Some focus areas for Canvas that you might not always see for smaller design system teams is support for accessibility, which Alan says is a “must” for them, and support for global languages (reading right to left and left to right).

Canvas is an open source design system but what’s unique is that most of its contributions and users are internal to Workday. This leads to an even bigger key difference between Canvas and other established, open-source design systems like Bootstrap or Material UI — those communities are massive, but they’re not working to get everyone upgraded.

As Alan says:

What design system metrics matter and why

Both Alan and I are excited about the growing focus on design system metrics in the industry, since they weren’t even part of the conversation two years ago. But from Alan’s perspective, there’s been a bit too much focus on early adoption metrics because they tell an incomplete picture about design systems.

This goes back to what he sees as his design system team’s purpose: to create a cohesive experience for their users across all products and platforms. It’s a goal that most design system teams have, no matter what their size and scale. So when you think about design system adoption through that lens, Alan explains:

There’s also the fact that adoption can also be teased into sub categories, like how many products or teams are using your components and to what extent. For example, if one product uses certain components very heavily, it can drastically skew your overall adoption metrics even if no other teams are using them. This type of scenario is something Alan’s team watches out for and have built an internal tool to track (we’ll get to that later), and it’s something Omlet is also built to identify and prevent.

What teams should care about

Alan and his team have experienced firsthand how important it is to monitor a variety of design system metrics to ensure Canvas supports a cohesive product experience across Workday. And based on what they’ve learned, the metrics all design system teams should be thinking more about should answer these questions:

• Which components are being used?
• How are they being used?
• Which teams are using them?
• Are there any components usage patterns?
• What do the usage patterns tell you about what components to update, add, or deprecate?
• Which versions are being used, and what do those usage patterns look like?
• Are teams moving through versions successfully?
• What parts of the system are causing problems, and who does it impact?

The way Alan thinks of it, these metrics not only help you get to the “truth” of how your design system is being used throughout your organization, but it also opens up more questions about your teams and how they’re working — with the system and each other. He emphasizes:

In addition to uncovering the truth about your system’s usage, these questions also bring objectivity to your metrics so you’re not basing decisions on intuition, or only looking at data that “looks great on a slide deck”.

Alan shared a great example from the early days of design system metrics at Workday that shows why objectivity matters. Back then, they used feedback from their support channels to prioritize what to update or build, which over time proved to be too narrow of a data set to base decisions on.

Going back to the idea that tracking adoption isn’t enough — this example shows how data isn’t just about measuring adoption, but also informing your decisions and prioritizing what parts of your design system to add to or update.

Building their own system to track Canvas usage and adoption

As Design System Developer, Alan is deeply involved in implementing the systems that help his team measure and optimize how Canvas is being adopted, used, and continuously updated across Workday. This includes those 40+ teams that are already using some version of the Canvas kit, as well as those who have yet to adopt it at all. What’s really cool is that his team is one of the rare examples of a design system team that built their own internal tool to track component usage and adoption.

What they built

Without getting into proprietary information, the underlying piece of their tracking system is GitHub search api. They use GitHub's search to track the vast majority of their usage data at a high-level, like which repositories are using Canvas code and what version of Canvas they’re using. They then go one level deeper by pulling down all of these repositories locally and parsing through the code — similar to what Omlet does. At that point, they’re digging into component usage down to the prop level to understand what they're passing along and who's using what where.

They collect this data daily and throw it into an S3 bucket, which is then passed along to an Elasticsearch dashboard to create visualizations (Elasticsearch is a free and open search and analytics engine). Though Elasticsearch is a great tool, Alan admits that they’re “trying to do something with it that not really what it’s made for.”

If you look at how their system works overall, it revolves around two main areas: 1) component usage tracking and 2) data visualization for monitoring and analyzing that usage data. We’ve been focused on those same two areas with Omlet, and today there are features to help teams do both.

The challenges and limitations

This internal tracking system is still growing and developing. And while it’s been an essential piece of how Alan’s team is able to monitor and improve Canvas usage and adoption, there are a few challenges and limitations that come with it.

It’s an imperfect system

Alan is the first to admit that there’s a lot more they want to do to enhance their tracking system, and that it’s current setup has imperfections they simply have to deal with for now.

Getting the data they need is manual, so scalability is an issue

While using things like GitHub search helps speed up their ability to locate repositories using Canvas, their tracking system overall takes regular manual effort. Alan is “still mostly pulling data manually”, which goes against the goal of scalability — a critical aspect of any system that has to serve multiple teams on a regular cadence.

An internal system = a time-consuming large-scale effort

It’s a huge accomplishment for them to have built this internal tracking system for Canvas. But for other teams to even think about replicating it for their own design system practice, they’d have to spend a lot of time and resources to get to where Alan’s team has.

Design system goals and trends they’re paying attention to

I’ll leave you with a few last highlights Alan and I talked about, which include some broader thoughts such as: what he’s looking forward to for the whole industry, the emergence of design system PMs, and what he and his team are looking forward to in their own work.

A growing focus on designs system metrics

It was really exciting to hear that Alan, someone who’s part of a very mature design system team and practice, is thankful for and excited about how Omlet can make his design system team’s life easier and also change how the industry thinks about metrics.

Design system PMs are becoming more of a thing

Alan has found a lot of value in having a PM on his team and believes that — especially when you have a large number of consumers or a large number of people working on your design system — it’s “an absolute must.” And in the same way he’s noticed this trending focus towards metrics in the industry, he’s thinking design system PMs will also become more common.

Going forward

Here are some finally thoughts and hopes Alan has for the future and how his team can approach their design system goals and metrics going forward.