How Workday uses data for Canvas Design System

Pelin Kenez Image

Pelin Kenez

Share on

How Workday uses data for Canvas Design System Thumbnail Image

As we’re building Omlet, we’ve been talking to lots of design system teams about what they’re focused on and what challenges they’re facing. And from a data-driven perspective, we’re really curious to know: How are they tracking adoption? What design system metrics matter? Where do you start? How are they measuring success and what role does data play?

I had a great interview with Alan B. Smith, Design System Developer at Workday, where we talked about these things and more. His team builds and supports Canvas, a mature, 6+ year-old design system that’s a central resource for hundreds of designers and thousands of developers at Workday. More than 40 teams use Canvas and 90+ active repositories are pulling from it — whether for single products, entire business units (using a mono-repo), or various extended component libraries.

Design system statistics for Workday's Canvas

Most of the time when I’ve talked to design system teams, it’s more common that they’re in building mode and don’t have many resources. But Alan is working on a mature system with a well established team, within a large organization that‘s invested in their work. So when I heard that he and his team are using data for prioritization and decision-making for Canvas, that made me super excited to talk to him and learn more about the role data plays in their design system practice.

I’ll share the main highlights and takeaways from our conversation, including:

  • His design system team structure and the Canvas design system
  • What design system metrics matter to them and why
  • The custom tool they built to track Canvas usage and adoption
  • The design system goals and trends they’re paying attention to

Design system team structure and Canvas Design System

Workday’s design system team is actually a very well-invested, advanced team with lots of experience. Their team includes:

  • 4 designers that are split between two teams in Europe and America.
  • 2 design managers, one for each design team
  • 7 engineers, one of which is a developer advocate who supports their internal community that promotes Canvas usage
  • 1 PM to keep everything on track

They operate on a very open model where they hold office hours for anyone throughout the company to meet with them, maintain support channels to help capture feedback and troubleshoot issues with Canvas, and are a part of the regular design clinic with the larger design organization at Workday.

Design system team structure for Workday's Canvas

As for the Canvas design system itself, it’s the only one Workday uses internally. There are also no extended models or sub-systems — Workday’s acquisition companies may come in with their own design system, but they all typically transition over to or adapt Canvas if they are not a standalone product. Some focus areas for Canvas that you might not always see for smaller design system teams is support for accessibility, which Alan says is a “must” for them, and support for global languages (reading right to left and left to right).

Canvas is an open source design system but what’s unique is that most of its contributions and users are internal to Workday. This leads to an even bigger key difference between Canvas and other established, open-source design systems like Bootstrap or Material UI — those communities are massive, but they’re not working to get everyone upgraded.

As Alan says:

Most open-source design systems have unmanaged scale. They are not actively trying to get everybody who uses them on the latest version. Whereas us, we have a very large managed scale where we actively care about people being up to date on the latest version. I think that's one of the key pieces about Canvas.

What design system metrics matter and why

Both Alan and I are excited about the growing focus on design system metrics in the industry, since they weren’t even part of the conversation two years ago. But from Alan’s perspective, there’s been a bit too much focus on early adoption metrics because they tell an incomplete picture about design systems.

The majority of the conversation across design system metrics is about adoption and it's like, yeah, but so what? So what if they're using your stuff? Adoption alone is not giving you a good indicator because it is not telling you about the health of your ecosystem and how well you're supporting it.

This goes back to what he sees as his design system team’s purpose: to create a cohesive experience for their users across all products and platforms. It’s a goal that most design system teams have, no matter what their size and scale. So when you think about design system adoption through that lens, Alan explains:

Sheer adoption is not good enough. For example, Canvas kit V4 looks drastically different than Canvas kit V6. And now we're on V8, V9 releases in a couple weeks. So even though some repositories are updated actively, they’re not updating their version of Canvas kit. That is not creating a cohesive experience for users.

There’s also the fact that adoption can also be teased into sub categories, like how many products or teams are using your components and to what extent. For example, if one product uses certain components very heavily, it can drastically skew your overall adoption metrics even if no other teams are using them. This type of scenario is something Alan’s team watches out for and have built an internal tool to track (we’ll get to that later), and it’s something Omlet is also built to identify and prevent.

What teams should care about

Alan and his team have experienced firsthand how important it is to monitor a variety of design system metrics to ensure Canvas supports a cohesive product experience across Workday. And based on what they’ve learned, the metrics all design system teams should be thinking more about should answer these questions:

  • Which components are being used?
  • How are they being used?
  • Which teams are using them?
  • Are there any components usage patterns?
  • What do the usage patterns tell you about what components to update, add, or deprecate?
  • Which versions are being used, and what do those usage patterns look like?
  • Are teams moving through versions successfully?
  • What parts of the system are causing problems, and who does it impact?

The way Alan thinks of it, these metrics not only help you get to the “truth” of how your design system is being used throughout your organization, but it also opens up more questions about your teams and how they’re working — with the system and each other. He emphasizes:

Your data should push you to ask better questions, like: How do we think about these users and these teams differently? Do developers feel comfortable talking to their designers? We pair the qualitative aspect of where teams are with our quantitative information, and ultimately get sense of what the key problems are that teams are facing.

In addition to uncovering the truth about your system’s usage, these questions also bring objectivity to your metrics so you’re not basing decisions on intuition, or only looking at data that “looks great on a slide deck”.

Alan shared a great example from the early days of design system metrics at Workday that shows why objectivity matters. Back then, they used feedback from their support channels to prioritize what to update or build, which over time proved to be too narrow of a data set to base decisions on.

We were prioritizing things based on people who were asking the most questions or engaging with us more. But it became clear over time that we were tailoring the system for an increasingly smaller percentage of our user base... and as a result, we weren't serving our larger demographic as well.

Going back to the idea that tracking adoption isn’t enough — this example shows how data isn’t just about measuring adoption, but also informing your decisions and prioritizing what parts of your design system to add to or update.

Building their own system to track Canvas usage and adoption

As Design System Developer, Alan is deeply involved in implementing the systems that help his team measure and optimize how Canvas is being adopted, used, and continuously updated across Workday. This includes those 40+ teams that are already using some version of the Canvas kit, as well as those who have yet to adopt it at all. What’s really cool is that his team is one of the rare examples of a design system team that built their own internal tool to track component usage and adoption.

What they built

Without getting into proprietary information, the underlying piece of their tracking system is GitHub search api. They use GitHub's search to track the vast majority of their usage data at a high-level, like which repositories are using Canvas code and what version of Canvas they’re using. They then go one level deeper by pulling down all of these repositories locally and parsing through the code — similar to what Omlet does. At that point, they’re digging into component usage down to the prop level to understand what they're passing along and who's using what where.

They collect this data daily and throw it into an S3 bucket, which is then passed along to an Elasticsearch dashboard to create visualizations (Elasticsearch is a free and open search and analytics engine). Though Elasticsearch is a great tool, Alan admits that they’re “trying to do something with it that not really what it’s made for.”

If you look at how their system works overall, it revolves around two main areas: 1) component usage tracking and 2) data visualization for monitoring and analyzing that usage data. We’ve been focused on those same two areas with Omlet, and today there are features to help teams do both.

design system adoption chart in Omlet.dev

The challenges and limitations

This internal tracking system is still growing and developing. And while it’s been an essential piece of how Alan’s team is able to monitor and improve Canvas usage and adoption, there are a few challenges and limitations that come with it.

It’s an imperfect system

Alan is the first to admit that there’s a lot more they want to do to enhance their tracking system, and that it’s current setup has imperfections they simply have to deal with for now.

The elastic search dashboard — if there's one part of this whole thing that I could fix or that I would like to replace, it would be the dashboard since it's been problematic and fairly unreliable.

Getting the data they need is manual, so scalability is an issue

While using things like GitHub search helps speed up their ability to locate repositories using Canvas, their tracking system overall takes regular manual effort. Alan is “still mostly pulling data manually”, which goes against the goal of scalability — a critical aspect of any system that has to serve multiple teams on a regular cadence.

You want your data to be accurate, but you also want to be able to collect it in a sustainable way. We have 95 repos and it would take up all of our time to see how Canvas is going for every team. You also need to run metrics on a regular cadence, because otherwise you're really cherry picking data and that doesn’t make a reliable data source.

An internal system = a time-consuming large-scale effort

It’s a huge accomplishment for them to have built this internal tracking system for Canvas. But for other teams to even think about replicating it for their own design system practice, they’d have to spend a lot of time and resources to get to where Alan’s team has.

It's an incredibly difficult problem. It’s taken a full-time dedicated person eight months to build the system and it’s still not done. There’s still the dashboard, and to get to a point we’re happy with would take another four months. So altogether this would take a year.

Design system goals and trends they’re paying attention to

I’ll leave you with a few last highlights Alan and I talked about, which include some broader thoughts such as: what he’s looking forward to for the whole industry, the emergence of design system PMs, and what he and his team are looking forward to in their own work.

A growing focus on designs system metrics

It was really exciting to hear that Alan, someone who’s part of a very mature design system team and practice, is thankful for and excited about how Omlet can make his design system team’s life easier and also change how the industry thinks about metrics.

I'm looking forward to a time when we're talking about adoption metrics, balanced with everything else that we should be talking about in the metrics world. And that's why when I learned about Omlet that I was like oh good, somebody's thinking about this. Somebody is actively building a tool that is going to make tracking metrics easier and this can move the conversation along.

Design system PMs are becoming more of a thing

Alan has found a lot of value in having a PM on his team and believes that — especially when you have a large number of consumers or a large number of people working on your design system — it’s “an absolute must.” And in the same way he’s noticed this trending focus towards metrics in the industry, he’s thinking design system PMs will also become more common.

I’m excited that the conversation is trending toward metrics. And I feel like part of that is also that design system PMs are becoming more of a thing. Because if you're gonna create a roadmap, you’ve got to know what's impactful. That's the only way you prioritize things. Having a PM helps with that predictability and making sure that you’re working on the right things.

Going forward

Here are some finally thoughts and hopes Alan has for the future and how his team can approach their design system goals and metrics going forward.

For us, it always goes back to: what is the core value that our design system provides? We help teams provide a cohesive experience for users across products and platforms, and we help them manage change over time. Hopefully there's a future where people can easily stay up to date with Canvas all the time. Ideally, we can make that so frictionless that it's super easy for them to keep up to date.

Share on

Big thanks to Alan for sharing his and his design system team’s unique experiences in building and supporting Canvas at Workday! These have been some super helpful takeaways for us and for any design systems teams out there tackling the same challenges.

If you’re interested in chatting with me about how your design system team tracks usage and adoption, we‘re always looking to learn about and highlight stories from what’s happening in this space. Contact me at hi@omlet.dev.