Customer Story: Wistia

Deploying 20-30x Per Day With Targeted Rollouts and Unlocked Teams

We sat down with Nick Quaranto, Director of Platform Engineering at Wistia. Wistia is a video marketing platform. Businesses use Wistia to manage and create videos, host webinars, generate leads, and measure video performance—all in one central platform. 

Wistia uses Flagsmith to help with release management and continuous deployment. They deploy 20-30 times per day with the help of feature flags, and their team is freed up for targeted rollouts.  

Let's start with a little bit about you and what your role is at Wistia 

I’m the Director of Platform Engineering. I've been here for three-plus years, so I’ve seen and shipped quite a few features and transitioned into helping manage and run teams. I’ve been uniquely positioned to see how bringing together continuous deployment and release management work in a world where we ship multiple times a day. 

No one wants to hold anyone else up from getting things released, and we have multiple streams of work going on, all inside of one big app. So I have to figure out how to manage it all. 

How did managing those workflows and deployments look when you joined?  

When I first started, we had our own homegrown flag system. We used that for our main app on Ruby on Rails. We found an open-source channel for flags that came with several caveats and we lived with those for a while. Eventually, though, we reached a breaking point and needed a different system because we were having problems during releases.

They were interesting issues. The core thing was that the homegrown tool didn't have a lot of flexibility in how we could define who we were rolling out to. We had to write code in order to say, “I want to target just people that signed up this month,” or “I want to target everyone with a letter ‘Z’ at the beginning of their account name.”

So any time the product team had interesting targeting or segmenting ideas, it would create a lot more work because our system didn’t support it. Then we had to add more time to projects to cover for that. 

We needed to figure out how to invert that control. So now we can tell Flagsmith about the properties in an account and target based on those. 

We could have written something like that. I'm confident we would be able to figure it out, but maintaining it and inevitably breaking it—as tends to happen with software—would have been pretty dang difficult.

The broad majority of teams are using it and we don't have any competing systems running which is the telltale sign that they don’t need anything else.

How did adopting and migrating to Flagsmith go for your team? 

We really liked the fact that Flagsmith has an open-source component if we need it. That made adoption feel less risky. But really, we wanted to get out of the business of building our own solution. 

At this point, Flagsmith is well embedded and the adoption's been high.

We only have one team that's not using it for a subsection of projects. The broad majority of teams are using it and we don't have any competing systems running which is the telltale sign that they don’t need anything else.

Releases feel safer and better overall. Before we used flags, it wasn’t uncommon to go to a post-mortem where someone would say “We should have put a feature flag around this.” 

And how are your team using feature flags now?

We have about 80 users right now that use feature flags and Flagsmith. The users are a mix of product, engineering, and design team members. We encourage teams to get their hands dirty. So, often an engineer will say, “I've got a new feature, it's under flag XYZ, go and turn it on for your account”. Now the right people know how to do that.

We also have an audit log now that we’ve moved away from a homegrown tool. That was huge because we know who’s done what in the code. This means that there aren’t unexpected changes. otherwise, if something unexpected happens in the code, or we had a flag change on a Sunday, we wouldn't know why. 

Did you have an overarching goal of how you wanted your releases to feel or how many releases and deployments you were going for per day? 

We deploy 20 to 30 times a day so it's important that the train doesn't stop. 

But really the main mission was addressing the developer experience and concerns. Like we wanted to unlock the product team and allow them to think wider about how they could do a rollout to users.

How does that unlocking play out for the product team? 

It’s been about lowering the risk and increasing confidence to some degree. Also, in our old system, we would have to manually tinker with it to get whatever new idea we wanted, which just wasn’t efficient. 

One example is that we segment a lot of features by plan type (Pro, Advanced, or Free) and in the old system you couldn't just pull all of the Pro customers.

Now, we can ask the system to give us all the Pro customers that have less than 10 users, or whatever it might be. Product Managers think this way built-in. They're like, “Okay, I need to find the edge cases and who the 80% are and then I'll worry about everyone else later.” They needed a release system that could handle that. 

The way that we were doing releases with that homegrown system was just not flexible enough for where we were as an org.  

Product Managers were met consistently with “No, you can't do that.” Ideas like segmenting would add a week onto the project estimate because we didn’t know how to make that rollout happen with our system.

That’s huge! Were there any fears or hurdles before you made the switch to feature flag software?

Yes. This is pretty mission-critical. If it doesn't work, the app doesn't work.

So that was number one. Knowing that there were a few different ways to mitigate risk helps. We can run the solution ourselves, we run on-prem if we need to. It’s open source. That all helps. 

Stability and reliability were big, too. We all agreed we weren't going to invest time in building out our own system anymore. So we needed a reliable system that could stand up to our needs. But choosing one that had forever lock-in? That was pretty nerve-wracking. We wanted a way that we could mitigate some of that.

Clearly, you have to give up some control if you're going to speed yourself up.

What helped mitigate and make up for that risk for you?

We use the cloud version, but having the open-source version available also helps. 

We also had a pretty reasonable and steady rate of adoption. Three teams adopted the software pretty easily. And then we took a while to create a step-by-step process to onboard the other teams. 

Once one team was on it, people were pretty happy and we could phase the rest on slowly. 

Did you have any difficulties configuring Flagsmith for the team? Did you come up with anything custom when you were deploying and configuring it for what you guys need?

We had some internal views that would show feature flags. So we had to port those across. For example, for a given account you can see which flags you had on. We moved that to be backed by Flagsmith. 

We also did a lot of Docs writing and upgrading as we moved over. We had several generations of feature flag information that had accumulated, so we created the steps for migration, and the supporting internal information. We wanted one straightforward system to manage it all. 

How do your deployments look now? How do you measure success there?  

For deployments we have KPIs. And we have an overarching trend where we are getting more done overall. Feature flagging has been part of enabling that.

Even if you look at our changelog, we’re just in a much healthier spot as an organisation. 

I’m taking a look now, and there are a ton of flags in here that I don't even recognise. There are more than I thought there would be! I think it's part of the overarching trend and without feature flagging, we would be on our own and that sounds terrible.

What about you, personally? How are your life and job different with feature flagging?

Well, qualitatively, an essential part of continuous delivery is how you roll things out and turn features on for users. So for me, I find it very difficult to imagine a world where we go back to not having very capable systems. 

I could continuously deploy without them. But could I do it safely? Before, we were flying blindfolded for a while. Or at least flying unsafely. And it feels a lot safer to me now.

The thing that has been hard to change is that it still takes effort to wrap stuff in a flag. Less is better with devs, especially on the front end. And we did not have a good front-end process for wrapping and feature flags for a while, which is way better now. There's an actual library you can pull in. You can very easily get a JavaScript flag.

Before it was a total nightmare filtering through everything, so we’re in a way healthier spot qualitatively. 

Quantitatively, we deploy 20-30 times per day, which is part of the overarching trajectory of healthier releases. 

Thank you so much, Nick, this has been immensely helpful and we really appreciate your time!

If you’d like to find out more about how Flagsmith could work with your teams, start by creating a free account, or contact us to find out more about how feature flagging could help your development processes.

‍

Read the case study with Wistia here.