Ben Rometsch Interview | Craft of Open Source Podcast

Flagsmith’s Founder and CEO, Ben Rometsch, believes open-source software is eating the world. In this special interview episode, Jason Bosco from Type Sense sits in the interviewer’s seat and conducts an exciting conversation with Ben.

This episode delves into the story behind Flagsmith and the pivotal moments that drove its creation. We explore his decision to embrace open-source from the start and the strategic considerations around its future monetization. Together, Jason and Ben discuss the significance of self-hosting for customers and the synergies it brings to software deployment.

Ben shares how he approaches positioning the product, bridging the gap between proprietary solutions and an open-source alternative. They also delve into the realm of developer marketing, exploring effective channels and lessons learned. Don’t miss out on the exciting glimpse into Flagsmith’s future roadmap and the enduring philosophy of simplicity and sustainability in software development. Tune in now!

—

As Ben was asking me things about Typesense, I was itching to ask him some questions back. We started chatting and figured we were calling a reverse episode this time where I ask the questions that I didn’t get a chance to ask Ben last time. I’ll be asking Ben questions about what he’s been working on with Flagsmith. We’ll see how the conversation goes.

I’m looking forward to it. Hopefully, people get to hear the story.

Ben, tell us what Flagsmith is in a couple of sentences and how it started. What are the eureka moments that made you build Flagsmith?

Flagsmith is a feature-flagging, remote config, and A/B testing platform. It is designed to help product teams ship faster and improve their code quality. It’s a bit of a meta-tool because it’s software that helps you write software.

If you look at that documentation, there was a seminal blog post written by Flickr in 2009 where that was the first time that it was not popularized, but it became an established practice in software engineering. Martin Fowler might have talked about it before that.

As far as I’m concerned, that Flickr blog post was the starting gun for the idea. The idea is that when you’re deploying software and you’ve got the deploy phase of the software tightly coupled to the release phase of the feature, that can generate all sorts of difficulty, especially for larger teams working on larger products. Feature flags and remote config allow you to decouple those two things. You can deploy code, but the code isn’t live. Those code paths of new code that you’ve written aren’t lit up and aren’t doing anything, but then when you toggle the flag, they become alive. Your features go live. That decoupling is the core concept.

How Flagsmith came about, there is quite a long answer to that question. I left university and got a job in a management consultancy writing software. After a couple of years, I went and joined a dot-com agency back in the day that managed to last about a year and a half before it flamed out. A good friend of mine was working at another agency that exploded or imploded.

We went and started a software agency many years ago. In America, you call it a consulting shop. We started writing software for people, websites, web applications, and then mobile apps. We do a lot of cloud transformation. It’s an independent company. It’s still independent. It is a relatively small headcount that fluctuates between 10 to 20 people.

Two things happened in that agency that resulted in Flagsmith. The first one was a few projects that we were working on with larger customers. The example I give to people is on sales calls. We were working with a very large British airline. They wanted us to build this operational dashboard for the airport that they mainly fly out of in the UK. We hadn’t worked with them before. They introduced us to how they build software, me and the team product owner. They were doing agile as far as they understood it. They were doing standups. They had sprints. They had a product owner. They had burned down charts and all this stuff and daily standups.

After we’d been on the first six weeks, writing this platform for them, and it was coming to the first release, we were like, “Who do we talk to about getting the thing live?” They were like, “We have a one-week release window every three months.” I was like, “I probably should have found that out sooner than I did.” They were like, “If you missed that one-week release window, then you have to wait two months and three weeks for the next one.” We were like, “That’s interesting. We have never heard of that before.” They’re this very engineering-led company with safety at the core of their engineering.

There were a couple of other projects that we did with larger teams. I wouldn’t call them dysfunctional, but they weren’t nailing it in terms of their engineering and velocity. We saw this common problem. Let’s say you have five things that maybe three different teams are working on and they all rely on each other in some way, or they’re all coupled in some way. If one of those things doesn’t make it production-ready at the end of the sprint, then you delay the go-live until the next sprint.

You know what’s going to happen in the next sprint. You’ve got two sprints worth of stuff you’re trying to deploy. Something else is broken. We’ve worked on projects where that has happened six times. After the fifth sprint has been missed, the board or the CTO was like, “What are you doing?” Everyone is freaking out and stressed. There is a ton of stuff that’s about to go live. There is a huge amount of work that’s about to go live. No one is quite sure what is going to happen when it goes live. The pressure is building and the release is getting more complex.

We were like, “This is nuts.” This was a couple of years ago. We were like, “We got all this automated tooling, testing, pipelines, and all this stuff, and we can’t get the fucking software release because one thing in one feature is broken and someone has decided that that is not acceptable.” Those sorts of projects that we are working on made us realize the value of feature flags as a concept. In partnership with that, agencies are in a terrible business in the feast and famine businesses. You’re either too busy or not busy enough. You’ve either got too many staff or not enough staff. It’s impossible to get that balance right.

Agency are in a kind of terrible business in the kind of feast and famine businesses. You’re either too busy or you’re not busy enough. You’ve either got too many staff members or not enough staff. It’s impossible to get that balance right.

We built an LED wall. We bought a massive Perspex pane and stuck LEDs to it. Neil still has scars from soldering this massive LED wall. We then built this mad software on a Raspberry Pi to drive this LED wall. It’s quite cool. We over-sourced that platform called Rio. Search GitHub for SolidStateGroup/rio. You’ll find the code to make your own LED wall.

We had loads of people contacting us going, “This LED wall, I made by myself.” That was cool. It was fun. It was enjoyable. We did some wacky stuff in terms of driving the panel and software, but in terms of commercial value created, it’s like a black hole. We thought, “We should probably try and do something with a commercial benefit that we could maybe become its own product potentially.” It was very much in the mold of 37signals who did the classic rotate into a product business.

A friend of mine, Josh, was working with us at the time. He’d been a VC for a couple of years at the top-of-the-funnel VC in London. He’d seen loads of businesses and what works and what doesn’t. He helped us put together a framework for identifying whether an idea would be worth trying. We’d come up with ideas and then we’d fill in the template or the form of like, “What is it? Who would buy it? How long would it take to build? What’s the competition like?” It was pretty simple, but that framework was helpful. We had a couple of ideas that bubbled to the top of that process. Feature flags were one of them.

The other thing about that project at the time was we thought feature flags are like bullion. We’ll be able to build that and have it completely finished in three months. How hard can it be? I told a friend of mine about the business. He works at a bank as an engineering manager. He was like, “People pay for billions?” I’m like, “They do.” Like all engineering, it’s never as simple as you imagined.

You could get to an MVP quickly. We built an API quickly and the platform looks fairly different from what it did. At its core, it was like, “Here’s an SDK. Make a call for some flags and get your flags back.” That is simple. It’s not like building a search engine. You’re looking at a long narrow slope rather than a vertical wall in terms of this as a project.

Also, the other main thing was Kyle, who wrote all the front end and did a lot of the design of the platform, it was his idea. He filled in the form. I read it and I was like, “What are the open-source competitors?” There were 1 or 2, but they weren’t fully formed maybe commercial open-source companies. They were side projects, individuals, or something like that. There’s the go one. They’re good, but as far as we were concerned, they weren’t fully formed that this is the one you use.

With search, there are 3 or 4 open-source search platforms that everyone knows about. They all may be slightly different designs and aim at slightly different use cases. If you were to ask an engineer what’s an alternative to self-hosting something like Elasticsearch, Algolia, or something like that, most people would probably rattle off two or three names that were in the heads of the global consciousness of engineers. As far as I’m concerned, there weren’t any of those names in my head at least. There were only three commercial solutions at the time.

There are good indicators of this idea. It’s relatively simple to get something valuable done. We could open-source it, which we always considered. Right out the gate, it was like, “It should be open source.” It was inspired by GitLab. I remember using GitLab when it was early and it had this shaky deployment story and looked terrible. It was like it was your own self-hosted Git thing. We were like, “We need to open source this.” I’m the underdog wearing the science glasses going, “I’ve got no idea what I’m doing when it comes to open-sourcing a project because I’ve never done it before.” That’s how it started.

I have a couple of follow-up questions about that. It sounds like when you went in, you had the idea that you would open-source it right away. Did you think about things around the business model at the time? Was it, “Let’s open source it and see what happens,” or did you already have a plan for how you would potentially monetize it in the future?

We didn’t put much thought into the commercial side of it. These sorts of projects generally got fairly liberal open-source licenses. They build a community, and then they monetize by providing a hosted SaaS platform. You don’t press a button and a SaaS-paid platform suddenly appears. I love Chargebee. That helps a ton, but it’s still rate-limiting and properly securing the platform. There is a ton of stuff, and we didn’t want to think about any of that. It felt like, “We’re not going to have to think too hard about there being a commercial opportunity here,” because it felt like it was so plainly obvious.

That’s been proven to be wrong in that the business is strong in the on-premise space rather than the hosted SaaS platform. That was naivety as well a little bit and not appreciating. It was the open source nature of it being an interesting component of that. We knew that there was an obvious commercialization path, but we probably should have put more thought into it.

That’s interesting. From your perspective as a business, do you have a preference when a customer, self-hosts or host with you or does it not matter to you?

Engineers always don’t like repeating themselves. From a purity point of view, having a single SaaS platform that everyone is running against is an elegant business. That’s the thing that you like. That was what we were originally not aiming at but felt like that was the obvious thing to do. If you look at GitHub, GitHub is not even open source, but you can run GitHub as an appliance inside your organization.

We don’t have a preference. There are pros and cons for either customer. The self-hosted side has made us behave much more rigorously around the deployment story and the deployment UX. We didn’t have much of a story around Kubernetes. It was like, “If you want to host it yourself, here’s the code. We’re not interested in doing anything else.” Now, it is like, “You want to host it yourself. Here’s this comfortable deployment sofa that we’ve made for you. That’s important. People are paying for that.” I don’t know why, but they perceive this purity in there being one platform that everyone uses. The problem with that is that then, the data is exfiltrating your organization. For a lot of organizations, they can’t do that.

It’s interesting that you mentioned the experience around self-hosting. That’s one thing that makes any software that has a SaaS component and also has an on-prem component that much better. You also have to worry about the deployment experience, which is not just for your customers, but also helps you internally when you deploy your own software. It simplifies things so much. I can relate to that as well. If we’re able to simplify anything in the deployment experience with Typesense, for example, it makes our own lives easier selfishly, and in the process, it also makes our self-hosting users’ lives that much easier.

One of the most insightful things we’ve discovered is that if we hadn’t open-sourced the platform, we’d decided straight away we were going to build it as a commercial entity. I know that I would’ve been the first one as well of the engineering team to be like, “Let’s go and use all this new wacky AWS stuff that no one’s ever heard of before. We’ll have SQS. We’ll have Lambda@Edge. We’ll use fifteen different AWS tools or products.”

Because we open-sourced it first, we were like, “We can’t do any of that because people want to be able to do docker-compose hub, and a minute later, they’ve got the platform.” You can’t do that if you are having to give Amazon $250 and wait 10 minutes for it to provide all this mad stuff. The infrastructure design constraint of the open source aspect meant that we, not by choice but by design, had to be completely agnostic about where we were running. You can run it by hand with Python, Node, and PostgreSQL on your laptop. You can run it on OpenShift with all this mad stuff going on. We only got to that point, not by accident, but the open sourceness of the project forced that.

We didn’t realize how powerful an idea that was until we started having people say, “Can I run it on Azure because I’ve got Azure’s government data center and we can run it on Azure.” It’s like, “We can run it on Azure.” We aren’t running these mad AWS services. It’s a database and an API. They’re like, “Can I run it on OpenShift?” We’re like, “Yeah.” It’s the same answer over and over again. It’s boring.

I’ve been working as an engineer for 25 years. Something like a classic ASP, that was the architecture. It was three-tier architecture. We’ve got a three-tier architecture. It’s boring, but you can’t be more flexible as a deployment thing. That was something that I had no idea about. I don’t think people talk about that, in fact, at all. For us, it’s been an important facet of the business.

I can relate to this as well. It simplifies your architecture in the sense that once you start using all these different external dependencies for the software that you’re building, each of them could give you some of the benefits. They’re good products by themselves, but it also adds a layer of complexity on top that you have to deal with, not just you, but every one of your users that are trying to self-host it as well.

I feel like the constraint of designing something that can be self-hosted on your laptop, a dedicated server that you co-locate somewhere, or one of the Cloud providers, having that constraint leads to a super simple architecture. It then keeps the product easy to work on as you add more features because the architecture itself is designed to be simple. I can relate to that.

Also, as a developer, if you want to develop on the platform as well if you’ve got to handle fifteen different external dependencies and get your time series database working right, you’re going to be like, “I’m not doing that pull request because I can’t get it running.” We are going in the other direction. We have a decoupled front end to the API and they hold the rest. In the front end, we wrote it in Node and Webpac. We’re about to wrap that back into the Django server.

We have four containers, the API, the front end, and the database, and we also use Influx for doing some time series stuff. I want to go down to two containers because we’re going to smash the API and the front end together so they’ll be in one container. I also want to encapsulate the Influx stuff so that we can store that time series data in PostgreSQL.

For people who aren’t sending 10,000 requests a second and don’t need Influx, they can store that in PostgreSQL. For me, if I’m looking at an open-source product and the docker-compose hub is alive and waiting to do stuff, that’s like, “This is awesome.” It’s simpler. The front end and the API being decoupled, as far as OpenShift deployments are concerned, are quite painful because you have to configure them. More firewall ports need to open. There are domain names you need to set up and all this stuff. We’re going in the other direction of fewer services and more monolith because it makes a lot of sense for a number of different reasons.

Around performance, were there any interesting things that you had to account for? You mentioned you’re using InfluxDB. I’m sure that came out of a performance concern.

That was a premature optimization on our part. That is an interesting question. From a scaling point of view, that idea that it’s the same code that’s running on your laptop in 30 seconds is also the code that’s doing 50,000 requests a second. At some point, that’s going to start grating. We’ve been putting off Redis and Celery, which is a tasks queue and scheduler in Python for as long as we possibly can because of small containers or small complexity. It’s more like mental overhead.

I’ve been fighting it off. The developers are like, “We need Celery now.” I’m like, “Is there any way that we can work this feature in without it?” I think of the hackiest solution you can imagine. Does that remove the requirement? That’s getting to the point where it’s like, “We do need it.” Having a task queue and a Redis cache is not particularly exotic in terms of the thing we’re doing.

What that’s informed us about in terms of the design from an open source point of view is I said to the team, “We can do that, but the platform is still working without Celery. It’s a hill that I’m going to die on.” If there is a feature that relies on that infrastructure, then that should be an optional feature. The platform works without Influx. Two features don’t work.

That has been a design philosophy that has bubbled up from this experience whereby things like infrastructure complexity should be an optional thing. If you don’t want to run Influx, you don’t have to, but you don’t get analytics. That docker-compose file doesn’t run Influx by default. It’s commented out. You need to do a bunch of stuff to set it up anyway.

I do worry that you can do that around features, but scaling is more difficult. You can’t do that around scaling. It’s a bit harder. Luckily for us, serving flags is computationally pretty simple. You don’t have to do mad stuff to scale it. They are well-understood problems, like database connections. They are things that have been understood for several years. We’ll see. There might come a time when we need to make some additional projects or something, which is, “This is the Flagsmith that does 50,000 requests a second. That’s different from the 1 that does 10 because most people need 10.”

Switching gears a little bit, how do you think about positioning the product? There are proprietary companies that are doing feature flags. I was looking at your landing page and it seems like you don’t mention explicitly that it’s an open-source equivalent of this or an open-source alternative of this. It sounds like that might be a conscious decision. Explain what the process behind was on how you positioned the product.

We had a board meeting. One of the outputs was, “Where’s the bloody competitor landing pages.” They’re being worked on. That’s been hard. Trying to figure out how the positioning of the business sits in the context of the open-source project is something that we’ve not struggled with, but it’s one of those things that you’re never quite sure if you’re doing it right.

It’s hard to figure out how the positioning of a business sits in the context of the open-source project. It’s one of those things that you’re never quite sure if you’re doing it right.

I can’t think of any particular specific examples, but I always think of an open-source project that is successful and then starts to manage a commercial open-source company. The GitHub logo gets smaller, and then it gets moved into the footer and then it disappears off the site. You’re like, “You’re doing that, are you?” I don’t care if people want to do that.

I feel like being a legitimate open-source project and commercial open-source business is a bit like pornography. You can’t define it, but you know it when you see it. A legitimate open-source project and business are like that. If it’s a greenwash open-source thing, it doesn’t pass the smell test. Do you know what I’m talking about?

Yeah. Where in the spectrum are you in? Are you leaning toward pushing down the open-source brand or do you want to elevate it more than what you do? Are your users explicitly preferring one or the other?

We put some telemetry in. That’s after a bunch of these episodes that I’ve been speaking to from people who are way more successful than we are like Paul of InfluxDB and David Cramer of Sentry. I was like, “Do you do telemetry?” They’re like, “We do telemetry.” It’s important data from an open-source point of view as well. Someone contacted us from a large organization. They were like, “Can you reset the password on my Flagsmith account?” This isn’t the support email that you are looking for kind of thing.

We’ve put together an internal philosophy on when we put stuff into the proprietary code and when we put stuff into the GitHub repository. I should publish it. It is around this idea that we should never kneecap the platform from a developer’s point of view in terms of what’s open and what’s closed. If it was one developer working by themselves, they should have no experiential difference between using the commercial version and using the open-source version.

Role-based access control is a perfect example, which a ton of projects gate on in terms of open and closed. You can still use the platform in the same way and get the same value out of it without role-based access control. If you want to stop the intern from putting the kill switch on your public website, then you need to pass because we want to be a sustainable company.

We’ve got what we describe as horizontal features and then vertical cross-cutting stuff. No developer’s going to be like, “They’ve got role-based access control.” The developers don’t care about role-based. They don’t care about it. They want to be able to use the platform. The business buying it or using it cares about role-based access control. At that point, it’s like, “Pay us.” Almost everything we’ve written has been open and almost none of it’s closed.

That’s interesting. There’s value if you’re able to share that framework that you were talking about. We did have an open-core model in the beginning. After a certain point, one thing that we realized though was that some of the features that were in the open core previously were starting to hurt the adoption of those features. Those features weren’t being used as much as the other features that were in the open source. At that point, we were like, “We don’t want to hurt feature adoption at this point.” Maybe we chose the wrong features to put in the core source back then. At that point, we decided we open source 100% of everything and then only focus on the SaaS version as the way to monetize the product. I’d be curious to hear your framework.

It does depend on what the project does. For example, InfluxDB, and this is common with database projects, is like, “If you want to run one, then you can run an open source. If you want to run two in a cluster, then pay us a load of money,” which is fair enough. You only want that redundancy if you’re doing something valuable.

For Flagsmith, it’s been fairly easy to make those calls. I could imagine there’s a certain class of projects where it’s not obvious where to draw that line. I feel for the people that run those projects because they’re always going to be pissing someone off or worried that you are losing out on a bunch of money because you’ve been too liberal or whatever.

We’ve been lucky because it’s like, “Role-based access control is perfect. If it’s a tiny team, self-host, and brilliant, go for it.” If you’re at the stage where you are controlling which of your developers have access to which environments in Flagsmith, then I feel like, “Pay us.” It’s also interesting as well that the contract seems to have become quietly accepted by almost everyone in our industry. It’s not everyone, but we’ve got people who are quite religious about it. It’s like “That’s fine. Let’s give and take in both directions.” That’s great. That’s one of the reasons why the commercial open-source movement has been popular.

The sales calls were on, “If you want role-based access, you have to pay us.” We’ve not had a single person say, “What are you talking about?” No one said that. They’re almost grateful because they don’t want you to give up because you can’t make money out of it or a project to die. They want you to be a sustainable business as much as you do. That’s been another thing that I’ve been infused by. We don’t have people moaning that it’s closed. No one has gone, “This isn’t fair,” or anything like that. They might be using the software and breaking the license. They might have hacked the platform to do that, but I don’t care about that, to be quite honest.

That’s a good problem to have. I learned after you brought up Sentry that Sentry can be self-hosted. Interestingly, I’ve only heard about Sentry as the SaaS platform. I didn’t realize you can self-host it. At least from all the messaging that I’ve been receiving about Sentry, it seems like it’s SaaS first and then you can also self-host it.

I had a good idea for a blog post. If you get on the Wayback Machine and look at their homepage over the last five years, calculate how many pixels on their homepage they give over to their GitHub links. He did talk about this when I interviewed him. He was like, “It’s hard to get set up. There’s a lot of stuff going on. When you’re self-hosting it, there are a lot of plates you’re spinning at one time. We have to do that for the product. If we didn’t have those plates spinning, then it wouldn’t be as functional as it was.” I’m paraphrasing. At some point, we almost cut the cord. It is like, “Most people aren’t going to self-host it. If you do want, to you can do it, but it’s hard.”

Switching to another topic, let’s talk about developer marketing. How do you approach it? What are the channels that you typically try to engage? What has worked and also what has not worked for you?

We’re looking to get a developer relations or developer advocate person on board. We haven’t done much in that direction. The only time we’ve been on a hacking news front page was when someone posted the homepage randomly one morning. We haven’t done much. We’ve done a couple of meet-up webinars. If COVID wasn’t happening, because I’m in London, we’d probably be doing a bunch more things like that, like meeting people face-to-face and showing the platform and how you can run it yourself and that side of things. We probably haven’t put enough thought into that side, to be honest. This has been a common story with the people I’ve interviewed for the show. We do get pull requests, especially around the SDKs. In terms of the core products, 99% of it is written by colleagues of mine.

Speaking of the SDKs, I noticed that you have probably different SDKs, at least from what I can count, in different languages. Do you hire for those particular skill sets when you write these SDKs or do you have a contract project? How do you do that?

Architecture makes life easier on the SDKs because all of the flags are evaluated on our server. It is different from some of the other folks in the market where they have much heavier SDKs. The SolidState team wrote six of them, and then Rust, Go, Swift and PHP are thin wrappers around the API. As a tiny team, we haven’t had the resources to make all of them as featureful as each other.

The Javascript and the Java SDK are way more advanced than the others, but because of our design, other than caching, there’s not much you’d care about. If you look at the Ruby one, for example, or.NET one, it’s one file with 400 lines of code. They’re syntactic sugar around the rest of the API. That’s been quite simple. It has been frustrating. We would like to make all of them as featureful as each other, but it’s a case of like, “Do people want caching in the Rust library or do they want multivariate flags?” I’d rather have multivariate flags, so let’s write multivariate flags.

That is interesting that you mentioned that. The Rust Library is a community contribution. The PHP library was a community contribution. The GO Library was a community contribution. It seems like the same pattern.

One thing that’s been interesting is the Flutter one. It was written by some guys that were self-hosting the platform in an advanced manner. They’d done all the Kubernetes deployment themselves. They contacted us. They were like, “Do you want the Flutter library? We feel bad that we’ve got a lot of value out of your products.” We were like, “Yeah. That’s brilliant.” One of the things that surprised me, and this happened several times, is where paying customers have been the people building out features specifically for the SDKs as well, which I’ve been blown away by.

I’ve seen that happen with us as well. It’s good because there’s a mutual exchange of value in both cases, and that leads to good communication as well. You have firsthand communication channels open with the contributors, and that always helps. Especially as you add features over time and keep all the libraries in sync, that has been so time-consuming for us.

That’s a thankless task as well.

Have you figured out the process to keep client libraries in sync across clients?

Our solution is not to keep them. If we add a new feature, it is generally encapsulated in the APIs, so the SDKs get that feature for free kind of thing. With the multivariate flags, we didn’t need to touch any of the SDKs because of our architecture. Those API endpoints don’t change. We don’t have this implicit work that needs to be done on the SDKs, but it’s a painful process.

What’s next for Flagsmith? What does the future roadmap look like, if you’re able to share what are some of the features you’re working on?

Having said everything I did about not using mad AWS services, we are building out distributed Edge API, which is fun to design and build. It’s going to Lambda@Edge and DynamoDB global tables. We are building it in such a way that the core platform will still function. It’s almost a plugin where if you want to be able to deliver a global latency API that can scale as much as Lambda and DynamoDB can, then you can go through all this extra work of deploying that. If you don’t, then that functionality will also live in the API. That’s resting on the design idea of making those things optional stuff that you don’t have to build out.

There are a bunch of features that we want to work on the roadmap, like four eyes approval for flags. We are having workflow approval. We need two people to toggle a flag in production, for example, time-sensitive flags and being able to switch features on and off or schedule those in the future, and a lot around the deployment and developer-user experience. It has way better tutorials and documentation. Feature flags are perfect for driving A/B tests as well, but there are a few different components that need to be put together to make that work to get the full end-to-end data flow. We want to do a lot more work around the documentation and show people how you can use it to drive A/B tests.

It’s interesting you mentioned A/B tests. We used to say we were abusing our feature flagging system to run A/B tests, but it works pretty well. In my past companies, we used to do that a lot.

These guys have reversed into feature flags from the A/B testing split very much. They’re coming at it very much more from an A/B testing angle. It is interesting. A lot we speak to want to do it but can’t for whatever reason. We want to do a lot of work to make that solution simpler. We’ve built out some integrations with segment and amplitude in Mixpanel and Heap where we, behind the scenes, send those flags into those platforms. It’s annoying because it demos well, but we haven’t built that into our sales demo. It’s a nice integration.

We’re talking to a couple of analytics companies about figuring out ways that we can share their cohort data, which would be cool. You’d be able to define a cohort in your analytics engine and then manage flags based on those cohorts, which would be cool. Thinking back to when we started building the MVP, it’s never-ending. The roadmap carries on going into the distance. It’s nuts. You must feel the same thing because the naive view of search is that you get to the end.

You think, “It’s an inverted index. The index has a bunch of tokens in which document IDs are stored.” What gets here is the long tail of features. It’s like, “Now we need to worry about this one little feature that still people are using, like adding one element to an existing array in a JSON document.” It is things like that. You keep adding all these little things, so it’s almost like 1,000 cuts over time, even much more. Some of them will look like it’s a small thing, but then, you realize that you’re underlying architecture, you need to go and rewire a whole bunch of things and refactor things to support that. I can see us having a roadmap for the next three years with all of the facets we’ve got so far.

It’s crazy. It’s weird how the software does that. There have been some projects that have been like, “It’s finished. That’s it.” I can’t think of any. SQLite maybe.

I feel like new features are coming out.

Maybe it isn’t SQLite. Maybe it’s something else. It’s curious. It’s a weird facet of software that it never ends. That’s one of the horrible things about running an agency. You get your customers going, “Have we finished yet?” This isn’t how it works.

One of the horrible things about running an agency is you get your customers constantly going on whether you’ve finished yet.

That’s true of any software. You can always come up with new ideas. Over time, keeping things simple and constraining, that’s where the constraints that we were talking about earlier can help, as long as you keep things simple and keep that as your foundation, that leads to good software over time. Ten years from now, when you look back and it looks like a beast to do anything, that’s where you give way for other smaller players to then start attacking smaller things. It’s a cycle. Let’s see. It’s a balance, but if someone can keep things simple for the long term, that creates good sustainable software. This was a great conversation. Thank you for sharing.

That was fun. I don’t know why I didn’t think about doing that earlier. Thanks for taking my chair. You did a better job than I do.

Thank you. This is a great show. I love knowing about all the different guests that you’ve invited so far. You have very insightful conversations.

Thank you.

Thank you for doing this.

Thanks for being in my spot. It’s super fun. Take care.

You, too.

Important Links

• Flagsmith
• Interview with Paul Dix: Co-Founder and CTO, InfluxDB
• Interview with David Cramer: Co-Founder and CTO, Sentry