Breaking changes on our GitHub app until it works

I’ll be totally transparent here, but not apologetic: I’m the CTO[1] and Watermelon has spent an awful amount of time broken. I was an SRE[2] before, working on systems that reported several nines, 99.999% uptime, and now I’m ok with one nine (90%). We deploy breaking changes and actually break things every other week. I’m sorry if you have been impacted, but it will also happen again.

‍

I am the one in charge of the product, it being a deeply technical one. It’s currently straightforward, we take your PR[3] and bring in information related from other services, and check the code to find errors and give you a preapproval. Sprinkle a little AI[4] here and there, drop a comment on your PR, add a tag to it, and boom, we have Watermelon. And you will ask me, what are the breaking changes that come up if that’s so simple? Well, let me introduce you to the world of VC[5] funded startups.

‍

In a startup that has raised money you are always in a race against time. Your investors want returns on their money and expect you to take risks to get there. Most commonly, they want more than 10x what they gave you. You need to move fast and break things[6]. And this is what we do. 

‍

Let me walk you through the process. My cofounder and I sit down, look at analytics and start making educated guesses on what the user might be doing or needing, generate a few hypotheses and go ask the user. This is very important, keeping the user in the loop before breaking or changing anything. We have several close Design Partners with which we bounce ideas with and make efforts to get to as many users as we can. We take notes on each of the conversations, most times calls but anything from a few messages helps. 

‍

Now armed with both qualitative and quantitative information it’s time to create solutions. The breaking changes we do now are nowhere near. We first try to create small changes that generate value. There are so many frameworks for picking what to work on that we decided to simplify: tackle from our side and see what sticks. I call this spray and pray. Now you might be thinking that we intentionally break things on each change but you’d be mostly wrong. We keep a tight metrics system that gives us a heads up before releasing to the public. We test everything internally, where breaking changes are almost welcome. 

As the team codes we make sure everything works, we help teams push better code to production so we should be the first to be doing that. But code is not where breaking changes are, we are trying a lot of ideas, so we change stuff, and we are a small team so we break parts. This is a feature, not a bug. 

‍

While breaking changes sound scary, we have embraced as part of the company, most of our experiments have been a total failure, some have been meh but the few that were a success are the ones that made us grow. We will continue failing and dropping breaking changes in production.

‍

Some examples of this are when we dropped the Discord integration after getting a couple dozen users onboarded, simply there was no API[7] for search and we couldn’t offer any value. Or when we ripped the commenting features we worked so hard on because no one used it. Or right now, when we announce no support for the IDEs[8]  the day we launch on JetBrains products (sorry intelliJ and PyCharm people!). And the one that generated the most fights inside the team: developing a billing system on two different occasions and then moving it to be a calendar link. But all of this stops aching after a couple weeks. It's an improvement, as I often say “less code is better code to maintain”. 

‍

Breaking changes are a bet on the future. Breaking changes are a step in the right direction.

Now, as a responsible CTO, it is my responsibility to tell you how to prevent breaking changes. A lot happens because there is no historical memory of how something worked or something was overlooked. Although breaking changes are part of semver[9], you try to avoid them. Well, this will sound like an infomercial but Watermelon was built for this.We help developers connect the dots on distant docs, old conversations and stale PRs and give you an AI summary before you sift through all that information. We also, recently, started reading your code to give you improvement recommendations starting with preventing console logs from hitting production. And my personal favorite, we give you hints if a PR is release worthy by comparing the description and title against the changes sent.

‍

[1]Chief Technology Officer
[2]Site Reliability Engineer
[3]Pull Request
[4]Artificial Intelligence
[5]Venture Capital
[6]Internal motto used by Facebook until 2014, as coined by Mark Zuckerberg
[7]Application Programming Interface
[8]Integrated Development Environments