Or "Wait a second.. what is that burning smell!?"
Now that things have calmed down a bit with WhyGoSolo, I'd like to give a bit of my perspective…
Whenever you launch a site, there are going to be problems. Some are going to be tiny and only detectable by the handful of people actively working on it. Others are going to be so large that you briefly consider giving up technology and living in a tree. Regardless, they all need to be dealt with… so the questions become: How and when?
In my book, there are three types of issues and each requires a different response:
First, there are tweaks. These can be spelling errors, broken images, bad punctuation, or a variety of other things. These are not functional things at all. These probably aren't preventing people from using the site. They're probably not breaking things. More than anything, they just annoy your users (or your boss) and hurt the credibility of your site.
The best thing about these fixes is that you can apply them just about any time without issue. You don't have to worry about the database or cached data or even users currently on the site. You can deploy the changes as often as you want with few problems.
Second, there are errors. These can be a variety of things… ranging from unexpected behavior to working exactly as expected but in a wrong way. These tend to discourage people from using the site. When your users see these, they say "wtf" or "why did it do that?" For the record, those are both bad. Odds are that the features are either getting/making bad data and blowing up or it's working exactly as designed and it's just wrong. The second one is much worse.
These are a bit more problematic… Most likely you need to both apply the fix and scrub any existing data or user sessions. The worst part is that users currently in the system can get into an unstable state and you can't being to predict what will happen to them. These fixes have to be scheduled and handled carefully… preferably while no one is using the site. Odds are that you only get one shot at this a day or even once a week.
Finally, there are showstoppers. These are huge. These are the ones that make your phone ring at 2am and put people's jobs at risk. It can be something simple like bad css that makes a page unusable or it can be something nasty like a security hole that lets anyone own your site. Both are bad, but there are magnitudes of difference between the two.
These are huge and problematic and not something to play arond with… These have to be applied as soon as possible because bad things are happening. Luckily, there's an upside. Since the site is already broken, you have a perfect opportunity to fix it. 😉
For WhyGoSolo, it's been an interesting mix of the above…
We've had literally dozens of tweaks and we have quite a few more. Some we've managed to roll out, some are done but queued for later, others are later on the todo list, and some we've just dropped.
The errors… we've had quite a few of those. We've closed a handful each day and tend to deploy and validate before we send out a blast of invites.
Finally, the showstoppers… we've had two. Both were resolved shortly after discovery, but they still hurt.
If you run into any issues, let us know – email@example.com – and we'll figure it out. All feedback is appreciated.
This post was originally posted on the WhyGoSolo blog and is crossposted here.