How One Check Box Lowered Conversions by 17%

This post was originally published on 12/7/2010.

At Skritter, we've been experimenting with email campaigns as a marketing tool. Specifically, we wanted to see if opt-in email marketing to customers that were trying the software was an effective way to boost engagement and sales. To do this, we set up four experimental groups and began sending out some short "how to use Skritter" emails (more on the results of that effort in another blog post). In the course of testing the emails, we needed to randomly show a check box field on our signup page that allowed customers to opt in/out of the emails. Initially we used the A/B test purely for the sake of helping us with the experimental design of the test, but what we found was interesting in its own right.

Two Designs

Here are the two signup page designs:

The control design.

The variant design.

The Result

I'll save you the effort of comparing the two: the only difference is the check box above the "Sign up" button. Surprisingly, adding that field to receive emails was surprisingly effective at deterring signups. After only a short period of time we were able to determine with 94% confidence that the variant was performing worse, to the tune of a 17% decline.

We hadn't expected that tiny email option to have such a large impact on sign ups! Lesson learned: never, under any circumstances, modify the appearance or function of a mission-critical part of your website without testing. That simple test helped us avoid a 17% decline in sign ups every month.

How to Overcome Your Fear of Hustling

The Horror

You know the feeling well: you've decided to attend a networking Meetup, you get to the location and only see one other car in the parking lot. You go into the building and your stomach sinks as you enter the appointed room. There's only one other person there, and it looks like the organizer. You quickly check your phone: it's 20 minutes past the starting time. How embarrassing. You're the only schmuck who turned up.  There  are unopened drinks on the table next to uneaten food. You look desperately around:  this poor organizer looks ready to talk to anyone and you see the next 2 hours of your life vaporizing in a mist of uncomfortable conversation as you try to politely leave.

You're Doing it Wrong

Most people I know would be uncomfortable with the situation I described above. In fact, I would have been uncomfortable with it until very recently, when I realized how much it was holding me back. I don't like being witness to, much less a part of a failing enterprise as much as the next person, but if you are going to start something new, you better get used to that feeling and stop letting it affect you.

Almost every new venture starts with discomfort. It's easy to forget that when you are starting a company because coding a site allows people to hide from that sort of discomfort for far too long. I would wager that if every web startup required the founders to knock on a 1,000 doors and pitch their product to the resident inside before being allowed to write code, very few businesses would get started, but those that did would be 1,000 times better off.

I'm talking about being willing to physically put yourself in socially uncomfortable situations to push your vision forward. I'm talking about hard sales, cold calls, meet and greets, and asking people for favors. I'm talking about being willing to politely ignore the unwritten rules of social decorum and get down to business. I think the more you are able to do that in your life, the more successful you will be at achieving your goals.

How to Do it Right

It's all well and good to talk about being brassy and hustling for your vision, but how do you go from being a shy, socially-hypersensitive person (like I used to be) to a self-determined success? Here are two tricks that I use:

1) Prediction Boxing

When you are faced with something uncomfortable, you will find yourself making excuses to avoid it. What you are doing is trying to avoid encountering the worst case scenario. Let's say you are going to cold call a potential customer. You have been putting it off for a week, but why?

There's a quick mental exercise I do which helps me put possible outcomes into perspective. It works like this: think about what you are going to do, and generate a scenario in which everything turns out impossibly well, and I mean push this to the extreme. To use the cold calling example, the extreme positive situation is that your customer not only buys what you are selling, but they 100 copies of it and refers 100 of their corporate friends. Bam, your company is now a success.

Now, think about the absolute worst case, and I mean the most god-awful worst way this could turn out. This time, your customer screams curses at you for the unsolicited sales call, tells all their corporate friends you're an idiot, and adds you number to their block list. Okay, that would suck pretty hard.

Finally, think about a situation in the middle. Maybe you get a curt hangup, or a polite denial. Maybe they tentatively agree to try your software for free with no obligation, maybe they try to get you off the phone by making up an excuse not to talk with you.

I have found time and time again that the middle situation is the most likely, and so long as it doesn't contain any truly horrible results, you should probably do what you are contemplating. Remember: think of the best, then the worst, then the middle situation, and if the middle doesn't seem bad, go for it.

2) Pre-commitment

This is a motivational hack that one of my good friends shared with me, and has proven effective time and time again: tell people what you are going to do, and tell them when you are going to do it. You will get the most out of this if you tell people you respect, and since we are talking about intimidating challenges in a business context, I would recommend your co-founders. Here's how I was able to push myself out of my comfort zone repeatedly to get stuff done:

Whenever my co-founders and I would come up with a marketing or sales suggestion that was uncomfortable, I would volunteer to do it regardless of my comfort level with the task and set a concrete deadline. This part is easy: it makes you appear valuable and important to your cofounders, and that feels good. The hard part comes later and you can build up the courage when that time comes. The most important thing is that you tell them exactly what you are going to do "I will cold-call 5 of our potential customers" and give it a deadline "and I will call them by this coming Monday."

By making a formal, concrete, public commitment, you have raised the stakes of failure, and made it that much harder to back out. Now if you fail, you can't hide: the people you respect most will know. You've put yourself in the position of having to succeed. Make no mistake, you don't end up hustling accidentally. I recently saw a talk by a guy talking about customer development in which he said that he would never have done it if the alternative hadn't been firing his best friends. One cold call may not push you to that extreme, but it's a good indicator of how difficult it is to make a habit of hustling.

If you're interested in learning more about the tips and tricks I use, you should read my cofounder's book: The Motivation Hacker. 

Conclusion

Starting something new is scary because failure is scary. You maximize your chances of success by improving your hustle and overcoming that fear. To use the example at the beginning of this post, there is no reason to be scared of an empty room. Worst case scenario: you lose a bit of time, learn what doesn't work, pack up early and try again.

Quit Your Job Now, Before It's Too Late

This was originally posted on 4/7/2013.

Six weeks ago I handed my employer my formal resignation. Two weeks ago I waved goodbye and left the office to begin work on my second startup.

At this stage in my life I have no children, no mortgage, a small passive income from my first startup, two rock solid technical cofounders, a working prototype for our idea, a monetization strategy, and enough cash saved up to bankroll some small business expenses. In short, there was very little risk to quitting. Yet I still struggled with the decision and on that Thursday as I walked into the chilly North Carolina air, I couldn't help but wonder if I was making the right decision.

Quitting is hard for social reasons. I worked at a small Django consultancy with an excellent group of folks who I respect and admire. But even there, among independent-minded, intelligent, professional, driven peers, most were confused why I would want to quit an excellent job. One coworker was frustrated that I was helping perpetuate the Silicon Valley brain drain, "don't move out there and be one of those guys," he told me during my last week. But by far the most common reaction to my news was "why?" accompanied by what I perceived to be quiet pity. "Oh dear, he thinks he's going to start the next Facebook."

To everyone yearning to quit their job and pursue their startup dreams, do your homework, then quit now. I'm not sure I would be capable of quitting my job with $100k in a home loan, 2 children, and a car payment. Quitting gets more difficult the longer you wait, so brace yourself and jump soon.

Know Thy Platform

This post was originally published on 5/5/2011.

Four months ago my two co-founders and I went to visit one of our investors for a quarterly check-in. Our investor runs an iOS development company from a renovated turn-of-the-century office atop a 28 story building. It would be an understatement to call his experience and surroundings impressive.

While sitting in front of a 9 foot window that looked out on most of downtown, we were discussing plans for our upcoming iOS app. Specifically, we were discussing how many features should be included in version 1. We had just started to translate our codebase to Objective C, and we were trying to define the scope of the project. Our development approach to date had been to build a quick and dirty v1 and use it to gauge user interest. The process had served us very well.

Like an all-knowing Buddha, our investor grabbed a nearby white board and drew a graph similar to the one at the top of this post.

"What does this graph tell you about the features you need in version 1?" he asked, pushing the graph towards us.

We all three thought for a moment. None of us liked the implications, but Nick eventually replied "I guess we can't build a minimum viable product then."

Our investor just nodded and gave us a smirk that seemed to say "sucks, doesn't it?"

What he knew was that the window of opportunity for a hot app on the App Store was measured in days, and especially for the kind of app we were building, a Hollywood Launch was the way to go.

When we actually starting building the app, we had our product scope waiting: it needed to be full-featured and polished. Prior to that meeting, we didn't know anything about the App Store as a platform. What our investor let us know was that building an MVP iPhone app would have been as stupid and useless for my co-founders and I as trying to iterate our way through FDA drug approvals for a new medicine.

Knowing your platform makes you a more effective entrepreneur. It means you have a better grasp of how your product will be shared and used. This means you can cut out unproductive iterations and get to pay dirt faster. Whether your entrepreneurial ambitions are to make enterprise software or fluffy mobile apps, qualified knowledge of your platform will significantly reduce the number of times you have to pivot strategies. And in some cases (like ours), platform knowledge can save the project completely. Had we made an iOS app that wasn't polished on day one, we would have thrown away our best chance to test the market and make money on our creation.

Even going between two similar platforms (web apps to mobile versions of web apps) requires entrepreneurs to have a firm grasp of the differences of the platforms. Most of the time you won't have a kindly investor at your shoulder to reiterate this point, which means you have to know thy platform without being told. Sucks, doesn't it?

You Can't Use Customer Discovery for B2C Products

This post is a simple reminder to everyone out there starting a B2C web startup: it's risky and no framework or theory is going to change that.

Steve Blank’s Customer Discovery framework has helped an entire generation of entrepreneurs focus on the most important aspect of their business from day one: finding paying customers. But Steve Blank focuses on B2B products, and even more specifically, B2B enterprise products. These are products for which the number of potential customers is relatively small and each account is relatively large.

But what if you are building a B2C product? This business model aims to build something that a large number of customers will use, and any one customers will have a tiny impact on your bottom line. I’m talking about consumer web products, Netflix, Prezzi, and Instagram all fall into this category.

If that is what you are trying to build, the Customer Development process really only has three steps:

1. Build a minimum viable product (MVP).
2. Show it to people and see if they use it.
3. Rinse, repeat, or pivot.

The only input the Customer Discovery process has for this type of product is to build as little as possible in step #1. But this is hardly new information. Building a minimum viable product has become de facto industry advice for entrepreneurs and was good advice before the 4 Steps to Epiphany was published.

The takeaway here isn’t that Steve Blank’s advice is flawed or that the Customer Discovery process is useless. The takeaway is that unlike B2B startups, there are no straight-forward steps you can take to validate your B2C startup idea quickly without writing code.

So if you are starting a B2C web product, remember there’s no magic bullet when getting started. The best thing you can do is just start building a stripped down minimum viable product and get people using it. Which is essentially what it’s always meant to build B2C consumer web products. The best thing you can do is get started!