Surprised to find bugs?

Code Collaborator just got a great review by game AI specialist Paul Tozour.

My favorite excerpt:

The ostensible reason for this process was to get more engineers familiar with different parts of the codebase that they wouldn't have had any exposure to otherwise.

What we found was far more surprising -- we uncovered an amazing number of bugs before they ever reached QA.

Furthermore, the bugs were coming from everywhere, not just the junior engineers. All the engineers, including all of the most senior developers, had plenty of room for improvement.

This seems to be a common theme among groups implementing a reasonable code review process for the first time. ("Reasonable" means not encumbered by heavyweight process.) We're always surprised that our code wasn't as good as we first thought.

Shouldn't it be obvious? Have you ever written a page of prose that couldn't be improved? That you yourself couldn't improve when you look back a month later, but that you friend could have found with a two-minute scan?

Writing code isn't different, nor should we expect it to be.

By the way, once you get used to having the safety net of fellow programmers catching all your little mistakes, it's scary to go back to isolation. You know the bugs are there!

Boiling waste and goo

In his review of Code Collaborator for the Jolt Award, Rick Wayne wrote the best description of our tool I've ever seen:

Smart Bear's Code Collaborator exploits that synergy by boiling the waste and goo out of the code-review process, bagging you the benefits of many eyes on each developer's code while neatly sidestepping much of the practice's traditional friction.

No hats!

"Oh, you have a small company? I guess you wear a lot of hats!"

Ug. I've always hated that phrase. Recently I figured out why.

To me it's always came off as a smug comment implying the speaker knows just what it's like and understands your pain. It's usually followed up with something like "I was employee #7 once and I was VP development and VP QA!"

Yeah. How can people live like that? Unfathomable.

Real entrepreneurs don't talk about hats, but why not? Why do I recoil at this attempt at empathy?

I think it's because in real (bootstrapped) start-ups there's no such thing as hats, titles, or departments. If you don't realize that development, marketing, sales, and even accounting are all inextricably linked, you'll probably fail. You can't make decisions about any one in isolation. You're learning how to sell, what to build, what to charge, who will buy, all at the same time. You're never even thinking in categories like "Marketing" and "Development."

So I think my disdain comes from the implication that "many hats" proves the speaker "knows just what it's like," when in fact the very idea shows you have no idea what it's like.

People like that will always be employee #7, never employee #1. (Or really employee #0, since everyone I know who has created a successful bootstrapped company paid themselves last, long after other people were hired on, long after profitability.)

Which is fine! But the experience gap is bigger than they think.

We're on the same page

Advertisements are typically designed to stand on their own. Expensive ad firms frame comps on black construction paper, focusing attention and emphasizing isolation.

But how would your advertisement stand up if it were displayed side-by-side with your competitor's ads?

This happens a lot actually. Your Google Ads are likely to come up next to competitor's. Product listings in catalogs and websites usually group products with their competitors. Magazine ads are often juxtaposed. Potential customers may even print out your website and physically place it next to your competitor's on their desk. In color.

How does comparative advertising affect your message?

In the age of user-driven information flow, you can assume that a person looking at your website is in the market for a product like yours.Therefore expending effort justifying your market space in general might be a waste of those precious few seconds when a person visits your website for the first time. Besides, extolling the virtues of your market validates your competitors as much as yourself.

Instead, what features (and benefits) does your product specially have to offer? What are your strongest claims, the ones your competitors have trouble competing against? If you're the fastest, explain how speed enables new features that would otherwise be impractically slow. If you're the most customizable, emphasize how your tool can support a process rather than dictating the process.

That's not to say you shouldn't address general market advantages at all. In fact, it's often possible to simultaneously fold general benefits into your specific ones.

For example, which of these ads is more compelling:

1. Relieves painful migraine headaches.
2. Relieves migraines faster than anyone else.

The second ad conveys both the general benefit ("relieves migraines") and the competitive advantage ("fastest").

This also points out that communicating the general market benefit is often unnecessary because the end user already gets it. If I have migraines, I already understand my pain and I understand the benefits of pain relievers. You don't have to tell me migraines are "painful," just get to the point!

As another example, this time from Smart Bear, we claim that one way Code Collaborator saves time with code review is that we show chat next to code, and the chat sticks with the code even when you upload fixes (where the line numbers shift around). On one hand, we're describing a general feature (chat in context with code) shared by all 3 of our major competitors, but at the same time we point out features that only we have (i.e. two competitors show chat in a separate window, not with the code, and no competitor is good enough to keep chat in the right place even after files are updated).

If you want to stick out from the crowd, explain why you're the best medicine, not why I need to take medicine.

P.S. Exception: If you're pioneering a new market space, this idea doesn't apply. In that case you need to explain and defend your very existence; in fact this was the case at Smart Bear for the first 5 years.

Limiting Options

The 1990's was the golden age of computer AI's for the board game Othello. OK, it was pretty much the only age.

Programmers love the thought of computers beating humans at "intellectual games." For me it's the "mad inventor" idea of creating something more intelligent than myself (whatever that means).

At the time, Checkers had been solved and even Chess was close. It was Othello's turn to fall.

What's interesting is how the winning strategy worked.

The typical computer board game strategy is to look many moves ahead and rate each resulting board position. Then you pick the next move that maximizes the ultimate board position you can achieve. The trick is in rating the "goodness" of a particular board position.

In Othello you win by having more tiles of your color than your opponent once there are no more possible moves. Therefore, typical metrics for Othello included things like "How many tiles of my own color do I have" (more is better), and "Tiles of my color at the edge of the board are more valuable than in the middle" (the edge is a better strategic position).

What's neat is that the winning strategy used a completely different "goodness" metric. Specifically the metric was: How many valid moves does my opponent have? Fewer is better.

The flip was that it's not primarily about how many tiles you have or even the positions of your pieces. Instead it's about limiting how many choices your opponent has. Limiting choice is more important than what the choices are.

This principle is common in defensive theories of sports. The defense can never cover all contingencies so instead it forces the offense into higher-risk, lower-percentage moves. In basketball you can't simultaneously cover the long shot and the charge, so you elect to give up the low-percentage three-point shot. In football you can't cover both in-routes and out-routes, so you force a throw into as much traffic as possible.

In business your competitors always have options. How can you limit their choices to things that are low-percentage or expensive? Make them have to spend more money, get more lucky, or be more creative.

An example is honoring a competitor's coupons. This eliminates the coupon as a way for your competitor to "beat" you; coupons are no longer a "choice" to develop a competitive edge.

You can think of patents as a way of limiting choice. If no one else can use your method, they'll have to think of another way. That means research time and money, and maybe it won't be as good or it will be more expensive to manufacture or support. (That is, if you believe in patents...)

We've done a few things at Smart Bear to limit options. For example, we're so strongly established as "the experts in code review," it would take an expensive, time-consuming effort for someone else to claim par, much less surpass what we've done. We literally wrote the book, we did the largest-ever study on code review, we have years of history, we have the most popular tool. It's possible, but very hard, for someone else to compete on that particular basis.

This matters because in my opinion, being "the expert" is the best qualifier for our particular market. Our customers would be less interested in "lowest cost" or "simplest installation" (even though our installation is simple). If my opinion is true, I've removed the single best choice from competition, and they'll have to find a less desirable, lower-percentage path.

Agile marketing interview at GeekAustin.org

I just got interviewed about "agile marketing" over at GeekAustin.org. It was fun to show how we apply principles from agile software development to business and marketing efforts.

If you're involved software or marketing in Austin, you should meet Lynn Bender, the founder and organizer at GeekAustin. Maybe come to the Agile Happy Hour in two weeks! They're always a blast.

The Anti-Bullet Test

I'm sick of generic feature/benefit bullet points. They're just too easy to make fun of. Here's a sampling from a website that will remain nameless to protect the guilty:

• Easy to use
• Robust features
• Innovative systems

Really, it's easy to use? As opposed to what, difficult and temperamental? Robust, huh? Great, because it looks tenuously held together, the slightest breeze threatening to crumble its delicate construction, so it's good to know that, actually, it's robust. Oh I'm sorry, its features are robust, whatever that means.

If you want to not stand out from the crowd, use generic statements. The ones everyone claim. Would anyone claim to be non-innovative?

So here's Jason's Anti-Bullet Test: For each feature/benefit bullet point, construct its negative and see if that statement is ridiculous. If it is, it means the bullet is obvious, or at least unoriginal. It's not strong, it isn't adding excitement, it's not differentiating you from your competition.

Let's apply the test. The negative of "Easy to use" is "Difficult to use." That would be a pretty funny statement! No one would ever claim that, so throw it out.

The negative of "Enables communication" is "Blocks communication." Crazy; no one would admit their tool does that.

The negative of "Stores files as big as 100 gigabytes" is "Cannot handle very large files." Not ridiculous, in fact this is sadly true of most computer systems of any kind. It passes the test.

Here's a good one from our own product: "Integrates with seven version control tools." Negative: "Not integrated with seven version control tools." Not particularly funny; in fact this statement is true of all our competitors. So this statement differentiates ourselves in a specific way.

If you're using generic bullets now, you'll find that replacing them isn't easy! You have to really think about what's strongest about your product, about how specifically it beats the alternatives, and how make it pithy. This is a useful exercise in itself.

One exception to the Anti-Bullet Test: You can use a generic if it's your single biggest differentiator, and if you're willing to put lots of energy behind it.

A good example here is "Fastest." The negative is funny ("Slow operation means lots of time staring into stagnant progress bars"). But if you make it your highest priority, it can work. Make your bi-line "The fastest ____." Prove it with benchmarks. Explain how speed is not only about saving operator time (the obvious benefit) but how it enables entirely new features. For example, perhaps operation X is typically so slow that people can't take advantage of it. But since your system can complete operation X in two seconds, suddenly it becomes a feature. Even if a competitor technically has the feature, you make it practical.

All this is just another way of saying: Be specific, be fully committed, and tell the truth.