Jolt Finalist: Code Collaborator

Code Collaborator has just been announced as one of six Jolt Award finalists in the category of "Collaboration Tools."

The competition is varied -- everything from wiki's to project management to collaborative development (us!). Hopefully, since this is a programming award, collaborative software development will be more relevant than general wiki's, but we won't know until March 5.

Not in the cards

Jeff Atwood recently posted an amusing treatise about shuffling cards demonstrating how sometimes the simplest algorithm isn't the right one.

Remember those open-ended interview questions that make you think deeply about a problem? Sometimes you actually need to do that in your job; it's not just an interview trick!

Card shuffling made me think of a similar, more common, and more complex problem of the "Shuffle Playlist" or "Random Slideshow."

(In the 90's we used to joke that "Every application grows until it becomes its own operating system." During the bubble we used to joke that "Every application grows until it is capable of sending and receiving email." Today I think the joke should be: "Every physical device grows until it is capable of playing mp3's or displaying pictures at random.")

The goal of the shuffled playlist isn't the same as shuffling cards. With cards it's sufficient to have a "randomized deck," meaning that every possible ordering of cards has an equal likelihood of being output by the shuffle algorithm.

The reason playlists and slideshows are more complex is: The goal is to make a human feel like the order is random, not to have it actually be random.

What's the difference? Imagine you have 5 songs. The algorithm for choosing the next song is to select one of the songs with equal probability and play it. Obviously this is completely random mathematically. Chi-squared will be happy.

But a human won't like it. It won't be uncommon (20% of the time) for the same song to be played twice in a row. Sometimes (1%) it will be played three times in a row! But that's not really what you want when you put songs on shuffle.

With an image slideshow it's worse. You're expecting images to change every 5 seconds (say), and sometimes it seems to get "stuck" (because the same image comes up twice in a row). Crappy slideshow player! Get a new one.

The usual fix for this is to take the entire playlist and randomize the order of the songs, rather than randomizing each selection of song. This ensures we have the perception of random distribution of songs. In fact it's not random at all by any mathematical definition. It's an even distribution, but not random one.

But that's OK, it's perception that counts. But we're not done! What happens when we go through the playlist once? You'd probably propose we just repeat the algorithm -- reshuffle the playlist and keep going another round -- but it's now still possible to repeat a song. The last song from the first shuffle might be the same as the first song of the second shuffle. A repeat!

In fact, the perception is probably marred even if any songs match near the "crease" between two shuffles. For example, if the last song of the first shuffle matches the second song of the second shuffle, that's probably still too close to "sound random." "I just heard this song!" exclaims the disgruntled, indignant user. And rightly so you bastards!

So now what? With larger playlists, you could start with a full shuffle, then take the last N (or N%?) of the first shuffle and check the first N of the second, and if there are overlaps you could swap songs from that initial "unsafe" region into the "safe" region, at random, being careful of course to swap only with songs allowed in the unsafe region.

"Wait," you cry, after having digested Jeff's excellent reasoning for why selective swapping is clearly for dummies, "that's not random!" No, it's not, and no one cares, because it's doing the Right Thing.

OK, so this swapping idea isn't great -- you can't do my version in place (because you have to remember the unsafe songs) and it's complex. Here's a simple, in-place algorithm:

1. S == number of songs, N == number of "unsafe" songs, meaning the last N of the first shuffle must not match any of the first N of the second shuffle.
2. Starting with the first shuffle, shuffle songs 0 through S-N. (This ensures that, at least, the first N songs are evenly selected from the set of songs allowed to be there.)
3. Now shuffle songs N through S. (This ensures that the unsafe songs are evenly distributed in the safe area.)

The keen eye will notice that if N > S/2, this algorithm fails. Indeed, you cannot satisfy the constraints in that case no matter what the algorithm because there's not enough "safe" songs to fill the "unsafe" region. As a simple example, if N=9 and S=10, we have only one song that is safe and nine unsafe slots to fill!

It's even worse for small playlists. With two items, the best you can do is alternate. With three, I would argue the best technique is to just keep any back-to-back's from happening -- anything more strict and you end up with zero variation, which doesn't feel like a "shuffle."

And don't think I'm being pedantic in calling out the cases of 2 and 3! That electronic picture frame on Grandma's sideboard might very well be displaying just 2 pictures, especially when you're first setting it up for her. First impressions are important!

So what's the lesson? In most applications, human perception is more important than mathematical correctness. And sometimes it's worth digging into your interviewee skills to cover a problem as deeply as you can.

In this case, a user unsatisfied with "shuffle" might not buy, might return the item, or might call tech support. A little thought and common sense could save many thousands of dollars.

Oh yeah, that's why great developers are worth a lot of money....

Choose your words

The headline in the Austin American Statesman:

Bush says Iran Still a Threat

The headline in the New York Times:

Bush Insists Iran Remains a Threat Despite Arms Data

Set aside political debate. Just consider how each of these headlines predisposes the reader while purporting to summarize the same event.

Marketing is about persuasion, hopefully more so than news outlets (!). How can you predispose your readers? How can you load the debate before it begins?

Every word in your headline counts. Use it to frame the debate before you potential customer even sees the competition.

I Will Survive

We make a peer code review tool for $400/person. What would happen if, today, Google released a peer code review tool? For free. And it's technically good, because it's Google. And because Google has tremendous visibility, everyone who's at all interested in code review goes out and tries it out. And it's open-source, so a community will be built around it and will add cool new features and plug-ins.

For free. From Google.

That's it for Smart Bear, right? No way we can keep charging for Code Collaborator, right? At least not at these prices.

I don't think so. Subversion is awesome and free and has a community, but Perforce charges $850/seat and their customers love them. Bugzilla is awesome and free and has a community, but JIRA, OnTime, Fogbugz, and TeamTrack have thousands of customers and continue to grow.

So how do you compete? You have to realize that it's not about selling software, it's about solving pain. No one sits around saying "What we need around here is another software tool!" No, even software developers themselves hate the idea of installing and learning a new application.

Here’s a hint: Imagine you have 100 developers under you and are charged with reducing bugs released to QA by 50%. Sounds fairly impossible (although our customers have seen these kinds of numbers). But how the hell do you do that without pissing off the developers or wasting time?

Your biggest concern isn’t whether tool A or B has the best diff viewer. It’s probably not even whether your initial tool cost is $0 or $30k. That’s nothing compared to the time, effort, training, and risk you’re taking on by doing any kind of peer review process. Not to mention your own reputation, and possibly your job.

The main blockade to code review isn't the tool -- it's the social change, the process change, the idea that someone else will be criticizing your code, the fear that pointy-haired bosses will be able to measure your "productivity" in terms of lines-of-code-changed and defect-density.

At Smart Bear we are experts in constructing practical code review, part of which involves a tool. All you can expect from a tool is to remove some drudgery -- integration with other systems means fewer mouse-clicks, organizing conversations and displaying them visually with the code is useful, collecting time-spent-in-review automatically so managers get their metrics but developers don't have to mess with stop-watches.

But if you're not good at reviewing code, it's still all a waste of time. Maybe you're bad at reviewing certain types of code? Maybe you'd get the most bang for your buck by reviewing just the code modules (where a little bug is a big deal) and not in GUI code under active development. Maybe some of these metrics are not actually correlated with anything, so you'll be chasing ghosts and making incorrect decisions if you look at them.

These are things everyone also worries about, and just installing a tool isn't going to fix it. But we can.

We give you a 164-page book for free that helps you make the right choices. There are articles and white papers on-line, for free.

And most importantly, you can call and we’ll talk about anything you want, for as long as you want. For free. That’s something no on-line forum or answer-by-Google-search can match.

So in light of all this, as the manager, will you go with the free Google tool with no assistance, no help, or do you want the direct phone line to the guy who literally wrote the book on how to construct truly effective code review processes?

How to be the most expensive product on the market

Seth posted a short, great article about how to completely win your space. It totally transformed the way I think about marketing. It's just a few paragraphs -- go read it and meet me back here.

A shallow interpretation of Seth’s point is that "service makes the difference," but that just leads to the usual phrases that have been repeated so many times they’ve lost meaning. "We put customers first." "Except the best, both before and after the sale." "We believe our role is to serve our customers." "Our customers are buying a solution, not a technology."

Blah blah. Everyone says that. What does it really mean?

Let's take that last vapid statement and make it concrete. It's true that no one wakes up in the morning and says, "Gee, I wish I could purchase, install, learn, and train my people on a new software tool!" Tools suck – they're confusing, they rarely do exactly what you want, and tech support is often a nightmare.

In our case, our customers have one of several specific problems that they'd like to make go away. Perhaps they spend 20% of their time in code review on boring, wasteful chores like collecting and sending diffs around or scheduling meetings. If it's something this specific, perhaps the tool can sell itself –- if it clearly removes the drudgery and during the trial the developers feel it saves more time than it wastes with its own idiosyncrasies, then it's worth it.

But usually our customers are in a more difficult spot. They need to reduce bugs or work with offshore teams, so they know code review is necessary. But how to do it?

This question -– how to do it -– is what keeps them up at night. They know they could easily blow 30% of their developers' time on this process. What if it doesn't result in reducing bugs? What if their developers hate it so much they revolt? How can they tell whether it's really working?

In fact, most people rightly believe the hardest thing about code review isn't the process, isn't metrics, isn’t reports, isn’t communication –- it's about (a) how do we make sure we're not wasting time and (b) how do we deal with social effects of ego-full, sensitive geeks critiquing each others' work?

This is where your "customer service" comes in. At this point it's not about tech support, it's not about what features your tool has, and it’s not how good your salesguy is at "closing."

I often sit down with a customer for a few hours. I help them as them the right questions of themselves. I help them determine which code they'll get the most bang-for-the-buck on so they can see some immediate results. I give them stories and even presentations about how gratifying code review can actually be, how to foster an environment where code review is genuinely about mentoring, learning, and getting rid of bugs together.

No tool will do that, and I also think no on-line form, Wikipedia page, or case study will do that either. Yes, our tool is important in removing drudgery and making reports, but it’s a means to an end.

The interesting part is all this mentoring stuff we do completely for free. If you tried to objectively quantify the true value to our customers of this advice and direction next to the tool, the advice is probably more valuable.

Maybe that's why it's the reason we consistently win over all other tools in the market. Because we're the only ones that can provide that service, and we do it for free. It tangibly demonstrates that we really do want our customers to succeed, not just buy some seats.

And of course when they do succeed, why go purchase something else? The free advice alone is worth any price difference.

How to build a checklist

CM Crossroads just published my article on how to build code review checklists. Half of which is, how not to build one.

It's not about in-scope/out-scope

The software project is behind schedule. The deadline is looming and everything finally acknowledges -- too late -- that we have to start making concessions.

So there are two choices, right? Release later, or reduce scope by throwing out features.

Not so fast.

"Remove from Scope" is not the answer
Too often we developers are stuck on this "remove it from scope" mantra. Yes, completely throwing out features is a way to get to the release date, but this isn't necessarily the best thing for the product or the customers.

Ultimately our job as engineers and not just worker-bees is to make trade-offs. Not just in design or algorithms, but in the end user use-case and, yes, even to support marketing and sales with their promises and messages.

Not cutting, but changing
Five years ago I was on a project with an impossible deadline. We had to get a complete custom reporting system integrated into a project for their v3.0 release, and for uninteresting reasons we had just four months to do it. There was a spec that couldn't possibly be completed in time, but the deadline was firm because a customer contract was in place.

The project ended up a complete success. The system was demonstrated to the customer on time and was accepted. It was so popular, it became the "big finale" for product demos.

The reason it was a success is because we never posed the question: "What parts of the spec do you want to throw out?" Instead we would ask: "How can we allow the user to accomplish what the original spec wanted him to accomplish, but in a different, simpler way?"

Sure that means less "stuff" -- fewer absolute number of features and fewer choices. But it doesn't mean tossing items J-M -- it means reevaluating the entire spec from the point of view of the use-cases and marketing requirements.

Happy days
There's another benefit to this approach.

We're all used to the tension between development and sales. Sales wants more, faster. Requirements pushed from the outside are ambiguous, and when you deliver it's never the right thing.

The normal reaction from development is: "You're changing the rules, therefore we simply refuse to give in. Either stop changing the spec or change the deadline. It's a zero-sum game, and we won't be responsible if you make the sum larger."

But that's not helping sales OR the hapless customer. (Customers don't have hap, apparently.)

What helps is arriving at a better solution. The project drivers will be more willing to negotiate if you're going to meet them halfway. They know they've added to your schedule. If you show the genuine desire to get their root problems solved, they'll be more willing to bend on exactly how you're going to solve their problems.

Besides, we can all agree that "Customer Success" is the ultimate goal, and clearly negotiating for their success is better than cutting them off.