Archive for the ‘Uncategorized’ Category

The database market wakes up from hibernation?

For most of the last 20 years that I’ve worked in databases, the industry has been pretty boring. Proof? It has been 10 years since Gardner group bothered to update a magic quadrant for operational/OLTP databases. You would think a $20B+ market would be worth some analysis, but not much has been going on.

20 Years ago your options used to be Oracle, IBM, Informix, and Sybase. Then Infomix hit financial troubles and was scooped up by IBM. Sybase did a deal with Microsoft that didn’t quite work out as planned and, after diversifying their remaining business, eventually was bought by SAP. 20 years later, your options were Oracle, IBM, and Microsoft.

Things have now gotten interesting. Looking at a ranking of database popularity, MongoDB is ahead of Sybase. Rewinding a bit further in database history to Ingres (ranked #30, and the last major RDBMS to fall away before my story started 20 years ago), some of the systems that have become more popular than Ingres include Cassandra, HBase, CouchDB, Netezza, Riak, Vertica, Neo4j, Couchbase, SAP HANA, and Greenplum (in that order).

The good news is that the market is waking up. All these new entrants are have created innovation. Customers are thinking differently and asking questions. And analysts are answering the questions: Gartner will do another magic quadrant.

Two interesting questions for the audience:

1. Will they call actually call it OLTP or will they describe it, more accurately in my view, as “Operational” database? I think there is still a split between operational and analytic workloads, but I don’t think OLTP is the correct way to describe many important operational workloads. I am optimistic that they will broaden the focus.

2. Any predictions on who is included and who is not? How many “traditional” databases vs NoSQL databases? If I were making the MQ, I would include:


Oracle (Oracle RDBMS and MySQL), IBM (DB2 and Informix), MSFT, SAP (both Sybase and HANA), Postgres


MongoDB, Cassandra, Couchbase, HBase, maybe Neo4j and Riak


MarkLogic (XML) and Intersystems Cache (Object)

For a total of 16 databases, half of which are relational. For open source technologies I believe Gartner will look at vendors and vendors with multiple products will only get one entry, so my actual prediction is 5-6 relational vendors and 6-7 non-relational.

I won’t try to predict who winds up where, but I’d love to hear opinions on who will and won’t be included and where folks will end up.

— Max

Sales compensation and lottery tickets.

Sales compensation is often a controversial topic. Big checks invite jealousy and second guessing. The best way I’ve found to calm down the jealousy and second guessing is to encourage the team to view sales compensation as being a lot like lottery tickets. Let me explain.

When you pick the right numbers for powerball, you might win $80 million. Do you “deserve” $80 million? It depends entirely on what you mean by deserve. Did you provide that much value to society in any direct tangible way? No. But if the winner doesn’t get paid, nobody would play the lottery. And the rules say you get paid, so at least in that sense you “deserve” to get paid.

I remember a story of an old time texas gambler who went to an illegal poker game. Something set off his danger sense so before going into the room, he asked the security guy at the door a question: “If I were to win a lot of money in this game, would I be able to leave with it?” He didn’t get the answer he wanted, so he didn’t play. Good decision.

I view sales compensation a bit like playing the lottery. You have to be able to count on being paid when you get lucky, or the whole system breaks down. When a sales rep earns a big commission, I pay it. I don’t think about how much they really deserve it. They may have gotten lucky by having the right account at the right time with the right supporting team, but somehow they showed up with the winning numbers. Occasionally someone who actively screwed up a deal gets lucky and closes it anyway. And they get paid. They may get fired if they screwed up badly enough or often enough, but even then they get paid.

I hear a lot about companies that find ways to not pay their sales reps. Almost every sales compensation plan has an escape valve where the company can decide not to pay. Some companies use it liberally. I know not because I do it but because I hear about it – in interviews of top performing reps whose employers found a way not to pay them. Once you’ve hit the winning lottery numbers and not gotten paid, it makes it hard to buy another ticket for that lottery. I sleep easier at night knowing that none of my sales reps are interviewing because we didn’t pay them what they earned.

How about jealousy? I have talked to a lot of technical people who are jealous of the big check their sales colleague got when a deal closed. I grew up on the technical side and only took on responsibility for sales teams later in my career so I have great sympathy, having felt that jealousy myself. Having now lived on the other side, my response is in two parts.

First, I explain that the sales team is, metaphorically, paid in lottery tickets, and this rep just got a winner. I remind them that being paid that way, and held accountable in the way sales is, has some downside too. When the deal closes they’re a hero and make a lot of money whether they “deserve” it or not. When the deal doesn’t close their income takes a hit and their job is at risk whether they “deserve” it or not. It’s not fair to have the downside without the upside.

Second, I ask if they would like that job. I do believe that’s the truest test: if you wouldn’t switch places with someone, don’t complain about them being overpaid. Heck, when I was a college student making $9 an hour doing tape backups on a VAX at night I was jealous of Alaskan fishermen making $2000 a week, but not jealous enough to quit school and do that. Not only does this usually end the jealousy, but sometimes we discover a hidden sales talent – I’ve had some hugely positive experiences with technical people migrating into sales roles.

Finally, one corollary is that if a rep always gets paid on the deals they close whether they “deserve” it or not, then a rep shouldn’t get paid on deals that don’t close whether they “deserve” it or not. It may sound obvious that a rep shouldn’t get paid on a deal that doesn’t close, but from time to time I am asked. Of course there’s always a good reason: the customer would have bought if the product had shipped on time, the customer would have bought if we’d fixed their bug faster, they would have bought if the company had agreed to xyz terms, the deal would have been bigger if xyz, etc. Often they are right, but I don’t care. To be precise, I care but not as much as I care about something else. What’s more important is the process being objective, so that the sales rep who does close a big deal doesn’t have to worry about whether they’ll get paid. Because once subjectivity and whether they “deserve” to be paid enters the picture, the whole system falls apart.

— Max

Lao Tzu’s advice for Kobe Bryant and Silicon Valley

This year has so far been an epic fail for the Lakers. With the acquisition of Dwight Howard and Steve Nash, they were widely expected to challenge the Thunder and the Heat as one of the NBA’s elite teams. Instead, they are 11th of 15 teams in the western conference, with a losing record. Adding insult to injury, they are 8 games behind the Clippers – never mind the best team in the west, they aren’t even close to the best team playing at Staples Center. 

Lots of things have gone wrong. Dwight Howard’s recovery from back surgery has been slower than expected, and Steve Nash’s broken leg has kept him off the court most of the season. But even if new coach Mike D’Antoni’s style doesn’t fit Pau Gasol very well, Kobe Bryant has been having a spectacular year. 

In his 17th NBA season, Kobe Bryant has been leading the league in scoring, averaging 29.5 points per game. While others greats have declined at this point in their career, Kobe is performing at his usual high levels in assists, rebounds, and steals too. While it might look like he’s carrying the team on his shoulders, a deeper look indicates otherwise. 

ESPN’s Chris Broussard recently looked into Kobe’s performance (I’d post a link but its subscriber only) and how it impacts the Lakers. What he found was that the Lakers are 4 and 11 when Kobe takes 20 or more shots a game and 8 and 3 when he takes less than 20 shots a game. The difference is not accounted for by quality of opponents; in fact the opposition was tougher when Kobe shot less and the Lakers did better. 

This is not a case of Kobe taking bad shots and missing most of them. He is shooting 48%, which is good. But if by doing that he is depriving other players of their very high percentage shots, it can be bad for the team. Of course it can be hard to know what is cause and what is effect; is Kobe shooting more when his teammates are shooting worse or the other way around? There are other statistical arguments that Kobe is helping his team this season; most notably the Lakers seem to be playing a lot better with Kobe on the floor than off the floor. 

This brings to mind some advice from Lao Tzu:


Right now Kobe is in that middle state of leadership. He is right in the middle of everything his team is doing, and he is doing it well. Maybe doing a little less would be better?

I think about these thoughts on leadership in the context of Silicon Valley. We worship hero-leaders who personify their companies. Many of the leading companies of Silicon Valley have been built by strong-willed iconoclastic leaders. Certainly in my days at Oracle it would be hard to say Larry Ellison spoke little, and Oracle did well. 

Has the world changed in the 2500 years since Lao Tzu’s time? Does his advice apply to basketball or software? Certainly I think we could move a little bit in that direction from where we are today.

I know at 10gen there are so many talented people my biggest contributions will be in growing the team, making sure we have money to keep paying them, keeping them pointed in a common direction and creating an environment and culture where we get the most from them. The art lies in staying close enough to understand what the team needs be but not so close as to get in the way of the team leading itself.

As for Kobe, maybe he needs a quota. He’s out of the game after his 6th foul, or his 20th shot?

— Max

Wake up Zuck!

You know that you are responsible for the user experience. You can’t possibly believe that users paying $7.00 to advertise to their friends makes people feel good about facebook. For that matter you can’t possibly believe it will make much money either. Dumb idea, just kill it.

I know its been a hard year. Somehow being worth $50 billion has become a failure – crazy world. Ignore that and make facebook great. None of us can know whether facebook will turn out to be worth $10 billion, $100 billion, or $1 trillion, but we all know that kind of crap that will ruin it.

— Max

Back with a math puzzle after a bit of a blogging break

It’s been too long. Here’s a short puzzle I saw that I thought was interesting:

How many prime numbers are there that start with a 1, alternate 1’s and 0’s, and end with a 1? Of course you should explain your answer…

The problem was originally stated in based 10, but I thought with a bunch of computer geeks in the audience, I’d ask about base 10 as well as binary and hexadecimal.

Have fun,

— Max

The value-based pricing trap

Software pricing

Sometimes pricing is easy. For example, the price of oil is pretty straightforward – there is a market and it sets the price. In many cases prices are just a little higher than production costs – this is the case for PCs or for some consumer electronics. Software is different and much more complicated.

Two things make software pricing complicated:

– It often isn’t directly substitutable. If C&H overprices their sugar, I’ll buy generic. What should I do when Microsoft overprices Excel? There are substitutes, but there is a significant switching cost.

– The marginal cost per unit to produce is very near zero. If Toyota sells extra minivans, they have to pay a lot to build them. If Oracle sells extra databases, they may have to pay a lot of sales commissions but they don’t incur additional cost to produce them.

Put these together and you get a pricing situation where there is a huge gap between the marginal production cost of extra units and the value to the user. Since the price should in general be somewhere between the marginal unit production cost (near zero in the case of software) and the value to the user (because above that nobody would buy), the range of possible prices to consider is very wide.

Given a wide range of possible prices at which you can make a profit and customers would buy, where should you price your product? Assuming your price must be the same for all customers, an economist would say to price your product so as to maximize the total profit generated. For a given price x, you can sell d(x) units. if they cost c per unit to produce, you select x to maximize d(x) * (x-c). In the case of software, c is near zero so you are maximizing d(x) * x. This is your total profit, and it is the same as the area of the largest rectangle you can fit under the curve that plots demand against price.

Custom per-deal pricing

Sounds good. Now you’re beginning to see why software is a great business (if you’re not sure on this point you will be if you visit Larry Ellison’s house). In fact the software business is even better than what we described: you can make even more money than you would using the maximization approach in the last paragraph. How? Your price need not be the same for all customers. Classically, you do this by segmenting the market. Airlines, for example, want to make prices as high as possible for business travelers who are viewed as price insensitive, and much lower for leisure travelers who are much more price sensitive; their great insight is that leisure travelers often stay over a weekend and business travelers often do not. Presto, the Saturday-night-stay requirement for discounted airfares. Not bad, but software does even better. Lets see how.

Regardless of the list price, each high end enterprise software deal usually winds up being negotiated by a sales person. I’ve seen many cases where the list price produced absurd outcomes. One of my favorites was a chain of pizza restaurants buying an order management package from Oracle that was priced based on the number of order lines processed – at a list price of 85 cents per order line including the advanced pricing option if I remember correctly. Try telling a pizza restaurant that it has to pay 85 cents to its software supplier whenever a customer adds pepperoni to their small pizza at a cost of 75 cents. They won’t like it very much.

Classically, enterprise software companies solve this problem with sales people who can offer big discounts. If your list price is effectively infinite and always needs to be negotiated (which naturally your sales people do perfectly) you will not just capture the largest box under the demand/price curve but you will capture the whole area under it.

Congratulations, you have achieved pricing nirvana. You are capturing all the value created by your software. Your sales team learns to work with the user to understand the value the software will create; this is wonderful because it makes them a “business partner” not sleazy sales people. But when it comes time to price negotiation, they understand what the value is and you can hold the line. There might be some slight inefficiencies: commissions to pay, customers hiring analyst firms to help them negotiate against you, you hiring business practices people to cross check the sales managers you hire, but this is all minor compared to the massive revenue you are extracting from your customers.

Caught in the trap

It sounds wonderful, doesn’t it? You get to capture every last drop of value created by your technology. Your shareholders should be very happy. But it’s not quite that simple. If you price your software at a level where you capture almost all the value in a deal, you create two problems:

– Your customers resent you

– Because your customers are capturing minimal value, they have very little incentive to grow their usage of your software. In fact, they have a strong incentive to find an alternative situation – even an inferior one – if they can capture even a little more of the value it creates.

Think about it. Imagine that new manufacturing software will save me $5,000,000 and costs $4,999,000 [assume both numbers are discounted for cash flow and adjusted for risk]. Would you buy it? Yes, because it saves you $1000. Did the vendor maximize their revenue in the transaction? Yes. Will you tell your friends that they absolutely should buy it or make it a top priority to install the same software in all your other manufacturing plants? No, because the net savings is very low so the project is barely worth the hassle. Would I switch to an inferior product that saves me $3,000,000 and costs $1,500,000? In a second. Don’t even talk about how fast I’ll switch when a competitor has a superior offering that is priced reasonably!

This is the trap most enterprise software companies are in. My use of the word trap is specific; I am not saying it is a mistake, rather something they have gotten stuck in and something which is very hard to get out of. Leaving money on the table will shrink revenues and the market will punish the stock price. The benefit of customers being more excited about your product and talking it up more is great for your future, but doesn’t help the business in the short term.

Enter open source


This fits perfectly with open source. MongoDB users don’t have to pay 10gen. They only pay us if we deliver value that they find compelling. They talk to each other about whether they got value from what they paid us. If they stop getting value, they won’t renew. Our reduced pricing power is, in my opinion, a huge benefit because it helps us avoid the trap of pricing too high.

What else can we do to avoid keeping all the value for ourselves and not leaving enough with customers?  We start with a reasonable list price. If list price is actually a great deal for most customers, much of this problem goes away. Second we are standardizing volume discounting – with very aggressive discounts on large deals, so that even in volume the standard price is a great deal. And finally, we need to explain to our sales team and our customers what we are trying to do.

— Max

Fun with right triangles

Last year my son enjoyed learning the pythagorean theorem. He was fascinated by a picture containing a proof and being able to calculate the length of one of the sides from the other two.

On vacation a couple weeks ago, we found a fun (well, for geeks anyway) driving game, which we called “Iron Chef Triangles”. The game is very simple: given a number, construct a pythagorean triple containing it – that is, find a right triangle whose sides are whole numbers one of which matches the given number. For example, if I say “5”, you might respond “3, 4, 5” (because 3 squared plus 4 squared equals 5 squared, so you can make a right triangle with sides of length 3, 4 and 5) or you might say “5, 12, 13” (because 5 squared plus 12 squared is 13 squared). If I say 8 you might say “6, 8, 10” or you might say “8, 15, 17”.

Try a few: how about 11? How about 20? See any patterns?

Once you’ve figured out how to easily do these in your head, here are two harder problems.

Hard: given right triangle whose sides aren’t necessarily integers, can you make a right triangle whose sides are integers with approximately the same angles? How close can you get?

Observation: the product of the three parts of a pythagorean triple is always a multiple of 60.

Unfairly hard problem: can you find two different pythagorean triples whose product is the same?

Have fun,

— Max

What our heroes tell us about ourselves








— Max

The 5 whys – and why do there need to be 5 of them?

An employee at my company (10gen, the company behind MongoDB) sent me this link to a video of Eric Ries talking about the “5 whys” and how technology problems are often people problems.

For those of you not familiar with the notion of “5 whys”, the idea is to ask why 5 times until you get to the root cause. While I think asking why and finding root cause can be important, a few issues jump out at me: first, why do you need to ask exactly 5 times?  Second and perhaps more importantly, when there are multiple answers to a given why, which one do you follow for further investigation.

In the example Eric gives, they start with a server crash and end with a manager who doesn’t believe in training. I would argue this is an attempt to reduce technical management to something that general managers reading Harvard Business Review can do, which is dangerous.

What really needs to be asked? The first why is straightforward.

1. Q: Why did the server crash?

A: There was a bad call to some new API

Now what’s the root cause? Is the problem with the API or the program that called it? Is it a problem of programmer competence, communication, or training? Or is it an architecture problem which is actually bigger than this specific API usage? Was the problem in design, implementation, testing, or documentation? Was it a process problem, a training problem, or a hiring/management problem?  I don’t think 3 or 4 more whys will mechanically answer that. One or two more whys might answer it, but the key is focusing on the right areas.

What technical manager at a startup really thinks that training will prevent a developer from calling an internally developed API incorrectly in a way that causes a crash? Its not impossible, but I think pretty unlikely. Better API docs could help, and better communications could definitely help. Or a more robust API, or more thought about whether that API needs to be public if it can’t be made more robust, or broader test coverage…

I think a better policy might be “2 or 3 whys, but the right ones”. I don’t think it will catch on as a management slogan, but I think it is more likely to yield useful answers.

— Max

Elections: just as irrational as human behavior in general

A friend of mine sent me a link to an article by Danny Hillis on elections. Danny is a smart guy who has done good work in many areas, but his analysis struck me as a) radically oversimplified and b) wrong in some important ways. But it got me thinking about how our changing understanding of human rationality (or lack thereof) in economics might spill over into political science. Let me start with how human behavior differs from classical economic models and move from there to how elections are effected.

Traditionally, economics was based on the notion of rational actors. Each individual makes decisions to maximize their utility functions (think about  a modern version of quantifying the notion of “greatest good” in Utilitarianism). Person A might value summers off more highly than a chance at millions of dollars and therefore go into teaching. Person B might dream of having their own private jet and be willing to work nights and weekends to keep even a small hope of their dream alive. But all of these are rational decisions based on utility functions which describe how much each economic actor values each benefit.

It seems reasonable (to a quantitative person such as myself) to think that human economic behavior works this way and the mysteries of the different choices people make is encapsulated in this unknown and in all practical respects unknowable utility function. Unfortunately for those of us who like simple mathematical explanations of human behavior, people don’t actually behave this way. In 1953 Maurice Allais wrote a paper (warning: it is mostly in French, 44 pages, and costs $10 to download) describing a paradox which showed that even if you look at simple financial wealth, people do not rationally follow a utility function. His paradox attacked the notion that if some percentage of the time (say 98%) two options produce the same outcome, we should base our choice between the options on the 2% of the time when they produce a different outcome (in economics-geek-speak, this is the Independence axiom in the Von Neumann-Morgenstern utility theorem). It seems rational (and official, now that it is part of a theorem), but in fact we care very much what happens in the 98% – we do not ignore the identical part of the two scenarios in evaluating them, irrational as that may seem.

Detailed example:

In scenario 1a, you receive $1 million 100% of the time. In scenario 1b, you take a 1% risk of losing your $1 million prize to gain a 1% chance of increasing its value to $3 million. 1% chance of nothing, 98% chance of $1 million, 1% chance of $3 million.

In scenario 2a, you have an 2% chance of receiving $1 million, in the other 98% you receive nothing. In scenario 2b, again you take a 1% chance of losing your $1 million prize to gain a 1% chance of increasing its value to $3 million. 1% chance of 3 million, 98% chance of nothing.

Some people (a reasonable number I would guess) would choose 1a over 1b but 2b over 2a. Mathematically speaking, both decisions are identical: you are giving up a 1% chance at $1 million to increase a $1 million prize to $3 million 1% of the time. The other 98% doesn’t matter. But psychologically, it matters a lot. When you are guaranteed $1 million dollars, risking it sounds irresponsible. And if that 1% chance happens, you will live forever with the knowledge that your greed to get $3 million cost you $1 million. In the other situation, if you don’t get anything, you aren’t surprised and you are fairly confident that it had nothing to do with your decision to increase the odds of getting nothing from 98% to 99%.

Now, on to politics.

Hillis lays out a simple left-right political spectrum. He puts two candidates on it and models voting behavior as people voting for whichever candidate is closer to their beliefs. From this, we can deduce all sorts of things, some of which – like elections being close – sometimes model reality pretty well. Then again, Herbert Hoover never participated win or lose in a close presidential election, and how many electoral votes did Walter Mondale get in 1984? 13. Which is 5 more electoral votes than Alf Landon received in 1936!

The problem is that elections are much more complicated than Hillis’s model. Lets put aside for a second that there are multiple issues involved so it is not a simple one-dimensional spectrum. Lets assume there are only two candidates in the election. Lets also put aside candidates personal charm or lack thereof and their potentially scandalous sexual, military, or substance consumption histories. This model is an still an oversimplification in two critical respects:

1. Voters do not have a simple choice between voting for the Democrat and voting for the Republican. They can stay home. Or they can volunteer or donate money or attempt to harangue their friends into voting for the candidate of their choice. Imagine an (exaggerated, to make a point) election where:

a) 60% of the electorate prefers Smith to Jones, but only by the narrowest of margins; they are generally dissatisfied with both candidates

b) 40% of the electorate strongly prefers Jones to Smith – in fact they thing Jones is close to their ideal candidate and Smith is so nightmarish a candidate they would consider leaving the country if he were elected

What happens in that election? Maybe a 48% turnout; 36% of population (90% of the Jones supporters) vote for Jones and 12% of the population (20% of the Smith supporters) vote for Smith. Jones wins in a landslide, with 75% of the vote. Smith, despite a 20 point margin in polls of registered voters, can’t even carry his home state.

Is this exaggerated? Yes, but intensity matters.

2. Just as the 98% that was unchanged effected the outcome in Allais’s paradox, candidates not on the ballot effect voters. Again, here’s a scenario:

The Yellow party and the Purple party each have 50% support. Both of them nominate a candidate who is free of scandal but not particularly personally charming. Each candidate represents the mainstream of their party – put them just slightly to the center of the middle of their party’s 50%, at say the 30th and 70th percentile respectively.

So far, it sounds like a close election.

However, in the Yellow Party primary Mr. Gold had to duke it out with Mr. Canary. Canary was just a few points further from center than Gold, so the primary was very close, and neither party had a majority of committed delegates at the convention. Mr. Gold did have a slight lead over Mr. Canary, so when the super delegates broke strongly for Gold (whom they felt was more electable) they felt they weren’t overturning the will of the people. But the Canary camp felt like if you included the Canary, Maize and Buff delegates (who endorsed Canary when they dropped out, but such endorsements aren’t binding on their delegates so they were counted separately) against the Gold and Ecru delegates (Ecru having thrown his support to Gold), Canary held a lead before the super delegates voted.

The Purple party primary, however, was a different story. Mrs. Plum was the only mainstream candidate; Mrs. Indigo and Mrs. Magenta were really fringe candidates who generated very little support. The party became unified behind Mrs. Plum in February.

How does this election turn out? Not close at all. About half the Yellow party voters are profoundly upset with the Gold campaign and their party. Many of those voters don’t have an ideological problem with Gold per se, but they are so disappointed with how the process played out for Canary (who wasn’t picked as VP) that they are not at all motivated to vote. Despite being weary from the primary, the half of the Yellow party that supported Gold turns out 60% of their voters giving gold 30 million votes, but only 40% of the Canary voters turn out at the general election. Those that do vote reluctantly for Gold, giving him another 20 million votes for a total of 50 million. Meanwhile, 60% of the Purple party voters show up, giving Mrs Plum 60 million votes and a 10 million vote margin.

If that seems farfetched, I can introduce you to some Hilary Clinton supporters who are strongly pro-choice, support gay marriage, and voted for John McCain.

Now, I constructed both of these scenarios while leaving in place many oversimplifications. Overlay multiple issues of varying intensity and varying degrees of organized lobbying with ethnic, religious, and gender identification and elections become very complex things. Certainly the practical modeling of individual factors is quite advanced; I am curious what the intellectual underpinnings are, and to what extent they have embraced the latest thinking in human economic behavior.

— Max

%d bloggers like this: