OK, you have waited long enough, here's my take on NoSQL

I have to admit I'm getting old, and I am now scaringly close to being a sad old git, and not only that, I'm a database guy also, and I have worked with SQL based relational databases for more than 20 years now.

So considering my age and heritage, I really should just dispose the NoSQL movement as something for those young kids with earrings and a baseball cap (and to add insult to injury, the cap is worn backwards) and that any serious database dude like myself, with my loads of experience (like the invaluable experience of having run Oracle on a Wang word-processor. Real valuable stuff I tell you). But no, you will not hear me do that. But also, you will not hear me say that NoSQL key-value stores will replace all SQL databases within 5 years (If I worked for an analyst and was paid dearly to say things like that, I might have, though. Any takers?).

My take is actually quite simple. The strength of the relational model is that it is inredibly generic. The lack of a specific order and hierarchy makes it even more generic. I think few people would argue that more or less all applications served by NoSQL could just as well be served by a SQL database, if it wasn't for two things:

• Scalability - The lack of the strict consistency rules of RDBMS heritage in most NoSQL implementations makes them much more scalable. The very nature of most NoSQL stores is distributed, and the lack of strict distributed consistency makes this distribution scalable beyond what is usually possible with an RDBMS, given the same platform etc.
• Performance - This is largely due to the above, i.e. a NoSQL store being more scalable makes it easier to cram more performance out of it.

Now, with all this in mind, am I saying that NoSQL has all the advantages of an RDBMS, but with much better scalability? Nope, no way, José.

The strict consistency requirements of an RDBMS is also an advantage. It's not so that, if I understand them correctly, the propoents for NoSQL stores thinks that consistency is bad, it's just that they don't want to pay the price in terms of performance for it. And to be frank, although in many cases data inconsistency is acceptable, it still has to be controlled, an uncontrolled consistency, i.e. you don't know how inconsistent your store is and in what way or anything, is not something we want. So even a NoSQL store is limited.

So it all comes down to performance then. We sacrifice consistency to gain performance through scalability. Right? If you agree to that, then I think NoSQL is not a long term solution. It's not that I am saying that "NoSQL is for kids, real databases needs SQL", that was the argument against SQL based databases in the 1980's largely, where Hierarchical databases still ruled, and SQL just had a too big overhead, or so it was thought. The differerence here is that SQL had higher functionality than the competing technologies of the 1980's, but not enough performance in many cases. But performance is bound to go up. All the time. And for much less money. At least for a while to come. Look at virtualization. I've been a proponent for that for quite a while, and just a few years back, the argument against it was that "performance sux". Well, compared to raw iron maybe it did, but that wasn't the point. The point was, did I get enough performance? And in many cases you did, with an environment that was a lot easier to manage and at a low cost.

What this means to me is that there is a place for NoSQL stores right now, where the performance and size requiements are really high, and where one is willing to compromize consistency. But a technology that limits functionality, features and ease-of-use at the price of performance will continue to be a niche technology. But that doesn't mean it's useless or anything, quite the opposite, I'm a pragmatist at heart, and whatever works, works. But if I had the choise of storing my data in consistent or in-consistent state, and if both solutions provided enough performance for my needs, I'd go consistent any time.

And then there is one more thing. The scalability of the NoSQL stores is largely due to it's distributed nature. And there are arguments out there that says that you cannot create a consistent, distributed, scalable datastore. I think you can, I'm convinced of it actually. There may be other compromises needed to achieve that, but that it can be done I am sure.

/Karlsson