Big Data at Strata Conference

I had the chance to join a great discussion on Big Data last week at Strata. Video below.

Based on the dialogue, I highlighted 3 points:

1) My view of the Big Data ecosystem that is forming

2) Why Hadoop-alone may mean failure for enterprises in 2012

3) Why the application server market is a better analogy than the database market, when it comes to Big Data

I will do longer blog posts on each of these topics soon.






Lastly, based on a few requests I received, here is the chart I used on the Big Data ecosystem:

Big Data Ecosystem
View more presentations from robdthomas.

Please Disperse...There is Nothing to See Here

This week, Oracle reminded me of a classic scene from The Naked Gun, where Lieutenant Drebin (the late Lesley Nielsen) grabs a bullhorn and states, "Please disperse...there is nothing to see here." Meanwhile, directly behind him, you see gunfire, explosions, helicopters, chaos, etc. It is great satire, demonstrating that despite very obvious conditions, some people remain oblivious and pretend that nothing is happening.

Oracle presented their own flavor of "Please disperse...there is nothing to see here." on their earnings call this week.

To paraphrase their comments: We make alot of money in relational databases. Therefore, we will do anything to protect this franchise. We will pretend that the world has not changed since our first government project, code named Oracle.

Their actual comments were:

Well, Hadoop... is going to be one of the feeder systems to the Oracle Database. We have announced interfaces that allow you to take your Hadoop systems and connect them to Oracle systems so you can load your Hadoop data into Oracle. So we think it's going to increase the usage of Oracle. Furthermore... we're going to come up with 4 new-engineered systems, one of which is SPARC Supercluster, and we're also going to have a big data engineered system that we're going to introduce at Oracle OpenWorld. So we think as more and more data gets computerized, a lot of that data finds its way into an Oracle Database and it allows our database business to grow at an accelerated rate.

After Hadoop finishes filtering the data, the place you want to put that data is in Oracle Database, and that's what a lot of our customers are doing. And we are exploiting the trend, the big data technology and the big data trend, if you prefer, by building a Hadoop appliance that attaches to the Oracle Exadata database or any Oracle Database for that matter. But you don't have to buy our Hadoop appliance if you can use whatever servers you want running Hadoop, and we provide the interface between Hadoop and the Oracle Database. The idea is all of your data should be -- you should be able to put all of your data regardless of data type into the Oracle Database where it can be stored securely and reliably. And you could search it and get answers to your questions very quickly.

Anyone that has spent any time looking at Hadoop/Big Data and has actually talked to a client, knows a few basic things:

1) Big Data platforms enable ad-hoc analytics on non-relational (ie unmodelled data). This allows you to uncover insights to questions that you never think to ask. This is simply not possible in a relational database.

2) You cannot deliver true analytics of Big Data relying only on batch insights. You must deliver streaming and real-time analytics. That is not possible if you are biased towards putting everything in a database, before doing anything.

3) Clients will demand that Big Data platforms connect to their existing infrastructure. Clients don't think that Big Data platforms exist solely for the purpose of populating existing relational systems. Big difference.

Once you consider what is happening in the market around Big Data, it becomes obvious that there really is something to see here.

Incentives

It is interesting to me that a best practice around incentive structures has never emerged. At a minimum, you would think that sales incentives would have converged to a common model. Yet, every company that comes into IBM seems to have a different approach. Why is this so hard?

Perhaps it is hard, because everyone tries to make it very precise, which often leads to hard/complex. This is a bit ironic, given that the best incentive is one that is easy to understand and easier to implement.

Here is a great excerpt from a recent Warren Buffet interview (full interview here):


Buffett:    I can end the deficit in five minutes.


CNBC:   How?


Buffett:   You just pass a law that says that any time there's a deficit of more  than 3 % of GDP, all sitting members of Congress are ineligible for re-election. 


While he said this tongue-in-cheek, there is alot of truth to the thought process. Why would this incentive work?

1) It is tied to the core motivation of the target party. In this case, a politicians desire to be re-elected.
2) It is numerical. No shades of gray on success or failure.
3) It is certain. If the target party does not achieve it, the consequences are clear.

I think if you were to look at your own organization or company, you would find that very few organizations set incentives that meet all 3 of these criteria.

Another post related to this topic can be found here.