looking ahead
2016 may go down as the year Hadoop crossed over from "early adopter" to mainstream PHOTO: Steve Halama

2017 will see some major changes in the big data ecosystem, driven in part by two trends.

Both trends offer opportunities for data-driven organizations to stay ahead of the curve — but only if they take the time to do some preparation now. 

1. Hadoop Gets Put to the Test – Here's How to Help It Pass

Apache Hadoop distribution vendors have crossed the adoption chasm — making unstructured data in Hadoop a reality. In 2017, we'll see the conversation shift away from whether Hadoop is here to stay and toward how companies can get the biggest bang for their buck. 

While analysts acknowledge that Hadoop has graduated from the “early adopter” phase to threshold of mainstream use, there are still bumps ahead in the road to Hadoop. A survey from research firm Gartner found that 49 percent of respondents cited "figuring out how to get value from Hadoop" as a key inhibitor to adoption.     

This leaves companies that have built Hadoop clusters scratching their heads, wondering how they can make good on Hadoop investments. But instead of giving up and trashing Hadoop clusters, now is the time to dive deep and identify the value around the reason they were built. 

Put on Your Evangelist Hat

To make the most out of a Hadoop cluster, start by embodying the role of a Hadoop evangelist. Beat the drum for why the Hadoop cluster should be used. 

Rather than taking on a huge project though, encourage business users across the organization to seek manageable projects that deliver on meaningful outcomes as opportunities to show the value of the Hadoop cluster projects. 

Support Simple Projects That Can Provide Great Value

Instead of aiming to achieve a 360-degree view of customers, shoot first for a 90-degree view. Arming customer service groups with social media data can have a positive impact on the customer journey and illustrate the value of Hadoop. Adding a bit of social sentiment to customer interactions can give customer service teams a slightly better view of whom they are serving. 

In the same vein, take a look at publicly available data that can easily be infused into data lakes to help color the content and provide more value. For example, injecting weather data into transportation dispatchers’ workflows can help automate processes and manage workloads. 

2. Data Prep Becomes a Feature – How to Prepare

Data prep vendors, from small data visualization vendors to giants like Oracle, are bombarding customers with data prep features and standalone products. 

However, in 2017 we’ll start to see data preparation become more of a feature rather than a market as big data analytics continue to evolve both in product offerings and market share. 

As such, there may be a consolidation in the marketplace as companies start to acquire product offerings in this area as well as customer lists from small, niche vendors. So what should organizations do to shelter themselves from the marketplace change?

If you already have a data prep vendor, make sure they are taking good care of you. Double check that licensing agreements are up to date and adopt revisions of the product as they come up. 

If you are considering buying a data prep solution, take a long-term perspective on your product hunt and consider these questions:

  • How may people do you hope to become self-sufficient with this tool?
  • What are the skill levels of your potential user base?
  • Will there be any components beyond data prep that might come in handy down the line?

After evaluating theses questions my final advice for choosing vendor is this: Rather than go for the Maserati of data prep tools, choose the Toyota equivalent. This way you have a tool everyone can drive and can be confident that it will be around for the next five years. 

With these two predictions, I think that 2017 will be a year of major changes to our ecosystem. Organizations will embrace all data while realizing that delivering autonomy and value to the stakeholders is the way to becoming a data-driven business. 

I am looking forward to what the new year will bring. What are your predictions on big data and big data analytics? Please reach out or comment below with your thoughts.