Editor's note: In part one of this two-part story on big data analytics in the cloud, several cloud providers discuss the emerging opportunity they see in the midmarket for big-data
Service providers hesitate to go as far as declaring that big data analytics will be the next big adoption driver for cloud, but the intersection of cloud and big data still offers some opportunity. The combination of resource pooling, universal accessibility and cost savings make the cloud an attractive hub for big data, several cloud providers said at the recent Cloud Expo 2012 trade show in New York.
"For big data to work, it requires massive amounts of storage and massive amounts of compute, and not everyone wants to buy that," said Jason Mendenhall, executive vice president of cloud and big data at Switch Communications, a Las Vegas-based colocation provider that hosts several large cloud providers and runs a 24-petabyte Hadoop testing environment. "One begets the other: Cloud allows for true big-data analytics to exist."
One begets the other: Cloud allows for true big-data analytics to exist.
Executive Vice President of Cloud and Big Data, Switch Communications
Big data analytics -- the practice of rapidly crunching anywhere from terabytes to petabytes of unstructured data to identify patterns and improve business strategies -- was first considered relevant only to the largest enterprises, and not just because they were the only ones that could afford the infrastructure necessary to support it. Smaller businesses traditionally haven't generated comparable volumes of data.
But cloud providers now say several trends -- such as mobility, machine-to-machine (M2M) communications and social networking -- are causing global data volumes to balloon. The spike has been so dramatic that cloud providers anticipate midmarket demand for big data analytics. Providers expect that the steep infrastructure costs needed to support on-premises big data architectures will drive those customers to look for cloud-based alternatives.
"What we're aiming at is a democratization of big data analytics," said Henry Fastert, chief technologist and managing partner at SHI International, a large reseller, managed service provider (MSP) and cloud provider based in Somerset, N.J. "The cloud is the best place to do this because you can assemble huge numbers of virtual machines to work in clusters. Now, you can certainly do this by buying physical infrastructure, but once you start talking about buying that type and amount of physical infrastructure, it gets expensive. Small companies literally just can't afford to do that."
Big data and cloud analytics: 'Everything is possible'
SHI's research and development lab is designing a big-data cloud analytics service to add to its Infrastructure as a Service (IaaS) portfolio, Fastert said. The service is expected to launch in early 2013 and will target small and medium-sized businesses (SMBs).
"[SMBs] have exactly the same needs as any other company but far fewer resources to put against that type of task," he said. "It is far and away the lion's share of the potential market in this space."
More on big data and cloud analytics
New cloud analytics and BI services to use big data
Gartner: The six elements of cloud analytics and SaaS BI
Cloud APM services win customer trust by monitoring app performance
Although AT&T is not exclusively pursuing the midmarket for big-data cloud analytics, it acknowledged that the cloud enables service providers to tap a new class of big data customers.
"Cloud is a great equalizer," said Steve Caniano, vice president of hosting, application and cloud services at AT&T. "[Without] the cloud, you'd have to have an IT department with highly skilled folks, capital and data centers where you could [deploy] heavy-iron equipment like supercomputers -- or pay consulting firms to do that work for you. That all sounds [time-consuming] and expensive … and in many ways, I think the cloud does start to level the playing field."
The cloud is one of several "transformative" technologies pushing demand for big data analytics in the midmarket, said Jonathan King, vice president of cloud solutions at Savvis. The St. Louis-based hosting and cloud provider recently announced a strategic partnership with Hortonworks, a big-data platform vendor that develops commercial products based on Hadoop.
"Everything is possible now," King said. "To get the amount of compute that you would need to run the types of jobs that are being run now would've cost a fortune before. But once the cost of compute came down, then you had players like Yahoo, Google, Facebook and LinkedIn start to develop [analytics] tools in their large-scale environments and then open source those tools.
"Then you had pervasive connectivity, smartphones and even more data points. Now, all that stuff coming together is not big data," he added. "Big data is saying, 'How do we solve problems with all this information?' And the connection to cloud is that cloud is helping to connect to this plethora -- no, plethora's not even a big enough of a word -- to this tidal wave of information."
Not all providers see big-data cloud analytics opportunity
Not every provider is convinced, however, that big-data cloud analytics is a universal trend. Joseph Corvaia, vice president of cloud computing at Evolve IP, a cloud provider based in Wayne, Penn., said big data helped customers identify more cloud use cases. But at this point, few are looking at the two technologies strategically.
"There's a lot of hype around the big data concept now, but the reality is not everybody has big data," Corvaia said. "I think the small to medium-sized businesses, while they don't necessarily have a need in most cases for big data [analytics], they are recognizing that there are a lot of different ways for us to use … cloud computing."
Continue reading part two: Cloud providers explain how they're modifying their networking, storage and server architectures to support big-data cloud analytics. Read the second half of this series, Big data analysis in the cloud: Storage, network and server challenges.
Let us know what you think about the story; email: Jessica Scarpati, Site Editor.