Ralf Haller April 15, 2014
In case you have been bombarded by Big Data event invitations recently you might be interested to hear what is going on behind the scenes here in Switzerland.
The Big Data topic has often been looked at in the past from scientific (University-driven) and technology perspectives, and events have been organised by this community. Our own first Big Data event 2 1/2 years ago went down that route but the attendees persuaded us to change direction. Since then we have focused entirely on success stories and use cases. As with all our other events, we have also continued not to allow product or service pitches.
PwC Switzerland has now launched the Big Data week Switzerland trying to capture that market segment, as their traditional cash cow consulting business (legal policies for financial services) is coming to an end and they have to desperately look for new income sources. Up to now they have been attending our events and claimed (until March!) that they still intended to come to our event in May. In parallel, however, they were contacting our sponsors and speakers for their own events week while still claiming in meetings with us to be supporting our event.
You could call this approach guerrilla tactics but, as PwC is hardly a small organisation, we think that is far too nice a name for it (the guys must read books like the Art of War).
At the end the people (you) will decide if Big Data needs a big week-long event series with a dozen PwC consultants and whisky tastings, talking about projects that they did not even run but only read about, or if a compact – neutral – afternoon session will do for you as well. From our perspective, we are looking and waiting to see what happens, quite relaxed, since one thing that Big Data definitely does not need is expensive consulting companies making lots of money in such projects. Informed end users will be able to run such projects entirely by themselves using technologies and services from Big Data expert firms (not accountants) and that is what our events will enable them to do!
Ralf Haller December 9, 2011
Yesterday IBM announced the acquisition of DemandTec for $440 million. The company has hundreds of customers such as in retail and government helping them analyze and derive conclusions from massive amounts of company-collected data.
“Big data” as the name suggests has to do with masses of data collected from just about everywhere. We are talking terabytes and more of data collected from all kinds of sources such as information-sensing mobile devices, aerial sensory technologies, cameras, microphones, RFID readers, wireless sensor networks, social media, buying patterns and so on. The fact that 90% of the data in the world today was created within the past two years makes this an even bigger challenge as it is a constantly moving target. Currently used relational databases and desktop statistics/visualization packages cannot deal with this unstructured data, requiring instead massively parallel software running on large computers, and often grids of dozens, hundreds or even more of servers.
Its not surprising that Google has been into this for quite a while since their search algorithm is doing exactly what big data is all about: collecting massive amounts of data and making decisions (search) based on analyzing it. Google is offering access to computing power even to enterprises now with its service BigQuery. More also in this Google blog. Google was also there right at the beginning with its framework MapReduce that was then used in projects by others such as Yahoo, leading to Hadoop, a story on this you find here.
Data collected are e.g. from web logs; RFID sensor networks; social networks; social data, Internet text and documents; Internet search indexing; call detail records; astronomy, atmospheric science, genomics, biogeochemical, biological, and other complex and/or interdisciplinary scientific research; military surveillance; medical records; photography archives; video archives; and large-scale eCommerce.
As IBM states: Big data spans three dimensions: Variety, Velocity and Volume.
- Variety – Big data extends beyond structured data, including unstructured data of all varieties: text, audio, video, click streams, log files and more.
- Velocity – Often time-sensitive, big data must be used as it is streaming in to the enterprise in order to maximize its value to the business.
- Volume – Big data comes in one size: large. Enterprises are awash with data, easily amassing terabytes and even petabytes of information.
A McKinsey report on Big Data mentions e.g. these points:
- The use of big data will underpin new waves of productivity growth and consumer surplus. For example, we estimate that a retailer using big data to the full has the potential to increase its operating margin by more than 60 percent.
- The computer and electronic products and information sectors, as well as finance and insurance, and government are poised to gain substantially from the use of big data.
- Policies related to privacy, security, intellectual property, and even liability will need to be addressed in a big data world.