Big Data in action – A real-time and historical analysis use

Blog 02

Source:, Posted by John Schitka

This truly wowed me and really had an impact on my perspective around how Big Data can change the very essence of how we approach things.

During my regular routine of scanning news articles over a coffee before settling down to real work the other day, I read a Time article on the first of the televised debates around Scottish independence. It highlighted what was said the night before; who won based on a poll, what was done right, and what was felt each side had to adjust going forward. Standard post analysis stuff I have fully come to expect. (read it here) The next day on Reddit I came across a timelapse visualization of sentiment based Twitter data captured during the debate. It totally blew me away and fundamentally changed the way I looked at things in terms of Big Data. I thought, OMG, holy bat droppings batman, this changes everything! [Yes I am that old and know Adam West as more the mayor of Quahog]. Suddenly I saw potential for definite impact scrolling before my eyes; the need and use case for both for real-time and post event analysis.

I thought, you can see the ebbs and flows of the yes and no sentiment as the debate progresses. (view it here) And I envisioned the potential and need for real time analysis. In real-time during the debate you can see the sentiment shift and with this real-time feedback, you can on the fly, make in the moment adjustments and alter emphasis. The support team viewing this real-time data can coach the speaker via an ear piece, iPad on the podium, hand signals or sign board in the front row – hit them again on that – go back to – avoid this – focus on that. Kind of the way F1 drivers, speed skaters and cyclists in a velodrome get in race advice from their support staff so they can make in race adjustments. It reminded me of a story on the use of Twitter presented at a peer group I was in some 5 years ago. In brief during a tradeshow someone in the office was monitoring Twitter and when somebody tweeted they were not the grasping the concepts from a presentation, this was conveyed to staff on the floor. They located the person, spent some one on one time, clarified the concepts and got a subsequent tweet praising the company. So the concept is not new just the scale, volume, velocity and automation of the process. And to think Twitter is just 8 years old. Of course you need technology capable of dealing with the volume and velocity of the data stream, and able to do real-time analysis on it.

Further, I recognized the need for, and value of, this information in post event analysis. You can see at what points during the debate yes or no sentiment surged and receded along with the size of the response graphically represented on a map. Not only can we correlate this to what was being said by whom at that point in time in the debate, but knowing the geographic location we can correlate it with a lot of other data. What are demographic of the area? What is the median age, level of education, income level, proportional type of occupation, religious adherence, political affiliation, level of charitable donation, average political donation and to whom, population density, marital status, divorce rate, number of out of country vacations, crime rate, average spend per shopping trip and even handedness, favorite beverage or most popular car color – you never know what kind of correlations will pop up. All this means complex, ad hoc, iterative work on massive data sets along with the Twitter and geospatial data. Sounds great and provides a lot if insight and drive future strategy and action. Of course you will need a solution capable of not only real-time analysis, but also able to handle complex queries on massive, diverse data sets, with a geospatial engine, natural linguistic capabilities, suited for ad hoc queries and with rapid response time to allow for iteration and what if analysis. And of course all this in a simplified manner, without a lot of complexity and fuss. Sounds a lot like what the SAP HANA platform offers along with partners such as Esri and intuitive visualization tools such as Lumira.

Learn how the SAP HANA Platform for Big Data can help you take a simplified approach to making Big Data work for you in deriving true value from Big Data for your business at

our clients

© 2015 Ameya Info Technologies.
Design  enrise