What’s this buzz? Big Data Analytics & Cognitive

By Rohit Kumar

“Big Data Analytics” and “Cognitive” are probably the hottest topics going around in areas of IT, Business availability of information and run smart strategy seekers and providers.

Very interesting here is, most of the time we although talk about Big Data, we assume we are talking about Analytics. When we talk about the Analytics we think we are half into Cognitive. We will cover this in more detail in later part of this discussion.

There is a chaos in industry with lots of tools available for various utilities related to big data management and Analytics. And all the new jargon and which promises lot of stuff, related to Cognitive.

Although, the core lies always in purpose and usability of it. All this new age Analytics and Big Data products which emerged in recent years, they were an instant hit in market the moment they had put their first feet in market. Cognitive was a slow walker though due to lot of uncertainty in people’s mind about it, use case alignment and time & money.

Like the saying goes, “There is always a room for improvement”. Markets have always been looking for progressive products. They have helped industries and businesses across globe reaching altogether new heights and even churn out new kind of products. In fact, global market is also result of such continuous improvements and innovations. Although, what was sought out initially was targeting to create monopoly or prominent place in market for businesses, actually made customers the king. So now this is current game of market.

Before we come back to actual discussion points relevant to title of this article. Let me just cut through the difference in such innovations then and now. This I assume would set right pace for the current discussion.

Like mentioned earlier in this blog, innovations and advancements have been continuous. Right from industrialization revolution (rather before that too) until now, continuous. The difference in having the current analytics products vs an upgraded machine or technology process or a software, which combines all department data or anything of this sort what we are used to get earlier, is important. That use to be black and white compared to these products.

Industry was able to get hold of these advancements and adoption was quick. They had a manual attached and how and when it must be used was explained simply. Really powerful solutions, numerous to count. Very convenient and mostly robust. Very good. Really?

Take a pause. Answer is yes. The quality and value in most of them is till the time extremely valuable and good. 

So, when everything is around why do we need something more complicated? We have robust ERP solutions e.g. SAP ERP Solutions taking care of everything. From manufacturing to supply, from ordering initiation to invoicing, finance, customer relationship, vendors, everything. We already have industry-based solutions also present and which are pretty much exhaustive in having all industry KPIs and processes already pre-packaged. Install, customize and go.

But if you go back to the previous paragraph, you see these products / solutions are covering, standardizing and optimizing business processes. There are though some opportunities with data mining and similar tools to dissect internal data and get great insights to it and even carry on planning and other such activities. Not saying not sufficient, not required. But in all what we are seeing, observing, analyzing, dissecting, planning is “looking inwards”.

We generate these data, mix it with some external information if possible and then use this data “by-product” from operations only.

The tools what we have been using until now have really brought us to systematic, disciplined and optimized state.

1. Processes which we are using, can we optimize more?
2. Can we put new variants to optimization processes and measurements? E.g. doing something at a certain frequency to do quality check, can we also analyses and see the effect of time to it. Say doing it at a particular stage precisely
3. Can we prioritize data and process based on real time inputs?
4. Can we observe in real time or when required patterns of reactions from interested subject groups in market?
5. Can we evolve into altogether new processes wiping out the previous one used?
6. Can we have a neural system which can learn and upgrade itself with time?
7. Can we track in multi dimensions every penny of our marketing cost with transparency?

These and many more are the questions which could be answered using big data and Analytics.

Will try to see some detail around all the seven questions above one by one. If possible with some instance. But let me first come to the part we left unattended at start, I mentioned earlier in this discussion that “mostly discussions towards analytics are actually big data discussions”. Strange but true.

Blame it on the concept being relatively new (not an infant though by now), or lack in knowledge exposure front or whatever. But fact is it exists. Game changer is getting clarity on this, for consulting IT providers and businesses both. Cognitive solutions remain the worst affected in terms of being understood with usability.

When we talk about adding real time data, blending unstructured data with operations data, storing petabytes of data, storing these volumes of data in structured ways and using fuzzy logic for these data – we are talking about Big Data handling and not analytics. Just to extract data from all the world and getting in some real time broadcast display option for social data or feeding tables real time or adding some extra fields in existing dashboards is not what Analytics is all about.

Analytics departments, at times as seen in consulting firms as well, are limited to train and plan for generating POCs. There is not many talking about creating customer segment wise brainstormed solutions, understanding their business and helping it. I have seen multiple customers talking about need for Analytics and proposing requirements and which talked about extracting storing and blending data to get little more insight on customers. Probably the most used use case around this analytic is getting customer feedback or misusing for marketing stunts.

Analytics is a thought process a strategy. Big data supports it. Just being able to get all data and do a flat reporting using this is just a technical stunt. Cognitive while is a trained and self-evolving / learning system created on top.

Purpose of Analytics is not only to optimize processes but provide a better insight to use update and device new processes and get rid of less value activities/processes. While Analytics would provide the manual means to use this information and work to generate results, Cognitive would use it automatically to convert into results. For this we need not only capabilities to bring in more data mindlessly just because we can do now, and / or everyone is doing in industry. Too much of data which cannot be used to churn out relevant information for use could become a nasty issue and high cost wastage as well. Always judiciously bring in big data wherever required but have a data and use case strategy first.

This missing strategy element and organizational goals only result into big data and analytic projects where 96-98% of effort and cost is planned for getting hold of every possible data and just 2-4% in actual analytic. Cognitive are not projects where companies should implement one fixed requirement and forget it. The solution should evolve around new dimensions and thought process where it could be further evolved recursively. One bank might go ahead and implement analytics solution to get analytics on customer trends to see better picture, to help existing customer and look for new customers. But getting into deep analytics, same data could also bring data towards identifying fraudulent customer transactions and predicting or raising alarm

for the same. Further use case might be where they could identify based on transactions, income etc. a solution to look for customers eligible for investment products. So deep diving and further extension of solution is what should be done. The solution should evolve automatically when taken to the level of cognitive computing.

We will try to elaborate this with talking in little detail about the above listed seven points one by one.

1. Processes which we are using, can we optimize more?

This is probably the most important aspect businesses should look forward to getting value out of Analytics.  But mostly the projects go towards gushing in more data to go for feedback from customers or related to market performances. Cognitive which is happening now slowly was a distant option earlier.

Let us talk about manufacturing domain. In this high competitive time, even a marginal and seemingly less important change could bring high rewards in terms of per ton or per piece production.  It might be because of less time in factory line or less wastage or more optimized use of raw materials.

In today’s world of analytic possibilities, IoT, combined with data streaming technologies, industries can run models to see most optimized way of running a manufacturing unit. That to dynamically adjusting as per requirement of the moment.

Not only commercial domains. Research labs can utilize combined big data banks and analytics on top of it with mathematical and / or learning models to analyses rapidly same data instead of actually carrying those experiments. One lab was able to use all possible data at once to calculate star and heavenly body positions for every space coordinate within seconds.

One telecommunication company was able to analyses together data from call center texts, customer calls and customer outlet feedback to understand and predict well in advance possible customers who might move to competitor. They streamlined processes and instead of top down approach where focus of customer-centric strategies was based on mathematics from finance department or inputs from sales team, they started focusing towards customer outlet data as this was most influencing to results. A process shifts to Bottom Up.

Internal organizational work patterns and relevant employee and project details could be mixed with skills details can help understand the unproductive black hole areas and time. It can also help understanding an organizational level preparedness direction and removing bottleneck processes. This can also help putting right skill at right place by optimizing skill matrix.

So, what here we are talking about using internal data to re-validate and optimize working and running of organizations. Employees and all stakeholders should be educated on this off course as wide spread fear of machine replacing man is the biggest hurdle coming as push back from employees in these cases. Cognitive done on top of these could actually take over some roles as humans would do and could over it use it to generate decisions rather than just good data points and dashboards.

2. Can we put new variants to optimization processes and measurements?

(e.g. doing something at a certain frequency to do quality check, can we also analyses and see the effect of time to it. Say doing it at a particular stage precisely)

As mentioned in so many examples previously in this write up. There could be other variants or candidates in existing analysis which could be tried tested and used with a combination of big data and deep analytics.

There was a case study of a car manufacturer who was running a successful factory line for assembling car parts. The process was a time tested and understood to be pretty mature. Only when analysis of all the data surrounding it was put and deep analytical models were created they came to know a small change in sequence of assembly in assembly line was not only making there people work more efficiently but it was actually decreasing ship to market time from assembly line. This is a good example of even moving ahead analytics only and introducing cognitive.

In various industries, the quality of product in a stage of manufacture is to be tested. This is very important in pharma, paint and other such chemical industries. But there at times are lot of variation in quality levels for same product at same manufacturing stage for different batches. This has baffled lot of companies and the million-dollar question is, mix of raw being such calibrated, especially in pharma, how this difference is there. The process steps are also very much calculated. These quality variations if exceed any threshold which is again very narrow, leads to deep analysis of sample and then at times this leads to impacting cost directly and indirectly and as a result profit. This would be still requiring human intervention to a larger extent, while Cognitive would even take away major manual effort and with precision and speed.

Research has shown, the variations are mostly due to the time difference in picking up sample in that particular manufacturing stage window. This very small factor of time has brought up to even 30% of high productivity to many such industries. Such is the power of using deep analytics. 

3. Can we prioritize data and process based on real time inputs?

Big data possibilities together with existing ERP data capabilities have opened all possible inputs for data. Sits on top is deep analytics, which can work as a deciding factor towards validating utility and most potential use case data.

In one previous example of telecommunication giant, we talked about using call center and retail outlet data with conventional ERP data. What take away from perspective of current point is, all the data was there earlier as well. All channels were providing data, analytics to understand the data sets lead to discovery of prioritizing data from customer outlets to highest priority and was found to be most valuable in that scenario. So now to decide things making plans source of data was prioritized and further additional data fields were added for sales outlet online forms to broaden this most important data.

There was a bank in Europe which tried to enhance online experiences by implementing a very well designed and user-friendly website. The moment they started the new one hit rate exceeded by around 10-20%.  This was a success only up to the time when someone thought of running a data analysis of usage of ATM, customer data, social feedback and hit vs action detail data. The results were stunning. The site which was designed for banking facility use mostly was being used least by their customers. The data deep analytics showed the most number of usage expected as opposed to planned design was seeking account opening and investment banking options details by new prospective customers. The bank in light of this optimized website to prioritize the same on website and also provisioned a quick addressing of the same via a new hotline number. Success rate was above 40%. How about pushing this further beyond the analytics dashboards to a situation where this is real time and more cognitive.

Big data brings in all channels of data to us. But some right analytics and its approach decides which you need primarily and where you should put you focus in terms of investment and labor. 

4. Can we observe in real time or when required patterns of reactions from interested subject groups in market?

There was a very famous case of failure where a so called automated at home vacuum packaged product sponsored a latest movie on television. After every 10-15 minutes there was an advertisement for the same. But still it failed to take off and failed very badly. Possibly, this was a wrong understanding of prospective market in India only, where people do not mostly eat same thing round the year. We normally change food varieties with season change and even based on religious events. So, this vacuum food packaging device (also coming for a fortune) was not at all anywhere an option for people at that point of time.

These blunders do happen, but most of the time these mistakes are not so frequent. A very calculated strategy, market research and launch is planned. At times even wrapper colors, bottle designs, timing of launch etc. are all planned and executed in very detail. But the fact is, everyone does so.

There is still space for similar products in market based on some other favorable attribute for that geography or demography.

It is very important to not only be able to sell products. But also, to be able to maintain customer loyalty. Such is the power that even car manufacturing companies now see themselves in service area, enhancing customer and brand loyalty. Selling a product, having a success hit for product is not only important consideration. Maintain loyalty, keeping an eye in real time (as much possible) to see any misinformation, loyalty issue or confusion is extremely important.

There was news of a prominent milk and dairy product company. They were able to respond in almost real time back to a dissatisfied customer on social media and even within 24 hours reach out to the individual to prove their point. This sounds simple. But think more than 125 million people are using that particular social site. If an average even there are just two comments or likes from a single user per day, it goes up to minimum 250 million such comments or likes per day. Considering frequency and deep connect people have with social sites, this actual number might be extremely high then what we estimated just now. In these millions of records being generated from one such site and then others as well, some normal retail user who buys one product in some remote part of India and then writes a comment. Being able to respond almost in real time to that customer is power of right deep analytics on big data.

This is the power of data insight and real time reputation management which could be achieved by strategically provisioned deep analytics. If the company in above example might just have focused on acquiring all data and tapping all information only and not having planned a strategic analytical solution to have right view on rightly prioritized data channel this would not have been a possibility. 

5. Can we evolve into altogether new processes wiping out the previous one used?

This point we have almost covered in point 2 of this write up.  This talks about understanding the ecology and related functionalities of an organization. This helps in cutting unnecessary flab and unproductive as much possible. This could fit very well even a focused area like manufacturing line and warehouse management or in goods supply chain to optimize them.

At industry level as well, this lead to new processes to work and help loyal customers. There is an example of a UK based power supply company. They empowered with sensor data and customer details were able to help their customers to minimize their power consumption to certain percentage per month. This sounds bad and as revenue loss but then this “working towards customer satisfaction and trust built up” helped them gain majority of customers.

Analytics used in right way could help optimize, cut off or create new dimension processes. 

6. Can we have a neural system which can learn and upgrade itself with time?

Si-Fi movies talk a lot about systems that can learn and have artificial learning capabilities.

This is not new in real life as well. We have lot of such devices and applications. This is known as Cognitive computing.

Organizations (commercial and non-commercial) can also have such facilities running using big data and analytics. Classic example would be Google itself.

There are other examples of machine learning e.g. UPS route map program runs on top of an analytical model running on huge operation, logistics and various inbound external data sets. It not only plans efficiently but learns the major issues also happened during any such previous scenario. Next time in similar situations this new item is also added to checklist. 

7. Can we track in multi dimensions every penny of our marketing and cost with transparency?

With analytics on top of data inputs from various sources in parallel this is already possible. The marketing plan, execution and after back feedback analysis could be done at any possible granularity. This so called campaign management data could be even mixed with other types of structured or unstructured data to get some unique results.

This gets a good control over marketing cost and overall campaign management.

These are some example statements made and discussed above.  But there could be millions of ways and use cases to be able to use it.

Key is there should be a balanced strategy running behind between what Analytics needs to be done and all possible data areas possible to be explored.

Planning a solution to explore all possible datasets to have a shallow analytical solution is creating lot of dissatisfaction with organizations going for it. Also, it is to be noted that this is a solution which needs to evolve with new use cases with time for any organizations.

Analytics and Cognitive need an organizational strategic initiative rather than just technical hand. Both working hand in hand miracles can happen and sky is the limit.

About the Author

Rohit Kumar has a Masters in Computer Science including Artificial Intelligence and Business Intelligence.  He has been working as Sr. Enterprise Architect into Business Intelligence Data, Business.  He has experience of consulting 30+ clients across globe in multiple industry verticals towards IT Transformation. Have experience with customers in various industries -  e.g. FMCG, RetArtificial Intelligence, Pharmaceutical, Telecommunication, Electronic, Education, Manufacturing, Healthcare, Logistics, Utilities, Banking, Real Estate, Artificial Intelligence, E-Commerce, Publishing.  Rohit also serves as guest lecturer for faculty development and PHD scholars for various universities while creating an extensive learning programs in SAP and Analytics for various organisations.

This blog post was contributed by Rohit Kumar, the author of “Machine Learning and Cognition in Enterprises: Business Intelligence Transformed”.