Attempting to learn more about the role of big data (here taken to datasets of high volume, velocity, and variety) within business intelligence today, can sometimes create more confusion than it alleviates, as vital terms are used interchangeably instead of distinctly. However, when investigating big data from the perspective of computer science research, we happily discover much clearer use of this cluster of confusing concepts.
Before we dive into the topics of big data as a service and analytics applied to same, let’s quickly clarify data analytics using an oft-used application of analytics: Visualization!
Looking at the diagram, we see that Business Intelligence (BI) is a collection of analytical methods applied to big data to surface actionable intelligence by identifying patterns in voluminous data. As we move from right to left in the diagram, from big data to BI, we notice that unstructured data transforms into structured data. The implication is that methods of data analytics are applied to big data, the methods of data preparation and data mining for example, to bring us closer and closer to the goal of distilling useful patterns, knowledge, and intelligence that can drive actions in the right hands.
Hopefully this clarifies these complex concepts and their place in the larger analytics process, even though it’s common to see pundits and outlets tout BI or big data as if they were ends in themselves.
AI-driven analytics is a complex field: The bottom line is that datasets of all kinds are rapidly growing, causing these organizations to investigate big data reporting tools or even approach companies whose whole business model can be summed up as “big data as a service” in order to make sense of them. If you’ve got big data, the right analytics platform or third-party big data reporting tools will be vital to helping you derive actionable intelligence from it. And one of the best ways to implement those tools is to embed third party plugins.
Big data challenges and solutions
When you have big data, what you really want is to extract the real value of the intelligence contained within those possibly-zettabytes of would-be information. To best understand how to do this, let’s dig into the challenges of big data and look at a wave of emerging issues.
For starters, the rise of the Internet of Things (IoT) has created immense volumes of new data to be analyzed. IoT sensors on factory floors are constantly streaming data into cloud warehouses and other storage locations.
These rapidly growing datasets present a huge opportunity for companies to glean insights like:
- Machine diagnostics, failure forecasting, optimal maintenance, and automatic repair parts ordering. Intelligence derived from these systems can even be fed to HR teams to improve service staffing, which further feeds to enterprise HR management and performance solutions (AI-based analytics reporting to ERP solutions)
- Assembled products shipped also feed directly to ERP on updating supply chain solutions, improving customer awareness and experience
To put it bluntly, the challenge we face is that no cloud architecture yet exists which can accommodate and process this big data tsunami. How can we make sense of the data wont fit in the enterprise service bus (ESB)? (ESB is a middleware component of cloud systems which will be overwhelmed if a million factories were to all try to extract intelligence from their sensors all at once.)
One solution with immense potential is ”edge computing.” Referring to the conceptual “edge” of the network, the basic idea is to perform machine learning (ML) analytics at the data source rather than sending the sensor data to a cloud app for processing. Edge computing analytics (like the kind platforms like Sisense can perform) generate actionable insights at the point of data creation (the IoT device/sensor) rather than collecting the data, sending it elsewhere for analysis, then transmitting surfaced intelligence into embedded analytics solutions (eg. displaying BI insights for human users).
The pressure to adopt the edge computing paradigm increases with the number of sensors pouring out data. Edge computing solutions in conjunction with a robust business intelligence big data program (bolstered by an AI-empowered analytics platform) are a huge step forward for companies dealing with these immense amounts of fast-moving and remote data.
Big data analytics case study: SkullCandy
SkullCandy, a constant innovator in the headset and earbud space, leverages its big data stores of customer data regarding reviews and warranties to improve its products over time. In a twist on typical analytics, SkullCandy uses Sisense and other data utilities to dig through mountains of customer feedback, which is all text data. This is an improvement over previous processes, wherein SkullCandy focused on more straightforward performance forecasting with transactional analysis.
Now that SkullCandy has established itself as a data driven company, they are experimenting with additional text analytics that can extract insights from reviews of their products on Amazon, BestBuy, and their own site. Teams also use text analytics to benchmark their performance against their competitors.
SkullCandy’s big data journey began by building a data warehouse to aggregate their transaction data, reviews. A breakthrough insight/intelligence in product development occurred thanks to the text analysis of warranties through which SkullCandy was able to distinguish between product issues and customer education. The fact that AI-based analytics can delineate between product and education in a text message is groundbreaking. A common pattern was that clients were returning a product as broken when in fact they simply didn’t know how to use bluetooth connectivity.
Data-driven product development also benefitted: Big data analytics allowed SkullCandy to analyze warranty/return data that showed that one of their headsets, which was used more during workouts than previously thought, was being returned at a higher than normal rate. It turned out that sweat was causing corrosion in terminals, leading to the returns. The outcome was to waterproof the product.
Among the many successes SkullCandy achieved, we also see a pattern of value derived from big data.
Big Data as a Service: Empowering users, saving resources
Strictly speaking, “big data analytics” distinguishes itself as the large-scale analysis of fast-moving, complex data. Implicit in this distinction is that big data analytics ingests expansive datasets far beyond the volume of conventional databases, in essence combining advanced analytics with the contents of immense data warehouses or lakes.
In order to get a handle on these huge amounts of possible-information, the AI components of a big data analytics program must necessarily include procedures for inspecting, cleaning, preparing, and transforming data in order to create an optimal data model that will facilitate the discovery of actionable intelligence, identify patterns, suggesting next steps, and supporting decision making at key junctures.
Intelligence drawn from big data has real potential to transform the world, from text analysis that reveals customer service issues and product development potential to training financial models to detect fraud or medical systems to detect cancer cells. Savvy businesses will empower users, analysts, and data engineers to prepare and analyze terabyte-scale data from multiple sources — without any additional software, technology, or specialized staff.
Fortunately, it is now possible to leverage all of these potentials and to avoid the cost and time of in-house development, by embedding expert third party analytics. Recognizing the tremendous task of big data analytics in conjunction with the value of outcomes, the natural propensity exists to use it as a service, and thereby reap the benefits of big data as a service as quickly as possible.
Chris Meier is a Manager of Analytics Engineering for Sisense and boasts 8 years in the data and analytics field, having worked at Ernst & Young and Soldsie. He’s passionate about building modern data stacks that unlock transformational insights for businesses.