Take the Value of Data to New Heights: The Shift to Cloud Analytics With AWS
Executive summary
Business leaders already recognize the growing importance of data and analytics. 83% say that big data projects give their organizations a competitive advantage and 97% of businesses are investing in digital transformation to harness the power of big data and BI.
An increasingly vital part of such a transformation is shifting computing power, storage databases, machine learning
At Sisense, our research shows that even in the toughest times, such as a global crisis like the 2020 COVID-19 pandemic, data and analytics are increasingly critical for organizations looking to successfully navigate change, identify opportunities, innovate, and overcome new challenges. Since the start of the pandemic, customers and prospects who report using data to address this new slate of aims are also the most likely to mention considering moving to the Cloud.35% who started embedding analytics into their products are likely or very likely to move to the Cloud. 31% who are using analytics to cut costs are likely or very likely to move to the cloud. And 31% who started tracking their sales funnel with analytics are likely or very likely to move to the Cloud.
According to Markets and Markets, the global Cloud computing market is forecasted to exceed US$350 billion in 2020, and it is predicted that 80% of organizations will migrate towards cloud, hosting, and colocation services by 2025.
We expect this trend of shifting to the Cloud to accelerate. In this whitepaper, we examine why, and we consider:
- The challenges organizations face as data volumes continue to grow
- How the Cloud meets this challenge: benefits of shifting to the Cloud
- How Sisense on the Cloud offers a powerful solution to the data challenge
- Partnering with cloud service providers: Sisense and AWS
- How shifting to the Cloud with Sisense and AWS helps organizations grow, innovate, and improve
The challenge: Embracing the growth in data with the shift to the Cloud
The figures from the previously quoted research, indicate that organizations of all stripes believe in data democratization, considering it vital for data to be available to the widest group of users, so more people can analyze it, gain insights from it, and use it to shape the future of their organizations.
To achieve these aims, it is essential that users be able to get the data they need at scale and as soon as possible, ideally in real-time, in a way that fits with their specific requirements. The volume of data is now so large that traditional IT-driven solutions are no longer fast, agile, versatile, or flexible enough to handle such scale and complexity.
Legacy on-premises architectures are limited in terms of capacity. As data size and complexity increase exponentially, companies must repeatedly increase their fixed capital investment for on-premises computing and storage power to support larger data warehouses, which is expensive. And, as more users request access to more structured data to perform their own analytics from the data warehouse, even more hardware and software licenses are required to ensure satisfactory performance for business users, further spiraling costs.
Then, there’s the issue of speed and efficiency.The explosion of unstructured data requires IT and database specialists who can optimize the data into a structured format so that it can be queried for analysis. It takes considerable time and resources to go through this process, leading to capacity planning challenges that can significantly slow down the time to insights. In an environment where fast insights, speed of response, and the ability to pivot quickly are vital, this kind of problem can be a serious impediment to success.
Moreover, making ad hoc and self-service analytics available to all requires an architecture that is more open and easier to access, without the need for technical experts. This is what the Cloud offers. Pairing a BI and analytics platform like Sisense with cloud services, enables customers to simplify complex data analytics and democratize access to their data.
Our partnership with AWS exemplifies how these positive synergies play out. Pairing Sisense with AWS enables customers to derive more value from data, build data products that increase adoption throughout their organizations, drive innovation, and deliver insights speedily and at scale.
The pressure is on for organizations to maintain their competitive edge by gleaning insights from as much data as possible at their disposal, which goes beyond analyzing performance to include predicting outcomes and anticipating potential opportunities. Analytics augmented with AI and machine learning models - something Sisense is a leader in - are at the vanguard of this trend, working seamlessly with the capacity, power, and flexibility that the Cloud provides to give customers the best of both worlds.
How the Cloud is the answer: the benefits of shifting to the Cloud
Having reviewed the Cloud’s impact on organizations, it is valuable to consider why and how it benefits users, and what makes the shift to the Cloud so advantageous. These benefits go hand-in-hand and create a virtuous circle of cause and effect within the BI and data analytics process.
Speed
First, speed is a significant factor. We have learned how moving analytics to the
Not only does speed give organizations a competitive edge in these challenging times, it can be vitally important for addressing serious issues such as data security breaches or safeguarding public health. Getting greater and faster access to data in real-time gives organizations the capability to learn and respond to events much more quickly than with legacy systems alone. The response to the COVID-19 pandemic is a significant example. Experts such as George Thiruvathukal, professor of computer science at Loyola University in Chicago, have stated that the success of COVID-19 tracking and tracing efforts depends on fast access to multiple data sources.
Organizations need to move at lightning speed and with an agility that pre-built systems and traditional IT methods struggle to support. To address this challenge and meet this need, Cloud is the preferred deployment channel for new analytics use cases.
Capacity and power
Cloud-based data warehouses and data lakes provide the best option for storing data, both structured or unstructured, at a minimal cost. Queries using SQL can seamlessly interact with cloud data warehouses and data lakes, allowing for a comprehensive end-to-end view into all types of data on the fly. The capacity of cloud-based tools also allows customers to easily scale the amount of storage in line with their demand.
Switching to the cloud delivers technology services and resources, such as computing power, storage
Lower costs
Shifting data and analytics to the Cloud obviates the need for users to maintain and upgrade hardware and software on-premises. Cloud service providers are responsible for the infrastructure. It is not a cost that the user has to bear. Neither does the user need to fund IT and personnel costs at scale to manage hardware and software necessary to handle the growing demand for capacity and power.
With this kind of maintenance undertaken by the cloud service providers, existing IT teams are freed from infrastructural responsibilities. They can focus instead on activities that add more value to their organizations.
Modernizing data strategies goes beyond deciding how data is stored and queried for lower total cost of ownership. A great example of this is Cloud-native Sisense on Linux, which is purpose-built on a containerized microservices architecture that captures the biggest benefit of scale, reliability, and delivery at the lowest overall TCO.
In short, the cost implication of shifting to the Cloud is lower TCO after the initial investment in the migration itself. According to Forrester, businesses note on average a 30% decrease in Cloud spend for initial implementation costs and ongoing savings of 15%, plus other “soft savings” such as operational savings, cross-cloud comparisons for migration and portability, and better usage of time for cloud-savvy employees.
Scalability, flexibility, and agility
Organizations’ data needs can change, depending on a varying combination of conditions that can alter at any given time. The COVID-19 pandemic is an extreme example of this because it has rapidly affected almost all organizations’ requirements, priorities, and objectives. Among infrastructure decision-makers whose firms have implemented public cloud or plan to, 38% say that on-demand capacity and scalability was an important reason for adopting a public cloud strategy.
Therefore, users need to be sure that the cloud services they deploy have the flexibility to scale up or scale down quickly, according to their requirements. They also need to avoid situations which lock them into an agreement with a provider that might not allow them enough capacity or, conversely, leaves them with unnecessary and under-utilized resources. When organizations can identify their requirements clearly and know reasonably well what they need now and in the future, AWS provides very competitive terms. Furthermore, where necessary, AWS Redshift can be calibrated to changing conditions as it offers on-demand pricing with no contracts and upfront fees, based on hourly usage rather than by the query, like Snowflake. So, when working with Amazon Redshift, for example, users can scale their data and infrastructure as they grow, while still enjoying the data exploration they need.
Flexibility extends into other vital areas that might drive a cloud purchase decision, such as the types of data that cloud services can handle. As discussed earlier, the proliferation of available data can be overwhelming and messy, with data generated in many forms. There is structured data from relational databases like spreadsheets and CRM software; and unstructured data, often qualitative, such as text, video, audio, mobile activity, social media activity, IoT generated data, and data from non-relational or non-SQL databases, and more. Cloud services can store these data sources raw in data lakes, and then handle them all in the most agile, on-demand fashion.
Augmentation of data analytics with advanced tools
The capacity, power, and flexibility of cloud services creates an environment in which advanced tools can effectively deliver augmented analytics, and ad hoc capabilities. The Cloud’s comprehensive services are optimized for high-performance computing, so organizations that have data stored and governed in a data lake and a data warehouse on the Cloud can fully take advantage of tools such as machine learning (ML), artificial intelligence (AI) and natural language processing or querying (NLP/Q).
Compared to on-premises deployment, cloud ML models can be trained to start generating predictive models that will drive competitive differentiation and business impact. Cloud providers deliver all types of AI services, such as computer vision, language, recommendations, and forecasting. They provide proprietary support for open-source frameworks to quickly build, train, and deploy machine learning models at scale.
There are hundreds of pre-built models to choose from, and organizations can leverage a broad set of powerful computing options from compute-intensive deep learning to high-memory instances. Or, organizations can build their custom models with support for all the popular open-source frameworks.
Enterprises getting results from their AI strategies and successfully progressing pilots into production are 81% more likely than their peers to have advanced data management capabilities. It is no surprise, then, that 82% of enterprises are increasing their spending on cloud services, according to a report by MIT SMR Connections.
These figures demonstrate that cloud services have become a vital, foundational technologyfor improving AI and machine learning outcomes. So, it’s also no surprise that companies are turning to the major cloud providers that offer strong and versatile solutions to meet organizations’ modern challenges and needs.
How does Sisense Cloud BI and analytics offer a powerful solution for organizations?
Sisense has developed a comprehensive end-to-end data and analytics platform that enables organizations to take full advantage of the new wave of analytics in the Cloud, giving them the ability to surface findings rapidly, take action, and win based on insights from their data. With almost half of all structured enterprise data already stored in public clouds, being able to seamlessly pair Sisense with cloud services is critical for many of our customers.
Based on a finding from Forrester, 47% of global data and analytics technology decision-makers either complement (27%) or have already replaced most or all (20%) of their on-premises BI applications with cloud options.
Using the Sisense Elastic Data Engine, users can analyze and draw correlations from their data, wherever it is from, whether it is on-premises or in the Cloud. Sisense delivers deep insights from all of this data, using native AI capabilities. The platform enables users to connect their data with cloud AutoML services to produce advanced predictions that business leaders can easily consume on Sisense’s interactive and highly intuitive, AI-powered dashboards. Organizations can also rapidly respond by scaling the growth in concurrency, availability, and stability of your data and analytics platform using any cloud hosting, hybrid
Plus, whether data is raw, structured, or unstructured in the Cloud, Sisense can connect to it to deliver insights. Sisense's native connectors deliver the ability to transform data into insights using code-first data discovery capabilities as well as visual drag-and-drop UX to build AI-powered self-service dashboards.
Sisense also supports analytics requirements at any phase of users’ cloud journeys. Users can use Sisense for Cloud Data Teams to rapidly transform and optimize raw data on the Cloud, in order to conduct advanced, ad hoc predictive analytics.
Sisense for Cloud Data Teams provides data teams with the ability to build data pipelines with Amazon Redshift and perform advanced analysis using languages they already know like SQL, Python, and R. Users can leverage the range of AWS products for storage, querying, and data management, like accessing as much historical data as you need on S3 to analyze real-time streams with Kinesis.
Then, users can get answers to complex business questions and surface insights and hard to detect patterns in data using AI with Sisense for BI & Analytics Teams.
And they can turn data into purpose-built analytics apps that can be white-labeled and embedded to create new revenue streams and competitive differentiation using Sisense for Product Teams.
Partnering with cloud services providers: AWS
As previously mentioned, AWS is the world’s largest cloud computing and services provider. A majority of Sisense customers use AWS, so our partnership with AWS enables these customers to combine
Sisense then turns a huge volume of complex data on AWS into interactive dashboards, self-service analytics, and intelligent BI apps, and users can rapidly and easily transform complex data into highly interactive actionable apps that can be embedded and delivered at scale.
Customer examples: How the shift to the Cloud with AWS helps organizations grow, innovate and improve
Working together with AWS to beat the COVID-19 virus
Perhaps the most important use of data and analytics in the Cloud during 2020 has been its role in the fight against the COVID-19 pandemic. G-Med is the largest global online medical community, with over a million verified physician members. This peer-to-peer network was set up to enhance clinical decision-making by crowdsourcing from physicians’ real-world experiences. Much of the information collected and shared by G-Med is unstructured data in the form of text files.
G-Med’s COVID-19 group has seen active participation by physicians from 160 countries and 130 specialties. Over 5,500 updates and items were already posted at the end of May 2020, and this number continues to grow. The ongoing daily discussions among group members range from medication and treatment efficacy, case progress, epidemiological and statistical data, to hospital practices and governmental decisions concerning efforts to address the COVID-19 virus.
So as one can imagine, G-Med faced significant challenges around being able to organize, analyze, and then visualize this rapidly growing and hard-to-consume data in a way that offered vital insights and on their desired timeline.
Sisense and AWS resources paired up as part of a hackathon held earlier this year, to meet this challenge of preparing and deriving meaning from such a big and unstructured collection of data. The result was the creation of a dynamic, agile, and innovative dashboard that responds to new data constantly. It was initially built in just two days.
Combining the capacity of AWS cloud services and Sisense analytics, data is sent through AWS Comprehend Medical via an AI framework, then processed using NLP capabilities to ultimately create an AI-powered, multilayer, and interactive global insights dashboard. The dashboard, now leveraged by G-Med’s global community of physicians, provides machine learning analysis of the organic COVID-19 peer-to-peer-discussions that happen on G-Med’s member site, enabling medical teams, institutes, and governments to search for updates in line with their particular specifications, rapidly gain a new and deeper knowledge of the virus, and enhance their ability to support clinical decisions during the global crisis.
The capacity and power of the cloud-based analytics that G-Med uses, has resulted in the creation of a dashboard that takes COVID19 big data analytics to the next level.
Understanding customers to drive innovation
The audio devices manufacturer Skullcandy uses Sisense advanced analytics, together with AWS cloud services, to understand customer sentiment so that it can predict return rates on new products before they get introduced, inform new product design decisions, and deliver better products. The company deploys a machine learning platform, combined with Sisense BI, to inform users about historical warranty costs, claims, forecasts, historical product attributes, and attributes of new products on its roadmap.
Sisense BI integrates text and sentiment data, using Python and its natural language processing libraries to understand what customers are talking about, and Sisense enables Skullcandy to use AWS Comprehend to understand how its customers feel about its products. In this way Skullcandy’s team is able to collect data from customer reviews and warranty claims to pull out key customer sentiments. The results include, new products getting extra attention from product designers and engineers before going to market, allowing them to respond to customer remarks, meet their requirements, and fix any potential product glitches. Analytics on the Cloud, therefore, becomes a key driver of product innovation.
Pairing Sisense and AWS to maximize the impact of analytics
Finxera provides a platform that gives non-financial institutions a simple, secure way to collect, store, and send money by enabling its customers to integrate banking services into their applications. It uses Sisense and AWS to optimize the analytics and insights from all the data it generates and collects, to enhance its services.
Sisense and AWS have helped us provide real-time analytics by providing insight across all of our data.
Praveer Kumar, Co-founder/CTO of Finxera
He continues, “The Sisense Elastic Data Hub capability has made it easy to have a live view into our data in Amazon Redshift, and the flexibility to add other data sources to have a consolidated view across our data.”
Enhancing access to data, accelerating time to insight, and boosting cost-effectiveness
At GE Cyber Security, the data and analytics team is responsible for driving the company’s digital transformation. Using Sisense and AWS Redshift, the team created a self-service analytics environment that enables users throughout the organization to gain access to faster, more accurate and consistent data, without the bottlenecks caused when taking the traditional approach of waiting for BI analysts to explore the data and produce dashboards. The time it takes to generate dashboards has been slashed from weeks to minutes.
Achieving this outcome involved migrating data workloads from legacy on-premises infrastructure to the Cloud. Now, GE Cyber Security’s data and analytics team hosts the company’s BI environments on AWS and has automated its analytics deployment, enabling it to create completely new Sisense environments at the click of a button.
As a result of this migration to the Cloud, the GE Cyber Security team has been able to save tremendous amounts of resources, time, and money. 80% of manual operational tasks have been eliminated through automated monitoring and remediation. Environment deployment and upgrades now take four hours instead of two days. Data storage costs have fallen by about 83% and the cost of infrastructure to host BI software has decreased by 54%.
Summary
Our world is getting increasingly faster, more complex, and more data-driven. Data is now both structured and unstructured, and is generated in a volume never previously experienced. The challenge for every organization is threefold: (1) gather and organize this mass of data; (2) efficiently analyze and successfully visualize the data to produce game-changing insights; (3) streamline, simplify, optimize and expedite analytics and insights to make them accessible to more users.
Meeting this challenge requires both the capability to analyze the data to derive valuable
As we have seen in this whitepaper, the solution lies in combining advanced analytics with migration to the Cloud. Sisense’s partnership with AWS provides customers with the capability to optimize the use of their data on the Cloud, by enabling them to:
- Analyze and correlate on-premises and cloud data for insights everywhere with Sisense’s Elastic Data Engine
- Rapidly transform and optimize raw data on the Cloud to conduct advanced, ad-hoc predictive analytics with Sisense for Cloud Data Teams
- Deliver answers to complex business questions and surface insights and hard-to-detect patterns in data using AI with Sisense for BI & Analytics Teams
- Turn data into purpose-built analytics apps that can be white-labeled and embedded to create new revenue streams and competitive differentiation using Sisense for Product Teams
- Transform sophisticated, AI predictions generated by data scientists using cloud AutoML services, into easy-to-consume business insights for all users within Sisense’s interactive and highly intuitive, AI-powered dashboards.
- Scale user growth and concurrency, availability and redundancy demands of their data and analytics platform on any cloud with Sisense’s cloud native microservices architecture, SaaS or Managed Service environment.
Sisense is also an AWS Data & Analytics Competency Partner, the highest possible recognition of our analytics technology leadership position. This combination provides the capacity, power, speed, know-how