Healthcare Data Kit

How Healthcare Organizations Successfully Manage Growing Data Challenges

At a glance

The amount of data created over the next three years will be more than the data created over the past 30 years, and the world will create more than three times the data over the next five years than it did in the previous five. IDC’s Global DataSphere Forecast shows that more than 59 zettabytes (ZB) of data will be created, captured, copied, and consumed in the world in 2020.

What then, should healthcare providers do to keep up with the ongoing explosion in healthcare data, so that they can successfully return their focus back to patient care?

Find out how modern, global healthcare providers are successfully integrating their growing datasets in a unified data model, monitoring KPIs, performing in-depth analysis, collecting insights from their data and returning the ultimate focus back to patient care with data analytics.


As stated in ScienceDirect, the amount of health-care data available was expected to reach roughly 25,000 petabytes by 2020.

Although the health-care industry has lagged behind sectors like retail and banking in the use of big data, it is expected to catch up soon due to recent innovations with patient confidentiality and data security. Healthcare businesses who have already established ways to analyze big data are already achieving positive results, which is prompting other stakeholders to take action.

What then, should healthcare providers do to keep up with the ongoing explosion in healthcare data, capture it’s full potential for decision making, and return their focus back to patient care?

Whether you’re just exploring the options for managing your organization’s expanding data environment or you’ve been on the fence about your organization’s need for a BI tool, this whitepaper serves as your trusty handbook for navigating business analytics for healthcare.

Find out how modern, global healthcare providers are successfully integrating their growing datasets in a unified data model, monitoring KPIs, performing in-depth analysis, collecting insights from their data and returning the ultimate focus back to patient care with data analytics.

Tackling complex data challenges in healthcare analytics

In recent years, healthcare organizations have grown increasingly aware of the potential beneficiary uses of the data which they generate and gather. From general hospitals to community clinics, data and analytics can be a powerful tool to improve care, increase patient satisfaction and drive operational efficiency.

However, adopting data-driven business and medical processes into the daily workflow of medical and administrative staff poses its own set of unique challenges. Among these, one of the major challenges is posed by data complexity– i.e., the fact that healthcare organizations must try to make heads and tails of large amounts of data originating from many disparate data sources.

Modern data analytics tools and technologies can help these organizations analyze complex data use to benefit institutions as well as their patients. However, selecting and implementing
these solutions requires one to understand the underlying data and processes in place.

This whitepaper will present the challenge of complex data in medical organizations, and present the ways in which modern technology can solve these problems.

What makes data complex?

First, let’s frame the concept of complex data. In the context of business analytics, there are two key drivers of data complexity:

  • The size of the data (is it millions of rows, 100’s of millions, or billion+)?
  • The number of disparate data sources (or data tables).

These elements drive complexity because the bigger the data, the more effort (cost) needed to query and store it. The more data sources (data tables) the more effort (cost) that is needed to prepare the data for analysis.

The data complexity matrix describes data from both of these aspects. When considering analytical solutions, different approaches are better suited for each data state.

Data complexity in healthcare

To understand why healthcare organizations are often dealing with complex data, let’s look at some of the common data sources they might typically be looking to analyze.

Common Healthcare Data Sources:

  • EMR (electronic medical records): these are essentially a digital version of the patient’s paper chart, used by clinicians to monitor the patient’s condition, treatments he or she is due for, etc.
  • EHR (electronic health records): a broader set of digital records pertaining to the patient’s overall health, including information regarding previous treatment administered by other healthcare providers, specialists, laboratory tests and more.
  • Patient monitoring systems: Equipment that monitors heart rates, blood pressure and oxygen rates, measures breathing rates, heart and brain function and many other functions—all use sensors that deliver unstructured data for notification.
  • Laboratory systems. Various information regarding the current condition of a patient can be represented with lab procedures. Structured data (such as iron levels in the blood) and unstructured information (lab slides) are produced in the lab and added to the EHR system.
  • Imaging systems: Imaging devices visualize the condition and sometimes the function of patients’ organs. They produce some of the most data-dense information in medicine.
  • Wave-form processors. This type of processing analyzes waves—heartbeats (EKG), brain function (EEGs) and blood volume surging through organs. This data-intensive processing adds more unstructured data to patient records.
  • Operations support systems. The many operations tasks that make lab tests, EKGs, and physical therapy possible.
  • Administrative data collected in healthcare management systems (HMS), which looks at the hospital’s overall operations and could include information regarding matters such as resource utilization and human resources.
  • Financial data, often stored in proprietary financial management systems for larger organizations.

This (partial) list should give some idea of the astounding amounts and disparity of data healthcare providers need to examine. This is compounded by the need to access this data rapidly, and furthermore the need to combine two or more datasets from multiple sources – e.g., to find correlations in clinical and financial systems.

These factors contribute to the high level of data complexity found in most modern healthcare services.

The path to insights becomes murky

It takes a long time to prepare because of data modeling challenges and the need to indexing and aggregate larger datasets before they can be queried for analytical purposes. Specialized skills and resources are needed throughout the analytical process, turning any project into a cross department, lengthy effort. This manpower cost is amplified each time changes are introduced to the data model order to investigate new analytic paths. Often additional third party tools are required as well in order to alleviate some of the pains associated with this process.

Complex datasets of the type described above require special attention in both the data preparation process and in managing the size of the data.

As a result analytical systems often comes with a very high total cost ofownership for modern healthcare services dealing with complex data. License fees are required for analytical tools, as well additional data warehouse and data preparation tools. Further costs are incurred by the overhead and specialized skills needed to integrate and maintain multiple tools from different providers that may or may not work effectively together.

A technological solution for complex data problems

Sisense provides healthcare organizations with business analytics software that was built specifically to simplify the analytical process for complex data, optimizing speed to handle larger datasets and unifying the process of analyzing data, thereby eliminating many steps in data preparation and easing the burden of preparing complex data for analysis in healthcare and other organizations.

This is achieved in large part thanks to two core technologies: In-Chip® technology, optimized for processing speed by leveraging modern CPU technology. This means the system can process 100’s of millions of rows using low cost, commodity hardware. A Single-Stack™ architecture fuses all the elements of a healthcare analytics solution, from data preparation to data management, querying and visualization, into a single, efficient software solution.

The combination of these two core technologies solves the bulk of the problems associated with complex data

Healthcare organizations use Sisense technology to combine data from the any disparate systems they work with and create operational dashboards which medical and administrative staff can consult for immediate answers to questions that arise in their day-to-day work, while healthcare analysts can dive deeper into the data to reach new insights and suggest data-driven courses of action – without the above described pains which are typically associated with complex data.

Key points for successfully implementing analytical solutions

While having, the right technology is useful, successful implementation of any new analytical tool within healthcare organizations is also a matter of adopting internal culture to turn data and analytics into part of the everyday workflow. Below are some factors to consider.

Executive and stakeholder buy-in

Adoption rates of analytical tools are often low due to a lack of confidence in BI tools and methods. Successful business analytics projects require making your hospital or facility a data-driven organization, and to do so the initiative must either come from the top, or at the very least enjoy the support of the organization’s leaders. It’s not enough to have the tools in place – for people to use them they need to understand the value and potential of analyzing and visualizing healthcare data, and this could require a cultural change that must be facilitated by upper management.

An effective data analytics team

One should not underestimate the importance of the human element: create a skilled and effective group of people who know their data and want to use it to improve decision-making processes, and who enjoy digging into the data to find new insights.

This means starting with people who have empathy for patients and excellent communications and management skills – but who are also passionate about data and how it can save time, money and lives.

Understanding the project’s goals

Given the availability of self-service healthcare analytics tools, it’s easy to take very deep dives into data that provide information but very little value. Business intelligence tools have varied capabilities, so the organization should define its objectives and medical or business questions it wants answered before spending time and effort generating reports. If you don’t know what you’re asking, you won’t get answers to questions that matter.

Broad access to data

While it’s important to have a strong analytical team to manage data modeling and governance, it’s just as crucial to give the non-technical easy access to data and allow them to draw their own insights from it. Broad usage of data by medical and operational staff, without these professionals having to pass through technical gatekeepers for every report they want to view, enables healthcare providers to truly maximize the value they derive from implementing analytical tools.

Case study – Health services agency delivers performance dashboards

Company overview

Res Consortium is a Management Consulting Organization built to improve performance of healthcare providers in the UK. By using Sisense as an embedded analytics solution as well as their own internal data analysis methodologies, Res Consortium delivers performance dashboards that allow customers to quickly and easily analyze and visualize big healthcare data. With three divisions that offer training and management skills, business systems such as Sisense, and a consultancy to interpret insights, Res Consortium is changing the way people approach healthcare performance.


Over the last 15 years, the National Healthcare Service (NHS) of the UK increased their spending from 70 billion British pounds to 150 billion British pounds. In order to improve their efficiency and cut costs, the NHS created dedicated internal organizations to measure performance of organizations against each other, and to publish and distribute the data in performance reports.

The problem was, the healthcare data was very complex— big, scattered and siloed– limiting the reports to focus on one area of measurement or one organization per report, and were published as static PDF documents.

Users were unable to compare to other organizations or integrate across different sets of data, giving them an isolated view of their performance. Data regarding each area, such as clinical performance, cost performance, and staff and patient surveys were reported in separate and heavy 50-page plus documents that required time and research to see a bigger picture across platform.

That’s where Res Consortium came in with the goal of providing performance dashboards that showed the data across platform in an intuitive way. In the past Res Consortium was producing dashboards using Excel with protection keys to protect sensitive patient data, but they started looking for a BI platform that could move the dashboards to a web-based environment as well as to more efficiently and quickly handle the amount of complex data typical in healthcare. With a background in engineering, Mike Stedman, the Operations Director at Res Consortium, understood the value in visualizing all data in an interactive way so that organizations could not only measure their performance, but understand problem areas.

If you want to change behavior, you need to talk about why before how. We use data visualization with Sisense’s performance dashboards to allow users to drill down into problem areas and compare to other organizations and see the ‘why’—thereby changing the narrative. Then, people are inspired and able to seek ‘how’ others perform better.

Mike Stedman, Operations Director, Res Consortium

While organizations within the NHS are using Res Consortium to try and identify problems areas and improve efficiency, typical clients of Res Consortium are also pharmaceutical companies that are trying to access the NHS by building and visualizing a story around a particular disease, therapy or medicine they are interested in improving: “Sisense allows us to monetize the ability to make data into a greater narrative in order to commercialize it and tell a specific story,” explained Mike Stedman.


After undergoing a Sisense evaluation, Res Consortium saw that Sisense would allow them to easily and quickly incorporate relevant public data, no matter how big or siloed, and mash it up with other datasets to create a useful, cross organization dashboard.

While the NHS provides a 60-page PDF document report with results, it is unengaging and has low readership. Res Consortium used the same data to create a Sisense dashboard that spanned over a number of years and that ranked over 8,000 practices and hospitals based on performance in over a dozen different areas with Sisense’s mapping function. Users can drill down into any hospital with such granularity, they are able to see various areas where performance is stronger or weaker, and even choose a prefered hospital or practice based on the results.

Users can independently analyze and uncover insights.

Sisense lets us take data and make it useful, dynamic and interesting. So, instead of just seeing their own data, users such as a Director of Workforce, can compare performance and can create benchmarks.

Mike Stedman, Operations Director, Res Consortium

Another benefit of Sisense is how easy it is to share dashboards within an organization and how intuitive the dashboards are—allowing users to independently analyze and uncover insights. “Sisense is an engagement and diagnostic tool. We use it to engage people in our organization, understand what the challenges are, and quickly see why something has gone up or down without spending hours on research. Sisense makes data current, accessible, and dynamic, and we can get a dashboard up and running and published in no time,” said Mike. Though patient records are confidential, Res Consortium deploys a protective non-public facing server with anonymized patient information and uses Sisense data governance to manage users and viewers.

Res Consortium also uses Sisense to track and monitor the usage of the dashboards in their own systems. By using Sisense internally, Res Consortium can better understand and monetize their clients’ usage by measuring how frequently users are accessing dashboards.


Mike explains how easy it is to inspire change once relevant information is quantified and relatable. “With Sisense dashboards, we show the context of how others are doing compared to you and in which areas, and that inspires healthcare providers to want to know why and how,” said Mike.

Sisense allows healthcare companies to frame the debate by showing users what’s important to them, in an easy to understand way.

The chief executive of Tees, Esk & Wear Valley NHS Trust started using Res Consortium’s dashboards as a mechanism to isolate challenges and highlight successes, which has served as a powerful springboard for discussing performance improvement with his board of directors. For example, this Tees, Esk & Wear Valley dashboard reveals that the organization is excelling in mostly all areas, except staff safety and abuse by patients and staff, allowing Tees, Esk & Wear Valley to identify and direct efforts on the problem area:

The ability to use Sisense to see beyond the data to the underlying correlation and causality, where the strengths and weaknesses lie, is incredibly empowering—especially in the healthcare industry where we treat individual patients
with varying personal needs.

Mike Stedman, Operations Director, Res Consortium

The most powerful example of a healthcare insight uncovered using Res Consortium occurred when a pharmaceutical company that specialized in Parkinson’s Disease approached Res Consortium. While Parkinson’s Disease is initially managed using oral tablets, as the disease progresses, patients experience rigidity and have trouble swallowing medication. It is only during those final years of care that doctors begin to prescribe patients medication in the form of patches or injections–which is significantly more expensive than oral tablets.

Yet, this particular pharmaceutical company wanted to prove that it was actually more cost effective to use the more expensive patches and injections earlier on in the disease, because the patient did not require as much hospitalization in the long run–benefiting both the patient and the healthcare system.

Res Consortium discovered this insight by analyzing hospitalization records and over 700 million lines of prescribing data in Sisense ElastiCube.

“By using the fast speed Sisense ElastiCube we were able to drill down into data at practice level, of over 8,000 practices, to see what doctors are prescribing and how much hospitalization their patients need,” Mike explained. After proving that the cost of one offsets the cost of another, their client is currently publishing a scientific paper based on the research Res Consortium provided.


Res Consortium is changing the way people understand healthcare performance–both by inspiring behavioral improvement and by helping the healthcare system be more efficient with budget and patient care. Using Sisense as an embedded analytics solution and as their own internal data analysis methodologies, Res Consortium is able to quickly turn heavy reports into interactive, visual dashboards, allowing their customers to easily analyze complex healthcare data and change the narrative across the entire healthcare industry.

Healthcare analytics with Sisense BI software

Sisense gives hospitals and other healthcare organizations a powerful, end-to-end business analytics solution to prepare, analyze and visualize complex data.

Simplifying business analytics for healthcare

Sisense makes it simple to explore big, unstructured healthcare data from multiple sources. Now you can centralize all your datasets in a unified data model, monitor KPIs, perform in-depth analysis and implement data-driven strategies – all using an easy, intuitive tool.

Improve care, monitor performance, reduce costs

Provide better service to patients or clients by closely monitoring multiple metrics and KPIs, with medical, financial and administrative data that’s always on time and accurate. Create automatically updated dashboards that give you a 360° view of your business, or drill-down into the most granular details to track individual performance. Identify cost factors to boost operational efficiency with highly effective healthcare business intelligence.

Single-stack™ solution for complex data analytics without IT

Sisense is a complete healthcare analytics solution with everything you need to crunch complex datasets: strong ETL and data modeling capabilities, advanced analytics and statistics, and an intuitive UI for creating dashboards, reports and visualizations. Our unique technology enables Sisense to rapidly process both big and disparate datasets without clusters, data warehouses, or intensive IT efforts.

Integrate many disparate systems: ERP, HMS, billing and more

Healthcare data comes in many sizes, shapes and forms. Sisense is the perfect tool to combine many structured and unstructured data sources and create a centralized data repository and single-source of truth across the organization. And since the Sisense data engine can join tables on the fly, the data preparation effort is dramatically reduced.

Trusted by leading healthcare providers

We wanted to maintain [data] governance so that the business actually has the right information.

Jayesh Vaswani, Brightstar Care

Sisense has a proven track-record of empowering indsutry-leading companies and healthcare organizations in the United States, Australia, Europe and Asia. We are dedicated to fast, successful BI deployment and development, and are willing to prove it by providing a free proof-of-concept on your data – in just 90 minutes.

Your data is safe with us

Sisense business intelligence software can be deployed on premise or in the cloud, and enables administrators to set granular permissions on user, database and row level to ensure data stays in the right hands. Keep sensitive information safe by using custom tables for data cleansing and transformation, without altering your production data.

Streamline regulatory and operational reporting

Generate compliance or other reports instantly based on the most up-to-date and consistent data. Create and share interactive BI dashboards across the organization to keep management, physicians and staff informed. Track KPIs to constantly improve both business and medical processes.

Start a free trial