As COVID-19 continues to spread, healthcare groups and companies of all kinds are under pressure to provide care in the face of increasing demand. Healthy Data is your window into how data can help organizations address this crisis.



COVID-19 required a worldwide coordinated response of medical professionals, data teams, logistics organizations, and a whole host of other experts to try to flatten the curve, improve treatments, and ultimately develop lasting remedies. Sisense has been at the forefront of the data-based response to this global crisis.



March saw the beginning of Project SiCo: The Sisense effort to make COVID-19 data more accessible and consumer-friendly. Dozens of Sisensers took part in project SiCo to create <a aria-label=" (opens in a new tab)" href="https://covid.sisense.com/#" target="_blank" rel="noreferrer noopener">this awesome COVID hub</a>. This article will focus on the AI Research (AIR) team’s effort, specifically an experimental combination of Sisense BloX (actionable <a aria-label=" (opens in a new tab)" href="https://www.sisense.com/blog/five-design-tips-to-deliver-a-seamless-embedded-analytics-experience/" target="_blank" rel="noreferrer noopener">embedded analytics</a>) and Quest (an advanced analytics add-on for Sisense) which we called the SEIR app.



Embed analytics in your products for competitive advantage: &gt;&gt; Read the guide on how to deliver innovative customer experiences with analytics



<a class="action-btn " href="https://www.sisense.com/whitepapers/stand-out-in-your-markets-how-product-managers-can-deliver-innovative-data-experiences-to-end-users-oem/" target="_blank" rel="noopener noreferrer">Read the Guide</a>



<h2 class="wp-block-heading">Envisioning the SEIR app</h2>



The SEIR app was designed to show the effect that social distancing had on the COVID-19 infection rate based on a model named <a href="https://towardsdatascience.com/social-distancing-to-slow-the-coronavirus-768292f04296" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">SEIR</a>. This model takes into account the population size of a certain country, social distancing rate, and the number of days that passed between the first confirmed case and when the social distancing policy was enacted.



The model outputs the 4 following time series:



<ul><li>S &#8211; The number of Susceptible individuals</li><li>E &#8211; The number of people Exposed to an infected individual and are potential carriers</li><li>I &#8211; Infected individuals</li><li>R &#8211; Recovered individuals</li></ul>



Each array contains a list of numbers that can be plotted as a daily time series.



The app was designed to enable users to choose different countries and make adjustments to the social distancing measures, and immediately see the corresponding Exposed and Infected curves plotted on the screen.



<figure class="wp-block-image size-full fancybox"><img decoding="async" src="https://cdn.sisense.com/wp-content/uploads/seir-2.png" alt="SEIR" class="wp-image-101631"/></figure>



<h2 class="wp-block-heading">Calling on Quest to handle complex calculations with ease</h2>



The large number of parameter combinations made it very difficult to precalculate all of the options and feed them to the Elasticube. The better option was to perform the model’s calculations only once the parameters were set. These on-the-fly Python model calculations are uniquely suited for Sisense’s Quest advanced analytics add-on.



All we had to do was upload the Python file as a Quest custom model and create a lambda function. The function would take care of all the SEIR calculations for us when called. Each Lambda function can be referenced through Quest’s in-product API.



However, there were two main challenges with using Quest alone:



<ol><li>The Input: The only dynamic parameter that Quest sends custom models comes from the widget’s current data, which made it difficult to send adjustable parameters such as the social distancing rate or the number of days since it began.</li><li>The Output: Quest can handle the results returned from models in several ways (e.g.&nbsp; showing those results as a table, plotting them as a new line on the chart, highlighting an existing point etc.). While it is able to add an additional series to a chart, it can’t handle two series for a single response, which made creating both the Exposed and the Infected plots very difficult.</li></ol>



<h2 class="wp-block-heading">Creating a custom interface with Sisense BloX</h2>



While the Quest runModel API was ideal for executing the SEIR custom model script, we needed an interface that would allow users to make parameter adjustments, execute the runModel API, and then plot the result on the chart.



What we needed was a widget type with sliders and input fields that could execute custom Javascript functionality on demand. Sisense BloX’s input and slider components were uniquely well-suited for accepting user inputs. Its custom actions could then use that information, and more, to communicate with API’s and manipulate the chart’s query result.



<h2 class="wp-block-heading">Technical deep-dive: Bringing SEIR to life</h2>



(Author’s note: The following section uses highly technical language and will be best understood by readers <a href="https://www.sisense.com/blog/a-guide-to-the-best-javascript-visualization-libraries/" target="_blank" rel="noreferrer noopener" aria-label=" (opens in a new tab)">fluent in Javascript</a>.)



<h3 class="wp-block-heading">Step 1: Creating the SEIR Custom Model</h3>



We started by facilitating Sisense Quest’s custom models capabilities in order to create an operational Lambda function. Sisense Quest’s most unique capability is probably the custom model mechanism.



Aside from the several native models that come out of the box, builders may define their own models using Python files.



Once uploaded, Quest generates an AWS Lambda function based on those Python scripts, which we can summon whenever the model is called. That communication is done via an internal API endpoint defined in the Sisense Quest microservice.



For this project, we began with a Python file containing the algorithm we’d like to use for our model. This file was created and adjusted by one of our amazing data scientists, though its content is fairly simple. It extracts our adjustable parameters out of the Lambda’s “Event” attribute and calculates the SEIR series based on them. 



Once the Python was set up how we wanted it, we created our custom Sisense Quest model. The visualization type and structure definitions were not important since we were not activating the model from Quest’s UI.&nbsp;



Before finishing the upload, we needed to open our browser’s developer tools, specifically the “network” tab:



<figure class="wp-block-image size-full fancybox"><img decoding="async" src="https://cdn.sisense.com/wp-content/uploads/model_id.png" alt="" class="wp-image-101618"/></figure>



Clicking “done” caused the browser to send a model creation request to the Quest service.&nbsp;



The response to that request included our new model ID.



<figure class="wp-block-image size-full fancybox"><img decoding="async" src="https://cdn.sisense.com/wp-content/uploads/image-33.png" alt="" class="wp-image-101698"/></figure>



Since our plan was to execute the model through Sisense Quest’s API, this ID was our way of identifying the relevant model whenever we wanted to run it.



(If you’re trying to replicate this process, now’s the time when you copy it and save it for later use. For more information about custom models and the best practices of their creation visit <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://documentation.sisense.com/latest/labs/quest/custommodel.htm" target="_blank">our Sisense Quest documentation</a>.)



<h3 class="wp-block-heading">Step 2: Creating the custom action</h3>



In this step, we utilized the model we’ve just created via a Sisense BloX custom action. (If you haven’t had the chance to work with custom actions before, I highly recommend reading <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://support.sisense.com/hc/en-us/community/posts/360031998193-BloX-Custom-Actions-Guide" target="_blank">this custom action guide</a>, which lays out the foundation for using them.



The action consists of 3 main stages.&nbsp;



<h4 class="wp-block-heading">Stage 1: Extracting the action input</h4>



While writing the logic for your action, identify the pieces of information that should be provided in the action’s input, whether they come from the Sisense BloX widget’s query, or from user input. In the following steps of this guide, we will include those in the action’s input payload.&nbsp;



In your code, extract those variables from payload.data.



<h4 class="wp-block-heading">Stage 2: Running the custom model</h4>



The SEIR model will be executed by using Quest’s runModel API endpoint, providing the model ID we saved from stage 1 along with the data our model needs:



<pre class="wp-block-code"><code> // Run SEIR custom model via Quest
 let modelResult;
 const myHeaders = new Headers();
 myHeaders.append("Content-Type", "application/json");
 
 const data = JSON.stringify({
 "modelId": modelId,
 "method": "POST",
 "payload": {
 "N": n,
 "p": socialDistanceRate,
 "p_start_day": pStartDay
 },
 "custom": true
 });
 
 const requestOptions = {
 method: 'POST',
 headers: myHeaders,
 body: data,
 redirect: 'follow',
 credentials: 'include'
 };
 
 fetch(`${sisenseUrl}/api/v1/quest/runModel`, requestOptions)</code></pre>



A few notes about the endpoint:&nbsp;



<ul><li>The modelId and Sisense url are constants that are defined at the beginning of the full action function (<a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://drive.google.com/file/d/19FygBQDUS6bty_PVCi4V62vh7TGALeD1/view?usp=sharing" target="_blank">attached</a>). Those constants should be adjusted to your environment and to the model you created in the previous step.</li><li>The request’s body should be a stringified object that contains the model ID, a payload object (which contains all of the parameters we’d like our model to receive), and 2 hardcoded values: custom: true, method: post</li><li>If you intend to embed the Sisense BloX widget in your own website using <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://developers.sisense.com/pages/viewpage.action?pageId=557127" target="_blank">Sisense JS</a>, make sure to include credentials in the request using the credentials: ‘include’ property.</li><li>If we were trying to communicate with any external URL, we would have to open CORS, but since Quest’s API is internal, there is no need.</li></ul>



<h4 class="wp-block-heading">Stage 3: Plotting the model results on the target widget</h4>



Once the model results are resolved, we can use them as we please. In this particular case, I manipulated another widget in the dashboard, and added 2 additional series, based on the model’s result. You can adjust this flow to your own needs by using the result in any other way.



<h3 class="wp-block-heading">Step 3: Creating a basic Sisense BloX Widget</h3>



Once the model and action have been created, we turned to the tool that will activate them: the Sisense BloX widget itself. To activate the custom action we have created, a user will have to click a respective action button, and that should be the element we add first.



In this example, I added the action inside an ActionSet element, as it makes it very easy to control the button’s style and location inside the Sisense BloX card. We then included all of the parameters that our action expects as properties of the data object. Our model will require the parameters n, socialDistanceRate and pStartDay, while the processing and plotting of the models result require the parameters widget id and series color.



Set the parameters’ values to either reflect the value of a widget panel (i.e. the parameter n), or to a hardcoded default value:



<pre class="wp-block-code"><code>{
 "type": "ActionSet",
 "class": "update-model-button",
 "actions": &#91;
 {
 "type": "SEIR",
 "title": "Apply",
 "data": {
 "widgetId": "5ea16f579e97a3002de5a31d",
 "color": "#000000",
 "n": "{value: Population}",
 "socialDistanceRate": 1,
 "pStartDay": 10
 }
 }
 ],
 "style": {
 "paddingBottom": "0",
 "marginRight": "15px",
 "display": "none"
 }
}</code></pre>



For every parameter that you would like to make dynamic and dictate by user input, create a Text input element. To enable this input field to feed the actions with its value, make sure to give it an id property whose value represents the path of the respective attribute in the action’s payload.



<pre class="wp-block-code"><code>{
 "type": "Input.Text",
 "id": "data.socialDistanceRate",
 "placeholder": "0-1",
 "defaultValue": "0"
}</code></pre>



For the sake of convenience, I wrapped each input in a Container element and I recommend you do it, as well. Finally, design and style your Sisense BloX widget to complement your dashboard.



<h3 class="wp-block-heading">Step 4: Kicking Sisense BloX up a notch</h3>



We now have an operational Sisense BloX widget whose custom action can trigger a custom quest model and plot its result on the target widget. At this point, we could style the Sisense BloX widget to our liking and be finished.



However, asking users to manually change the values of the social distance rate, or the number of days until social distance was applied in an input field was cumbersome and didn’t give the slick experience we were looking for.



Since those 2 parameters are numeric ranges, it was natural to use sliders for them.



However, sliders are not considered inputs in Sisense BloX, and therefore, invoked actions do not collect their values. To overcome this issue and take the slider values into account, we implemented the <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://support.sisense.com/hc/en-us/community/posts/360046992953-BloX-Auto-Slider" target="_blank">Auto Slider solution</a>, as well as additional CSS, to give it a <a rel="noreferrer noopener" aria-label=" (opens in a new tab)" href="https://www.sisense.com/blog/position-your-analytics-app-for-maximum-impact/" target="_blank">Sisense-app</a> look and feel.



<h2 class="wp-block-heading">Building better advanced analytic apps</h2>



The SEIR app was a bold experiment not just in helping save lives via social distancing, but in pushing what the Sisense data and analytics platform was capable of. Sisense BloX puts actions and insights together and allows for an unparalleled level of customization for both visuals and functionality. The Sisense Quest add-on gives you greater flexibility to use advanced analytics in a variety of ways, supercharging your insights and allowing you to get more out of your data.



These are just a few ways you can use Sisense to do amazing things with your data. We hope the how-to elements of this article get your imagination going and help you build boldly. Be sure to jump into our active <a aria-label=" (opens in a new tab)" href="https://support.sisense.com/hc/en-us" target="_blank" rel="noreferrer noopener">builder community</a> to see and hear about the exciting projects that other builders are working on and get feedback on your own projects. There’s nothing we can’t do when we work together, from flattening the curve to building amazing new data and analytics products.



Embed analytics in your products for competitive advantage: &gt;&gt; Read the guide on how to deliver innovative customer experiences with analytics



<a class="action-btn " href="https://www.sisense.com/whitepapers/stand-out-in-your-markets-how-product-managers-can-deliver-innovative-data-experiences-to-end-users-oem/" target="_blank" rel="noopener noreferrer">Read the Guide</a>



Adi Lewenstein has spent almost a decade in the analytics and BI space as a BI consultant in Sisense&#8217;s CS department and as a growth hacker in Sisense Labs. She has spoken at numerous industry events and today serves as a full stack developer in Sisense&#8217;s AI R&amp;D team, implementing the AI-driven capabilities she has once desired as a user.

Combatting COVID-19 with Embedded Advanced Analytics

LinkedIn

Twitter

GitHub

curve-image-unique-image-unique

curve

3-dark-2-image-unique-image-unique

3 DARK 2

Get the latest in analytics right in your inbox.

Article