Overview

Sisense provides flexible design choices for supporting high availability (HA) and scalability for your Sisense configuration. To configure Sisense for high availability or optimized performance by scaling out Sisense servers, you must build in redundancy, thus reducing potential downtime or bottlenecks.

Note: High availability must be enabled in your license. For more information, please contact your Sisense representative or submit a request to Sisense Support  through our community website.

In a simple configuration, you interface with the Sisense Web Application that interacts with the ElastiCube.

This configuration represents the Sisense full stack solution. At least one instance of each of the following components must be active to enable you update your data and allow your users to query that data from the dashboard:

Sisense Web Server

The Sisense Web Server hosts the Sisense Web Application that provides the user interface and hosts the API endpoints. If the Sisense Web Server fails, your users cannot access the Sisense Web Application to view dashboards or use the Sisense APIs.

MongoDB

MongoDB is installed with Sisense and supports the Sisense Web Application. The MongoDB  contains dashboard, filters, and user information necessary for ensuring data consistency across all web servers. In addition, MongoDB is used for authentication when you make calls to Sisense’s APIs.  If the MongoDB fails the Sisense Web Application will fail.

ElastiCube Server

The ElastiCube Server is installed locally on your computer and provides access to ElastiCubes. If access to the ElastiCube Server fails, queries from the Sisense Web Application will fail.

Understanding High Availability in Sisense

While Sisense is fully-functional in an environment without high availability, a multi-node configuration is necessary for scalability to support large amounts of concurrent users and redundancy in case of a failure of one of the Sisense components.

In a multi-node configuration, Sisense components are replicated. Replicating each of these components provides redundancy and fault tolerance against the failure of any single component. The replicated components are combined into nodes. There are two types of nodes, a build node and query nodes. The query nodes, which handle user queries from the Sisense Web Application, are replicated to support high availability. The build node is typically not replicated as its failure only prevents building new ElastiCubes not issuing queries from the Sisense Web Application.

Build Node

The build node is responsible for building ElastiCubes and distributing the build to query nodes via the Sisense Orchestrator Service. The Sisense Orchestrator Service is an automated service that you configure on the build node to synchronize and distribute builds to the query nodes. For more information, see Distributing ElastiCube Builds to Query Nodes.

Query Nodes

Query nodes are responsible for supporting queries from Sisense users. These nodes contain a web server, MongoDB, and an ElastiCube server. ElastiCubes are distributed by the build node to the query node. The query nodes’ ElastiCubes are combined into ElastiCube Sets to support high availability by separating the web and ElastiCube servers across multiple query nodes. If a build node is distributing a build to one ElastiCube server, Sisense automatically directs any queries to the other ElastiCubes in the ElastiCube Set.

The query node can be configured as a single application stack where each node hosts a Sisense Web Application, ElastiCube Manager, and a MongoDB. In this configuration, if the machine hosting the components fails, the whole query node will fail. Queries will then be redirected to the next available query node. Another option is to host each component of the query node separately in a distributed application stack. In this configuration, if a component of the query node fails, the rest of the query node is not affected. For an example of a single application stack, see Scenario 1 and for an example of a distributed application stack, see Scenario 2.

Load Balancer

In addition to query and build nodes, to support a high availability configuration, you must handle load balancing on your side prior to directing traffic to one of your Sisense nodes. Load balancing spreads requests across multiple query nodes according to an algorithm you define and the current status of the query node.

When implementing ElastiCube Sets, Sisense’s query nodes operate in active-active mode. This means that each of the query nodes is active and can handle requests when the node is not building and its components are available. For example, traffic could be spread 50-50 across two web servers and if a component fails, a load balancer should redirect traffic to the other available web server.

Within an ElastiCube set, you can define how queries are routed to the available ElastiCubes. For more information, see Routing Queries in ElastiCube Sets.

Creating a High Availability Environment

There are many possible configurations you can implement to support high availability. What determines the type of configuration you choose based on three parameters:

  1. Redundancy: Which components require redundancy?
  2. Concurrency: How many users do you need to support?
  3. Resources: How many systems can you add and maintain to your configuration?

Regardless of which configuration you choose, there are three steps to achieving high availability in Sisense:

  1. Install Sisense on each machine in the build and query nodes. The Sisense installation includes the Sisense Web Application, ElastiCube server, and the MongoDB.
  2. Replicate the MongoDB instance on the build node and distribute it to each of the query nodes. For more information, see Replicating MongoDB.
  3. Configure the Sisense Orchestrator Service on the build node to distribute the latest ElastiCube builds to each of the query nodes. For more information, see Distributing ElastiCube Builds to Query Nodes below.

Distributing ElastiCube Builds to Query Nodes

The Sisense Orchestrator Service is an automated service that distribute builds and balance queries across an ElastiCube Set. The Sisense Orchestrator Service is installed during the default installation of Sisense. To distribute ElastiCube builds across all query nodes, you must configure the Sisense Orchestrator Service. When an ElastiCube is building, or receiving a build from the build node, the Sisense Orchestrator Service redirects requests to another available ElastiCube in the set.

Note: After configuring the Sisense Orchestrator Service, the directory C:\ProgramData\SiSense\PrismServer\ElastiCubeData\ and the relevant ElastiCube folders within it must be shared with permissions for ‘Everyone’ so Sisense can build or update the ElastiCubes in those folders.

The Sisense Orchestrator Service is defined in a JSON file called config.json in the Sisense.Orchestration Config folder (Sisense/Sisense.Orchestration/Config/) of your Sisense Installation folder of your build node.

The config.json file has three main objects that you must define. The first object is the ElastiCube object that defines which ElastiCube is to be built on the build node and which ElastiCubes are located on query nodes.  The second object is the Tasks object that determines how and when the build ElastiCube is to be distributed to the query nodes. The final object that you must define is the Schedule object that defines when an ElastiCube is to be built and distributed.

To distribute your ElastiCube builds through the Sisense Orchestrator Service, you define and save the config.json file. After saving the file, the Sisense Orchestrator Service automatically begins to build ElastiCubes in your ElastiCube Set. You can review the progress of the build in the /Sisense.Orchestration/Logs folder in a log file whose file name you define in the JSON file.

Note: Sisense recommends stopping the Sisense Orchestrator Service while importing data and reactivating it after the importing process is completed.

The objects that you must include in the config.json file and their descriptions are provided in the table below:

KeyValue
GeneralThe General object contains two objects, logLevel and emails that define what types of logs Sisense generates and where to send them to.
LogLevelThe type of info returned in the log. The possible values you can enter are Info, debug, and Error.
EmailsThis object defines who receives an email and their email address when an email is triggered according to the events you define in the mail object.
EmailThe email address where emails are to be sent depending on the value of the When key.
NameThe name of the recipient of the email.
CubesThe cubes object contains all your ElastiCubes in your configuration and their location. As the Sisense Orchestrator Service is installed on the build node, the build cubes are local, so you must define the name of the ElastiCube. For ElastiCubes on a query node, you must define the ElastiCube name, URL of the remote server, and its directory.
Build ElastiCubesThe Build ElastiCube object defines the ElastiCubes that will be used as Build ElastiCubes nodes.
ecubeThe name of the ElastiCube.
Query ElastiCubesThe Query ElastiCube object defines the name of the remote ElastiCube, its URL address, and directory.
ecubeThe name of the ElastiCube.
URLThe URL address of the ElastiCube server. If the URL is to a secure address, the value should include the username and password credentials. For example, “ssh://username:password@10.50.1.128:/C/ecubes”. See Scenario 1 for an example.
localPathDirectory of the ElastiCubes.
TasksThe Tasks object contains a task array that defines which ElastiCube should be built, the type of build, and to which ElastiCubes the build should be distributed to. Currently, Sisense only supports one Task object.
TaskThe name of the task array. Currently, you can define only one task. Defining multiple tasks in the config.json file may cause builds to fail.
BuildThe build object defines the ElastiCube to be built and distributed.
CubeThe name of the cube to be be built.
QueueThe value of this key is the type of build Sisense should attempt and in what order. There are two possible values delimited by a comma:

accumulate: Attempts an accumulative build.
entire: Attempts an entire build.
schemaChanges: Attempts to update the build only if changes were made to the schema since the previous build.

Sisense recommends the following value: "accumulate","entire"

In this example, Sisense attempts to do an accumulative build first, and if it fails, Sisense attempts an entire build. If the build fails for all builds, Sisense retries again according to the next scheduled build.
DistributeDetermines which cubes the latest build should be distributed to. The value should be the ElastiCube name for your ElastiCubes on your query nodes. For example, [“cube1, cube2, cube3”].
You can add multiple ElastiCubes delimited by a comma separator.
ReattachReattach is an object that contains two objects, DeleteOldDbfarm and Cube.
DeleteOldDbfarmA boolean value that determines if an old ElastiCube is to be deleted after it has been updated.
CubeThe name of the ElastiCube to be reattached or not.
MailIndicates when an email alert is triggered. Email alerts can be triggered for the following scenarios:

None: No email is ever sent.
Build: An email is sent when a build is complete regardless if the build was successful or not.
Fail: An email is sent when a build fails.
SchedulerThe Scheduler object contains an array of tasks that define when a task is to be task is to be initiated.
TaskDefines which task to perform and the order. Currently, Sisense only supports one task.
ScheduleThe time that a task is to be initiated in Cron format. Some examples: -To run a build each night at midnight, enter the value "0 0 * * * *" -To run a build every hour, enter the value "0 * * * * *" See the full Cron format reference here.
EnabledA boolean value that indicates if the task is to be executed or not.

Use Cases

The following section presents two use cases and an example config.json file that demonstrates how to support these use cases. For assistance with additional HA configurations, contact your Customer Service Manager.

Scenario 1 – Single Stack

In this example, there is one build node and two query nodes. Each query node is hosted on a separate machine while the components that make up the query node are hosted on the same machine as a single application stack.

In this example, the build cube is called “BuildTest1”. This ElastiCube is distributed to two ElastiCube servers defined in the cube1 and cube2 objects. The build cube is distributed first to cube1 and then to cube2 as defined in the tasks object. The first time the Sisense Orchestrator Service tries to build the BuildTest1 ElastiCube, it attempts an accumulative build and if that fails, an entire build. After each build is replicated, the previous version of the ElastiCube is deleted as specified in the reattach object. This task takes place hourly. If the distribution fails, Sisense sends an email to johndoe@sisense.com with a debug log attached.
{
"general":{
"logLevel":"INFO",
"emails":[{
"email": "johndoe@sisense.com",
"name": "John Doe"
}] },
"cubes":{
"localCubeBuildTest1":{
"ecube": "BuildTest1"
},
"cube1":{
"ecube": "BuildTest1",
"url": "\\\\10.49.1.68\\ecubes",
"localPath":"d:\\ecubes"
},
"cube2":{
"ecube": "BuildTest1",
"url": "\\\\10.70.0.32\\ecubes",
"localPath":"e:\\ecubes"
},
"cube3":{
"ecube": "BuildTest1",
"url": "ssh://administrator:Password@10.70.1.133:/C/ecubes",
"localPath":"e:\\ecubes"
}
},
"tasks":{
"task1":[
{
"build": {
"cube": ["localCubeBuildTest1"],
"queue": ["accumulate","entire"] }
},
{
"distribute": ["cube1","cube2","cube3"] },
{
"reattach": {
"deleteOldDbfarm":true,
"cube":["cube1","cube2","cube3"] }
},
{
"mail":"build"
}] },
"scheduler":[{
"task": "task1",
"schedule": "0 * * * *",
"enabled": true
}] }

Scenario 2 – Distributed Stack

In this example, there are three query nodes in which each component is located in a different remote location. This scenario, while costly to implement, demonstrates a distributed application stack configuration where each component is replicated and located on a separate server independent of the other components. If any component fails, the rest of the architecture remains intact. For example, the web server is on one server, the ElastiCubes are stored on another server, and the MongoDB is located on another server.

The build cube in this example is called “BuildTest1”. This ElastiCube is distributed to three remote ElastiCube servers defined in the cube1, cube2, and cube3 objects. The build cube is distributed first to cube1, then cube2, and then cube3 as defined in the tasks object. The URLs are standard URLs for cube1 and cube2 while the URL for cube3 is secured through SSH. When the URL is secured, you must provide the URL and the required credentials to access it.

The first time the Sisense Orchestrator Service tries to build the BuildTest1 ElastiCube, it attempts an accumulative build and if that fails, an entire build. If both builds fail, Sisense attempts to update the schema only.  After each build is replicated, the previous version of the ElastiCube is deleted as specified in the reattach object. This task takes place At 14:15 on the 1st of every month. After a build is successful, Sisense sends an email to johndoe@sisense.com.

{
“general”:{
“logLevel”:”DEBUG”,
“emails”:[{
“email”: “johndoe@sisense.com”,
“name”: “John Doe”
}] },
“cubes”:{
“localCubeBuildTest1”:{
“ecube”: “BuildTest1”
},
“cube1”:{
“ecube”: “BuildTest1”,
“url”: “\\\\10.49.1.68\\ecubes”,
“localPath”:”d:\\ecubes”
},
“cube2”:{
“ecube”: “BuildTest1”,
“url”: “\\\\10.70.0.32\\ecubes”,
“localPath”:”e:\\ecubes”
},
“cube3”:{
“ecube”: “BuildTest1”,
“url”: “ssh://administrator:Password@10.70.1.133:/C/ecubes”,
“localPath”:”e:\\ecubes”
}
},
“tasks”:{
“task1”:[
{
“build”: {
“cube”: [“localCubeBuildTest1”],
“queue”: [“accumulate”,”entire”, “schemachanges”] }
},
{
“distribute”: [“cube1″,”cube2″,”cube3”] },
{
“reattach”: {
“deleteOldDbfarm”:true,
“cube”:[“cube1″,”cube2″,”cube3”] }
},
{
“mail”:”fail”
}] },
“scheduler”:[{
“task”: “task1”,
“schedule”: “15 14 1 * *”,
“enabled”: true
}] }

Add Query Nodes

In some configurations, replicating components can improve performance by scaling out and preventing any potential bottlenecks. Each component you replicate must be added to the ElastiCube Set and the config.json file. You can continue to add query nodes according to your requirement so long as the nodes are included in the config.json file.

To add a query node:

  1. Stop the Sisense Orchestrator Service. Open Windows Services, select Sisense.Orchestrator, and click stopbutton.
    orch1
  2. In the Sisense Web Application, click Admin in the upper-right corner and select the ELASTICUBES tab on the left.
  3. Hover over the ElastiCube you want to add to the ElastiCube Set and click on the menu that appears. Click Add to ElastiCube Set and select the set that you want to add the ElastiCube to.
    addtoset
    OR
    In the ElastiCube Set table, click pencil  and select the ElastiCubes to be added to the set. Click Save after you have selected the relevant ElastiCubes.
  4. Edit the config.json located at Sisense/Sisense.Orchestration/Config/ and add the new ElastiCube to the Cubes object.
  5. Save the config.json file.
  6. Restart the Sisense Orchestrator Service.

Remove Query Nodes

You can remove query nodes by removing them from the ElastiCube Set and the config.json file.

To remove a query node:

  1. Stop the Sisense Orchestrator Service. Open Windows Services, select Sisense.Orchestrator, and click stopbutton.
    orch1
  2. In the Sisense Web Application, click Admin in the upper-right corner and select the ELASTICUBES tab on the left.
  3. In the ElastiCube Set table, click pencil  and clear the checkboxes of the ElastiCubes to be removed. Click Save after you have selected the relevant ElastiCubes.
  4. Edit the config.json located at Sisense/Sisense.Orchestration/Config/ and delete the relevant ElastiCubes from the Cubes object.
  5. Save the config.json file.
  6. Restart the Sisense Orchestrator Service.

Limitations

If your user queries a query node (Query Node 1), then the build node begins to replace a build on that node (Query Node 1), there are two scenarios in which the data may be inconsistent.

  1. When generating PDFs or Images, the widgets may not match the dashboard depending.
  2. In a Pivot widget, the subtotal value may not match the expected value.

This occurs because the widgets exported to PDF or subtotal values inside Pivot widgets run new queries against the node. If the data was originally taken from query node 1, then subsequently queries were run against another query node (Query Node 2), the data may have changed leading to inconsistent results.