Distributed View in WSO2 Status Dashboard

As a second project of my internship at WSO2, I got an opportunity to work on the status dashboard for WSO2 Stream Processor.

What is WSO2 Status Dashboard?

We can understand the performance of the system via

  • Throughput
  • Latency
  • CPU, memory utilizations

We can Monitor in various scales

  • Node Level
  • Siddhi App Level
  • Siddhi Query Level

Okay, hope you all got a basic idea of the WSO2 Status dashboard and its functionalities. Monitoring the Distributed setup using WSO2 Status Dashboard is a new feature of status dashboard.

So What is Distributed Set Up?

The most common deployment pattern for WSO2 Stream Processor is the HA Deployment (Minimum High Availability Deployment) that offers high availability and high performance with the minimum amount of resources. However, there are a some real time scenarios where the HA (High Availability) deployment is not sufficient to handle the scenarios. The Fully Distributed Deployment pattern is introduced to handle to such scenarios. The main benefits to this approach are high availability and fault-tolerance.

In simple words, a distributed system is that splits into multiple running instances, all instances work together in a cluster to appear as a single instance to the end user.

Distributed deployment pattern is suitable in such scenarios where single SP instance fails to manage high amount of data as it allows complex event processing in a distributed manner. eg: High volume of data will be distributed among multiple SP instances instead of having them accumulated at a single instance. For detailed information about distributed system, refer here.

Monitoring Distributed System in WSO2 Status Dashboard

Before we start:

  • Fully distributed system must be available.
    To configure the distributed system, Please follow the instructions mentioned here
  • Download the latest WSO2 Stream Processor
    You can find the latest pack here

How to start WSO2 Status Dashboard?

  1. Extract the downloaded zip and navigate to the <SP_HOME>/bin directory (<SP_HOME> is the extracted directory).
  2. Issue the following commands to start the WSO2 Status Dashboard.
  • For Windows: dashboard.bat
  • For Linux: ./dashboard.sh

Once WSO2 dashboard is successfully started, a log similar to the following is printed in the CLI.

The server log prints the Status Dashboard URL in the start up logs as shown below.

Let’s get started!

  • Access the WSO2 Status Dashboard via the https://<HOST_NAME>:<STATUS_DASHBOARD_PORT>monitoring status dashboard URL.
    The default URL is: https://192.168.1.2:9643/monitoring
  • The Stream Processor Studio opens as shown below.
  • You can login to status dashboard by giving ‘admin’ as a user name and password
  • Once you login to the status dashboard, the nodes that are already added to the Status Dashboard are displayed
    If no nodes are displayed, you can add nodes for which you want to view the status by following the procedure below:
  • Click ‘Add New Node’ button, this opens the following dialog box.
  • Enter the following information in the dialog box and click ADD NODE to add a gadget for the required node in the Node Overview page.
  1. In the Host parameter, enter the HTTPS host ID of the node you want to add.
  2. In the Port parameter, enter the HTTPS port number of the node you want to add.

If the node you added is reachable, automatically status dashboard determine the runtime environment and will proceed further or else the following dialog box will be opened.

Since this blog is all about the Distributed view, am gonna add manager node :)

Node View

  • CPU Usage: The CPU resources consumed by the SP node out of the available CPU resources in the machine in which it is deployed is expressed as a percentage.
  • Memory Usage: The memory consumed by the node as a percentage of the total memory available in the system.
  • Load Average: The average system load calculated over a given period of time of 1, 5 and 15 minutes
  • Siddhi Apps: The total number of Siddhi applications deployed in the node.
    * Green color indicates active number of siddhi applicatios
    * Red color indicates inactive number of siddhi applications

This dispalys the change that has taken taken place in the CPU usage, memory usage and the load average of nodes since the status was last viewed in the status dashboard.

Positive changes are indicated in green (e.g., a decrease in the CPU usage in the above example), and Negative changes are indicated in red (an increase in the memory usage and the load average in the above example).

Viewing status details

Sample View

Description

Managers
The active manager node in the cluster is indicated by a green dot that is displayed with the host name and the port of the node. Similarly, a grey dot is displayed for passive manager nodes in the cluster.

Workers
When user adds the active manager node, automatically it retrieves the worker node details that are connected with that particular deployment and you can see the metrics of each worker nodes

Purpose

  • Whether the request load is efficiently allocated between the nodes of a cluster.
  • Whether the cluster has sufficient resources to handle the load of requests.
  • What are the nodes connected with the particular deployment

Recommended Action

  • If there is a disparity in the CPU usage and the memory consumption of the nodes, redeploy the Siddhi applications between the nodes to balance out the workload.
  • If the CPU and memory are fully used and and the request load is increasing, allocate more resources (e.g., more memory, more nodes, etc.).

Viewing Parent siddhi application details

This opens a separate page for the manager node as shown in the example below

Description

The Active/Deployed parent siddhi application deployed in the cluster is indicated by a green dot that is displayed with the name of the parent siddhi application. Similarly, a orange dot is displayed for Inactive/ Un-deployed siddhi applications in the cluster.

  • Worker Nodes : This gives the details of worker nodes that are inside resource cluster.
    Yellow Color indicates total number of worker nodes in the resource cluster (Eg: 2 in the above example which means for that particular distributed deployment we have two worker nodes in the resource cluster).
    Green Color indicates the number of worker nodes used in the particular parent siddhi application deployment. (Eg : 1 in the above example which means even though we have two worker nodes in the resource cluster,only one work node used for this specific parent siddhi application’s deployment).
  • Child Apps : This gives the total number of child apps of the particular parent siddhi application
    Green Color indicates number of deployed child apps
    Red Color indicates number of un-deployed child apps
  • Groups : This gives the total number of execution groups of that parent siddhi application. (Eg : In the above example, Testing Parent siddhi application contains 1 execution groups)

Purpose

  • Summary details of each parent siddhi applications that are inside the specific active manager node.

Viewing Specific Parent Siddhi Application

This opens a page with information specific to the selected Parent Siddhi application as shown in the example below.

Code View

View

Description

This allows you to check the queries of the Siddhi application if any further investigations are needed based on the kafka diagrams and performance.
For detailed instructions to write a Siddhi application, see Converting to a Distributed Streaming Application.

For detailed information about the Siddhi logic, see the Siddhi Query Guide.

Distributed Siddhi App Deployment

View

Description

Let me explain above example,
Kafka topic ‘topic_1’ is a source, that publishes events to ‘SampleTest-001–1’ and ‘SampleTest-001–2’ child apps. Then both child apps publishes events to ‘SampleTest.filteredStockStream’ kafka topic. Then ‘SampleTest.filteredStockStream’ publishes events to SampleTest-004–1child app. Finally SampleTest-004–1 child app publishes events to kafka topic ‘topic_2’.

Child App Details

View

Description

  • Group Name : The name of the group that particular child siddhi application belongs to
  • Worker Node: HTTPS host and HTTPS port of the worker node where that particular child siddhi application is deployed. By clicking that host and the port, you can get the worker specific details. This blog mainly focus on the manager view. To monitor each worker nodes that are connected with the distributed setup, please refer here

Okay now, It’s time for live demo :)

That’s all :)

Hope you all understand how we can use WSO2 Status Dashboard to monitor Distributed Setup :)

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gowthamy Vaseekaran

Senior Software Engineer @ hSenid Mobile | Former Software Engineering Intern @ WSO2 | Software Engineering Undergraduate @ IIT