1. Setup the monitoring Warp 10™ instance
  2. Installing Sensision
    1. Running Sensision service
  3. Using the metrics

Monitor Warp 10™

Sensision service is available for collecting Warp 10™ and his host metrics.

The complete list of the metrics exposed by Warp 10™ is available here. Below are few examples of what you can monitor:

  • Host metrics: CPU, memory, disks, network, ...
  • Warp 10™ metrics: number of datapoints fetched/deleted, number of Geo Time Series™, ...

The easiest way to monitor, gather and analyze Warp 10™ metrics is to use another Warp 10™ instance.

monitoring

Setup the monitoring Warp 10™ instance

First, install a Warp 10™ instance where you will do the monitoring, gathering the metrics from the main instance and analyzing them via WarpScript.

If needed, look at the getting started page for the detailed instructions to install Warp 10™

Generate monitoring tokens

The monitoring metrics from the main Warp 10 will we pushed in the monitoring Warp 10™ as any other GTS. As such, before pushing them, you need to generate both a READ and a WRITE token for the monitoring application.

To generate these tokens, use the worf command via Warp 10™ init script:

root@XXX:/opt/warp10-X.Y.Z/bin# ./warp10-standalone.init worf appName ttl(ms)

In this example, we choose monitoring as the monitoring application name and generate one year (31536000000 ms) tokens:

root@XXX:/opt/warp10-X.Y.Z/bin# ./warp10-standalone.init worf monitoring 31536000000
default options loaded from file:/opt/warp10-X.Y.Z/bin/../etc/conf.d/.00-secrets.conf.worf
{"read":{"token":"lO5XXXXXXXXXXXXXXXXXXXXDXk","tokenIdent":"606aa9b8f6f47136","ttl":31536000000,"application":"monitoring","owner":"3b8ab607-4aac-4b13-9251-94f152924766","producer":"3b8ab607-4aac-4b13-9251-94f152924766"},"write":{"token":"8D_XXXXXXXXXXXXXXXe7B","tokenIdent":"a219f98201b29e01","ttl":31536000000,"application":"monitoring","owner":"3b8ab607-4aac-4b13-9251-94f152924766","producer":"3b8ab607-4aac-4b13-9251-94f152924766"}}

Installing Sensision

In order to gather the monitoring metrics from the main Warp 10™, you only need to install and configure Sensision service in the main Warp 10™ server.

In these documentation page we are going to suppose that you're using Warp 10™ Standalone. If you are using the distributed mode, you will need to install Sensision in all the Warp 10™ nodes but ideally also on the Kafka and HBase servers.

Download Sensision archive on Bintray.

Running Sensision service

Untar the Sensision service file.

tar xf sensision-service-X.Y.Z.tar.gz
cd sensision-X.Y.Z/bin

If you already have Warp 10™ running, your JAVA_HOME environment variable should be already set, but you can also set can set it in the Sensision init script sensision.init.

Execute the Sensision init script sensision.init. This script must be run as root. It will create the folder /var/run/sensision and all the stuff surrounding it. Then the init script starts the Sensision service with a configuration by default.

./sensision.init start
root@XXX:/opt/sensision-X.Y.Z/bin
Config file does not exist - Creating it from template...
Init Sensision..
Fix permissions..
'WARP10_CONF' is not defined. Skip generation of Sensision configuration - Copy template as it is
Starting is going to be stopped - Update configuration '/opt/sensision-X.Y.Z/bin/../etc/sensision.conf' manually and restart Sensision

Sensision has been only partially initialized, at it lacks the information on the Warp 10™ instance it must push the metrics to. You need to edit the /opt/sensision-X.Y.Z/etc/sensision.conf configuration file to set it.

Look at the Default queue part of the configuration file:

#
# Default queue
#

sensision.qf.url.default=http://127.0.0.1:8080/api/v0/update
sensision.qf.token.default=@warp:WriteToken@
sensision.qf.topn.default=250
sensision.qf.period.default=1000
sensision.qf.batchsize.default=100000
#sensision.qf.proxy.host.default=
#sensision.qf.proxy.port.default=
#sensision.dedup.maxsize.default=xxx
#sensision.dedup.maxage.default=xxx

And set sensision.qf.url.default to the Update endpoint of your monitoring Warp 10™ instance, and put the monitoring WRITE token you generated before in the sensision.qf.token.default variable.

Then you can restart Sensision:

./sensision.init start

If everything is ok, you can see that a Sensision target file for your main Warp 10™ instance is created on /var/run/sensision/targets/:

root@XXX:ls -al /var/run/sensision/targets/
total 8
drwx-wx-wt  2 XXX XXX 4096 sept. 27 09:52 .
drwxr-xr-x 10 XXX XXX 4096 juil. 21 16:29 ..
-rw-rw-r--  1 XXX XXX    0 sept. 27 09:52 7ffffea13dad1efc.60000.af4ec5da-f721-493d-aaa0-25b3b8e70d05.38261.target
-rw-rw-r--  1 XXX XXX    0 sept. 27 09:52 7ffffea13dad4e0c.60000.f519167d-c4f7-435f-ae8e-2b87f67b102f.42893.warp.target

In this example, the target file for your Warp 10™ is 7ffffea13dad4e0c.60000.f519167d-c4f7-435f-ae8e-2b87f67b102f.42893.warp.target. The number before .warp.target is the port opened by Warp 10 to expose its metrics, you can manually poll it to see if it is working:

root@XXX:curl 127.0.0.1:42893/metrics
1506498948335000// warp.revision{component=standalone} '1.2.10'
1506498948335000// warp.script.mobius.sessions.scheduled{} 0
1506498948335000// warp.script.repository.macros{} 0
1506498948335000// warp.script.repository.jars{} 0
1506498948335000// warp.script.bootstrap.loads{} 6
1506498948335000// warp.script.function.count{function=DEF} 6
1506498948336000// warp.directory.gts{} 458810
1506498948336000// warp.script.function.time.us{function=DEF} 324

Using the metrics

The complete list of the metrics exposed by Warp 10™ is on GitHub.

You can manipulate those metrics in any way you want. Some suggestions:

  • Use QuantumViz custom elements and some simple WarpScript FETCH to get and plot the last values of the metrics you want to monitor.

  • Use the Grafana Warp 10™ plugin and some WarpScript code to build a fully featured monitoring dashboard.

  • Make your favorite monitoring tool do some WarpScript requests (they are simple HTTP requests, almost any tool is able to do it) to get the metrics and add them to you favorite dashboard or monitoring tool.