Summer's here

Summer's here - and I can prove it, thanks to my SmartThings integrations with InfluxDB and Grafana…

If you've been following the previous blog posts, you'll know that I've set up a SmartApp to send stats from ST to a Zato ESB, and in this one I'll talk about sending that data on to InfluxDB.

Why not change the SmartApp to write directly to InfluxDB? It should certainly be possible, but if we've got the SmartApp working why change it? InfluxDB has some rules around quoting tag values with spaces, and that would mean figuring out the Groovy code in the SmartApp to appropriately quote the strings. We'd then have to format the POST for our call to HubAction.

Debugging is also a pain - temperature stats only come in when the temperature changes, so it could be a long wait to test changes. Since we're doing everything else locally in Python and already have the SmartApp sending to Zato, why not stick with that?

The first step is to set up the InfluxDB end. Let's create a database called "home" for our home monitoring stats with a default retention policy of a year:

> create database home
> use home
> create retention policy annual on home duration 52w replication 1 default

The volume of stats is low enough that this won't be a problem - unlike the server stats that poll every minute or more, ST temperature readings register only when the temperature changes. There may only be a handful of readings each day.

We'll use a measurement called "temperature" where the only tag will be the sensor name and the only field will be the value. We don't need to do anything in InfluxDB to set this up, just send data. For example:

curl -X POST http://localhost:8086/write?db=home --data-binary 'temperature,sensor=test value=25'

Getting the ST data into Zato was described in a previous post, so assuming that's working we just need to change the ProcessEvent code in the smartthings.py we set up previously.

This is where we really reap the benefits of Zato. It's a great platform for trying out APIs because it's so flexible. Traditional XML based ESBs often have trouble with unusual formats - they expect data to be going in and out in as JSON, XML or SOAP messages and there are hoops to jump through with other formats.

InfluxDB uses a mix of query string and POST parameters, where the POST body is just a string of data - hence the --data-binary in our curl example above.

In Zato we have some choices.

We could set up an outgoing plain HTTP channel that has the URL of our InfluxDB database. That would mean manually building the output POST and dealing with the encoding issues ourselves.

Alternatively we could use the InfluxDB Python client library, which allows us to build a JSON structure with the data that we're sending to InfluxDB. As this requires initialising the client with the URL and credentials, we can't use an outgoing channel for that information, but we could either hardcode it or store in it the Redis database.

If we're feeling really ambitious we could even build a new outgoing connection type that stores the configuration and then converts the input using the library. Zato already has outgoing connection types for FTP, WebSphere MQ and SQL amongst others, and InfluxDB would be a useful addition. Hmm.. I'll come back to that idea…

The beauty of Zato, though, is that we can do a quick and dirty hack to get things working by directly calling the InfluxDB URL using the Python requests library, which is an easy to use wrapper around the urllib library:
class ProcessEvent(Service):
 def handle(self):

  if self.request.payload['eventname'] == 'temperature':
 
  sensorname = 
       self.request.payload['devicename'].replace(' ', '\ ')

  postdata = "temperature,sensor=" + 
             sensorname + 
             " value=" + 
             self.request.payload['value']

  response = requests.post(
     url = 'http://localhost:8086/write?db=home',
     data = postdata,
     headers = {'Content-Type': 'application/octet-stream'})
We're manually escaping the sensor name string, which isn't ideal - it would be better to rely on the proper InfluxDB library to ensure that all escaping rules are followed, but this will do for now if we know that our sensor names may have spaces in but no other special characters.

As always, for brevity I'm ommitting any error handling and authentication.

Once we've got all the data into InfluxDB it's a piece of cake to build a Grafana dashboard to display the data - it's pretty much point and click:


And there you have it - indoors it got a bit stuffy last week, but I can't complain - the Great British Summer™ doesn't last long so I've got to enjoy it while I can.

Labels: , , ,