If you’ve perused the Wavefront blog, you likely noticed that some of us Wavefronters are obsessed with sending data to Wavefront from almost anything that generates metrics. In addition to our primary focus on monitoring cloud-native applications and infrastructure like java code, Hadoop, Kubernetes, servers, and AWS Lambda, you can also find blogs about Wavefront integrating with thermostats, weather stations, automobiles, and more.
Recently, I’ve been working on sending metrics created by a Raspberry Pi to Wavefront (Try Free Here). Raspberry Pi is a single-board compute platform that’s great for extracting metrics data from sensors. A wide array of sensors is available, from environmental sensors, motion sensors, navigation sensors, light sensors and more. In fact, there are millions of Pi’s being used in industrial internet of things (IIoT) applications.
(Also, check out the Raspberry Pi Liota lab code here: Maker Space Raspberry Pi Liota/Wavefront Lab - VMware {code} )
Most Pi sensors include Python or Bash shell sample code. In fact, I recommend only buying sensors which have code samples. Once you have this code, pulling the data off the sensor is relatively easy.
Raspberry Pi needs an OS, and the Raspbian Linux distribution is a great choice. It includes a full network stack, for wired or wireless networking. This makes it easy to open a TCP socket and forward the sensor data generated in a format that Wavefront understands.
I’m setting up examples of this in preparation for the Tech Shop I’ll be running at VMworld. This will be one of many use cases showing Wavefront at VMworld US 2018. You can see everything that’s planned to go on with Wavefront at VMware during VMworld US 2018
In the example below, I working with relatively common BMP280 chip temperature and barometric pressure sensor with a Raspberry Pi board, and hacked some Python code, which will runs on the PI, to get data into a format that Wavefront can ingest via the Wavefront proxy (see the documentation for more information about the Wavefront data format, and the Wavefront proxy).
The Parts List
First, you’ll need to compile the parts. I recommend the following gear, which is what I used for this example.
- Raspberry PI 3 B: https://www.adafruit.com/product/3055.
- NOOBS software for getting started. Don’t try a Raspbian ISO like I did. It won’t work. Critically, you have to install the NOOBS software in a specific way. Read this to find out.
- A basic startup kit with a breadboard and jumper wires: https://www.amazon.com/Electronic-Starter-Kit-Raspberry-Pi/dp/B00IT6AYJO
- A BMP280: https://www.adafruit.com/product/2651 You may have to do some soldering. Get it pre-installed with pins if you can. (this one is also good and comes with pins preinstalled)
- A free Wavefront trial account at http://wavefront.com/sign-up/
Useful Reading
How did we get things done before Google search? Now when you need something done, you can always see how someone else before you did it. Here are a bunch of things I read to help me get this all working.
First, you should understand the pin-out of Raspberry Pi, if only to understand which pin you use for power. You also need to understand how to communicate with the sensors. With the Pi, it’s no longer only about reading voltage levels. You’ll need a basic understanding of I2C, which is a way chips talk to each other on a board. There’s a decent PHP library that supports all of this. Google “I2C” and you’ll get it pretty quickly. It essentially provides an addressable bus you can do arbitrary reads and writes.
It also helps to understand the temperature sensor chip. This is where I found the BMP280 tutorial: http://faradaysclub.com/?p=1325. This is essential to getting data off the chip, since it’s NOT obvious. Special thanks to pyromd76, whoever you are.
The Python code mentioned in the above article did have some syntax errors and bugs. These are corrected on my GitHub: https://github.com/BillRothVMware/wavefront-raspberry-pi/.
Get the Data
At first blush, pulling data off the BMP280 is relatively easy. In order to do this, we run some Python code on the PI, which pulls the data off the sensor, and then formats it for sending to Wavefront.
Here’s the code:
# BMP280 address, 0x76(118)
# Read data back from 0x88(136), 24 bytes
b1 = bus.read_i2c_block_data(0x76, 0x88, 24)
That’s the fun of I2C. It’s a bus, and when you attach a sensor, it has an address you can use in Python on the PI (or any language) pull data of the sensor. The above call takes 24 bytes from a specific address. Simple, right? (Your BMP280’s bus address might be 0x76 or 0x77. Check the docs).
What gets pulled off by the Python code on the Pi is the raw sensor data and it needs to be massaged into the right format. You’ll then need the code to deal with the data that came off the chip, which is by no means straightforward. I pulled it directly from the code mentioned above. It seems to work, but as of this writing, the pressure calculations seem a bit off. Free Wavefront t-shirt to the first person who can fix it. 😉
Send Data to Wavefront
Sending data to Wavefront is also pretty easy. Once the Python code on the Pi has pulled the temperature and pressure readings, all you need to do is to send them in the Graphite-influenced Wavefront format, mentioned above. Its as simple as passing in a name, a number, and where the data came from. For example:
Bills.living.room.temp.f 72 source=myhouse
Below, I pull the temperature in Celsius, Fahrenheit and the pressure in millibars, and output into Wavefront format.
# Output data to screen
# print "Temperature in Celsius : %.2f C" %cTemp
print "pi.bmp280.temp.c %.2f source=pi1 dc=willow-glen user=broth@vmware.com" %cTemp
print "pi.bmp280.temp.f %.2f source=pi1 dc=willow-glen user=broth@vmware.com" %fTemp
print "pi.bmp280.pressure.hPa %.2f source=pi1 dc=willow-glen user=broth@vmware.com" %pressure
Output
The output from the code above will look something like:
pi.bmp280.temp.c 25.58 source=pi1 dc=willow-glen user=broth@vmware.com
pi.bmp280.temp.f 78.04 source=pi1 dc=willow-glen user=broth@vmware.com
pi.bmp280.pressure.hPa 1168.91 source=pi1 dc=willow-glen user=broth@vmware.com
Note that no timestamp is used. This implies to Wavefront that it should be recorded as “right now”. You can include a time stamp in Unix Epoch seconds after the data if have a specific date to use.
To make this work, I then ran this code in a shell script, and piped it to netcat, using a method like the one I described in my previous blog about getting data from the Nest Thermostat into Wavefront.
Show Data in Charts and Dashboards
The data format in Wavefront is as simple as sending the name of the data, the data, and where it came from.
I then built a simple dashboard in Wavefront to show the results using basic queries, like
ts(makerspace.pi.bmp280.temp.f)
This says take the Time Series (ts()) named makerspace.pi.bmp280.temp.f and visualize it. The default chart is a line graph, seen below.
To visualize a query, go to Dashboards->Create Chart. We’ll start with the query builder, which makes it easy to get started with time series data. To do this, we only need use the name of the time series, in this case makerspace.pi.bmp280.temp.f. When this is pasted into the Query Builder, a chart like will appear:
Wavefront has a wide array of functions for analytics, time series manipulation and more. Read the docs for more info about the Wavefront query language and all the functions available to you.
The real value of Wavefront becomes apparent when charts are aggregated into a dashboard – a single view of all your important metrics. At the bottom of the chart view, you will see a “Save To New Dashboard” button, or a place to name an existing one. This will take you to a flexible online editor for arranging charts in your dashboard. Check out the online tutorial for this by starting a trial at http://wavefront.com/sign-up/.
Below is a screenshot of the simple dashboard that I built for my various metrics data:
Out of the box, Wavefront has over 150 integrations for the full stack of cloud application systems, like StatsD, DropWizard, AWS, Azure GCP, databases, app servers, containers and tools like PagerDuty and Slack. These integrations come with pre-built dashboards to help you get started quickly. For more information, sign up for a Trial of Wavefront at https://wavefront.com/sign-up/
Conclusion and Future Work
It’s really is easy to get any stream of metrics data into Wavefront, as summarized above. Forwarding metrics data from a Raspberry PI and Raspbian is relatively simple. I haven’t tried anything lower level like Arduino sketches, but this is on my to-do list. As long as you can open up a TCP port, you should be able to stream metrics data into Wavefront. Have fun, and if you do any of your own integration code, let us know by posting a comment, or leave a comment on the Public Wavefront Slack Channel.
