Instrument Your Virtual Instruments! (Part 2)
If you're reading the part 2, then it must mean that I managed to pique your curiosity, and that I potentially called on to some awareness about metrics collection. Prometheus and the likes really make sense as the number of applications, servers, testbeds or any computing unit connected on your company network increases:
- they offer a centralized method for metrics storage and management.
- they scale ridiculously well, and you can have clusters (replicated/distributed machines) running them.
- they make your application simpler because you can exclude all the logic used to store and display metrics, as well as handle and raise alerts.
- they are open-source (at least a fair number of them), flexible (setup on premise or cloud), easy to install, extensible... and they run on cheap machines (#RaspberryPi !)
To me, the reason why monitoring solutions like Prometheus are pretty much ignored in the LabVIEW world is because we usually love bringing other technologies to LabVIEW (not the other way around), and because we love rewriting everything in LabVIEW - at least I used to be like that. But I hope this will encourage you to think differently and trust Prometheus for your next project!
PromVIEW, the client library
I previously mentioned how the metrics collection in your application is still to be done through metrics exporters.
In many cases (essentially in the web/cloud products and projects), such exporters already exist and you just have to install and cofigure them. Such exporters usually run as a separate service that collect the metrics from the app you'd like to monitor (e.g. Apache or MySQL), and expose them to a HTTP interface. Prometheus only needs to scrape these exporters to extract the data they contain before storing them in its internal time-series database.
In the other cases, you will need to instrument your application code using a client library written in a language compatible with your app: you're basically creating a custom exporter inside your app. There are about a dozen libraries available, but you guessed right, none of them is in LabVIEW. So I decided to bootstrap PromVIEW, the LabVIEW Prometheus client library!
Prometheus luckily documents how to write client libraries, so it's not like PromVIEW was born from nothing. However the LabVIEW paradigm made a few things a bit more complicated than expected:
- The metrics access scope must be very large (globally accessible in the application), and so are collectors and registries.
- Exposing the metrics as a HTTP endpoint is not particularly difficult using the NI Web Server when you do it manually, but what about providing a way to automate the creation of this HTTP endpoint?
By-Reference Design
There are not hundreds of ways to access an object anywhere in the application context. Notifiers, queues, data value references. They are all references, and even though they are known to break the dataflow, they're powerful when used right. PromVIEW uses the Extensible Session Framework from NI to provide globally accessible metrics, collectors and registries by their name.
While accessing references is a slow process in LabVIEW, it does not affect PromVIEW too much, mainly because the metrics are accessed/modified when Prometheus calls the HTTP endpoint that causes the metrics values to update, and this is usually done every few seconds to minutes, so there should not be a contention due to heavy access to the references in your app.
Automating the LabVIEW Web Services
The goal is simple: provide a menu, or button or any no-brainer way that allows you to automatically create the web service (WS) that exposes your application metrics. And to achieve this, you have... nothing! Sadly, there's no supported way to programmatically create or edit a web service in your project.
Fortunately, knowing how LabVIEW project providers work, I was able to reverse-engineer the LV web service provider so I could meet the requirement I fixed. To make it simple (project providers will soon be on TheLabVIEWLab...), the VI Server is able to create a web service, then it takes some magic to find the reference to that newly created web service project item, so I needed to use the LV Web Service project provider VIs to edit the WS properties, and to add the HTTP GET resource Prometheus needs to scrape your metrics.
From Execution...
When running you app and the WS next to it, a web page containing text formatted such as Prometheus can read it, parse it, ingest it and add the resulting metrics in the time-series DB.
So, with PromVIEW doing the tedious work for you (by providing LabVIEW palettes and a way to create a web service in 2 clicks), instrumenting your code should be easier than ever before. If you made it to this point, you now have:
- An app running whatever your business logic is
- A custom Prometheus exporter powered by PromVIEW running in your app
- A web service running on the same machine, that exposes your metrics as you designed it
- A Prometheus instance somewhere ready to scrape your metrics
- Probably a data visualization tool to display your metrics in a meaningful way
- Possibly alerts and notifications set up to inform/warn people of interest about anything going wrong with your app, or any other apps/services Prometheus may collect metrics about.
Enabling Prometheus to scrape your app data should be a breeze. Edit the Prometheus configuration file to add your app exporter as a new target to scrape. After restarting the Prometheus service, the administration board should look like this:
To Dashboards...
Grafana is a super popular tool to display data found in (time-series) databases. And as it natively has a Prometheus connector, within a few minutes you can have panels containing graphs and other widgets showing your data.
Here is an example of a graph showing the PromTest1 metric collected from LabVIEW (the basic PromVIEW example) !
About PromVIEW
PromVIEW comes with examples that you can easily run, and some documentation about how they are built and how to run them.
PromVIEW is already available on VIPM.io, but also on Github if you want to contribute or post bugs. Finally, TheLabVIEWLab now has a Resource section where you can find PromVIEW info and links, and a place to find future libs/packages!
Note: PromVIEW requires LabVIEW 2020 (as it's my current version, and because it uses an interface, feature only available since LV2020). Planning for backward-compatibility is currently not in my to-do list as LV 2020 Community Edition is now a thing!