Possibly add Riemann logging ability… this way a farm of printers can send these streams to a remote system for logging and monitoring with dashboard alerts and metrics etc.
Riemann is an agentless monitoring system that would work perfectly for something like this and is recommended by and supported by Docker:
https://blog.docker.com/2016/03/realtime-cluster-monitoring-docker-swarm-riemann/
Why Riemann in particular?
There are a variety of reasons to use Riemann. Here are a few of the most obvious.
Riemann’s push-based model lets you detect problems and verify that fixes are working in near real-time.
Traditional monitoring systems such as Nagios usually work with a poll-based model: They wake up periodically (say, every 5 minutes), run a series of health checks to verify that everything in the system is in the desired state (e.g. your services are up), and report the results to you via e-mail or another method. Riemann requires that services send their own events, and consequently outages or network partitions can be detected as soon as the events stop flowing. Likewise, we can collect and process events of any type, opening the doors to proactively providing value through monitoring instead of simply fighting fires.
Riemann’s stream processing allows for powerful transformations of incoming data.
For instance, even if an exception is logged 1000 times in an hour, Riemann can roll up all exceptions of that type into just a few e-mails with information about how many times the exception occured. This helps combat alarm fatigue. Another example- Riemann makes measuring metrics in percentiles quite trivial. This allows to gain true insight into the operations of our systems without having important information masked behind aggregations such as averages.
Riemann is easy to get data into or out of, allowing interesting use cases such as storing event data for analysis later on.
Clients for Riemann can send events using a thin protocol buffers layer over TCP or UDP. Riemann can emit events and metrics from its index into a variety of backends including Graphite, InfluxDB, Librato, and more. Riemann also has built-in support for notification via SMS, e-mail, Slack, etc.
Riemann’s architectural model is simple, preventing the potentially ironic situation of having monitoring software that is difficult to operate and/or failing often.
Riemann is designed from the ground up to be straightforward to operate and reason about. While this has some tradeoffs (for instance, it makes it impossible to distribute a single Riemann instance safely), in practice Riemann’s philosophy is largely that imperfect information right now is better than perfect information which never arrives. If the Riemann server crashes, usually a simple restart will remedy the problem; no finicky reconciling of distributed state needs to occur.
Likewise, the push-based model helps to alleviate the “who monitors the monitoring software” question (i.e. how do we detect when our monitoring itself has issues). We can simply set up a few Riemann servers and forward events to the downstream servers. If the upstream server goes down, the downstream servers will notice this and alert you. Especially running Docker across clouds, this could be particularly valuable.
It’s rather fast.
From the landing page: “Throughput depends on what your streams do with events, but a stock Riemann config on commodity x86 hardware can handle millions of events per second at sub-ms latencies, with 99ths around 5ms. Riemann is fully parallel and leverages Clojure and JVM concurrency primitives throughout.”
