Recently, I put together a solution to speak notifications to me in my office regarding the status of various components in my house. Currently, I am only notified when my garage doors open and close, but I have plans to expand this down the road.
In a nutshell, it's a Python application that runs on a Raspberry Pi and reads notifications off a queue. The notifications are then fed into Amazon's Polly service and the resulting audio is played through a small set of speakers hooked up to my Pi.
On Saturday, however, we had a power outage due to a nasty storm that came through, and after the power came back on, my application no longer worked. The Pi would boot just fine, so it didn't get surged in the storm, but my text-to-speech (TTS) application stopped speaking.
Digging into the logs a bit, I saw that the application couldn't connect to an API I built, which is hosted on an AWS EC2 instance. The API provides the necessary configuration information for the RabbitMQ broker to which the application connects, so without that information, the application didn't know how to connect to my broker.
The recurring error message I saw was 'getrandom() initialization failed.':
The image doesn't show the full stack, but this same error message bubbled up through the requests library that I use to make the HTTP calls to my API, so I searched for this error message alongside the "requests" package name.
I didn't find much with this search, because I failed to scroll up and see that the original error wasn't coming from the requests package, but instead, was coming from OpenSSL. Once I refined my search to include "OpenSSL" instead of "requests", I found the answer for which I was looking here.
The accepted answer was to just install the rng-tools package on the Pi, which I did by running:
sudo apt-get install rng-tools
and to my surprise, it fixed everything. However, it made no sense. Why in the hell was the application working just fine, and then just stopped after a reboot? And why did installing a package that I didn't need from the start fix the issue? What the hell is going on?
For those answers, I had to dig a little deeper, and the answers all lie in the concept of entropy, which was hinted at in the accepted answer on that forum post I just mentioned.
I'm not going to dig too deep here, as I don't believe I'll even come close to doing the topic of entropy justice, but from what I've gathered, there's an "entropy pool" that applications can pull from for generating random numbers, which is heavily used in security/cryptographic situations. Since my API utilizes certificates and I was making HTTPS requests, the underlying OpenSSL library was pulling from this pool to generate random numbers for what it does.
This pool is built from a number of different sources, but it can take longer to build on a headless system, such as my Pi. I don't have a mouse, keyboard, or monitor hooked up to my Pi, because I don't use it for any sort of development or desktop applications. Right now, it just sits on my network and "speaks" when it's told. That's it.
What was happening was the entropy pool was being depleted, but it wasn't being built back up quickly enough, so any code that needed to generate random numbers was blocked.
Installing the rng-tools package helps build the pool up more quickly, so that's why installing that package fixed the issue.