Log2ram is a package out there that builds ram drives for highly volatile files (i.e. logs, db, etc…). Part of the package is writing the logs back to disk once a day (configurable via Cron).
Would there be interest in integrating this into the HestiaPi, with the interest of saving the SD card from dying an untimely death?
My opinion would be no, this isn’t a good idea for the HestiaPi, at least when running the default openHAB. The two biggest limiting factors are the single core CPU and RAM. There simply isn’t enough RAM so at some points, particularly when the machine is starting up, the swap gets used.
Moving the logs to RAM is going to decrease the amount of RAM available which will cause the swap partition to be used even more. I’d not be surprised if doing this would increase the number of writes to the SD card as a result, not reduce it.
ZRAM might be an option as it compresses the file system that’s in RAM letting one put more in there without consuming as much RAM. But then the fact that we have only a single CPU might slow everything down because the CPU has more to do. And even taking a little bit of RAM for logs is going to increase the use of the swap. You can look at the openHABian scripts for details on how to do it.
I think the best option is to just retain backups of your HestiaPi SD card, or at a minimum backups of any changes you’ve made to the configs. Unless and until HestiaPi switches to something with more RAM using tempfs/zram type partitions is not going to be an option. Note: I’m not necessarily advocating for this.
In the work I’ve done to improve the boot times and flexibility for users to modify the configs through the browser instead of editing files on the Pi itself, I’ve also modified the logging to greatly reduce the amount that gets logged to the SD card in the first place, at least from OH. It’s way low priority, but I also have plans to add a button to BasicUI to turn the logging on/off entirely to further reduce the writes. There isn’t much else writing to the SD card on the HestiaPi.
Thank you for the thorough response, it seems like you thought through it a good bit.
It sounds like the boot process is very intense, and requires all the CPU/Memory that it can muster.
I have had a couple full pi’s die after getting 2-3 years use out of them, and it appears the SD card gets corrupt. My belief is that the /var/log files eventually wear out the memory cells in the SD card.
I recently came across the log2ram project, and have installed in on a number of my large pi’s, and i will see if i get longer life from the SD cards.
Since i am starting to build a HestiaPi, i was carrying the same concerns about the SD card being destroyed in 2-3 years. So that is why i posted this idea.
In watching it bootup with the touchscreen and all peripherals attached (And I have log2ram installed w/ defaults), i am watching “top” to see if swap is used. It appears near the end, I got about 3.5kb of swap used (Granted, this is my first boot, and this is 1 observation.)
The trade-off i am seeing, is that swap may be used a lot during boots. But boots are an infrequent event, whereas logging is happening almost every minute of every day.
I have it installed on my image, and will experiment with having log2ram installed, and can post an update in a few months of my results.
Thank you again for the thoughtful response.
Do you mean the hardware actually dies or simply the SD gets corrupted. I have many running around 24/7 (not HestiaPis) and never had one fail at a hardware level apart from the occasional SD corruption on power failure. Never under normal circumstances (on a legit SD with a legit power supply).
SD cards will eventually wear out for sure. There is a limited number of writes that a given SD card can support.
Unfortunately I don’t have a good measurement for how much I’ve reduced the logging over all but turning off events.log and reducing openhab.log to something close to a couple of MB per month, maybe less. I’ve tried to strike a good balance between minimizing writes and keeping track of what is going on. Looking at the logs from the past day, the default logging for v1.2 (if there are no errors) would indicate about 100 writes per day from openHAB. There will be more or less depending on changes to settings and how quickly the temperature changes so YMMV. But that’s a long long way away from one or more writes per second or even one per minute.
If we assume a three year life span with one write per minute, we should get 42 years out of the same SD card with 100 write per day.
Other writes that take place on the system are pretty few and far between (depending on how Mosquitto is configured). With a decent sized SD card, it should last much longer than three years even with v1.1. With the changes I’ve made to logging, it should extend the life of the SD card by orders of magnitude.
However, power loss is a real concern for SD card corruption. I’m not sure there is any way to prevent that totally short of running off of a read only file system which really isn’t practical.
This topic was automatically closed 91 days after the last reply. New replies are no longer allowed.