Hi,
Are there any known issues with the software locking up, maybe a memory leak or something? It takes a week or so, but the app stops working.
I just logged in with SSH and rebooted to “fix” it.
Can’t see what the screen is doing, as I broke it … again.
Never heard of that.
Use an app like “Fing” to check if the phone can ping HestiaPi and what ports it sees open.
Can you visit the web UI from a browser from that same phone while the issue is present? It should be:
http://HestiaPi_IP:8080
Thats really weird to be able to SSH into it from a laptop but not from the app.
It’s frozen up again, quicker this time. This time I tried to connect by HTTP, it seemed to work 2nd time, but I think I was just seeing the cache. Cleared the cache, no page.
Tried to SSH in, no joy. Tried ping. Dead. I was going to try and run “top” to see what was going on, but …
Is the mains connection coming from the heating system or the house?
Do you experience the issue when you try to start/stop the heating or at other random moments?
Did you check the system logs near the time of failure? It could give valuable clues.
I am having a very similar issue, except the whole thing locks up and I can’t even SSH in to reboot. The reset button works at least which seems to reduce the chance of the microSD corruption compared to switching it off at the mains but I can’t keep resetting it every day. I tested a new install on a new microSD yesterday with the same thing happening today.
The mains in my instance is coming from my boiler but it’s wired to be always live. It seems unrelated to what the actual boiler is doing.
The PI seems to really struggle to read the file system too. For example if I’m using ssh and nano or tail a file, it may decide to just stall for what seems like permanently, though I have had it come back to life after a few minutes (where as the complete lockups from before are definitely that). Starting another session and running the same command weirdly has roughly the same chance of completing it, regardless of another session attempting to do the same thing…
run
df -h
to see if you run out of space for a strange reason and
top
to check if the 3 numbers at the top line right corner are all below 1 to see if the Pi is struggling with something.
Mine are relatively low.
load average: 0.47, 0.45, 0.51
Did you get the SD card from us (Kingston with lifetime warranty)? Can you check it on another computer if it has any “bad sectors”.
For me at least:
Space is fine. top: 2.02, 1.17, 0.80. Openhab was pretty consistently 60-80% cpu usage at first but now seems to have gone down to ~6%.
SD is from you but trying a faster 64gb samsung microsd right now with seemingly no difference.
Looks like something is wrong in OpenHAB unless you are referring to the first 10-20 minutes of the booting which can be even higher!
If you’ve modified the OpenHAB installation in the provided image I would recommend reflashing the default image (v10.3 as of today) and try for a few days.
The high cpu usage would have likely been within those first 20 minutes. I had a slightly modified system which it happened from and I installed from stock (10.3) and ran the update scripts and it continued happening so I am sure reinstalling is not the answer.
I have looked through the log files and they are confusing me. At some point it appears the hourly cron job fires and then the system seemingly reboots (from the log files), but the screen is just frozen.
Dec 3 20:17:01 raspberrypi CRON[14972]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 3 20:17:10 raspberrypi systemd-modules-load[75]: Inserted module ‘i2c_dev’
Dec 3 20:17:10 raspberrypi fake-hwclock[62]: Mon 3 Dec 20:17:01 UTC 2018
Dec 3 20:17:10 raspberrypi systemd[1]: Started Apply Kernel Variables.
Dec 3 20:17:10 raspberrypi systemd[1]: Started File System Check Daemon to report status.
…
Dec 3 20:17:10 raspberrypi kernel: [ 0.000000] Booting Linux on physical CPU 0x0
…
Dec 3 20:17:38 raspberrypi dhcpcd[271]: wlan0: no IPv6 Routers available
Dec 3 20:50:41 raspberrypi systemd-timesyncd[195]: Synchronized to time server 145.239.118.233:123 (2.debian.pool.ntp.org).
Dec 3 20:50:41 raspberrypi systemd[384]: Time has been changed
Dec 3 20:50:41 raspberrypi systemd[1]: Time has been changed
Dec 3 20:50:41 raspberrypi systemd[1]: apt-daily.timer: Adding 2h 24min 41.121902s random time.
Dec 3 20:50:41 raspberrypi systemd[1]: apt-daily-upgrade.timer: Adding 13min 11.563490s random time.
In this instance the system froze presumably around 20:17 and I hit the reset button at 20:50. Could the issue be that the reboot is actually happening at 20:50 but the time is starting from the previously crashed time until it updates the time? The log files seem to show a similar pattern happening each time. The alternative explanation is that the reset button is not actually doing a proper reset, but that seems less likely.
The next thing to work out will be if it is the hourly cronjob that is responsible for the freeze or whether it’s just coincidence. I am veering towards the latter.
Raspberries don’t have a system clock. The software can keep the time though. After a reboot but before time synchronisation, the clock is running from the last saved time, hence it’s lagging behind. This can screw up your log a bit, especially if all the interesting things happen at the same recorded time (like Dec 3 20:17:10). New entries can be logged with earlier timestamps but after a later timestamp. Best practice is to post a full log for us to see what is happening.
The Dec 3 20:17:10 raspberrypi kernel: [ 0.000000] (...) should be the first recorded entry of booting. Probably around 20:17:10 was the last recorded time.
At 20:50:41 the software clock was synchronised with the timeserver.
What is logged around 20:17:10?
Dec 3 20:17:01 raspberrypi CRON[14972]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 3 20:17:10 raspberrypi systemd-modules-load[75]: Inserted module 'i2c_dev'
Dec 3 20:17:10 raspberrypi fake-hwclock[62]: Mon 3 Dec 20:17:01 UTC 2018
Dec 3 20:17:10 raspberrypi systemd[1]: Started Apply Kernel Variables.
Dec 3 20:17:10 raspberrypi systemd[1]: Started File System Check Daemon to report status.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Create Static Device Nodes in /dev.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting udev Kernel Device Manager...
Dec 3 20:17:10 raspberrypi systemd-fsck[94]: rootfs: clean, 109551/3797024 files, 1132572/15338880 blocks
Dec 3 20:17:10 raspberrypi systemd[1]: Started File System Check on Root Device.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Remount Root and Kernel File Systems...
Dec 3 20:17:10 raspberrypi systemd[1]: Started udev Kernel Device Manager.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Remount Root and Kernel File Systems.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Flush Journal to Persistent Storage...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Load/Save Random Seed...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting udev Coldplug all Devices...
Dec 3 20:17:10 raspberrypi systemd[1]: Started Load/Save Random Seed.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Flush Journal to Persistent Storage.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Set the console keyboard layout.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Local File Systems (Pre).
Dec 3 20:17:10 raspberrypi systemd[1]: tmp.mount: Directory /tmp to mount over is not empty, mounting anyway.
Dec 3 20:17:10 raspberrypi systemd[1]: Mounting /tmp...
Dec 3 20:17:10 raspberrypi systemd[1]: Mounting /var/tmp...
Dec 3 20:17:10 raspberrypi systemd[1]: Mounted /tmp.
Dec 3 20:17:10 raspberrypi systemd[1]: Mounted /var/tmp.
Dec 3 20:17:10 raspberrypi systemd[1]: Started udev Coldplug all Devices.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Show Plymouth Boot Screen...
Dec 3 20:17:10 raspberrypi systemd[1]: Received SIGRTMIN+20 from PID 139 (plymouthd).
Dec 3 20:17:10 raspberrypi systemd[1]: Started Show Plymouth Boot Screen.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Encrypted Volumes.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Paths.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Forward Password Requests to Plymouth Directory Watch.
Dec 3 20:17:10 raspberrypi systemd[1]: Found device /dev/serial1.
Dec 3 20:17:10 raspberrypi systemd[1]: Found device /dev/ttyS0.
Dec 3 20:17:10 raspberrypi systemd[1]: Found device /dev/disk/by-partuuid/02a42963-01.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting File System Check on /dev/disk/by-partuuid/02a42963-01...
Dec 3 20:17:10 raspberrypi systemd-fsck[160]: fsck.fat 4.1 (2017-01-24)
Dec 3 20:17:10 raspberrypi systemd-fsck[160]: 0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
Dec 3 20:17:10 raspberrypi systemd-fsck[160]: Automatically removing dirty bit.
Dec 3 20:17:10 raspberrypi systemd-fsck[160]: Performing changes.
Dec 3 20:17:10 raspberrypi systemd-fsck[160]: /dev/mmcblk0p1: 176 files, 45013/87078 clusters
Dec 3 20:17:10 raspberrypi systemd[1]: Started File System Check on /dev/disk/by-partuuid/02a42963-01.
Dec 3 20:17:10 raspberrypi systemd[1]: Mounting /boot...
Dec 3 20:17:10 raspberrypi systemd[1]: Mounted /boot.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Local File Systems.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Raise network interfaces...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Set console font and keymap...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Preprocess NFS configuration...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Create Volatile Files and Directories...
Dec 3 20:17:10 raspberrypi systemd[1]: Started Set console font and keymap.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Tell Plymouth To Write Out Runtime Data.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Preprocess NFS configuration.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target NFS client services.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Remote File Systems (Pre).
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Remote File Systems.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Create Volatile Files and Directories.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Network Time Synchronization...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Dec 3 20:17:10 raspberrypi systemd[1]: Started Update UTMP about System Boot/Shutdown.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Network Time Synchronization.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target System Time Synchronized.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target System Initialization.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target System Initialization.
Dec 3 20:17:10 raspberrypi systemd[1]: Listening on D-Bus System Message Bus Socket.
Dec 3 20:17:10 raspberrypi systemd[1]: Started Daily Cleanup of Temporary Directories.
Dec 3 20:17:10 raspberrypi systemd[1]: Listening on triggerhappy.socket.
Dec 3 20:17:10 raspberrypi systemd[1]: Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Sockets.
Dec 3 20:17:10 raspberrypi systemd[1]: Reached target Basic System.
Dec 3 20:17:10 raspberrypi systemd[1]: Starting System Logging Service...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting LSB: Switch to ondemand cpu governor (unless shift key is pressed)...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Configure Bluetooth Modems connected by UART...
Dec 3 20:17:10 raspberrypi systemd[1]: Starting Avahi mDNS/DNS-SD Stack...
Dec 3 20:17:10 raspberrypi systemd[1]: Started D-Bus System Message Bus.
Dec 3 20:17:10 raspberrypi liblogging-stdlog: [origin software="rsyslogd" swVersion="8.24.0" x-pid="221" x-info="http://www.rsyslog.com"] start
Dec 3 20:17:10 raspberrypi kernel: [ 0.000000] Booting Linux on physical CPU 0x0
and another:
Dec 2 14:17:01 raspberrypi CRON[19690]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 2 14:17:11 raspberrypi systemd-modules-load[74]: Inserted module 'i2c_dev'
Dec 2 14:17:11 raspberrypi fake-hwclock[60]: Sun 2 Dec 14:17:01 UTC 2018
Dec 2 14:17:11 raspberrypi systemd[1]: Mounted Configuration File System.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Apply Kernel Variables.
Dec 2 14:17:11 raspberrypi systemd[1]: Started File System Check Daemon to report status.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Create Static Device Nodes in /dev.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting udev Kernel Device Manager...
Dec 2 14:17:11 raspberrypi systemd-fsck[95]: rootfs: clean, 109476/3797024 files, 1131130/15338880 blocks
Dec 2 14:17:11 raspberrypi systemd[1]: Started File System Check on Root Device.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Remount Root and Kernel File Systems...
Dec 2 14:17:11 raspberrypi systemd[1]: Started udev Kernel Device Manager.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Remount Root and Kernel File Systems.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting udev Coldplug all Devices...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Load/Save Random Seed...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Flush Journal to Persistent Storage...
Dec 2 14:17:11 raspberrypi systemd[1]: Started Load/Save Random Seed.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Set the console keyboard layout.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Flush Journal to Persistent Storage.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Local File Systems (Pre).
Dec 2 14:17:11 raspberrypi systemd[1]: tmp.mount: Directory /tmp to mount over is not empty, mounting anyway.
Dec 2 14:17:11 raspberrypi systemd[1]: Mounting /tmp...
Dec 2 14:17:11 raspberrypi systemd[1]: Mounting /var/tmp...
Dec 2 14:17:11 raspberrypi systemd[1]: Mounted /var/tmp.
Dec 2 14:17:11 raspberrypi systemd[1]: Mounted /tmp.
Dec 2 14:17:11 raspberrypi systemd[1]: Started udev Coldplug all Devices.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Show Plymouth Boot Screen...
Dec 2 14:17:11 raspberrypi systemd[1]: Received SIGRTMIN+20 from PID 131 (plymouthd).
Dec 2 14:17:11 raspberrypi systemd[1]: Started Show Plymouth Boot Screen.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Paths.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Encrypted Volumes.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Forward Password Requests to Plymouth Directory Watch.
Dec 2 14:17:11 raspberrypi systemd[1]: Found device /dev/serial1.
Dec 2 14:17:11 raspberrypi systemd[1]: Found device /dev/ttyS0.
Dec 2 14:17:11 raspberrypi systemd[1]: Found device /dev/disk/by-partuuid/02a42963-01.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting File System Check on /dev/disk/by-partuuid/02a42963-01...
Dec 2 14:17:11 raspberrypi systemd-fsck[153]: fsck.fat 4.1 (2017-01-24)
Dec 2 14:17:11 raspberrypi systemd-fsck[153]: 0x41: Dirty bit is set. Fs was not properly unmounted and some data may be corrupt.
Dec 2 14:17:11 raspberrypi systemd-fsck[153]: Automatically removing dirty bit.
Dec 2 14:17:11 raspberrypi systemd-fsck[153]: Performing changes.
Dec 2 14:17:11 raspberrypi systemd-fsck[153]: /dev/mmcblk0p1: 176 files, 45013/87078 clusters
Dec 2 14:17:11 raspberrypi systemd[1]: Started File System Check on /dev/disk/by-partuuid/02a42963-01.
Dec 2 14:17:11 raspberrypi systemd[1]: Mounting /boot...
Dec 2 14:17:11 raspberrypi systemd[1]: Mounted /boot.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Local File Systems.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Set console font and keymap...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Raise network interfaces...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Preprocess NFS configuration...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Create Volatile Files and Directories...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Tell Plymouth To Write Out Runtime Data...
Dec 2 14:17:11 raspberrypi systemd[1]: Started Set console font and keymap.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Preprocess NFS configuration.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Tell Plymouth To Write Out Runtime Data.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target NFS client services.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Remote File Systems (Pre).
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target Remote File Systems.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Create Volatile Files and Directories.
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Update UTMP about System Boot/Shutdown...
Dec 2 14:17:11 raspberrypi systemd[1]: Starting Network Time Synchronization...
Dec 2 14:17:11 raspberrypi systemd[1]: Started Update UTMP about System Boot/Shutdown.
Dec 2 14:17:11 raspberrypi systemd[1]: Started Network Time Synchronization.
Dec 2 14:17:11 raspberrypi systemd[1]: Reached target System Initialization.
Dec 2 14:17:11 raspberrypi systemd[1]: Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
Dec 2 14:17:11 raspberrypi kernel: [ 0.000000] Booting Linux on physical CPU 0x0#
and a third:
Dec 1 17:17:01 raspberrypi CRON[18545]: (root) CMD ( cd / && run-parts --report /etc/cron.hourly)
Dec 1 17:17:12 raspberrypi systemd-modules-load[71]: Inserted module 'i2c_dev'
Dec 1 17:17:12 raspberrypi fake-hwclock[62]: Sat 1 Dec 17:17:01 UTC 2018
Dec 1 17:17:12 raspberrypi systemd[1]: Mounted Configuration File System.
Dec 1 17:17:12 raspberrypi systemd[1]: Started Apply Kernel Variables.
Dec 1 17:17:12 raspberrypi systemd[1]: Started File System Check Daemon to report status.
Dec 1 17:17:12 raspberrypi systemd[1]: Started Create Static Device Nodes in /dev.
Dec 1 17:17:12 raspberrypi systemd[1]: Starting udev Kernel Device Manager...
Dec 1 17:17:12 raspberrypi systemd-fsck[94]: rootfs: clean, 109500/3797024 files, 1129106/15338880 blocks
Dec 1 17:17:12 raspberrypi systemd[1]: Started File System Check on Root Device.
Dec 1 17:17:12 raspberrypi systemd[1]: Starting Remount Root and Kernel File Systems...
Dec 1 17:17:12 raspberrypi systemd[1]: Started Remount Root and Kernel File Systems.
Dec 1 17:17:12 raspberrypi systemd[1]: Starting Load/Save Random Seed...
Dec 1 17:17:12 raspberrypi systemd[1]: Starting udev Coldplug all Devices...
Dec 1 17:17:12 raspberrypi systemd[1]: Starting Flush Journal to Persistent Storage...
Dec 1 17:17:12 raspberrypi systemd[1]: Started udev Kernel Device Manager.
Dec 1 17:17:12 raspberrypi kernel: [ 0.000000] Booting Linux on physical CPU 0x0Is anyone else facing this? I don’t see anything suspicious in the logs.
These logs are all from after boot. The problem, if logged at all, could be in the entries above these quoted.
Perhaps you can pastebin a complete log?
So I thought it was a long shot but as the hourly cron job appears to consistently be the last thing that logs before it freezes, I disabled the fake-hwclock job and so far it has run 4 days without crashing whereas it was very rare to reach the 2nd day previously. I only had to restart it yesterday because it stopped turning the heating pin on (another problem I seem to get frequently…). Obviously needs a little longer to see if this is purely coincidence as I can’t see any reason why fake-hwclock would cause a lock up.
Are you using the original hardware and original image? Try running the update script too first before testing any issues.
Have you set the one or two setpoints accordingly higher than the current temperature? Obviously it does not turn any heating on otherwise.
Did restart solve the problem? If not please describe the conditions under which the issue happens or open an issue on github.
For what it’s worth, mine hasn’t done this since I last posted. Keeping an eye on it though.
Spoke too soon, froze again.
I’ve wired it in to a socket bypassing the boiler (aside from the comm wire) to see if it makes any difference but I’m not expecting anything.
Again, I am using original images and have run the update script.
The Hot water is controlled separately by the boiler and needs to be on for the central heating to kick in, whether this interferes with the communication wire for the central heating when it turns off I am not sure. But there is definitely no click or activity from the boiler with the temperature set point set correctly on occasion and restarting the HestiaPI fixes it.
I am suspecting a faulty Pi. This is really strange and as you have used another SD too and had the same issue, the Pi is the only thing that could cause what you face. Can you spot any physical damage to a component on the Pi. Not sure I can be more specific. Take a couple of minutes and compare it with another Pi W you may have or bring a picture of it on your screen.
