Memory leaks with Domoticz

Hello,
since marsh 2022, I used Deconz on my Domoticz server, I can see a memory leak increasing up to slowing totally Domoticz and I need to restart. Actually, I used Deconz on a deported Raspberry between december and marsh but I did not notice the memory leak.
If I restart the phoscon gateway, everything goes back normal during few weeks.
Someone has an idea to help?

Hello I think it’s more a domoticz issue than a deconz issue (or perhaps a plugin issue)
I have domoticz and deconz too on the same machine

For information, domoticz make lot of change ATM on the python plugin management.
On my side I m using the build 14315

Can you try to just restart the plugin instead of restarting the deconz sofware ?
I m updating my version to the same build than you to test.
If you think it s a plugin issue, better to use this github https://github.com/Smanar/Domoticz-deCONZ

Perhaps some infos here Python framework memory leak · Issue #5234 · domoticz/domoticz · GitHub

hello,
thanx for your answer. Actually, I updated my domoticz version this week to 14352 (because I thought that it could solve my problem).
Before, I used to use the 14315. You probably right, I will try to stop the plugin during 1hr. We’ll see.

After 2h with plugin stopped, the memory leak continues.

Ok so restarting the plugin using the domoticz hardware tab (using the “update” button) do nothing.
But restarting deconz make a difference ?

This chart is the memory on the machine with Domoticz or the machine with deconz ? Both are raspberry ?

Still nothing on my side with last version.

Both Deconz and Domoticz are on the same raspberry.

So this was your previous installation, now you have both on the same machine ?

Ok so restarting the plugin using the domoticz hardware tab (using the “update” button) do nothing.
But restarting deconz make a difference ?

Can you just confirm that ?

If restarting totaly the plugin on domoticz (disable and re enable it for exemple) do nothing but restarting deconz do, It can be a deconz issue.
Else it s surely a domoticz or plugin issue and better to use the plugin github or domoticz forum (I m on the english and french one)

(For information restarting deconz, clear lot of thing in the plugin too in same time)

BTW what is your deconz version ?

Hello do you have some news with your issue ?

Hello,
I tried to stop all scripts and all materials (including deconz) on my Domoticz last week-end and ram continues to increase. It is not understandable.

So it continue to increase with all python plugins disabled ?
I haven’t see recent memory leak discover on their forum.

You have disabled too python script ? (The last memory leak I know)

yes, all lua and python scripts.

So the issue can come from third app.
Can be deconz, can you try the command
top

You will see a table with the cpu and memory used by application, just compare on some days to find the guilty.

What is your deconz version ?

Package: deconz
Version: 2.16.01-raspbian-buster-stable

So ATM 6.9% by domoticz and 7.6% by deconz, will see in some time

hello,
after 17h since the last screenshot, we see that only deCONZ has increased:

Ha yes, 1 % is not big, but yes It seem the guilty is Deconz for the moment.
I m at 7.5% the 26/06 on my side (same machine than you)

I will ask to others devs.

6.9% by domoticz > 7.2 %
7.6% by deconz > 8.6 %
So 1 % in 2/3 days ? (on your first capture it increase by 1 % by day minimum, 16% to 17 %)

On my side 7.9 % the 28/06 so 0.4% in 2 days.

There is a devs that have noticied a possible increased CPU% over time, but for the moment nothing about a memory leak.

I’d now tend to confirm this as an issue. I currently run a customized version of the plugin to heavily reduce DB writes in order to analyze an observed increase of CPU% over time. This is occurring on my production system as well as at a good frient of mine.

I started deconz last about 2 weeks ago where it begun with around ~3% CPU usage and 6% RAM usage. See for yourself where I ended up now

grafik

Average load has also more than trippled, hardware is an RPi Zero 2 and I used to run deconz on the predecessor back in the days without any problems.

Just for the sake of comparison, this is how it is supposed to look like (2 hrs after deconz restart, memory is increased as I’m now running the vanilla plugin)

grafik

While trying to unsuccessfully hunting down this behavior, I spotted some other oddities with strace, which got already fixed in the meantime. What remains is also the strace observation, that subsequent syscalls start to get slower and slower. In the below example, those are the top values where the first number is in seconds and it’s hughe (taken from a ~5mins strace)!

     0.282925 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000056>
     0.283216 poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}], 6, 0) = 0 (Timeout) <0.000076>
     0.283577 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000068>
     0.283602 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000064>
     0.283691 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000063>
     0.284479 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000071>
     0.284547 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000083>
     0.285975 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000063>
     0.286700 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000064>
     0.287339 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000055>
     0.287499 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000054>
     0.294066 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000059>
     0.294361 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000055>
     0.312549 poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}], 6, 0) = 0 (Timeout) <0.000091>
     0.505603 poll([{fd=3, events=POLLIN}, {fd=5, events=POLLIN}, {fd=10, events=POLLIN}, {fd=11, events=POLLIN}, {fd=13, events=POLLIN}, {fd=14, events=POLLIN}], 6, 0) = 1 ([{fd=13, revents=POLLIN}]) <0.000071>
     0.506594 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000056>
     0.522900 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000062>
     0.778827 write(3, "\1\0\0\0\0\0\0\0", 8) = 8 <0.000078>

Bottom line’s that somewhere, there’s a heavy foot on the break. It seems to more drastically observable on enviromnents with less powerful resources (like in my case the RPi Zero 2), but takes days to get apparent.

Today, I’ll update to the new release and start some (hopefully more enlighening) memory analysis. Maybe that gives some directions after a few days…

Thx, I m happy because on my side, my fu**ing energy provider make a “energy micro cut” every 2 days, impossible to make logging for more than 3 days …