I have no questions, just wanted to report a possible memory leak bug:
Large System (6xQuattro 15k with 22xMPPT RS450/200 and 500kWh batteries, Ekrano GX)
Running 3.64 and VenusOS has run out of memory.
System load is high, I could only capture this screenshot before the watchdog rebooted the system.
Logs of /var/log/venus-platform/current are full with:
@4000000068a1839a0b13f1f4 OUT OF MEM
After the automatic watchdog reboot everything is back fine. Let’s see for how much longer.
Any mods or nodered flows?
No and yes.
In the old pre-v2 Gui we were running SetupHelper+Tailscale, but this is now gone and moved to an external management box. So no mods left.
The Mod-Check on 3.64 fails anyways because we have set an SSH password which counts as modding I guess.
We have one Node-Red flow running, but this is absolutely small-scale, just 3 Nodes making changes to the grid set-point.
This specific flow is running on a number of our installations, no problems anywhere so far.
I am not sure what to think, from the screenshot I posted you can see that dbus-daemon and venus-platform are using larger amounts of RAM than usual.
I think that I have a similar problem. My system is composed by a Multiplus GX set in ESS mode with a DYI battery with a JK inverter BMS (connected via CAN) and a VM-3P75CT power meter (connected via ethernet). I use the modbus TCP server to interface the ESS with Home assistant and I use these two DBUS projects to grab the solar inverter production:
Since 2-3months I noticed that the multiplus II is randomly not reachable for about 5min every 3-4days. After further investigation I saw that the problem is a reboot caused by the watchdog and the problem is the out of memory of the GX system.
Here the watchdog processlist captured this just before the reboot (I copy just the first lines with the most memory/cpu hungry process):
In this case I was running the venus version V3.64:
Mem: 498300K used, 10844K free, 6660K shrd, 624K buff, 23248K cached
CPU: 52% usr 38% sys 0% nic 0% idle 8% io 2% irq 0% sirq
Load average: 27.24 12.52 6.09 7/282 32660
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
744 724 root S 97656 19% 28% /opt/victronenergy/venus-platform/venus-platform
48 2 root RW 0 0% 12% [kswapd0]
819 801 root R 32588 6% 10% {dbus_characterd} /usr/bin/python3 -u /opt/victronenergy/dbus-characterdisplay/dbus_characterdisplay.py
32645 789 root D 11936 2% 6% {dbus-modbus-cli} /usr/bin/python3 -u /opt/victronenergy/dbus-modbus-client/dbus-modbus-client.py
32655 32638 root R 2704 1% 4% top -b -n1
499 498 messageb S 172m 35% 2% dbus-daemon --system --nofork
1414 1406 root S 45580 9% 2% /usr/bin/flashmq
810 779 root S 29284 6% 2% {dbus_systemcalc} /usr/bin/python3 -u /opt/victronenergy/dbus-systemcalc-py/dbus_systemcalc.py
1436 1432 root S 22540 4% 2% /opt/victronenergy/dbus-modbustcp/dbus-modbustcp
759 730 simple-u S 10040 2% 2% /bin/simple-upnpd --xml /var/run/simple-upnpd.xml -d
1336 1325 root S 7800 2% 2% /opt/victronenergy/vedirect-interface/vedirect-dbus -v --log-before 25 --log-after 25 -t 0 --banner -s /dev/ttyS1
752 725 root S 1912 0% 2% multilog t s25000 n4 /var/log/venus-platform
31693 2 root IW< 0 0% 2% [kworker/1:2H-kb]
32491 2 root DW< 0 0% 2% [kworker/0:0H+kb]
1136 770 root D< 125m 25% 0% /opt/victronenergy/gui/gui -neatvnc 127.0.0.1
1411 1408 root S 46988 9% 0% {mqtt-rpc.py} /usr/bin/python3 -u /opt/victronenergy/mqtt-rpc/mqtt-rpc.py
754 722 root D 43840 9% 0% {vrmlogger.py} /usr/bin/python3 -u /opt/victronenergy/vrmlogger/vrmlogger.py
1156 1122 root S 35432 7% 0% python /data/dbus-opendtu/dbus_opendtu.py
747 741 root S 34164 7% 0% {netmon} /usr/bin/python3 -u /opt/victronenergy/netmon/netmon
760 748 root S 32688 6% 0% {localsettings.p} /usr/bin/python3 -u /opt/victronenergy/localsettings/localsettings.py --path=/data/conf
1141 1120 root S 30028 6% 0% python -u /data/dbus-huaweisun2000-pvinverter/dbus-huaweisun2000-pvinverter.py
791 781 root S 28212 6% 0% {dbus_shelly.py} /usr/bin/python3 /opt/victronenergy/dbus-shelly/dbus_shelly.py
Second case with the latest venus V3.65 version:
Mem: 503324K used, 5820K free, 1264K shrd, 284K buff, 18408K cached
CPU: 0% usr 33% sys 0% nic 2% idle 60% io 2% irq 0% sirq
Load average: 28.41 14.66 7.73 6/282 6460
PID PPID USER STAT VSZ %VSZ %CPU COMMAND
48 2 root SW 0 0% 20% [kswapd0]
806 786 root D 22400 4% 1% /opt/victronenergy/dbus-fronius/dbus-fronius
1388 1383 root D 46604 9% 1% /usr/bin/flashmq
1129 757 root D< 86680 17% 1% /opt/victronenergy/gui/gui -neatvnc 127.0.0.1
1442 1439 root D 22584 4% 1% /opt/victronenergy/dbus-modbustcp/dbus-modbustcp
739 714 root D 97656 19% 1% /opt/victronenergy/venus-platform/venus-platform
5869 774 root Z 0 0% 1% [dbus-modbus-cli]
754 752 root D 20772 4% 1% /opt/victronenergy/hub4control/hub4control
4183 2 root DW< 0 0% 1% [kworker/1:0H+kb]
5778 2 root IW< 0 0% 1% [kworker/0:3H-kb]
720 711 root D 43952 9% 1% {vrmlogger.py} /usr/bin/python3 -u /opt/victronenergy/vrmlogger/vrmlogger.py
738 721 simple-u D 10040 2% 1% /bin/simple-upnpd --xml /var/run/simple-upnpd.xml -d
1217 1199 root D 29868 6% 0% python -u /data/dbus-huaweisun2000-pvinverter/dbus-huaweisun2000-pvinverter.py
1486 1468 root D 3268 1% 0% /opt/victronenergy/mk2-dbus/mk2-dbus --log-before 25 --log-after 25 --banner -w -s /dev/ttyS3 -i -t mk3 --settings /data/var/lib/mk2-dbus/mkxport-ttyS3.settings
6454 6444 root R 2704 1% 0% top -b -n1
535 1 avahi D 2916 1% 0% avahi-daemon: running [venus.local]
810 794 root D 32592 6% 0% {dbus_characterd} /usr/bin/python3 -u /opt/victronenergy/dbus-characterdisplay/dbus_characterdisplay.py
781 764 root D 29288 6% 0% {dbus_systemcalc} /usr/bin/python3 -u /opt/victronenergy/dbus-systemcalc-py/dbus_systemcalc.py
6456 6455 root D 2860 1% 0% [ip]
1223 1207 root D 35416 7% 0% python /data/dbus-opendtu/dbus_opendtu.py
1288 1287 root D 2832 1% 0% /opt/victronenergy/can-bus-bms/can-bus-bms --log-before 25 --log-after 25 -vv -c socketcan:can0 --banner
816 811 root D 24320 5% 0% {dbus-acsystem.p} /usr/bin/python3 /opt/victronenergy/dbus-acsystem/dbus-acsystem.py
1928 1460 root D 10936 2% 0% ssh -o ExitOnForwardFailure=yes -o ConnectTimeout=20 -o ServerAliveInterval=10 -o ServerAliveCountMax=3 -o TCPKeepAlive=yes -o StrictHostKeyChecking=yes -p 22 -N -R 0:localhost:22 -i /usr/share/ssh-tunnel/id_rsa ccgxlogin@supporthosts.victronenergy.com
493 491 messageb D 199m 40% 0% dbus-daemon --system --nofork
1333 1327 root S 7800 2% 0% /opt/victronenergy/vedirect-interface/vedirect-dbus -v --log-before 25 --log-after 25 -t 0 --banner -s /dev/ttyS1
5017 2 root IW< 0 0% 0% [kworker/2:1H-kb]
6328 2 root IW< 0 0% 0% [kworker/3:0H-kb]
1389 1385 root S 46968 9% 0% {mqtt-rpc.py} /usr/bin/python3 -u /opt/victronenergy/mqtt-rpc/mqtt-rpc.py
I see that both the “dbus-daemon --system --nofork” and “/opt/victronenergy/venus-platform/venus-platform” process use a lot more ram than they normally do.
Any idea on what can be the problem here?
With the latest version V3.66 the memory leak seems resolved 
