Venus OS v3.67~2 available for public testing

Beta testing Venus OS beta testing
,
1

Good morning today!

v3.70 beta testing is temporarily on hold to validate v3.67 before making that an official version.

The latest v3.70 beta version install files are still available for download for those who need them, here. To install, use the off line .swu method documented in the GX manuals.

Full change log below, and first please make yourself aware with the beta testing instructions:

Instructions 1 of 3: Venus OS beta testing & how to join/install

In case you don’t know what this message is about, please start with reading this link, which explains the Venus OS beta program. Venus OS is the software running on all our GX devices, such as the Cerbo GX.

Instructions 2 of 3: How to report an issue?

Before posting, please check if your issue already exists, in which case please contribute by replying to that issue or up-voting it.

Only once you are sure its a new possible issue, start a new topic.

Lastly, before posting, preferably first revert to the latest official release, to double check if the issue you’re seeing was present there as well. Regressions require a different treatment than other issues and bugs. And let us know if you did that, and what the difference is. A perfect report contains:

  • How its expected to work
  • How it doesn’t work as expected
  • Details on the system
  • What you’ve already tried to make sure its not a setting or other issue.

Please take note that that this works quite differently than it used to do on the old version of Community:

  • For a new issue, open a new topic. Instead of adding a new reply at the bottom.
  • To add to an already open issue, aka topic, reply to that. Or just upvote it if you have the same issue. This will help us a lot in determining its importance and manage our priorities.

Instructions 3 of 3: using Node-RED, Kevin’s SetupHelper, GuiMods, dbus-serialbattery or other add-ons?

Node-RED or Signal K

In case you are running Node-RED or SignalK, then please at least say so in your bug report.

SetupHelper, GuiMods, dbus-serialbattery

Modifications such as the popular GuiMods, SetupHelper, dbus-serialbattery and other 3rd party add-ons: do not report your issues here. Please do it elsewhere instead; and make sure to check out the related issue tracker first.

These topics and beta testing and bug reports are only for clean systems.

Change log

v3.67~2

  • Add support for Gobius C tank sensors, wirelessly (Bluetooth)
  • Add support for the new Ruuvi Air sensors. Note that this only involves read-out of temperature and other . Read-out of the air quality parameters will be part of the -future- Venus OS v3.70 release.
  • Lynx BMS NG: Add cell IDs for the cells having the lowest and highest voltage as well as the cells having the lowest and highest temperature. The cell id consists of the last part of the Batteries serial number, then a dash, and then the number of the cell.The IDs are displayed in the GX menu’s like here:
    image
    image1482×516 51.1 KB

    And here on the VRM Portal on the Advanced page in the graphs:
    image
    image1094×455 38.8 KB
  • ESS: Fix bug in peak shaving function when the “Self-consumption from battery” setting is set to “Only critical loads”.
  • DVCC: Fix bug where charge current limit on the Multi in systems without a BMS, or systems with a VE.Bus BMS, would not be correctly applied: switching the Multi off, and then adjusting the Maximum charge current limit in the DVCC menu, would not be applied when the Multi was turned back on. Instead the Multi would charge at the maximum it was configured for.
  • Fix Error mk2vsc-61 which could occur during during Remote VE.Bus firmware updates / remote configuration.
  • Fix the Genset Daily run time menu showing empty lines in the Classic UI.
  • GX IO-Extender: Fix invalid “Output fault” being raised when switching the bi-stable relays rapidly.
  • Fix bug in combination with Wakespeed alternators, that resulted in all data continuously appearing and disappearing again from the GX display when the battery is full in combination with a few more specific conditions.
  • Inverter RS: Add State of charge & Time to go information
  • Lower level / core OS security updates & patches
2

Still getting unexplained memory loss using 3.6x. Below 3.55 rock-solid stable. When upgrading to 3.67 the memory usage increased, probably due to newer GUI binary, but then as soon as I open the GUI v1 interface and make even a small change like Grid Setpoint change it triggers a memory leak that just goes forever. below a capture illustrating the effect.

image
image559×404 21.9 KB

3

Hi @IngoDeJager , thanks for reporting this again.

Is this a clean system or a modified one? You can see that in the new UI, Settings → General → Modifications.

We’ve not been able to reproduce this issue, and as you’ll know are not very keen to dive into modified systems - lots of valuable hours have already gone into troubleshooting systems to then find out it was something very specific due to the modification.

and ps. regardless of what is exactly happening there: this won’t be a show stopper for releasing v3.67, as it wasn’t introduced per v3.67 either.

4

Hi @mpvader , I would say both.

Initially I removed all my additions and it made no difference at all so I put it back. If you are willing to have a look you are welcome to make any change you need to. I can remove both my additions and test again but I am 100% sure it’s not my scrips. It only happens once I use the V1 interface and make a change.

5

What are you using to monitor the memory usage? (I might have asked this before)
I have multiple Cerbo GX’es running v3.66 for >30 days and can’t say I have issues that would point to a memory leak, even though I’m using GUIv1 a lot.
With a simple NodeRed function I’m monitoring the OS uptime and the NodeRed uptime.

Record issue-free uptime for both VenusOS and NodeRed is 102 days on a Pi4 running v3.63.
(All others have been upgraded & rebooted in the meanwhile)

6

I am using Home Assistant and a SSH integration. It logs into the device every so often and run a few commands to extract the data. The commands are pure linux shell commands so is nothing that the Cerbo can’t handle. The output is then fed back to HA as sensor values. All you need is to set a root password for it to be able to login. No other changes required.

PS. I need to add that I would never have found this if I didn’t start looking at free memory due to the previous venus-platform memory leak which did reboot the Cerbo. This issue doesn’t restart the device, the free memory drops to around 20MB to 25MB and then hovers around there. The CPU usage does go up and it gets hotter, all things someone not actively logging these changes will not notice.

7

I quickly hacked together something in Node Red.
Doesn’t contain logging or anything but should indicate when things go south.
Could maybe serve as an input to trigger alerts, or to visualize a trend.

[
    {
        "id": "84326389e55610fc",
        "type": "inject",
        "z": "890d9817cf94867b",
        "name": "",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "10",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 150,
        "y": 180,
        "wires": [
            [
                "a3f84e6b648f5d00",
                "5a9d999a920964a5",
                "316f4168bcd064fb",
                "91d57b96e8ed0bf0"
            ]
        ]
    },
    {
        "id": "91d57b96e8ed0bf0",
        "type": "function",
        "z": "890d9817cf94867b",
        "name": "Loadavg",
        "func": "let loadavg = os.loadavg(),\n    onemin = { payload: loadavg[0] },\n    fivemin = { payload: loadavg[1] },\n    fifteenmin  = { payload: loadavg[2] };\n\nnode.status({text: loadavg[0] + \", \" + loadavg[1] + \", \" + loadavg[2]});\nreturn [onemin, fivemin, fifteenmin];\n//return msg;",
        "outputs": 3,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [
            {
                "var": "os",
                "module": "os"
            }
        ],
        "x": 380,
        "y": 100,
        "wires": [
            [
                "f056a9d9b907d6b1"
            ],
            [
                "18dacfef8fdf6d31"
            ],
            [
                "b5efd8dc021f6f23"
            ]
        ]
    },
    {
        "id": "a3f84e6b648f5d00",
        "type": "function",
        "z": "890d9817cf94867b",
        "name": "OS Uptime",
        "func": "Uptime_In_Seconds = os.uptime();\nfunction format(sec){\n  function pad(s){\n    return (s < 10 ? '0' : '') + s;\n  }\n  var seconds = Math.floor(sec % 60);\n  var minutes = Math.floor(sec % (3600) / 60);\n  var hours = Math.floor(sec % (3600*24) / 3600);\n  var days = Math.floor(sec / (3600*24));\n  var dayspadded = days > 0 ? days + (days == 1 ? \" day, \" : \" days, \") : \"\";\n  return dayspadded + pad(hours) + ':' + pad(minutes) + ':' + pad(seconds);\n}\nmsg.payload = format(Uptime_In_Seconds);\nnode.status({text: msg.payload});\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [
            {
                "var": "os",
                "module": "os"
            }
        ],
        "x": 390,
        "y": 160,
        "wires": [
            []
        ]
    },
    {
        "id": "5a9d999a920964a5",
        "type": "function",
        "z": "890d9817cf94867b",
        "name": "NR Uptime",
        "func": "Uptime_In_Seconds = process.uptime();\nfunction format(sec){\n  function pad(s){\n    return (s < 10 ? '0' : '') + s;\n  }\n  var seconds = Math.floor(sec % 60);\n  var minutes = Math.floor(sec % (3600) / 60);\n  var hours = Math.floor(sec % (3600*24) / 3600);\n  var days = Math.floor(sec / (3600*24));\n  var dayspadded = days > 0 ? days + (days == 1 ? \" day, \" : \" days, \") : \"\";\n  return dayspadded + pad(hours) + ':' + pad(minutes) + ':' + pad(seconds);\n}\nmsg.payload = format(Uptime_In_Seconds);\nnode.status({text: msg.payload});\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [
            {
                "var": "process",
                "module": "process"
            }
        ],
        "x": 390,
        "y": 220,
        "wires": [
            []
        ]
    },
    {
        "id": "316f4168bcd064fb",
        "type": "function",
        "z": "890d9817cf94867b",
        "name": "Memory",
        "func": "let tmem = os.totalmem();\nlet fmem = os.freemem();\nconst pmem = parseFloat((100 - (fmem / tmem) * 100).toFixed(2));\n\n/*switch(scale) {\ncase \"Byte\":\n    break;\ncase \"Kilobyte\":\n    tmem = parseFloat((tmem / 1024).toFixed(3));\n    fmem = parseFloat((fmem / 1024).toFixed(3));\n    break;\n*/\n//case \"Megabyte\":\n    tmem = parseFloat((tmem / (1024 * 1024)).toFixed(3));\n    fmem = parseFloat((fmem / (1024 * 1024)).toFixed(3));\n//    break;\n/*case \"Gigabyte\":\n    tmem = parseFloat((tmem / (1024 * 1024 * 1024)).toFixed(3));\n    fmem = parseFloat((fmem / (1024 * 1024 * 1024)).toFixed(3));\n    break;\ndefault:\n    break;\n*/\nmsg1 = { payload: tmem + \" MB\" };\nmsg2 = { payload: fmem + \" MB\" };\nmsg3 = { payload: pmem + \" %\" };\n\nnode.status({text: \"Memuse: \" + pmem + \" %\" + \" of \" + tmem + \" MB\"});\nreturn [msg1, msg2, msg3];\n",
        "outputs": 3,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [
            {
                "var": "os",
                "module": "os"
            }
        ],
        "x": 380,
        "y": 280,
        "wires": [
            [
                "35685ebd11e961b2"
            ],
            [
                "19a0b53bc34e6d50"
            ],
            [
                "d15fca1ad397737b"
            ]
        ]
    },
    {
        "id": "d15fca1ad397737b",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "MemUse",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 560,
        "y": 340,
        "wires": []
    },
    {
        "id": "19a0b53bc34e6d50",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "FreeMem",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 560,
        "y": 280,
        "wires": []
    },
    {
        "id": "35685ebd11e961b2",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "TotalMem",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 560,
        "y": 220,
        "wires": []
    },
    {
        "id": "b5efd8dc021f6f23",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "15 min",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 570,
        "y": 160,
        "wires": []
    },
    {
        "id": "18dacfef8fdf6d31",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "5 min",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "payload",
        "targetType": "msg",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 570,
        "y": 100,
        "wires": []
    },
    {
        "id": "f056a9d9b907d6b1",
        "type": "debug",
        "z": "890d9817cf94867b",
        "name": "1 min",
        "active": true,
        "tosidebar": false,
        "console": false,
        "tostatus": true,
        "complete": "true",
        "targetType": "full",
        "statusVal": "payload",
        "statusType": "auto",
        "x": 570,
        "y": 40,
        "wires": []
    }
]
8

Now try using and making a few changes in the old V1 GUI interface. I always change the grid set point to another value and back again leaving the UI running in my browser in the background.

9

I’ve got GUIv1 open in a separate browser window, pretty much all the time, and never encountered an issue.
Out of curiosity: do you have GUIv1 open in an HTTP or HTTPS window?

10

http.

For reference: CerboGX, CAN Battery, 2 x VE.Direct MPPT’s, RS485 Grid Meter, Quattro 5k. Using both Modbus and MQTT to access the Cerbo. No NodeRed, Normal image. WiFi OFF, LAN connected.

11

Hi both, thank you for the added information. We still can’t reproduce this, but good to have more details.

12

So far I can’t reproduce the memleak either.
Memory usage on my Cerbo’s is 30ish % and seems to be stable.

13

And that is why this one is so confusing to me and seemingly unreproducible by anyone else.

14

hey all. v3.67 has been released officially yesterday.

Thanks for all help and support.

1 Like
15

I noticed this was available for update on my Cerbo yesterday, however when checking again this morning it does not appear to be available. 3.66 is up todate. Has the firmware update been retracted?

16

All good now. Have now updated.

1 Like
17

In order to be sure about his, try to log onto Cerbo at longer (or shorter) intervals and see if the memory issue is slower (or quicker).
Maybe some task is leaving behind something, or is not fully freeing the memory before the next launch.
Just saying…

18

You almost hit the nail on the head, it’s not the logins but the browser session. My browser has the v1 gui open permanently, as soon as I close my browser session the memory leak stops. See the data from yesterday where I closed the browser session. The gui process ended up at 331mb and is still 331 mb today. This clearly shows while the gui v1 keeps the browser updated it doesn’t release all memory and over time free memory drops to 15mb to 20mb and hovers around there.

image
image472×327 19.1 KB

2 Likes
19

Glad that I could help you pinpoint the problem. :+1:
Now let’s hope that the problem can be solved. :grin:

LE:
Do you have a certain page on gui v1 that you keep open?
Maybe it will help understanding from where the problem starts.

20

I usually start by making two changes to ESS Grid Setpoint eg. 20W to 30W, confirm/apply, and back to 20W again with confirm/apply then escape to the main screen and leave it there in a tab running in the background. While the screen updates the browser in the background the gui process CPU runs at ±18% and the memory starts to drop about 1M every 20 minutes or so, could be longer. If I close the browser the memory ‘leak’ stops and it stays on whatever level it was during that time. The gui process CPU also drops down to ±6% again as it doesn’t have to update the browser anymore.