MP II domestic ESS : two freak events of charge to 100% using grid

[ph83 (Pascal Hingamp)]

Hi,
I’ve been running a domestic ESS for 6 months with no trouble (setup below). Optimized ESS without BatteryLife, and minimum SOC set to 40%, and more recently 60%. Dynamic ESS not activated. No scheduled target SOCs. My PV available power is unable to charge more than 25% of SoC over a single day.
In the last week, I have got two VRM alerts corresponding to two unscheduled SoC peaks >95%, energy mainly pulled from the grid. Looking through the Remote Console parameters, I noticed on both occasions that some ESS parameters had changed to totally outlandish values, eg today it was the ESS “Grid set point” that was set à 1900W instead of the normal 100W value that I had set . During the previous freak SoC peak event, this value was observed at 61400W and the “grid maximum feed in” was set at 56900W (I corrected both values after the first freak event).
Would anyone have observed such freak events, possibly due to Venus OS parameters having drifted without human intervention ? The only explanation I can imagine would be corruption of the filesystem, but it seems so unlikely that just some random parameters are affected, whilst everything else in the system including software appears stable.
I guess I could start by replacing the current SD card with a fresh one using a backup image of the system ?
Any help or suggestions most appreciated
PS: interestingly, another user seems to have a similar experience
Setup : MP II 48/3000 with ESS assistant, Venus on raspberry, 100A JK-BMS with dbus-serialbattery connection, 16 x Eve 280 LFP, array of 7 second hand PV 250W panels / 4 APS YC500I micro-inverters, grid meter Shelly EM50 via MQTT.

(alternates between “BL Disabled (Low SoC)” and “BL Disabled”)

[hominidae]

@ph83

I noticed on both occasions that some ESS parameters had changed to totally outlandish values, eg today it was the ESS “Grid set point” that was set à 1900W instead of the normal 100W value that I had set . During the previous freak SoC peak event, this value was observed at 61400W and the “grid maximum feed in” was set at 56900W (I corrected both values after the first freak event).

As you are using a grid meter also integrated via MQTT, maybe there is a problem with another mqtt broker instance, that is somehow bridged with your VenusOS or VRM instance?
Do you have an installation with Homeassistant or another EMS/SmartHome system with a mqtt-Broker deployed at your location?
If so, you should definitely kill that connection between the two.

[xsnrg (Jim)]

This is happening to me right now. Same situation. I have PV that does not fully charge my batteries unless the sun is out, so cloudy winter means it will go a few days of less than full SoC. I am using ESS with Optimized without Battery Life, and my current SoC is 85% with a voltage high enough in the batteries that the SoC is likely accurate. I am getting the SoC both from the battery BMS and from a smart shunt, and they are in line. Something caused the MultiPlusII to receive a low battery signal (I am guessing it was not actually a low voltage signal, but this is what it shows up as). I am running a Pi with Venus 3.53~2, batteries are connected on CANbus to the Pi and the inverter is connected to the Pi with a MK3. The Multi is running v552.

It appears the battery has commanded the Inverter to charge, as there is nothing else that would be controlling it. Fine if this is the case, but is there a way to actually see or log these commands from the BMS to better know what is happening?

I have disconnected the grid as we have a good solar day starting today, and I want to use the solar.

[ph83 (Pascal Hingamp)]

You read right through my installation - indeed I’m feeding the ESS data into Homeassistant via MQTT. This is for read only purposes, ie to keep historic stats of the ESS parameters, I don’t use any HA automations or switches to control the ESS via MQTT (I didn’t even test if that is possible, as the VRM or Remote Console on the Venus OS do that job perfectly fine). Do you think my ESS observed freak events might have been the result of MQTT transported commands from HA ?
Also is there a way other than MQTT to export the ESS state to HA ? Or can MQTT be configured as “read only” mode ?

[ph83 (Pascal Hingamp)]

Interesting alternative to an MQTT origin, a BMS command over CANbus. In my case, the BMS is connected to the Pi via a Serial to USB converter. I don’t know if commands can be sent from BMS to Venus OS by default, by changing a Venus OS parameter, or at all. If you don’t have logs, how did you determine that a low batt signal triggered the charger ? I’d really like to find a way of diagnosing what chain of events led to the unplanned full charge…

[xsnrg (Jim)]

I only have the low battery event, which was an alert in notifications. No other indications. As the battery was not actually low in voltage or SoC, this seemed odd enough that it might be related. Beyond speculation though, I have no concrete evidence.

My hypothesis is some event was sent from the BMS that Venus perhaps did not handle or log correctly? It may have been the BMS requesting balance voltage or something (no idea really). The BMS does send information to VenusOS. All sort of good info, just not enough to help with this problem:

I wonder if there is more info available in part of the MQTT data. I have it enabled as well, and track things in OpenHAB, much like you are doing in HomeAsssistant. No data, in my case, is sent back through MQTT, it is only read.

[hominidae]

@ph83

You read right through my installation - indeed I’m feeding the ESS data into Homeassistant via MQTT. This is for read only purposes, ie to keep historic stats of the ESS parameters,

Please think about how mqtt pub/sub action works. Normally, as venusOS is supplying the data, it is not actvely feeding it - as you describe it - but rather HA is subscribing to it.
So it’s mora a kind of pull active pull into HA.
However, the most common approach here is, that HA is not subscribing to each topic individually. Normally HA has its own mqtt broker instance and typically uses a bridge to keep both mqtt instances - HA and VenusOS - in sync.
When wrongly configured, this bridge is bi-directional and when one instance gets a restart, like after an upgrade, the topics can be populated with a value that is not the most recent one.
Especiallly when these are published with retain flag on the side of origin.

I’m not a HA expert - normally a ordinary subscribe activity is OK. It’s a bridge that can cause these problems, even when one side is not actively publishing data to the other.
So check if your HA connection is via a bridge instead of individual subscribe actions.

[ph83 (Pascal Hingamp)]

I’ve checked both the Venus and the HA configurations, and my memory was wrong (note to myself: write down all configurations done, as you’ll forget in 6 months…).

• MQTT is only used for 1/ grid and 2/ PV metering : using Github mods the Venus raspberry subscribes to relevant topics where these values are published by respective meters on an independent MQTT server (separate raspberry)
• Homeassistant reads the Venus data via its ModBus TCP service (using a specific HA integration )

Therefore I can confirm there is no MQTT bridge in place, only Venus subscribed to a couple of PV/grid power related topics.

However I did notice I had configured the HA ModBus TCP integration in “write” mode, which does provide the capacity to change Venus configurations. I have not knowingly used these write capacities, but just in case I have now disabled the “write” option in HA, leaving only the reading of the Venus ModBus values by HA.

I’ve just checked via SSH the log files on the Venus file system, and have found the following very surprising entries :

root@raspi2:~# cat  /data/log/localsettings/@400000006777e1601907ff44.s | tai64nlocal
2024-12-30 16:11:14.968009500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 43, New: 42
2024-12-30 16:19:33.734265500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 42, New: 41
2024-12-30 16:27:56.535821500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 41, New: 40
2024-12-31 08:23:49.897480500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: -1.0, New: 56900.0
2024-12-31 09:21:16.179716500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 40, New: 41
2025-01-01 09:37:46.739555500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 41, New: 42
2025-01-01 09:53:08.162306500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 42, New: 43
2025-01-01 10:07:31.477530500 INFO:root:Setting /Settings/Devices/serialbattery_JK_BMS_JK_PB1A16S15/SocCalc changed. Old: 43, New: 44

The MaxFeedInPower is being changed to a weird unexpected value of 56900.0 W
Looking through these logs shows that lines with “CGwacs” correspond to occasional setting of wild values, such as 61400.0 W for AcPowerSetPoint

root@zeworks2:~# grep -rh "CGwacs" /data/log/localsettings/* | tai64nlocal
2024-12-31 08:23:49.897480500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: -1.0, New: 56900.0
2025-01-01 22:24:23.546174500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 100.0, New: 61400.0
2025-01-02 09:12:14.010448500 INFO:root:Setting /Settings/CGwacs/BatteryLife/MinimumSocLimit changed. Old: 40.0, New: 95.0
2025-01-02 09:57:08.807264500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 61400.0, New: 100.0
2025-01-02 09:58:34.205022500 INFO:root:Setting /Settings/CGwacs/BatteryLife/MinimumSocLimit changed. Old: 95.0, New: 60.0
2025-01-02 12:27:18.695789500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: 56900.0, New: 3000.0
2025-01-02 12:27:25.231425500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: 3000.0, New: -1.0
2025-01-07 15:35:32.511356500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 100.0, New: 5500.0
2025-01-07 15:35:34.341800500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 5500.0, New: 58500.0
2025-01-07 15:35:36.338296500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 58500.0, New: 1900.0
2025-01-07 15:35:38.556208500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: -1.0, New: 1900.0
2025-01-07 19:09:21.976356500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: 1900.0, New: -1.0
2025-01-07 19:09:23.916406500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: -1.0, New: 1000.0
2025-01-07 19:09:26.940449500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: 1000.0, New: -1.0
2025-01-07 19:09:33.537218500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: -1.0, New: 1000.0
2025-01-07 19:09:39.972377500 INFO:root:Setting /Settings/CGwacs/MaxDischargePower changed. Old: 1000.0, New: -1.0
2025-01-07 19:10:32.836302500 INFO:root:Setting /Settings/CGwacs/AcPowerSetPoint changed. Old: 1900.0, New: 100.0
2025-01-08 00:07:29.119840500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: -1.0, New: 1000.0
2025-01-08 00:07:35.022739500 INFO:root:Setting /Settings/CGwacs/MaxFeedInPower changed. Old: 1000.0, New: -1.0

What I don’t understand from these logs, is who on earth ordered these changes. The CGwags appears to stand for “Carlo Gavazzi Wired AC Sensors”, but I don’t use these sensors: my grid is metered via MQTT subscription to a topic published by a Shelly CT probe… Is it possible that internally in the Venus engine, whatever grid metering is in place, it gets fed into a pseudo CGwags DBus ? Is there a way in the logs that I can nail down how those CGwags parameter changes came to be ?

[hominidae]

@ph83

What I don’t understand from these logs, is who on earth ordered these changes. The CGwags appears to stand for “Carlo Gavazzi Wired AC Sensors”, but I don’t use these sensors: my grid is metered via MQTT subscription to a topic published by a Shelly CT probe…

normally, data-feed-in to VenusOS via mQTT is not standard.
I believe, with your Shelly you need a non-victron add-on in venusOS.
Many variants of these exist for non victron meters and most simulate an EM24, hence the CGwac tree is used for this.

[nickdb (Nick)]

The issues you are experiencing are due to custom modifications/software on your system.
I would suggest you reset to default your GX and start again, which will remove anything that could be misbehaving.
With unsupported mods like this, the best place for queries is the modifications section.

Venus OS, will never randomly change anything. Only automation (like DESS) and custom mods and add-ons can do this.

[ph83 (Pascal Hingamp)]

Many thanks @hominidae for the insights. Indeed simulating the Victron supported meter would probably be the easiest way to integrate. I’m still wondering what is the route to CGwags though, because the Shelly DIY service appears to publish the metered values on DBus paths such as “/Ac/L1/Power”, which might be expected where CG sensors would end up dumping measures to, rather than going from a general path such as “/Ac/L1/Power” back up to CGwags related code. Nonetheless this remains the best explanation !
Now that I’ve killed the “write” feature of the HA integration, I’ll observe for a while to see if those freak parameter changes no longer occur. If they do, then indeed it’s down to the MQTT Shelly mod.

[ph83 (Pascal Hingamp)]

Cheers @nickdb for the sound advice, it confirms the bad behavior lies in one of the mods (HA interaction via ModBus TCP, or Shelly/PV via MQTT). I would like to unplug the MQTT mods, but that would end all hope of running an ESS for PV daytime storage - unless I ditch the Shelly and invest in a supported Carlo Gavazzi Wired AC Sensor, which I admit is tempting.
I also note that I should have posted in the “modifications” section, I don’t know if this thread can be moved there altogether ? [Edit : found the topic switching menu at the top ]