question

jacola avatar image

ESS 3 phase 3 with external control, one phase sometimes ignores setpoint

I might repeat myself but i am scratching my head and the problem is not solved.
My ESS is 3 phase, with Multiplus-II 5000 on firmware 474, Venus on 2.60. I use external control.

Sometimes one of the Multiplus units on one phase apparently decides to ignore the setpoints and starts doing odd things .. with the result being a wild ride between feed-in and consumption.

The situation resolves after some time, but that can take an hour or so. The other multiplus'es follow the instructions without issue.

this is the problematic phase only, SP is setpoint.
The overview of all 3 phases for a longer interval is here:
you can see that normally the inverter follows the instructions well ..


Any advice appreciated.

Johannes


Multiplus-IIESS
1 comment
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

here we are again, producing 2500W instead of charging 400 .. not funny .. this is only one phase, the others show similar behavior. out of the blue for me ..

5 Answers
Daniël Boekel (Victron Energy Staff) avatar image
Daniël Boekel (Victron Energy Staff) answered ·

Hi @jacola

is it ok if we look into your system? if so, what is the VRM id or system name? (a link to the VRM page is also good)

10 comments Share
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

sorry i answered as "Answer" .. 41839

since today (portugese time) about 10:30 the inverters do "what they want" again, despite giving them setpoints. green is actual power as reported by the inverter, yellow is setpoint. note 10:42 all 3 phases show strange behavior .. at this time now things are back to ok.

we have cloudy weather today with substantial short term changes in production ..

Hi Daniël, did you come to any conclusions ? The problem happens intermittent, and not always on all phases. For me it almost looks as if the inverters go "on their own". Interesting is that the situation eventually fixes itself. If i should log any other values let me know please ..

Hi @jacola

As you know, we're looking at it, I'll answer you as soon as I know more.

jacola avatar image jacola Daniël Boekel (Victron Energy Staff) ♦♦ ·

Hi Daniel, here another log of a longer problem happening today.
The trigger of the situation is probably the following:
phase 1 inverts, phase 2 and 3 invert. a pump goes on (remember: nothing connected to load side, all is grid assist) drawing 5.5kw (again, on grid side) at 14:13
So the setpoints change, obviously.

From my viewpoints ESS at this timestarts to go in a havoc cycle, producing unrelated to my setpoints and fortuntaely, today did not violate grid feed rules (it produces together with production and that can drive too much triggering a protector, putting us off grid, that happened before).
It recovered around 14:38 for no obvious reason, but fortunately :)

here the graph

let me know if i can provide any other help to find this.

johannes

jacola avatar image jacola Daniël Boekel (Victron Energy Staff) ♦♦ ·

Hi Daniel,

to avoid any doubt about how i interface to ESS mode 3, (which i do via Modbus/TCP), i added a completly independent monitoring via MQTT.
A simple software which just captures the (actively) reported setpoint and power power per phase and throws it to the reporting platform every 5 seconds. here are the subscribes:

client:subscribe("N/xxxxxxxxx/vebus/261/Hub4/+/AcPowerSetpoint",2)

client:subscribe("N/xxxxxxxxx/vebus/261/Ac/ActiveIn/+/P",2)

client:subscribe("N/xxxxxxxxx/vebus/261/Ac/ActiveIn/+/S",2)

The result shows that the ESS itself has those crazy ideas about not following the setpoint. usually for about 20 mins before getting "sane" again by itself. Green is the setpoing .. you can see hat ESS does out of it .. especially bad are the situations where the result is a fast discharge of the battery by producing a lot (and if that happens while we have production we blow the feed-in limits).

This really needs to be solved.

Johannes



Hi @jacola


You mention this happens when the battery is fully charged, could it be that there is a DC overvoltage that causes the inverter to feed-in to prevent this overvoltage?

jacola avatar image jacola Daniël Boekel (Victron Energy Staff) ♦♦ ·

@Daniël Boekel (Victron Energy Staff)

Hi Daniël, that was in one event, the above one was with an SOC of 18%. It happens quite often (usually one or multiple times per day), and i haven't found any correlation with anything until now. happens over all battery charge states, and also happens both with the batteries being charged and discharged (as you can see from the various posted graphics).
Happy to provide you with grafana access so you can soom to any area of the graphs if that would be helpful, but i would expect you can get the same data via VRM too ?
Is there anything i can do to help with the diagnosis ? I was about to write software to detect the situation so i then could try to command the ESS to do something (if it still listens in that phase).


Johannes


situation is really bad now, Can you please check what's the problem?

Hi Pedro, Victron, i might have an idea. Here is a completely unbearable situation, which starts when the battery is getting full. The ESS software starts to oscilllate between massive "production" while i have full sun power and excessive energy, driving my feed-in way above the allowance .. the only thing i could do is really stop sending any setpoints and waiting. As usual, graph shows communicated setpoint and the results.

I am going to set up another independent monitoring using MQTT (currently i read back the data via modbus/tcp, and verify also that the grid connection shows those results so they are real and not fake).

Johannes

So there are definitely 2 problems ..
the one reported where ESS in mode 3 just goes for its own decisions and does not follow the setpoints, and then the case when the battery is full - instead of leaving it alone, ESS goes to heavy discharging, trying to cut me off the grid with attempted overcurrent :(


Here is another great graphic of ESS not following the setpoints .. the green line is the setpoints


i am pretty desparate to get this solved.
The reported setpoints and other values are taken from MQTT from the venus, and directly sent to a victoriametrics/grafana instance. Setpoints are provided via modbus/tcp. so they do arrive in venus otherwise they would not be visible on this chart.

jacola avatar image
jacola answered ·

Hi Daniel, absoluteley !

https://vrm.victronenergy.com/installation/41839/dashboard ..

i guess 41839 is the number ;)

Feel free to play, and i am more than happy to play along if you need local help, i should stop sending setpoints or the like .. linux/modbus/control specialist here ;)

If you want to see actual setpoints and the resulting data, i have that all on a victoriametrics/grafana instance, i can send you a login for that.

Johannes


Share
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pedrolima avatar image
pedrolima answered ·

Dear Daniel,


My customer Johannes is having some really annoying problems with this situation. Voltage from the grid is sometimes quite on the limit, and this behavior makes it even worse, triggering some equipments off due to a out-of-limits AC voltage.


Im glad Johannes runs an industrial electronics company, so he knows bugs like this happen sometimes. But if he was a regular customer, that pays a lot of money for this inverters, batteries (he has around 40kWh of Pylontech's), and this happens it would be a big mess (and the customer would be right to complain or even maybe go to court over this).


Johannes was planning to upgrade the system further, adding more inverters, batteries, panels, etc and converting to an off-grid system, to avoid the grid issues, but Im pretty sure if this software reliability problems are not sorted very soon, he will think twice about going off-grid using Victron.


Let me know if we can help in any way.


Best regards

2 comments Share
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Hi Pedro the situation is getting really bad now.
As of today when the battery is 100% charged, the ESS starts to "oscillate" between inverting thousands of watts (into the grid which already has to accept the solar production) and charging, constantly. Even if i stop to send setpoints, this "out of the blue starting to invert" continues for a minute before the system finally goes to idle.
See below graph. Despite ESS mode 3, the system things i t can do what it wants and invert, causing grid feed-in problems .. the green line is the actual performance (below shaded line is inverting, above is charging). remote console shows 99%/100% charged battery.
I need the system also with a charged battery to balance phases, so i should be able to set one phase to inverting if necessary and others to charge .. but i would not expect the system to deliberately invert without command.

Johannes

Can someone from Victron please reply?!?


We have no idea what the problem is, clearly its a software fault.


Can someone let us know if we can do something on our side?!?


This are expensive equipments, this lack of replies and lack of support should not happen!

merlepeter avatar image
merlepeter answered ·

What is the ESS setting for Multiphase regulation ? Is it single phase or all 3 phases ? What is the external device ?

1 comment Share
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

i am controlling via modbus/tcp. the ESS is set to mode 3 (external control), so i should be able to command every phase separately by setpoint. and it mostly (almost always) works well, just "sometimes" the ESS goes havoc and does whatever it wants, ignoring the setpoints given by the external control (which is a little linux appliance, a barix barionet, connected with an ET340 and controlling loads also).

To be 200% sure there is no problem with that device, i have set up yet another system, using MQTT, to just monitor the ESS. This other "Monitoring only" sytem receives both the setpoints as well as the actual data from ESS via MQTT, push, and records those to a victoriametrics cloud instance with grafana frontend. So i consider that 100% neutral, independent, just monitoring.
It shows the same data .. the ESS eventually ignores the setpoints and just does what it wants. those "havoc" phases can take 10-20 minutes, and in that time, unfortunately, ESS might deliberately go into 4k "production" on each phase, which adds to the solar production and can (and does) blow my line due to (negative!) overload. It clearly violates the feed-in regulation.

Next i am going to try to stop this, by setting the ESS to "charge only" when that happens, let's see if it accepts this.

But i have to say i am quite frustrated with this .. an ESS which decides by itself to feed-in and empty the battery, despite other commands, is useless.
As all three multiplus show the same behavior at the same time interval (usually all three start to ignore the setpoints) i have to assume it's the ESS and not the multiplus, but hard for me to diagnose this further. Sincerely hoping for Victron staff to get this diagnosed and fixed. I was about to upgrade the system substantially ...


jacola avatar image
jacola answered ·

well, not really an answer, just more information to those interested or which run into the same problems.

The situation continues to be UNRESOLVED. Last week my installer was here to update the Multiplus Inverters as that was advised by Victron.
The problem remains.

To recapitulate: every now and then the multiplus inverters (i have to assume it is them) do not follow the setpoints (ESS Mode 3) any more but do go "havoc", and in worst case they go to full power inverting/production, causing excess grid feed-in and being cut off the network. Sometimes they just go to "0". In both cases the situation resolves itself within about 20 minutes, if we are not thrown off the grid of course. There are NO grid faults reported, nothin in NS log.
Here is a graph of the latest occurence, yesterday. At 16:43 all three phases seem to get to that point where they dont charge and dont produce, not following the setoint. At 16.53 they go "havoc". i did a multiplus subsystem reset via remote console at 16:58 which brought them back to 0 but phase 2 and three continue with havoc mode.
That data is collected by listening to the setpoint and actual power values published via MQTT by Venus, so that should be what the system should see.

Anyone else has similar problems ?
Is there any solution to listen on the VE.Bus to see what is going on there ?
For me the problem MUST URGENTLY be solved, i might be forced by the power provider to shut the inverters down otherwise as they produce excess grid feed in eventually, beyond my contracted power limit (when added with the solar production).


Any advice ? Anyone with similar problems ? If you think you might have this too, talk to me i can help you to log like this into a cloud server.

Johannes



Share
10 |3000 characters needed characters left characters exceeded

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.