Importing prices into Dynamic ESS with NodeRED

Node-RED
, , ,
21

Hi Andrew,

I am running in Trade mode but have not really let DESS do anything yet. I am just running ESS with DESS off.

My house has 3 phase supplied but I have 2 x Multiplus 5000 and the third phase has Oven and Ducted aircond on it. Basically lighting is on phase 2 and power on phase 1. Ducted AC draws about 5.1kW.

I have a Goodwe 5000 W inverter fed with 4.2kW of solar on the input of phase 1 multiplus with an energy meter. The Goodwe can’t be controlled with frequency variation so it is on the input side. Quite a small system compared to yours. We have 24kWh of battery.

I have a 3 phase energy meter on grid supply.

Even though the loads are complex and drastically unbalanced, only AC PV on one input of phase one multiplus, phase two multiplus fed with only grid, phase 3 with no multiplus at all, somehow ESS keeps the net power use to zero and it is brilliant to watch it work.

I can have 5.1kW being drawn from phase 3 for AC, and both mulitplusses push back half of that each on phase 1 and phase 2 to even it up to net zero. Amazing.

The software is so smart that even though Victron say 3 phase needs 3 multiplusses with idential hardware and software, my system works brilliantly.

DESS is just the next step in this path of automation where we may be able to actually claw back some money due to the wild variation of prices. I can only see it getting worse in the short term so I want to take advantage of it if I can. I have been having some fun pushing back some energy during massive price spikes but the spikes dont last very long because I imagine the automated systems are quickly evening out the supply based on the spiked price. I dont know that is the case for sure but it stands to reason.

So far I am really happy with how this works. I may engage DESS soon and see what it decides to do. I am not sure how smart the actual Amber systems are but I have pieced my system together with batteries from Grays On Line (Samsungs) that were very cheap but of course unsupported by Amber. I like the Victron stuff too and that is not supported by Amber either. I figure I am having more fun this way in any case. :smile:

22

I wonder if the weird behaviour with the prices/graphs is to do with DESS being switched off? :thinking:

Sounds like we’ve been on similar journeys! I’ve got self-built 15kWh batteries using “DIY” kits from Alibaba - 280Ah EVE cells, 200A JK Inverter BMS in a nice metal chassis. We have a weird (for Australia) two-phase supply and have a similar setup with two multi-plusses in parallel currently on the one phase (they’re in the shed, which only has a single phase supplied). With the magic of the Victron grid meter, the ESS will zero out the draw at the meter. Magic! :slight_smile:

23

Hahaha…DESS just failed its first test. We have negative feed in pricing currently, -4 c/kWh and DESS decided to ramp my feed in to maximum…Oh well…may have to think a bit more about this…

24

Yes, I’ve been having that exact problem – I’ve tried feeding this back to Victron, but haven’t really got to the bottom of it yet. It still doesn’t really know what to do with negative prices!

Green mode seems to be a bit better here, but still. It’d be great if the Victron staff could be a bit more engaged with us to work out a few of these issues.

25

Hi @ndrew ,
I am trying to implement a speculative price forecast flow in node red that extends the know price range (array) by 12 or more hours and fill those (not yet published) prices with the last/latest known price. Reason is that DESS refuses to start charging early in the day because it assumes it can’t sell after Midnight that same day. Extending the range even with a flat price or an intelligent guess of sorts would solve that without the need of tweaking any other DESS settings.
Could you help me out with a basic guideline how this could be achieved? Prices are already imported correctly by DESS, al I want is to take the newest price at the end of the range and copy/ pad that for 12 additional hours once a day early in the morning (or by a trigger such as a very low/negative current ‘now’ price)
See also: Phasing out the Node-RED Dynamic ESS implementation - #7 by UpCycleElectric

26

@ndrew see also: Phasing out the Node-RED Dynamic ESS implementation - #8 by UpCycleElectric

27

Jan, I apologise for the slow reply – acknowledging your question and work, I will do some further digging, but I believe you (and me) are coming up against some limitations of the DESS implementation. I think I will shortly need to start looking at locally-managed solutions (EMHASS looks promising, but we will see).

28

Hi All

I’ve imported the flow from the gist above and think I have updated all the various SiteIDs + API tokens for both VRM and Amber.
It looks like I’m successfully retrieving the prices from Amber.
I’m currently getting an error when attempting to update the DESS prices. “Error fetching VRM data”.
I’ve ticked the option to store response in the global context which is below:
{“dynamic-ess-settings”:{“483236”:{“success”:false,
“error_code”:“validation_error”,
“errors”:{“buyPriceSchedule”:“The price schedule is invalid.”,
“sellPriceSchedule”:“The price schedule is invalid.”},
“options”:{}}}}

The prices output from the function are:
{“buyPriceSchedule”:“[{"days":[0,1,2,3,4,5,6],"schedule":[{"from":"00:00","to":"00:05","price":0.14},{"from":"00:05","to":"00:10","price":0.15},{"from":"00:10","to":"00:15","price":0.16},{"from":"00:15","to":"00:20","price":0.15},{"from":"00:20","to":"00:25","price":0.16},{"from":"00:25","to":"00:30","price":0.18},{"from":"00:30","to":"00:35","price":0.18},{"from":"00:35","to":"00:40","price":0.15},{"from":"00:40","to":"00:45","price":0.15},{"from":"00:45","to":"00:50","price":0.16},{"from":"00:50","to":"00:55","price":0.16},{"from":"00:55","to":"01:00","price":0.15},{"from":"01:00","to":"01:05","price":0.15},{"from":"01:05","to":"01:10","price":0.15},{"from":"01:10","to":"01:15","price":0.13},{"from":"01:15","to":"01:20","price":0.08},{"from":"01:20","to":"01:25","price":0.13},{"from":"01:25","to":"01:30","price":0.1},{"from":"01:30","to":"01:35","price":0.1},{"from":"01:35","to":"01:40","price":0.18},{"from":"01:40","to":"01:45","price":0.08},{"from":"01:45","to":"01:50","price":0.11},{"from":"01:50","to…”,
“sellPriceSchedule”:“[{"days":[0,1,2,3,4,5,6],"schedule":[{"from":"00:00","to":"00:05","price":0.04},{"from":"00:05","to":"00:10","price":0.06},{"from":"00:10","to":"00:15","price":0.07},{"from":"00:15","to":"00:20","price":0.06},{"from":"00:20","to":"00:25","price":0.06},{"from":"00:25","to":"00:30","price":0.08},{"from":"00:30","to":"00:35","price":0.08},{"from":"00:35","to":"00:40","price":0.06},{"from":"00:40","to":"00:45","price":0.06},{"from":"00:45","to":"00:50","price":0.06},{"from":"00:50","to":"00:55","price":0.06},{"from":"00:55","to":"01:00","price":0.06},{"from":"01:00","to":"01:05","price":0.06},{"from":"01:05","to":"01:10","price":0.06},{"from":"01:10","to":"01:15","price":0.04},{"from":"01:15","to":"01:20","price":-0.01},{"from":"01:20","to":"01:25","price":0.04},{"from":"01:25","to":"01:30","price":0.01},{"from":"01:30","to":"01:35","price":0.01},{"from":"01:35","to":"01:40","price":0.08},{"from":"01:40","to":"01:45","price":-0.01},{"from":"01:45","to":"01:50","price":0.02},{"from":"01:50"…”}

I note the final values of each array has “…” but i think that is just as displayed from the debug output. Is there a way to confirm this?

29

G’day, Greg.

This occurs secondary to Amber’s recent-ish change to 5-minute billing. DESS only supports 30-minute increments. I fixed this locally when it first happened but didn’t think to update the Gist. I’ve done that now. Thanks for the reminder! :slight_smile:

It’s probably just as easy to update your already-imported flow – it’s simply a matter of specifying the resolution in the API call. In the “Get forecasted prices” node, append ?resolution=30 to the URL, so it looks something like https://api.amber.com.au/v1/sites/YOUR_SIDE_ID_HERE/prices?resolution=30. Amber’s API will then return compatible data for DESS.

Obviously, the prices are changing faster than every thirty minutes, so we mitigate that by polling Amber every 5 minutes (set by the inject node).

Sing out if you have any further issues.

30

Hi Andrew

Ah gotcha, great pickup. I’ve made that change but still have an error on the vrm-api node.

I’m still getting the “Error fetching VRM data” and in the debug log:

{“url”:“``https://vrmapi.victronenergy.com/v2/installations/483236/dynamic-ess-settings?",“options”:{},“headers”:{“X-Authorization”:"Token mytoken”,“accept”:“application/json”,“User-Agent”:“nrc-vrm-api/0.2.13”}}

I’ve just been going through the vrm-api docs and found an error with their request builder

{idSite} doesnt actually update and I was getting child site error for idSite - there I was thinking I had to create a child site for the installation for DESS… wow hours gone ohwell!

Using the correct site ID I get a successful response so that doesnt seem to be it.

Curious… what version are you running?

I’ve got victron-vrm-api 0.2.13 on v3.66-1 after having no luck with the official release either.

Otherwise, any other ideas for getting more debug info that actually says what the error is?!

31

That’s unusual. I’m on the v3.70 betas, but this has been working for the better part of 12 months. The victron-vrm-api Node-RED package is 0.2.15 - I suppose it’s possible there’s an issue with 0.2.13, but I feel like it’s unlikely. Enabling verbose output on my node shows the endpoint is the same on my end.

Apologies if this is asking you to suck eggs, but just to confirm we’re not missing anything basic:

  1. You’ve got a valid VRM API key from VRM Portal - Victron Energy
  2. In Node-RED, you have a “Change Dynamic ESS settings” VRM API node, and:
    1. it is configured with a VRM instance (a name and an API token).
    2. The VRM API type is “Installations”
    3. Your site ID is correctly specified
    4. “Installation” is set to “Modify Dynamic ESS configuration”

If that’s all there and correct, you can add a debug node after the “Modify Dynamic ESS configuration” node and redeploy the flow. That will at least show us what the msg is that the VRM API node has.

32

Hi guys,

Good to see this topic is still getting some attention. I’m about to revisit our own hybrid DESS Trade solution and have a few questions you might be able to enlighten me on.

  1. Is it possible to use VRM-API to replace the VRM DESS schedule (the one readable by installation stats, visible on the VRM dashboard and actively driving DESS in mode 1 auto) by a modified custom schedule, in part or even entirely?
  2. If the mode 1 schedule can be modified/replaced, how can we prevent VRM DESS overwriting it back to the schedule generated by Victron’s own cloud based scheduler that gets updated every hour or more often even? Can this be prevented by certain settings (without turning DESS mode 1 off)?
  3. If we cannot prevent VRM DESS from overwriting our custom schedule, can such an ‘overwrite’ event be detected to activate a countermeasure to ‘overwrite the overwrite’ with the custom schedule again. Will such a thing even work without unforeseen negative side effects?

Reason for asking is the ‘standing issue’ that VRM DESS mode 1 scheduler ignores the planned SoC target (b_goal_SoC and b_goal_hour) even though it clearly has the ability to do so.

The most confusing result I have seen is that the VRM-API node, when activated (inject node) to update the schedule, will actually produce a schedule object that does adhere to the planned SoC target, while at the same time the actively in use schedule as can be read back through installation stats does not adhere to the planned SoC target.

What I am looking for to implement is a means to change that behaviour by back feeding the returned schedule object (after some reformatting where needed) to the local DESS schedule actively in use in VenusOS / ESS assistant. With VRM DESS mode 1 active.

33

yeah i’m wondering if there is an issue with this package version. Out of interest, did you specifically choose 0.2.15 or did it just come with whatever version you had loaded on the GX. Also, what are you using for a GX? I’m using a Cerbo…

Happy to be asked because it has to be something basic right?

  1. Yep definitely have a valid VRM key - even tried issuing a new one just to be sure.
  2. I’ve tried importing your flow and just modifying the API node and config node, also tried creating a new config within the imported API node, and recreating them all from my palette.
    It is configured.
    It is installations
    it is my site ID
    It is modify (also tried get DESS just to check)

I have a debug node after it but nothing ever hits it so I’m not sure how to retrieve a log from inside the API node.

Thanks for your time and thoughts attempting to get this rolling!

34

Hi Andrew, Any luck with the negative pricing?

35

Alas, no. DESS is a hot mess, unfortunately. It is not a priority for Victron and getting it to function like it should is difficult. I’ve asked what Victron plan for it in the long term and can’t get answers.

I’ve had some success with Node-RED flows turning off the MPPTs during negative grid pricing; DESS certainly isn’t planning based on the negative prices or making its own decisions to throttle down the MPPTs.

I’m going to move to a basic ESS and control it externally with separate software. Summer’s job is to trial EMHASS and if that doesn’t give me joy, write something of my own.

36

OK! so it seems that there is something going on with the API call being received by VRM as i got annoyed and just kept smashing the inject button and after about 15 hits it actually came back with a success and updates the prices but it is few and far between. No real error message to go off either which I raised an issue on github about. It was suggested I update to 3.70.45 with 0.3.4 for the vrm-api. At least now there are some more indicators for the error but still with a large miss rate. I actually dont think I’ve been getting pricing updated anymore so the battle continues. I noticed another super weird thing recently… If I look at VRM on my home or work wifi, it displays fine / as you would expect. However if I am on 4G with good reception and no other browsing issues experienced - I can log into VRM, but there is no data, even my site seems to come and go. I’m not convinced it is a Victron issue… although I’m not convinced it isnt either! Just seems super odd given the node-red issues are inconsistent but same request with curl is steady.

37

What error are you getting? I’ve had a NodeRED flow running to update prices for many months now, and it seems to be throwing a 422 error as of the past few days (also on 3.7~45)

IMG_0217
IMG_0217760×196 27.8 KB

38

I haven’t had a chance to look closely myself, but I’d suggest you take a look at the 3.70~45 release notes, as there are breaking changes for the Node-RED VRM API nodes.

40

Hi Greg, did you solve this issue with the VRM API not returning data? I am having the same issue atm.

David

41

Not solved yet unfortunately.

Suggestion is to roll back and/or wait until vrm-api 0.3.5 is out.

Venus 3.70~49 seems to be out and includes:

victron-vrm-api node v0.3.6

  • Restore previously removed Dynamic ESS stats endpoint, with one addition/change: the endpoint now always returns 24 hourly data points, maintaining compatibility with existing flows. Also a deprecation warning has been added. The new end point, “Fetch Dynamic ESS schedules” introduced earlier during v3.70 is the recommend method for getting DESS schedules.

  • Fix PATCH operations for Dynamic ESS configuration (now correctly uses msg.payload)

I’ll probably update tonight and cross some fingers…..