Phasing out the Node-RED Dynamic ESS implementation

Node-RED
, ,
1

We are contemplating on phasing out the Node-RED implementation of Dynamic ESS .

While it was valuable during initial development, the current VRM implementation already offers way more functionality (e.g. Octopus (UK), Tempo (FR) and Vario (CH)) and is better kept up to date.

The VRM-API node seems the better alternative, though that one still requires some changes to be up to date as well. The idea is there to keep the configuration of Dynamic ESS in VRM (and partly in Venus OS), while the scheduled can be retrieved, altered and filled using the VRM-API node and the Dynamic ESS node that comes with node-red-contrib-victron.

Also the 15 minute interval changes haven’t been looked into for the Node-RED implementation, while they are already done for the VRM implementation.

If you are using the current implementation, I’d like to hear your specific use-case for not just using the VRM implementation instead.

2 Likes
2

Initially I was working on something resembling DESS myself but Victron was quicker to release :slight_smile:
Currently I’m on VenusOS v3.55 and running a more or less Hybrid DESS setup, using the VRM-API in Node Red, Dognose’s DESS Hack (scheduled to be included and improved in v3.60 iirc) and some custom Node Red voodoo.

Among the things I’m overriding:

  • forced grid consumption / charging at negative grid prices
  • scheduled charging at low grid prices to anticipate a large load later (charging my 22.5kWh electric motorcycle mostly)
  • manipulating grid target (depending on grid prices, anticipated load & solar forecast etc) to force consumption from grid and all solar to the battery
  • triggering external actions

The VRM API is thus very much appreciated.
Even though I’m currently not using the Node Red DESS implementation, it could be useful for the very special cases where the VRM version is not compatible with certain overrides (in future versions).
Or is simply down, as occurred a few times.
Granted, the Node Red version also needs internet for prices and load/solar forecasting - but that can be worked around.
A use case there would be Privacy, for people who run offline and/or don’t want to send data to a processor on the internet.

3

In Portugal both solutions are not ideal, since the dynamic energy prices could have different different “fixed values” depending on the time of the day. The NodeRed solution do a better fit but I loose the predicted energy prices, the VRM one doesn’t support this scenarios.

If in VRM we have a mix of the Fixed Price and Dynamic Price solution it would do the job: define custom formulas depending on the time of the day.

1 Like
4

@BartChampagne could you share some of that node red voodoo?
I’m just starting with node red (see : Dynamic ESS: manual set energy price for each hour - #7 by UpCycleElectric) to automated a boost charger on our trade (test) installation.
Next step concerns the phenomenon that in the morning before new pricing becomes available, prices already tend to be low a few hours earlier but DESS won’t start charging because it does not see any selling opportunity beyond midnight. And once pricing roles in is too late to charge up for it.

2 Likes
5

I’m reluctant to share my voodoo “as is” , not because I don’t want to share but more because it’s (in my opinion) a bunch of clunky and messy code that I should clean up & rewrite some day when I have spare time.
It all works for my specific setup but I have no idea nor can I guarantee how my code will respond to your setup.
I have some very clean & nifty code as well but that code was paid for by a customer so I can’t just throw it out in the open.
If you can elaborate on what you’re looking for specifically, I can check to clean up that part of my code before sharing it.
If you’re looking for a Node Red example to force charging from the grid, it’s a tricky one these days.
To achieve that on v3.55, my implementation requires The Hack to be installed, then it (ab)uses the “Custom Control” node to write to the com.victronenergy.settings'|/Settings/DynamicESS/AdhocChargeRate entry - just like you would do in GUI v1.
Since the entire DESS mechanism will change in soon-to-be-released v3.60 and The Hack will no longer work on that version, I’ll need to rewrite my code anyways.

And if you’re just starting with Node Red, I suggest you learn it first by making baby steps before importing unknown and complex code.

With regards to Dynamic prices: it’s called ‘Day Ahead’ market, so you should get prices for tomorrow not much later than 14:00 today.
If you only get your prices on the day they’re intended for, you’re either not working on the Day Ahead market or there’s something off with your data source.

6

Ok, lets start with the day ahead market.

  • The ‘look ahead’ window jumps from about half a day to one and a half day around 13:00 when new prices roll in.
  • On many days the ‘midday low price’ window runs somewhere between 10:00 and 17:00
  • Every morning DESS trade plans to charge the battery midday but only to the level that it knows it can sell back profitably within it’s known look ahead window, which is upto midnight that same day until the new prices roll in.
  • Therefore DESS typically misses the opportunity to charge up much more in the late morning, not aware of the sell opportunity the next day early morning.
  • And by the time to the new prices roll in, it’s too late obviously.

So my question is, how can DESS be made aware that there is a reasonable and quite stable additional sell opportunity within the next 24 hours, in the morning?

Screenshot_20250510-095634
Screenshot_20250510-095634800×1340 71.7 KB

Like right now, DESS is missing 2,5 hours of low price charging opportunity

7

One solution that I think would work reasonably well, is to extend the look ahead window to always be 24 hours ahead at least, and populate the hours for which actual prices are missing, with estimated prices, just a copy of the same hour from the previous day before would already be good enough. And by the time the new actual prices roll in, there is still plenty time for DESS to adjust accordingly.

If possible, at 01:00 in the morning, when the look ahead window is 23 hours, extend it to 35 hours by adding last days pricing to the end of the look ahead window.

The thinking being that history might not repeat, but will usually rime.

Alternatively, the same could be done every hour, always keeping a full 36 hour look ahead window available.
And also instead of copying the hourly price from the day before, simply populating the ‘missing’ actual prices with the ‘last price known’ could also work. Anything, even a ‘bad estimate price’, is better then the current ‘no price known’ situation.
…
Edit: adding 8 hours of the ‘last known price’ at midnight as an intelligent guess looks pretty promising to me. As can be seen in picture below. By the way, these are crazy prices this weekend: minus 28ct to charge the battery, with taxes and costs, wow.

Screenshot_20250510-132038
Screenshot_20250510-132038800×1340 74.7 KB

1 Like
8

While ‘getting my feet wet’ with Node-RED, I found the Node-RED DESS better suited for a ‘no-solar, trade only’ DESS, even if only for it’s ability to set a dayly b_goal_SOC (90) and b_goal_hour (18) parameters. I am currently using that to have Node-RED calculate an alternative schedule while still running VRM DESS (mode: auto) and using the alternative schedule to trigger the ESS state into ‘Keep batteries charged’ during the morning/midday hours when prices are generally lowest (and then some, will post the flow soon).

What I found missing for Node-RED DESS is the ability to set ALL the ‘VRM dynamic ESS node’ parameters as that would allow to sync those with the VRM DESS settings, why only TG/FG max, Buy/Sell price and Green mode and not the rest?

What I will dearly be missing in VRM DESS if forced to depart from Node-RED DESS completely when tibber changes over to 15m pricing (soon?), are those dayly b_goal_SOC/hour parameters. Maybe the devs can have another look at making the VRM ESS ‘Scheduled charge levels’ behave with DESS Trade (currently disfunctional).

Node-RED_DESS_Flow_081106
Node-RED_DESS_Flow_081106896×2041 460 KB

And yes: feet are sufficiently wet by now. @BartChampagne @dfaber
PS, I set prices to simply p*1.21 in Node-RED, makes evaluating the graphs a bit more intuitive. In VRM DESS the correct tibber prices are used.

Here is the zipped json (7-Zip).
It requires:

  • node-red-contrib-boolean-logic
  • node-red-dashboard
  • victron-dynamic-ess
  • Your own vrm-id vrm-token

Configuration:

  • MP-II 5000 GX
  • 90kWh 16 * 12S Li-NCM (Porsche Taycan) parallel. 16 * 128 = 2048Ah. 3V - 4.2V gives 95% capacity. Min-Soc at 20%
  • BMV-712 Battery Monitor
  • BMV-712 DC-System
  • 4 * Huawei R4875G1 CCCV 40A 50.4V on ACout-2
  • 35A 1-phase 230Vac (max 40A ac / 9kW)

UpCycle_DESS_Trade_v0.01_112359.zip (20.6 KB)
Just for learning, use at own risk

VRM DESS Trade just switched to ‘Keep batteries charged’ based on Node-RED DESS and hybrid workflow:

VRM_ACTIVE_120709
VRM_ACTIVE_120709891×1961 150 KB

GUIv2_ACTIVE_120655
GUIv2_ACTIVE_120655854×503 51.3 KB

Node-RED_DESS_Flow_ACTIVE_120703
Node-RED_DESS_Flow_ACTIVE_1207031920×1184 311 KB

DESS_ACTIVE_121158
DESS_ACTIVE_1211581702×957 198 KB

ADVANCED_ACTIVE_121742
ADVANCED_ACTIVE_121742900×1378 193 KB

PS, that negative ‘PV Inverter’ keeps the AC loads till clean, while not adding to any solar counting (because: negative). Also, the MP-II actively adds DC charge current right up to the maximum ACin current limit (set to 42A gives stable 39.4A AC current. Not sure why there is that discrepancy there. Any devs know @dfaber ?
Screenshot_20250603-122445
Screenshot_20250603-1224451340×800 52.1 KB

@mpvader : could you pls assist moving (a copy of) this post to a better suitable topic of it’s own, where I can continue with a log of it’s further development?

9

Well the main obvious reason, One shoe doesnt fit everybody , that should be enough

When you say “phase out” does that mean remove all together from the cerbo, this would be a complication to me as I use a basic node red on the cerbo to use as a gateway to my Pi which does all the calculations for my system.

I could never get VRM DESS to work exactly the way I wanted, always had curtailment, always had charging in peak times, battery charging with solr when solar should have been exporting as thats more profitable, and a million other reasons.

So I learned and built my own system which I think is far superior to my needs, its built around specific needs and it deliversd perfectly, maximum profit while maintaing consumption

Not to mention, many of us love to play around with stuff like this, where is the fun in just using a generic system, I learned node red because of the shortcomings of the VRM DESS.

I must admit I have only used the VRM DESS for very short periods over the last few months while I have been developing my system (DECS), but it still fell short of my needs.

One aspect I am working on at the moment is that DECS fetches next day tariffs (and it could actually fetch all octopus smart tariffs), then compares them based on historical consumption, next day solar forecast, calculates which is going to be the most profitabe for the next day, then will auto switch to that tariff, then that days data willl be fed back into the profit/loss calculations.

Now I admit I dont know if the VRM DESS is capable of that yet, and maybe it is, but even so it wouldnt be optimised for my needs.

10

I’m very interested in that as well, my current solution is more a ‘hack’ than an ‘elegant’ solution (see above) for the fact that DESS (Trade) is otherwise always too late to start charging in the hours before the new prices roll in (and therefore only charging for what it knows it can sell same day while it is reasonable to assume next morning will provide additional ‘sell opportunities’.

11

Im currently migrating my system off the cerbo, its grown quite big and started taxing the cerbo, once I am fully migrated, work on the auto switching flow will start again, here is some of the preliminary work where it is comparing the two tariffs and working out which is the best one to be on in a basic form (consumption and solar activity not integrated yet), all colour coded for easy reading, comparing Go to Agile (Go is the fallback tariff for me)

Once historic consumption and solar/battery metrics are implemented there will likely be another table with the calculations of assumed prices throughout for the next day and a recommendation of which tariff to switch to (not really thought that too far ahead yet) with ÂŁÂŁ numbers

But what it will do is start acting on the best (most profitable) tariff immediately after midnight so the full 24hrs will be assigned to that tariff and the system will act accordingly, and the switch will take place that day.

Its quite a compliocated flow, initially will go with manual switching, but already working on an auto switching flow too.

Screenshot 2025-05-28 201849
Screenshot 2025-05-28 2018491653×1074 137 KB

12

I saw your other post with the flows, OMG, that looked complicated :hushed_face::smiling_face_with_sunglasses::rocket:
I’d be quite happy already if I could extend the current 24 to 48 hour price forecast ‘window’ with 12 to 24 hours of heuristic price estimation data. Do you have any suggestion how to implement such a thing?

13

The idea is to add most of the existing functionality to the VRM API node. The configuration part will of DESS will then be taken from VRM, which saves us from having to store it in Node-RED too.
Using the VRM-API node you can already query and update the configuration of DESS in VRM, so that will remain possible

The VRM-API node will spit out the schedules that can then be fed into the schedule paths of Venus (just as the current node does). In between the VRM-API node and filling into Venus, you can then alter the schedule as pleased.

As far as I see all of the functionality remains, but it will need some tweaks in a flow here and there.

Only that daily b_goal_hour and b_goal_soc is a good point of @UpCycleElectric I will look into that.

1 Like
14

A, Ok, so I see, i think I misunderstood the original post, you are not removing node red from the Cerbo completely, just removing some functionality of it over to VRM.

15

Indeed, this is only about phasing out this node: victron-dynamic-ess (node) - Node-RED

1 Like
16

A migration guide of sorts would be quite helpful, one of the advantages of Node-RED DESS is the much less daunting learning curve with most of the logic and data structures readily available to figure out what is going on ‘under the hood’. VRM DESS, not so much so.

Also, there are distinc advantages to be had in running two DESS variants parallel in hybrid mode, as demonstrated above. I was planning to run VRM DESS in Green mode with Node-RED DESS in Trade mode with an arbitration flow in between. Unless Trade Mode improves significantly (as in targeting max SoC strategies just as well as min SoC strategies concurrently), phasing out Node-RED DESS altogether would be a big step backwards, IMHO.

17

Happy to share the flow I wrote, its a stand alone flow at the moment, but pulls data from the api of course which I think are user input variables (I originally had them as hard coded), so you would need to find a way to get your api and account info into the flow. Would that help ?

You could adapt it tou your needs perhaps.

By the way, I have no node red coding experience (although getting more familiar with it), everything I have built has been done by AI, I just tell it what I want to achieve, its not actually as simple as it sounds, but I couldnt have done any of my system without AI.

18

My experience is less than a month to date, see msg May 9, it was quite doable once I put myself to it.

I started with AI (grok) but couldn’t stand the hallucinations. Having a bit more hands-on experience will make it easier to get value out of those LLM’s, especially when the flow logic itself gets more complicated and AI only needs to create meta code because I found it horrible for the nitty gritty implementations (when I had absolutely no clue where to start yet).

19

I started with Grok, but it got rubbish, I now use Gemini Studio, and its very good, far superior, the key is understanding how to write the prompts.

20

With Grok you need a paid version for it to work I think. Anyway, I can’t find any DM options on this forum, I think those are disabled. How can I contact you? I’m using victron@av-e.nl for this forum.