Importing prices into Dynamic ESS with NodeRED

I’ve spent a bit of time setting up a Node-RED flow to import prices from my (unsupported) local provider in Australia - Amber Electric - using their API. With thanks to @dfaber’s post and @podarok’s contribution also, I’m feeding the provider’s prices into Dynamic ESS successfully:

// Extract the intervals from the payload
const intervals = msg.payload;

// Create empty arrays to hold buy and sell schedules
let buySchedule = [];
let sellSchedule = [];

// Helper function to convert UTC time to local time
const toLocalTime = (isoTime) => {
    const date = new Date(isoTime);
    const options = {
        hour: '2-digit',
        minute: '2-digit',
        hour12: false,
        timeZone: Intl.DateTimeFormat().resolvedOptions().timeZone // Local timezone
    };
    let time = new Intl.DateTimeFormat('en-US', options).format(date);

    // Fix for midnight time formatting
    if (time.startsWith("24:")) {
        time = time.replace("24:", "00:");
    }

    return time;
};

// Iterate through each interval and sort data into buy and sell schedules
intervals.forEach(interval => {
    const from = toLocalTime(interval.startTime); // Convert to local time
    const to = toLocalTime(interval.endTime); // Convert to local time

    // Extract and format price
    let price = parseFloat(interval.perKwh);

    if (interval.channelType === "general") {
        // Convert buy price to dollars and round to the nearest cent
        price = Math.round(price) / 100;
        buySchedule.push({ from: from, to: to, price: price });
    } else if (interval.channelType === "feedIn") {
        // Invert sell price, convert to dollars, and round to the nearest cent
        price = Math.round(-price) / 100;
        sellSchedule.push({ from: from, to: to, price: price });
    }
});

// Create the final output structure
const buyPriceSchedule = [{
    days: [0, 1, 2, 3, 4, 5, 6], // All days of the week
    schedule: buySchedule
}];

const sellPriceSchedule = [{
    days: [0, 1, 2, 3, 4, 5, 6], // All days of the week
    schedule: sellSchedule
}];

// Assign the formatted schedules to the payload
msg.payload = {
    buyPriceSchedule: JSON.stringify(buyPriceSchedule),
    sellPriceSchedule: JSON.stringify(sellPriceSchedule)
};

return msg;

This function node feeds a “Change Dynamic ESS Settings” node, which successfully imports the price data from my provider.

If I look at VRM → Settings → Dynamic ESS, I see the buy and sell prices correlating correctly to the information my electricity provider’s API generated. What doesn’t line up, though, is the graphed “Schedule” data in the Dynamic ESS section of the dashboard - the prices are not what has been specified on the Dynamic ESS settings page by the Node-RED.

I confess I don’t understand Dynamic ESS well enough just yet - I’m really just experimenting at the moment and it’s switched off for now because it hasn’t been making good decisions - but I’m concerned that it’s not acting on up-to-date information based on the discrepancy between the graph and the settings page.

My provider passes on wholesale prices (our national energy market operates on a 5-minute basis), and I trigger the flow to reload every 15 minutes so as not to overwhelm the system or the provider’s API.

Does anyone have feedback about this approach and why there might be this discrepancy?

Thanks so much in advance, this is a great product that I’m sure will only get better! :smiley:

Andrew

1 Like

You need to let 28 days of data for DESS to work properly.

Rule of thumb - set something like 70% SOC, green mode and let it work

1 Like

Thanks mate — looks like you’re doing some really cool stuff with it!

I’d like to see it trade — or at least, charge intelligently — to minimise grid cost. For example, if SoC is a bit low, grid price is low now but expected to be high in a few hours, it makes sense to buy from the grid now at a low price and then discharge to loads when the grid is expensive.

All of that can be controlled with some simple enough Node-RED work, but it’d be cool to see it happen like magic inside VRM.

Still not entirely clear why the prices I’m feeding into DES are not being reflected on the graphing for the schedule.

I have trade mode(feed-in to grid) in DESS + automated prices pull from planned grid cuts. Working somehow ok.

For the actual outage I’m disabling feed-in

Prices, once you set them - visible after refresh of the graph. Use next-prev day pagination to refresh it. Also keep in mind DESS has 60s update interval as well as hourly schedule change from server

@ndrew hello, I have an ESS system just installed and would also like to transfer Amber price information into DESS. I wonder would you be willing to share your Node where the data from Amber is imported into DESS?

I am just starting to work with Node-Red would appreciate any help with the integration with Amber.

Many thanks…

Of course! Please feel free to import my flow into Node-RED and customise it for your environment, it’s here on Github:

You can paste that JSON into the “Import” dialogue in Node-RED. You will need to make a few changes to point it to your VRM installation and your Amber account.

If you haven’t already, get yourself a VRM API token (from VRM: Preferences → Integrations → Access tokens) and note the installation ID from your Dashboard URL (you’re looking for the numbers after /installation/XXXXXX).

When you login to the Amber web app, you can enable “Developer Mode” which shows their API documentation. Generate an API token (call it “NodeRED” or something) and keep a copy of it safe somewhere. Authorise the documentation with that token by clicking the “Authorise” button. You can then retrieve your Site ID from the GET /sites endpoint by clicking “Try it Out”.

You’ll have to now enter your Amber details into the flow you imported by editing the HTTP request node called “Get forecast prices”. In the URL, replace <SITEID> with the site ID you got from the Amber API. Enter your API key as the Bearer token.

Edit the VRM API node called “Change Dynamic ESS settings” and enter your Installation ID in the corresponding field. Under the VRM field, choose “Add new config-vrm-api” and give it a name and your VRM API key. Save it all and deploy.

That should more or less work! The inject node will fire every five minutes, which should keep VRM up-to-date as Amber fluctuates prices throughout the day. As pointed out by @podarok upthread, if VRM doesn’t show the prices you’re expecting to see on the Dynamic ESS graphs, try navigating back and forward on the date selector - that seems to update the data for me.

Sing out if you need any help! :slight_smile:

2 Likes

Many thanks Andrew. Very much appreciated. I can see my data in the Amber dev site so that is a start.

I have access to the VRM API so that is also working.

I am having some trouble accessing the Amber site from the HTTP Request node.

You’ll have to now enter your Amber details into the flow you imported by editing the HTTP request node called “Get forecast prices”. In the URL, replace <SITEID> with the site ID you got from the Amber API. Enter your API key as the Bearer token.

I have entered the API key as Bearer token ok but not sure how to " In the URL, replace <SITEID> with the site ID you got from the Amber API".

That screen looks like this

I am embarressed to admit I don’t know how to use the programming you provided in the Node. At this stage I have only attempted using the the simple Node programming as shown in the screen shot but haven’t attempted anything lower level than that. In fact I don’t even know how to get to the lower level programming for the Node…

I have managed to deploy something fairly trivial to work on my system in Node Red so that was at least a start on this journey.

When I can get some time I am keen to get the HTTP Request node talking to Amber. Any tips greatly appreciated.

Ah! I think the import process has borked my placeholder in the URL. Replace the URL with https://api.amber.com.au/v1/sites/<SITE_ID>/prices - and enter the site ID from the Amber Developer page in place of <SITE_ID>

Fingers crossed - that’ll do the job. Let me know how you go! :slight_smile:

1 Like

Thanks so much Andrew. I now have that section working thanks to your advice. When I get some time I will try to get this thing pieced together and working.

1 Like

Hi Andrew,

I now have prices in the system! Whooohoo. I do have a slight problem in that the prices seem to 12 hours out. In other words at 6am I am seeing prices that are more likely to be for 6PM and vice versa. I have the Amber interface working and those prices look right to me.
I have not really delved into it too much without checking with you first. Is that what your price issue was earlier and if so, how did you resolve it?

Many thanks…

We’re making progress, Trevor!

This is a little odd. The function converts the UTC time in Amber’s API response to local time, so I’d suggest we start looking at your local settings first.

Can you confirm your Cerbo’s time and time zone are set correctly?

Hi Andrew, Yes, the Cerbo time zone is set to Sydney. I thought it maybe took timezone from location but it is set to Sydney and it is correct. When I set the Timezone to UTC the graphs looked about right so I think the prices are probably UTC, hence the slip.

I will look into it when I have time but sensational to at least have prices in the system.

That’s odd - there’s a bit of code in the function to convert prices from Amber’s UTC times to local time based on what your Cerbo’s timezone is set to. I take it your location is set in VRM and the timezone is set via the console (Settings → General → Date and Time → Timezone)? UTC is disabled on my Cerbo.

Try adding this to the bottom of the “Set Dynamic ESS prices” node code, just above return msg; to get it to display your timezone in the debug node:

msg.timezone = Intl.DateTimeFormat().resolvedOptions().timeZone;

Deploy the flow (top right of the screen) and then in the sidebar (you may need to show it by clicking the three-lines icon in the top-right → view → show sidebar) you should see some debug output. You’re looking for the debug output from “DESS prices debug”, which if you expand the object, you should see timezone: Australia/Sydney if the timezone is set correctly on the Cerbo.

If you make any changes to the timezone settings on your Cerbo, it might take a reboot for Node-RED to pick it up – I don’t know – but you’ll certainly have to re-trigger the flow to load the Amber data in with the right times. Let me know how you get on!

Thanks Andrew…Well I am learning things which is good. I used the snippet you provided and set the debug to “see” msg.timezone and sure enough, the timezone is definately Australia/Sydney. I have checked the Cerbo and it is definately Sydney.

The price graph shows correct current time. The price data is showing in the UTC location on the graph…

If I see the output of the conversion function on the debug window it shows one day, and 12 hours of buy and sell data. Does that sound right to you? I can see the array for the 7 days but I don’t see an output for any day but the current one and then not 24 hours but 12 hours data.

Anyway, I am learning a little about JS which is a fun thing to do.

Quick update…it appears the data going into the “Update Dynamic ESS” node is correct. Something is going astray after that…

Another quick update. I decided to check the CerboGX …Settings → Dynamic ESS and scroll down to buy and sell prices. They are accurate!

When I look at the graph they are still wrong. Holy smokes!

Many thanks for the guidance with this. I figure I have an interesting problem.

Have you reloaded the VRM page, or scrolled back and forward a day on the graph? I had some weird issues with VRM displaying “old” prices early on that @podarok pointed out go away when you use the pagination.

1 Like

Yes, understood. I have rebooted many times and scrolled as suggested. It is a very interesting problem where I now know prices in the system are correct but the graph seems to be wrong.

Anyway, maybe the DESS will work correctly in any case based on the prices at least being ok.

I’m not sure how the graph handles negative numbers either. Right now on the east coast we have 5c/kwh general use and -4c /kwh for feedin. What a crazy energy market we have here!

Indeed! The grid really can be an ally during the day, particularly if you’ve got large loads you can shift to these low-price periods (EV charging for example).

Are you running DESS in Green or Trade mode?

I’ve found it takes a little while (some hours, perhaps) of just leaving it be for it to start making sensible decisions. It still doesn’t seem to know how to deal with negative buy prices (i.e., when Amber is paying you to use power) — it calculates it as a “cost” rather than cr edit — but by and large running in green mode, on days when we’re not thrashing the ducted A/C, our 30kWh of batteries and 10kW of solar are keeping our Amber bills to < $1/day (and we’re a 40kWh/day household… somehow!)