API Question: Regarding raw data from the new VRM Solar Forecasting algorithm

Question

question

joshuadodds asked Jul 18, '23

API Question: Regarding raw data from the new VRM Solar Forecasting algorithm

Great feature! Have been looking for a reliable source for this data for some time and to have it "baked-in" now with Victron equipment is great.

Just one request that would make this data A LOT more useful to those of us with custom integrations and software working in concert with Victron equipment. Can we get this data from the solar projections published on the MQTT bus or available on the an accessible API endpoint somewhere so that we can periodically query this forecast data for other decisions we might want to make with advanced Dynamic ESS installations?

Is this data already available somewhere and if so could someone point me where to find/query it?

If anyone is interested in exactly what type of advanced DynESS integrations i am speaking of, Have a look at these 2 projects I have been slowly developing over the last year:

https://github.com/JoshuaDodds/cerbomoticzgx

https://github.com/JoshuaDodds/venus-nextgen

I would love to get the data from the solar forecasts directly into these two projects and start writing more advanced decision algorithms for the Dynamic ESS module i have been building (the energy broker and evCharging modules especially)

solar

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) answered · Jul 19, 2023 at 05:54 AM

Hi, thank you; and glad you like it!

The data is available via the VRM Portal REST APi.

See its documentation, as well as some hints regarding consuming the data from NodeRED on bottom of this blogpost: https://www.victronenergy.com/blog/2023/07/05/new-vrm-solar-production-forecast-feature/

3

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

joshuadodds commented · Jul 21, 2023 at 06:35 AM

@mvader (Victron Energy) Hi and thanks for your reply. I dont believe the VRM API docs have been updated with which widget or endpoint to use to retrieve this data. If so i cannot find it. I am looking at this documentation here:

https://vrm-api-docs.victronenergy.com/#/

Could you point me directly to a link which describes which part of the API to query to get this solar forecasted data? I have tried a few of the widgets but cannot find anything yet which contains this data.

0 ·

joshuadodds joshuadodds commented · Jul 21, 2023 at 06:37 AM

Spoke too fast.... I think i found it in the link you sent. This appears to be the right part of the API to call.

https://www.victronenergy.com/blog/wp-content/uploads/sites/10/2023/07/image_232350427.png

I will try this and see if it works.

0 ·

joshuadodds joshuadodds commented · Jul 21, 2023 at 07:26 AM

@mvader (Victron Energy) After diving into this some more it appears that the REST API i need to call is this one:

https://vrm-api-docs.victronenergy.com/#/operations/installations/idSite/stats

But it's still not clear how to query the solar forecast data from here when not using node red and just querying the endpoint directly with an http get request. Is there any updated documentation still needed for this?

0 ·

Modifications Space

Node Red Space

Answer 1 · 2023-07-26T18:04:21Z

joshuadodds answered · Jul 26, 2023 at 06:04 PM

So, just in case anyone else has had any trouble or was struggling with retrieving this data from the VRM API, i wanted to share my solution. I am running for several days now with very accurate and spot on forecasts matching actual solar production within 1-2 kWh margin of accuracy. The method i have used to achieve this is documented quite well (I think) in the python module I will link below. Essentially its a query that is sent every 5 minutes which adds current realtime solar production (scraped from the cervo mqtt data bus) to a solar forecast call to the VRM API with a start time of $now and an end time of $sundown. This has resulted in a very accurate forecast which matches my actual daily production remarkably accurately.

Here is the module which implements this:
https://github.com/JoshuaDodds/cerbomoticzgx/blob/main/lib/solar_forecasting.py

Hope someone finds it useful!

-jd

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 2 · 2023-07-27T20:52:59Z

alkazar answered · Jul 27, 2023 at 08:52 PM

Hi Joshua,

how did you manage to get future data from the API? I receive data only with empty start and end parameters and than its showing data from yesterday:

I just added timestamps for start (tomorrow mornig) and end (tomorrow evening) and it don't show any data anymore:

Would you be so nice to share an example URL which is working.

1690490760918.png (58.1 KiB)

1690490948106.png (43.1 KiB)

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

joshuadodds commented · Jul 29, 2023 at 09:41 AM

Hi There,

AFAIK, they have not added a +24 hour forecast yet (tomorrow's prediction) its just a current day forecast that they have implemented so far.

This code explains how i do it and so far it has been quite accurate.

https://github.com/JoshuaDodds/cerbomoticzgx/blob/main/lib/solar_forecasting.py

Read the comments at the top of the module which explain in plain english what data I send for start and end dates. It's essentially asking for the forecast between (now - 60 seconds) and sundown and then adding that forecast to what has already been produced for the day. It requests and makes this calculation every 5 minutes and results in a forecast that is within 1 to 2 kwh accurate throughout the day.

It would be nice if Victron could share the algorithm they use to populate the 2 number on the VRM Portal dashboard so that we could replicate that. I don't think they are in the API call itself so they are deriving them somehow from the raw data that CAN come from the REST API.

@mvader (Victron Energy) any comments or anything to add?

0 ·

alkazar joshuadodds commented · Jul 31, 2023 at 04:59 PM

Hey, I already read it, my problem is that the API does not send any data if I send the request containing START and END timestamps. For example: A request without START and END answers with data from yesterday (1690675200000 = GMT: Sunday, 30. July 2023 00:00:00; 1690761600000 = GMT: Monday, 31. July 2023 00:00:00)

{
    "success": true,
    "records": {
        "vrm_consumption_fc": [
            [
                1690675200000,
                3255.4685891768995
            ],
            [
                1690761600000,
                10810.130578603099
            ]
        ],
        "solar_yield_forecast": [
            [
                1690675200000,
                870.3880082984999
            ],
            [
                1690761600000,
                25379.2442356206
            ]
        ]
    },
    "totals": {
        "vrm_consumption_fc": 14065.599167779998,
        "solar_yield_forecast": 26249.632243919103
    }
}

If I use the same timestamps in my request I don't get an any data. Even if I use timestamps from today, no data. That is what I dont understand.

{
    "success": true,
    "records": [],
    "totals": []
}

0 ·

1690822553407.png (10.1 KiB)

question