Use different weather model for solar forecast

Q&A and troubleshooting DESS
1

@dfaber :
In the past few days I noticed that the solar forecast of DESS is way too optimistic most of the time. It seems that the forecast algorithm is using data from the US weather model, which tends to be very inaccurate in Europe.
Would it be possible to use weather data from a different source, like ECMWF?
There are some weather forecast websites available, where you can compare the forecasts from many different calculation models of many different countries and organizations, and the US forecast seems to be the least reliable for Europe. This is particularly true in weather situations in mid-winter, where there is high air pressure, but fog is absorbing the sunlight almost entirely. Above the fog (1000 meters or higher) the sun is really there, but to no avail for most of us, who reside in a valley.
Kind regards and keep up the good work!

1 Like
2

Hello

I assume you are talking about the solar production forecast.

On my end, pending for the Victron forecast training (if it works), I use three different sources:

  • one is the Victron native forecast
  • two are pushed from Home Assistant with MQTT: Solcast & Forecast Solar (both have ready to use plugins for HA)

Depending on some specific parameter of my system, at 10pm, I consider for the next day production’s, either:

  • the smallest production of the three forecasts (I position the system in the worst case production situation)
  • the average of the three forecasts.

Indeed, this implies using Node Red & some programming logics + some Home Assistant knowledge.

So to answer your question: if you find a source with trustable data & related API (ideally free), this could be surely done using NodeRed.

Matt

3

Hi Matt,
thanks for the info.
The approach to use three different forecast models and then to use the worst case scenario or the average of the three seems very sensible to me.
I myself am not using Home Assistant, nor am I too much into Node-Red programming, but I found a website at https://open-meteo.com/ which offers weather forecast data from various sources via an http API, free for non-commercial use.
If I find the time, I will try to fiddle around a bit with Node-Red and this API. Maybe the solar production forecast can be enhanced a little bit.
Cheers,
Baloo

4

Hi

Thanks for the link.

I played around and, helped by ChatGPT, I’m now able to get the data into NodeRed from Open Meteo.

Just to check against my past production & related forecast, over the last 2 days, I checked 10 irradiance models available from Open Meteo:

  • SolCast (from HA) is providing results that are not far from the various models average
  • Victron forecast has always been over-optimistic (my system is still young, thus maybe still learning…)
  • all models were wrong for the 1st January (this day, I actually got more production than forecasted)

Here is the flow:

image
image1750Ă—208 20.9 KB

For the HTTP request node, just put the open meteo API URL you have generated from their web site - with solely the Global Tilted Radiation GTI forecast and the models you are interested in + proper setting for the orientation & tilt of your PV array. Also, do not forget to change the output type to have an analyzed JSON model in the node setting.

image
image1016Ă—1250 67.9 KB

Here is the code in the Function node (ChatGPT-designed). Adapt the powerPeak and performanceRatio const according to your PV system.
Out of the individual production for each model data, I also added in the JSON output, the minimal/maximal/average production values over all models. Minimum and Average are excluding models returning zero (I hardly trust zero production can happen, and I prefer excluding it from calculations).

// Données JSON reçues (remplacez msg.payload par vos données si elles sont injectées)
let data = msg.payload;

// Vérifiez que les données nécessaires existent
if (!data.hourly || !data.hourly.time) {
    node.error("Les données horaires sont manquantes dans le payload.");
    return null;
}

// Extraction des données horaires
let hourlyData = data.hourly;
let timeStamps = hourlyData.time;

// Identification des modèles météorologiques disponibles
let models = Object.keys(hourlyData).filter(key => key.startsWith('global_tilted_irradiance'));

// Paramètres de l'installation photovoltaïque
const powerPeak = 3.5; // Puissance crĂŞte en kWc
const performanceRatio = 0.9; // Coefficient de performance

// Fonction pour calculer la production journalière
function calculateDailyProduction(irradianceValues) {
    // Conversion des irradiances horaires de W/m² à kWh/m² et somme totale
    let totalIrradianceKwhM2 = irradianceValues.reduce((sum, value) => sum + (value / 1000), 0);
    // Calcul de la production en kWh
    return totalIrradianceKwhM2 * powerPeak * performanceRatio;
}

// Fonction pour calculer la différence de jours entre deux dates
function getDayOffset(currentDate, targetDate) {
    const current = new Date(currentDate);
    const target = new Date(targetDate);
    const diffInMs = target - current;
    return Math.round(diffInMs / (1000 * 60 * 60 * 24)); // Convertir millisecondes en jours
}

// DĂ©terminer la date du jour courant (premier timestamp du JSON)
const currentDate = timeStamps[0].split('T')[0]; // Extraire la date (AAAA-MM-JJ)

// Organisation des données par modèle et par jour
let results = {};
models.forEach(model => {
    let dailyData = {}; // Stockera les résultats par jour pour ce modèle

    timeStamps.forEach((time, index) => {
        let day = time.split('T')[0]; // Extraire la date (AAAA-MM-JJ)
        if (!dailyData[day]) dailyData[day] = []; // Initialiser le tableau pour ce jour
        dailyData[day].push(hourlyData[model][index]); // Ajouter la valeur horaire correspondante
    });

    // Calculer la production journalière pour chaque jour de ce modèle
    results[model] = {};
    Object.keys(dailyData).forEach(day => {
        const dayOffset = getDayOffset(currentDate, day); // Calculer l'Ă©cart en jours
        const dayKey = dayOffset === 0 ? "D+0" : (dayOffset > 0 ? `D+${dayOffset}` : `D${dayOffset}`);
        results[model][dayKey] = calculateDailyProduction(dailyData[day]).toFixed(2); // Production en kWh (arrondi à 2 décimales)
    });
});

// Calculer les statistiques globales (min, max, moyenne) pour chaque jour à partir des productions calculées
let globalStats = {
    global_tilted_irradiance_min: {},
    global_tilted_irradiance_max: {},
    global_tilted_irradiance_average: {}
};

// Récupérer toutes les dates disponibles dans les résultats d'un modèle (les jours sont identiques pour tous les modèles)
let days = Object.keys(results[models[0]]);

days.forEach(dayKey => {
    let productionsForDay = models.map(model => parseFloat(results[model][dayKey])); // Récupérer les productions journalières pour tous les modèles
    
    // Ignorer les valeurs égales à zéro pour le calcul du minimum et de la moyenne
    let nonZeroProductions = productionsForDay.filter(val => val > 0);

    // Calculer le minimum, le maximum et la moyenne pour ce jour
    let minProduction = nonZeroProductions.length > 0 ? Math.min(...nonZeroProductions) : 0; // Si tout est zéro, min reste zéro
    let maxProduction = Math.max(...productionsForDay);
    let avgProduction = nonZeroProductions.length > 0 
        ? nonZeroProductions.reduce((sum, val) => sum + val, 0) / nonZeroProductions.length 
        : 0; // Si tout est zéro, moyenne reste zéro

    // Ajouter les statistiques au résultat global
    globalStats.global_tilted_irradiance_min[dayKey] = minProduction.toFixed(2);
    globalStats.global_tilted_irradiance_max[dayKey] = maxProduction.toFixed(2);
    globalStats.global_tilted_irradiance_average[dayKey] = avgProduction.toFixed(2);
});

// Ajouter les statistiques globales aux résultats finaux
results.global_tilted_irradiance_min = globalStats.global_tilted_irradiance_min;
results.global_tilted_irradiance_max = globalStats.global_tilted_irradiance_max;
results.global_tilted_irradiance_average = globalStats.global_tilted_irradiance_average;

// Retourner les résultats dans msg.payload
msg.payload = results;
return msg;

Here is the type of result (in Node Red debug) - this is for the API URL set with 0 past_days and 2 forecast_days, 10 irradiance models:

image
image920Ă—1516 121 KB

Note that I used the “D+x” or “D-x” naming for the days, “D+0” being the current day. This allows to have a stable path in the JSON structure.

I am from now using the D+1 averaged irradiance to my daily flows and process, to get an additional data.

Enjoy !

Matt

1 Like
5

Hi Matt,
I am absolutely baffled by the short time in which you came up with a solution!
Great work! Thanks a lot for this!
I shall definitly try this out the next days, but I will have to install the large Venus OS image first.
@dfaber :Perhaps this could also be integrated into the Venus OS firmware at a later time, although I am not sure if open-meteo.com would still treat this as non-commercial use. But maybe there are other free sources for weather forecasts, that I have not found yet.