Could somebody confirm / deny that the DESS scheduler has been acting up today with multiple ‘down’ periods varying from a missed quarter up to more than an hour at times?
It would be great if we could have an uptime / downtime indicator on VRM on the homepage, right next to the product news section. Even just a timer indicating the most recent schedule update would do.
Still ongoing issues with the VRM DESS scheduler? please confirm @Barbara
I haven’t seen any complaints on the internal group. Though, that isn’t conclusive.
Thanks but can somebody please check the logs/stats then please? I have seen over a dozen missed schedule updates over the past 24 hours, live and in our logs, some in blocks of well over an hour, with no obvious regularity detectable, and no sign of internal nor external connection issues either. There have been quite a few of these incidents over the past year and longer, and under normal circumstances the system will handle them just fine. Unless you are trying to make changes and do some fine tuning. Then it is very valuable to have some sort of direct access to a conclusive source of information on the actual operational state of the scheduler before starting a wild goose chase locally. Dont you guys run some sort of a dashboard with warnings for that or is it just black box until people start complaining?
Our system subscribes to mqqt115, maybe that helps.
I am still experiencing ongoing schedule update skipping.
Now I do appreciate the system handles this with grace based on the up to 12 hours of forward schedule planning, which is a great and robust feature, but that does not negate the fact that any DESS setting changes are also depending upon a timely delivery of schedule updates.
And under special circumstance such as active development and testing of custom Node-RED DESS integration functionality it become quite a drag to A) stare at the screen not knowing whether the quarterly update will fire and B) not getting any factual response from Victron on the status of the VRM DESS scheduler backend. And I know there are all kind things happening behind the scenes with respect to VRM DESS Scheduler that are not communicated to me/us customers, so it is not a far stretch to think these things are related. Therefore I find it bordering on the absurd that I cannot get a straight answer whether there are unplanned and possibly even unnoticed performance issues with that massive bank of mqqtxyz.victronenergy.com brokers, or that I just happen to be subscribed to a flaky mqqt115.victronenergy.com, or that these missed updates are part of known but uncommunicated work/maintenance and the mqqt brokers or the systems behind the brokers that do the actual schedule calculations. Or that all of that is just one big coincidence and I am too stupid to figure out there is an actual problem on my end (and I tried to find any real hard but couldn’t).
Bottom line this is not even about technical issue, it is ALL about Victron being non-communicative for unknown reasons, could be accidental, work overload, or deliberate even, I have no means of knowing. But I cannot fathom it to be so extremely difficult and time-consuming to go check some logs/dashboards on mqqt115.victronenergy.com that Victron cannot even shine at least a shiver of light on this ongoing issue.
Literally no one else has an issue. And we have installers with many clients on related groups. I cannot escalate something only you are reporting.
So what you are saying is that because nobody but one person running a system connected to that (one-out-of-many) mqqt115.victronenergy.com broker is noticing that there are issues to begin with. Which is quite normal because you’d have to be actively monitoring the schedule updates to notice, and me being likely the only person actively developing a DESS based NODE-RED application that makes me notice, you guys at Victron simply argue that it is not worthwhile to look at the logs, because there are not enough complaints about it.
You know, this is called the “squeak management system” and known to be one of the most devastating ways of dealing with highly complicated technical developments and accompanying issues. You should be so happy I am giving you the opportunity to have look to see if there are undetected issues with your backbone systems, instead of painting me as a lone wolf crying wolf. Unbelievable.
And just to prevent you and others thinking I am just making all the above up out of thin air, here a logscreen copy of what I am working on (and indirectly makes me the only person in a position to notice and to be so kind to provide Victron with a friendly warning signal that there might be something persistently wrong at their cloud backbone.
I am not asking you to drop everything you are doing or send me your firstborn am I, I am only asking you to look at the logs. For crying out loud.
I will leave this subflow running for the weekend, and report back the log results. Anybody interested in monitoring their own system for this issue can simply import the flow. it does nothing but logging the active windowslot with timestamps. Normally you would only see the windowslot toggle between 0 and 1, when ‘windowslot > 1’ your system has missed a quarterly schedule update:
[
{
"id": "20dca72999465eb2",
"type": "tab",
"label": "SLOTLOG",
"disabled": false,
"info": "",
"env": []
},
{
"id": "308a3e3bf21503f5",
"type": "group",
"z": "20dca72999465eb2",
"name": "SLOTLOG",
"style": {
"label": true
},
"nodes": [
"4eda5dc95fb33308",
"e27ce897075b368c"
],
"x": 54,
"y": 39,
"w": 592,
"h": 82
},
{
"id": "4eda5dc95fb33308",
"type": "victron-input-custom",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"service": "com.victronenergy.system/0",
"path": "/DynamicEss/WindowSlot",
"serviceObj": {
"service": "com.victronenergy.system/0",
"name": "com.victronenergy.system (0)"
},
"pathObj": {
"path": "/DynamicEss/WindowSlot",
"name": "/DynamicEss/WindowSlot",
"type": "number",
"value": 1
},
"name": "",
"onlyChanges": true,
"roundValues": "no",
"rateLimit": "0",
"outputs": 1,
"conditionalMode": false,
"outputTrue": "",
"outputFalse": "",
"debounce": "",
"x": 290,
"y": 80,
"wires": [
[
"e27ce897075b368c"
]
]
},
{
"id": "e27ce897075b368c",
"type": "function",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"name": "slotlog",
"func": "const ts = Date.now();\nconst logkey = (\"slotlog.\" + ts);\nconst logobject = { timestamp: ts, windowslot: msg.payload }; \nglobal.set ( logkey, logobject );\nreturn null;",
"outputs": 1,
"timeout": 0,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 570,
"y": 80,
"wires": [
[]
]
},
{
"id": "14c26b05e56668a1",
"type": "global-config",
"env": [],
"modules": {
"@victronenergy/node-red-contrib-victron": "1.6.60"
}
}
]
const ts = Date.now();
const logkey = (“slotlog.” + ts);
const logobject = { timestamp: ts, windowslot: msg.payload };
global.set ( logkey, logobject );
return null;
Well, as it turns out. we didn’t have to wait all weekend, 4 quarters missed already. Do I need to make my point more clear than this, really? (Or are you just going to silence/block me as you threatened me with for ‘getting clever’, in private DM already)
{"1773432988389":{"timestamp":1773432988389,"windowslot":2},"1773433801359":{"timestamp":1773433801359,"windowslot":3},"1773434701580":{"timestamp":1773434701580,"windowslot":4},"1773435602051":{"timestamp":1773435602051,"windowslot":5},"1773435717214":{"timestamp":1773435717214,"windowslot":0},"1773436502914":{"timestamp":1773436502914,"windowslot":1},"1773436618053":{"timestamp":1773436618053,"windowslot":0},"1773437403824":{"timestamp":1773437403824,"windowslot":1},"1773437518930":{"timestamp":1773437518930,"windowslot":0}}
I had to update the flow to reduce noise. Now it captures only those events that have missing quarterly updates. This ran since yesterday, the logs from earlier in the weekend showed the same patterns).
This is too regular to be a one-off or a coincidence (skipping precisely a full hour of updates, in 3 hour intervals is a dead giveaway). Either there is something programmed to skip full hour updates at distinct hour, or there are issues with the scheduler. Either way, the effect is the same: during those ‘stale’ hours, changes in DESS settings are not propagated to the operational system as they only take effect after a schedule update. The former is an issue of lack of information sharing, the latter is a system issue, either way this IS an issue. I find it hard to accept that I am made to carry the burden of proof by a volunteer forum mod that already told me to stop ‘pushing’ issues or ‘risk’ being put on the ‘ignore list’. There is something even more problematic with that, than whatever is going on with the DESS scheduler IMHO. @nickdb @Barbara
My request to Victron to check the scheduler logs stands, or provide an update on why the scheduler behaves this way. That is a completely reasonable request, even if I am the only person observant enough to notice this scheduler behavior this past week.
[
{
"id": "20dca72999465eb2",
"type": "tab",
"label": "SLOTLOG",
"disabled": false,
"info": "",
"env": []
},
{
"id": "308a3e3bf21503f5",
"type": "group",
"z": "20dca72999465eb2",
"name": "SLOTLOG",
"style": {
"label": true
},
"nodes": [
"4eda5dc95fb33308",
"e27ce897075b368c"
],
"x": 54,
"y": 39,
"w": 592,
"h": 82
},
{
"id": "4eda5dc95fb33308",
"type": "victron-input-custom",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"service": "com.victronenergy.system/0",
"path": "/DynamicEss/WindowSlot",
"serviceObj": {
"service": "com.victronenergy.system/0",
"name": "com.victronenergy.system (0)"
},
"pathObj": {
"path": "/DynamicEss/WindowSlot",
"name": "/DynamicEss/WindowSlot",
"type": "number",
"value": 1
},
"name": "",
"onlyChanges": true,
"roundValues": "no",
"rateLimit": "0",
"outputs": 1,
"conditionalMode": false,
"outputTrue": "",
"outputFalse": "",
"debounce": "",
"x": 290,
"y": 80,
"wires": [
[
"e27ce897075b368c"
]
]
},
{
"id": "e27ce897075b368c",
"type": "function",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"name": "slotlog",
"func": "const ts = Date.now();\nconst slot = msg.payload;\n\nconst logkey = \"desslog.\" + ts;\nconst lastslotkey = \"desslog.lastslot\";\nconst lastupdatekey = \"desslog.lastupdate\";\nconst lastslot = global.get( lastslotkey ) || 0;\nglobal.set( lastslotkey, slot);\nif ( !global.get( lastupdatekey ) ) { global.set( lastupdatekey, ts ) }\n\nif ( slot === 0 ) {\n global.set( lastupdatekey, ts);\n if ( lastslot > 1 ) { \n const logobject = { timestamp: ts, lastslot: lastslot, slot: slot };\n global.set( logkey, lastslot );\n global.set( logkey, logobject );\n }\n}\n\nreturn null;",
"outputs": 1,
"timeout": 0,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 570,
"y": 80,
"wires": [
[]
],
"info": "{\"lastslot\":0,\"1773576248125\":{\"timestamp\":1773576248125,\"lastslot\":5,\"slot\":0},\"lastupdate\":1773586921357,\"1773586921357\":{\"timestamp\":1773586921357,\"lastslot\":5,\"slot\":0}}"
},
{
"id": "9768aa66068a3cb0",
"type": "global-config",
"env": [],
"modules": {
"@victronenergy/node-red-contrib-victron": "1.6.60"
}
}
]
const ts = Date.now();
const slot = msg.payload;
const logkey = "desslog." + ts;
const lastslotkey = "desslog.lastslot";
const lastupdatekey = "desslog.lastupdate";
const lastslot = global.get( lastslotkey ) || 0;
global.set( lastslotkey, slot);
if ( !global.get( lastupdatekey ) ) { global.set( lastupdatekey, ts ) }
if ( slot === 0 ) {
global.set( lastupdatekey, ts);
if ( lastslot > 1 ) {
const logobject = { timestamp: ts, lastslot: lastslot, slot: slot };
global.set( logkey, lastslot );
global.set( logkey, logobject );
}
}
return null;
{"lastslot":0,"lastupdate":1773669058899,"1773597922023":{"timestamp":1773597922023,"lastslot":5,"slot":0},"1773608691845":{"timestamp":1773608691845,"lastslot":5,"slot":0},"1773630170102":{"timestamp":1773630170102,"lastslot":5,"slot":0},"1773640964735":{"timestamp":1773640964735,"lastslot":5,"slot":0},"1773651804363":{"timestamp":1773651804363,"lastslot":5,"slot":0},"1773662693288":{"timestamp":1773662693288,"lastslot":5,"slot":0}}
This wont stay turned off. I change it, it says saved but then a refresh and its still on.
Anyone else having this issue?
Cheers.
Red.
I never turn it off so I can’t help with that. Could it be you misunderstood what this toggle does? It enables another toggle in the right top corner of the VRM dashboard, where you can turn DESS off (at the bottom of this screenshot)
If that doesn’t work either, I do know there have been issues before that some settings set in VRM fail to ‘stick’. You could try two things: VRM beta (https://betavrm.victronenergy.com/) might behave different than standard VRM. And what gives most control is to install VenusOS large and set it from there with the victron-output-dess node. Simply download this flow (buttom lower right corner) and import to node red, should work directly. You can check in VRM
[
{
"id": "3f9bb5e3d4d9df52",
"type": "group",
"z": "77e8033910d87d1a",
"style": {
"stroke": "#999999",
"stroke-opacity": "1",
"fill": "none",
"fill-opacity": "1",
"label": true,
"label-position": "nw",
"color": "#a4a4a4"
},
"nodes": [
"2e40edcd310ecf61",
"97bde198ea98e622",
"d1adee21dbcd2616",
"934a4c58baba2a6f",
"7d805ff101481e8e",
"4f239e4e8064317e",
"8d6d1aec9763b4f8"
],
"x": 34,
"y": 39,
"w": 652,
"h": 202
},
{
"id": "2e40edcd310ecf61",
"type": "inject",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"name": "0 - Off",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "0",
"payloadType": "num",
"x": 130,
"y": 120,
"wires": [
[
"97bde198ea98e622"
]
]
},
{
"id": "97bde198ea98e622",
"type": "victron-output-dess",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"service": "com.victronenergy.settings",
"path": "/Settings/DynamicEss/Mode",
"serviceObj": {
"service": "com.victronenergy.settings",
"name": "Venus settings",
"communityTag": "dess"
},
"pathObj": {
"path": "/Settings/DynamicEss/Mode",
"type": "enum",
"name": "Dynamic ESS mode",
"enum": {
"0": "Off",
"1": "Auto",
"4": "Node-RED"
},
"mode": "both"
},
"name": "",
"onlyChanges": false,
"roundValues": "no",
"rateLimit": 0,
"outputs": 0,
"conditionalMode": false,
"condition1Operator": ">",
"condition2Enabled": false,
"condition2Service": "",
"condition2Path": "",
"condition2Operator": ">",
"logicOperator": "AND",
"outputTrue": "true",
"outputFalse": "false",
"outputOnChange": false,
"debounce": 2000,
"x": 210,
"y": 80,
"wires": []
},
{
"id": "d1adee21dbcd2616",
"type": "inject",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"name": "1 - Auto (VRM DESS)",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "1",
"payloadType": "num",
"x": 180,
"y": 160,
"wires": [
[
"97bde198ea98e622"
]
]
},
{
"id": "934a4c58baba2a6f",
"type": "inject",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"name": "4 - Node-RED (DESS)",
"props": [
{
"p": "payload"
},
{
"p": "topic",
"vt": "str"
}
],
"repeat": "",
"crontab": "",
"once": false,
"onceDelay": 0.1,
"topic": "",
"payload": "4",
"payloadType": "num",
"x": 180,
"y": 200,
"wires": [
[
"97bde198ea98e622"
]
]
},
{
"id": "7d805ff101481e8e",
"type": "victron-input-dess",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"service": "com.victronenergy.settings",
"path": "/Settings/DynamicEss/Mode",
"serviceObj": {
"service": "com.victronenergy.settings",
"name": "Venus settings",
"communityTag": "dess"
},
"pathObj": {
"path": "/Settings/DynamicEss/Mode",
"type": "enum",
"name": "Dynamic ESS mode",
"enum": {
"0": "Off",
"1": "Auto",
"4": "Node-RED"
},
"mode": "both"
},
"name": "",
"onlyChanges": false,
"roundValues": "no",
"rateLimit": 0,
"outputs": 1,
"conditionalMode": false,
"outputTrue": "true",
"outputFalse": "false",
"debounce": "2000",
"x": 520,
"y": 80,
"wires": [
[]
]
},
{
"id": "4f239e4e8064317e",
"type": "victron-input-ess",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"service": "com.victronenergy.settings",
"path": "/Settings/CGwacs/Hub4Mode",
"serviceObj": {
"service": "com.victronenergy.settings",
"name": "Venus settings",
"communityTag": "ess"
},
"pathObj": {
"path": "/Settings/CGwacs/Hub4Mode",
"type": "enum",
"name": "ESS mode",
"enum": {
"1": "Optimized mode or 'keep batteries charged' and phase compensation enabled",
"2": "Optimized mode or 'keep batteries charged' and phase compensation disabled",
"3": "External control"
},
"mode": "both"
},
"name": "",
"onlyChanges": false,
"roundValues": "no",
"rateLimit": 0,
"outputs": 1,
"conditionalMode": false,
"outputTrue": "true",
"outputFalse": "false",
"debounce": "2000",
"x": 490,
"y": 140,
"wires": [
[]
]
},
{
"id": "8d6d1aec9763b4f8",
"type": "victron-input-ess",
"z": "77e8033910d87d1a",
"g": "3f9bb5e3d4d9df52",
"service": "com.victronenergy.settings",
"path": "/Settings/CGwacs/BatteryLife/State",
"serviceObj": {
"service": "com.victronenergy.settings",
"name": "Venus settings",
"communityTag": "ess"
},
"pathObj": {
"path": "/Settings/CGwacs/BatteryLife/State",
"type": "enum",
"name": "ESS state",
"enum": {
"1": "BatteryLife enabled (GUI controlled)",
"2": "Optimized Mode /w BatteryLife: self consumption",
"3": "Optimized Mode /w BatteryLife: self consumption, SoC exceeds 85%",
"4": "Optimized Mode /w BatteryLife: self consumption, SoC at 100%",
"5": "Optimized Mode /w BatteryLife: SoC below dynamic SoC limit",
"6": "Optimized Mode /w BatteryLife: SoC has been below SoC limit for more than 24 hours. Charging the battery (5A)",
"7": "Optimized Mode /w BatteryLife: Inverter/Charger is in sustain mode",
"8": "Optimized Mode /w BatteryLife: recharging, SoC dropped by 5% or more below the minimum SoC",
"9": "'Keep batteries charged' mode is enabled",
"10": "Optimized mode w/o BatteryLife: self consumption, SoC at or above minimum SoC",
"11": "Optimized mode w/o BatteryLife: self consumption, SoC is below minimum SoC",
"12": "Optimized mode w/o BatteryLife: recharging, SoC dropped by 5% or more below minimum SoC"
}
},
"name": "",
"onlyChanges": false,
"roundValues": "no",
"rateLimit": 0,
"outputs": 1,
"conditionalMode": false,
"outputTrue": "true",
"outputFalse": "false",
"debounce": "2000",
"x": 490,
"y": 200,
"wires": [
[]
]
},
{
"id": "f69566642ed056ae",
"type": "global-config",
"env": [],
"modules": {
"@victronenergy/node-red-contrib-victron": "1.6.60"
}
}
]
Hi Jan.
Thanks for the information.
I understand the function and wanted to remove it from being accessed.
I tried the Beta site and could change the setting and it saved as it should, seems to possibly be associated with the DESS setup bug they had a few days ago.
Thanks again.
Cheers, Red.
Ah ok. I believe if you turn off DESS, the on/off toggle may disappears until you re-enable the toggle in the other settings page again. It used to do that anyway. Anyway, maybe the the flow will come handy someday, I added DESS / ESS state input nodes to provide a bit more information.