Could somebody confirm / deny that the DESS scheduler has been acting up today with multiple ‘down’ periods varying from a missed quarter up to more than an hour at times?
It would be great if we could have an uptime / downtime indicator on VRM on the homepage, right next to the product news section. Even just a timer indicating the most recent schedule update would do.
Still ongoing issues with the VRM DESS scheduler? please confirm @Barbara
I haven’t seen any complaints on the internal group. Though, that isn’t conclusive.
Thanks but can somebody please check the logs/stats then please? I have seen over a dozen missed schedule updates over the past 24 hours, live and in our logs, some in blocks of well over an hour, with no obvious regularity detectable, and no sign of internal nor external connection issues either. There have been quite a few of these incidents over the past year and longer, and under normal circumstances the system will handle them just fine. Unless you are trying to make changes and do some fine tuning. Then it is very valuable to have some sort of direct access to a conclusive source of information on the actual operational state of the scheduler before starting a wild goose chase locally. Dont you guys run some sort of a dashboard with warnings for that or is it just black box until people start complaining?
Our system subscribes to mqqt115, maybe that helps.
I am still experiencing ongoing schedule update skipping.
Now I do appreciate the system handles this with grace based on the up to 12 hours of forward schedule planning, which is a great and robust feature, but that does not negate the fact that any DESS setting changes are also depending upon a timely delivery of schedule updates.
And under special circumstance such as active development and testing of custom Node-RED DESS integration functionality it become quite a drag to A) stare at the screen not knowing whether the quarterly update will fire and B) not getting any factual response from Victron on the status of the VRM DESS scheduler backend. And I know there are all kind things happening behind the scenes with respect to VRM DESS Scheduler that are not communicated to me/us customers, so it is not a far stretch to think these things are related. Therefore I find it bordering on the absurd that I cannot get a straight answer whether there are unplanned and possibly even unnoticed performance issues with that massive bank of mqqtxyz.victronenergy.com brokers, or that I just happen to be subscribed to a flaky mqqt115.victronenergy.com, or that these missed updates are part of known but uncommunicated work/maintenance and the mqqt brokers or the systems behind the brokers that do the actual schedule calculations. Or that all of that is just one big coincidence and I am too stupid to figure out there is an actual problem on my end (and I tried to find any real hard but couldn’t).
Bottom line this is not even about technical issue, it is ALL about Victron being non-communicative for unknown reasons, could be accidental, work overload, or deliberate even, I have no means of knowing. But I cannot fathom it to be so extremely difficult and time-consuming to go check some logs/dashboards on mqqt115.victronenergy.com that Victron cannot even shine at least a shiver of light on this ongoing issue.
Literally no one else has an issue. And we have installers with many clients on related groups. I cannot escalate something only you are reporting.
So what you are saying is that because nobody but one person running a system connected to that (one-out-of-many) mqqt115.victronenergy.com broker is noticing that there are issues to begin with. Which is quite normal because you’d have to be actively monitoring the schedule updates to notice, and me being likely the only person actively developing a DESS based NODE-RED application that makes me notice, you guys at Victron simply argue that it is not worthwhile to look at the logs, because there are not enough complaints about it.
You know, this is called the “squeak management system” and known to be one of the most devastating ways of dealing with highly complicated technical developments and accompanying issues. You should be so happy I am giving you the opportunity to have look to see if there are undetected issues with your backbone systems, instead of painting me as a lone wolf crying wolf. Unbelievable.
And just to prevent you and others thinking I am just making all the above up out of thin air, here a logscreen copy of what I am working on (and indirectly makes me the only person in a position to notice and to be so kind to provide Victron with a friendly warning signal that there might be something persistently wrong at their cloud backbone.
I am not asking you to drop everything you are doing or send me your firstborn am I, I am only asking you to look at the logs. For crying out loud.
I will leave this subflow running for the weekend, and report back the log results. Anybody interested in monitoring their own system for this issue can simply import the flow. it does nothing but logging the active windowslot with timestamps. Normally you would only see the windowslot toggle between 0 and 1, when ‘windowslot > 1’ your system has missed a quarterly schedule update:
[
{
"id": "20dca72999465eb2",
"type": "tab",
"label": "SLOTLOG",
"disabled": false,
"info": "",
"env": []
},
{
"id": "308a3e3bf21503f5",
"type": "group",
"z": "20dca72999465eb2",
"name": "SLOTLOG",
"style": {
"label": true
},
"nodes": [
"4eda5dc95fb33308",
"e27ce897075b368c"
],
"x": 54,
"y": 39,
"w": 592,
"h": 82
},
{
"id": "4eda5dc95fb33308",
"type": "victron-input-custom",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"service": "com.victronenergy.system/0",
"path": "/DynamicEss/WindowSlot",
"serviceObj": {
"service": "com.victronenergy.system/0",
"name": "com.victronenergy.system (0)"
},
"pathObj": {
"path": "/DynamicEss/WindowSlot",
"name": "/DynamicEss/WindowSlot",
"type": "number",
"value": 1
},
"name": "",
"onlyChanges": true,
"roundValues": "no",
"rateLimit": "0",
"outputs": 1,
"conditionalMode": false,
"outputTrue": "",
"outputFalse": "",
"debounce": "",
"x": 290,
"y": 80,
"wires": [
[
"e27ce897075b368c"
]
]
},
{
"id": "e27ce897075b368c",
"type": "function",
"z": "20dca72999465eb2",
"g": "308a3e3bf21503f5",
"name": "slotlog",
"func": "const ts = Date.now();\nconst logkey = (\"slotlog.\" + ts);\nconst logobject = { timestamp: ts, windowslot: msg.payload }; \nglobal.set ( logkey, logobject );\nreturn null;",
"outputs": 1,
"timeout": 0,
"noerr": 0,
"initialize": "",
"finalize": "",
"libs": [],
"x": 570,
"y": 80,
"wires": [
[]
]
},
{
"id": "14c26b05e56668a1",
"type": "global-config",
"env": [],
"modules": {
"@victronenergy/node-red-contrib-victron": "1.6.60"
}
}
]
const ts = Date.now();
const logkey = (“slotlog.” + ts);
const logobject = { timestamp: ts, windowslot: msg.payload };
global.set ( logkey, logobject );
return null;
Well, as it turns out. we didn’t have to wait all weekend, 4 quarters missed already. Do I need to make my point more clear than this, really? (Or are you just going to silence/block me as you threatened me with for ‘getting clever’, in private DM already)
{"1773432988389":{"timestamp":1773432988389,"windowslot":2},"1773433801359":{"timestamp":1773433801359,"windowslot":3},"1773434701580":{"timestamp":1773434701580,"windowslot":4},"1773435602051":{"timestamp":1773435602051,"windowslot":5},"1773435717214":{"timestamp":1773435717214,"windowslot":0},"1773436502914":{"timestamp":1773436502914,"windowslot":1},"1773436618053":{"timestamp":1773436618053,"windowslot":0},"1773437403824":{"timestamp":1773437403824,"windowslot":1},"1773437518930":{"timestamp":1773437518930,"windowslot":0}}
