I am trying to debug a lost-telemetry-when-offline issue, and I am wondering what’s the expected free “disk space” available to VRM data as seen on “Settings” → “VRM” menu.
I have a bunch of installations, both Cerbos and “generic gx” devices, aka Raspberry Pis running Venus OS. All “generic gx” devices are set up on a 32GB SD card. All devices are running v3.66 of the Venus OS software.
Here’s a table of available “disk space”, as seen through the aforementioned menu for some of my installations:
DEVICE
OS type
free “disk space”
Cerbo GX (some NodeRed flows)
LARGE
4774 MB
Pi
normal
10248 MB
Pi
normal
5238 MB
Pi (light NodeRed flows)
LARGE
10310 MB
Pi (no NodeRed flows)
LARGE
5238 MB
Numbers don’t make any sense, plus seems like the “disk space” allocated for VRM records (as I presume is seen through the aforementioned menu) is kind of an arbitrary space?
Also, seems like there shouldn’t be more than 30MB daily for offline records, which would calculate to 160 days of offline data for even the first device that seems (?) to have the less available “disk space”. Why am I also seeing a “maximum of 2-3 days of records” for stock Cerbo GX devices with no extra SD card installed?
Lastly, since the aforementioned menu displays some “arbitrary, offline records” free “disk space”, is there an official way to check for ACTUAL free “disk space” on these units?
On the official Venus GX spec sheet, it reads that “When there is no internet connection available, the Venus GX will store the data internally, up to 48 hours.”.
However, Cerbo GX documentation changes the phrase to “several days”, however doesn’t mention something more specific.
So, what gives? Is there some arbitrary limitation of offline VRM data when not using external storage with a retention period of 48hours? If so, why?
On a specific really remote installation running on a rPi, remote console reports “5123.43M bytes” free space, system configuration is minimal (just a BlueSolar controller up there) and VRM log interval is set to 5min. However, I only get up to 2 days of VRM data back, whenever there is a network outage on the mountain top. This is bonkers. And there’s no additional SD card slot on the rPi.
It should be an easy test to intentionally disconnect the GX and take df outputs before and after an hour, two hours etc.
It should give a pretty accurate view of what is actually being consumed.
I am not aware of any restrictions/limitations that VRM might have that could prevent the data getting back in sync, there could feasibly be some.
I have seen that it can take some time after a relatively short disconnect to upload data.
Just to confirm.
There is no limitation to data retention, only the disk capacity of the GX and your ability to upload it.
VRM retains different stats for differing amounts of time, 6 months for some, so if your stored data exceeds this, only then will it be ignored.
It may take hours or days for data to appear as processing of live data is prioritised and if the system is busy this can take quite some time.
The installation I’m having connectivity issues with, is at the top of a mountain, in the middle of nowhere. That’s why the Victron ecosystem was selected, because I thought the intermittent connectivity issues wouldn’t lead to loss of data, but that doesn’t seem to be the case.
I know VRM retains various data for differing amounts of time, but here I’m talking about the data that is being retained on the GX device while offline. I know this data might take some time to upload, however every time there’s a network outage, I find that only 48hours of data has been uploaded. Hence the question.
Latest network outage event was almost a month, all daily yields are available through Victron Connect (by accessing the charge controller), but nothing on the VRM itself.
This specific system (the one linked above that’s been having network issues) is running on a rPi, no extra SD card slot unfortunately. There’s plenty of free space on the system SD card, Venus OS should be able to utilise that.
Maybe it does, when I manage to get up there I’ll give it a shot. This installation made me have a look at a few other installations I’ve set up, that’s why I’ve also posted available “disk space” for a few of them. So now I’m trying to understand how Venus OS is managing the space in general, except from solely trying to find a solution for this particular one.
It makes sense there is a small reserve though so we can fill the rest with other with stuff without jeopardising the logs.
Would be neat to be able to edit its size
Without data that shows what the log region is doing, it’s difficult to diagnose. You could probably achieve a log with nodered running an exec command for df.
I can only confirm that there is no VRM limitation and the restriction should only be the capacity available for logging.
guystewart
(Guy Stewart (Victron Community Manager))
13
The GX (specifically Venus) will not use the free space to store VRM data.
If you want to store longer than “several days”*** that are specifically allocated for this, then you need to use external USB or MicroSD card storage.
The built in memory is only intended to be used as a small buffer for a normally internet connected system.
The free space in the other partitions is reserved for other operating system functions, swaps, space for firmware files, etc.
*** - the exact amount of storage in time depends on the specific system, how many components, and how much data they are generating. ~48 hours is a good rule of thumb.
According to Guy’s reply, there indeed seems to be an arbitrary limitation of how Venus stores VRM data while offline, although undocumented. However afaik, this limitation seems to be exactly 48 hours on my Venus OS systems, regardless of the size of the system and the connected components, so it looks like it is a hard-coded thing.
Guy’s reply (although not perfectly clear) seems to have put a nail on this coffin, thanx for your attention.
