question

abegweit avatar image
abegweit asked

Data Retention - Venus OS on Raspberry PI

Hi, I have a Raspberry PI 3B with Venus OS v2.71 without any modifications. The PI is connected by Cat5 to the router and logs data in 1min intervals.

Everything is working and I am seeing data on the VRM portal.

However, if my router loses internet access for a period of time, and then regains access, The Raspberry PI re-establishes a connection to VRM but I do not see data in VRM for the period time that there was no internet access.

Is there anyway to configure the Raspberry PI to retain data when it loses connection to the VRM portal and have all that data upload when the VRM connection is re-established?

Thanks

VRMRaspberry Pi
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

4 Answers
Kevin Windrem avatar image
Kevin Windrem answered ·

It should already be doing this. It uses space on /data partition to store VRM data when off line if there is no external (USB or SD) storage. Usually there is plenty of space on internal storage. You can check in the VRM portal to see where the storage is and how much there is. It also shows how many records have been stored if the GX device is off-line.

Uploading the stored records to VRM does take a while. You might just need to be patient.

4 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

abegweit avatar image abegweit commented ·
Thank you @Kevin Windrem. VRM and the Raspberry PI implementation are new to me. I've logged into VRM and cannot see where to check on the storage. Can you help point me in the right direction?
0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem abegweit commented ·
The storage indiction is on the Venus display:

Menu (Device List) / Settings / VRM online Portal


At the top of that menu, Logging enabled should be: Enabled

The storage information is at the bottom of the menu.

0 Likes 0 ·
abegweit avatar image abegweit Kevin Windrem commented ·

Thanks @Kevin Windrem that worked. I checked and I have 5G free disk space available.

So now I'm wondering if my data loss is due to a reboot of the Raspberry PI.

I have the option set to reboot if no contact for 1hr and I am missing data for 1hr and 3min.

I'm wondering if the Raspberry PI had a SW fault of some kind that blocked communication and then after the 1 hr delay timed out, it re-booted itself and in doing so fixed the fault?

I'm not sure there is anything else to do on this but if you can offer any suggestion that would be welcomed. It's great to know that the data is stored and I have lots of storage available.


screen-shot-2021-08-24-at-42318-pm.png


Thanks again

0 Likes 0 ·
screen-shot-2021-08-24-at-42318-pm.png (163.4 KiB)
Kevin Windrem avatar image Kevin Windrem abegweit commented ·
I know there have been issues with Wi-Fi disconnecting at random and not recovering, so your explanation is plausible. I seem to remember discussion here that this was fixed but don't recall which version.
0 Likes 0 ·
Warwick Bruce Chapman avatar image
Warwick Bruce Chapman answered ·

I experience this issue as well on data loggers we have built with Rapsberry Pi's. They do not log when the internet connection goes down.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mcarse avatar image
mcarse answered ·

I did a little testing this morning with my Raspberry Pi 3+ monitoring install. I am running v2.89.

I blocked the Pi's ability to connect out to the Internet, and connected to the VenusOS remote console on my local network. VenusOS did report that it could not contact VRM. I allowed the "Stored Records" count to climb to 20. Then rebooted. The records were retained after the reboot. I then unblocked the Pi to allow it to connect to the Internet. Within about a minute, all stored records had been uploaded to VRM.

Matt


2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

johanndo avatar image
johanndo answered ·

There is another issue that many forget.

The pi has no real time clock, if you reboot the pi without internet connection it cannot get date and time from ntp, so the energy is logged at 1970. Once connection is established, this data will be uploaded and disappear in the past. You end up with a gap in your logging, because of the wrong time stamps.

1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Warwick Bruce Chapman avatar image Warwick Bruce Chapman commented ·
Interesting. More testing required this side.
0 Likes 0 ·

Related Resources

Additional resources still need to be added for this topic

Raspberry Pi running Victron’s Venus firmware - Blog Post

Venus OS Large image: Signal K and Node-RED - Install

raspberrypi install venus image


Additional resources still need to be added for this topic