question

Bill avatar image
Bill asked

Raspberry Pi Venus OS local file system question

I noticed that when I burnt the Venus OS v2.30 (and other older versions) it makes 4 partitions on the micro sd card of a bit less than 1.8 GB. It all fits on a 2 GB sd-card with room to spare. The main partition is the smallest and only seems to have a few hundred megs left over after the OS files. I literally don't know how much data the GX writes locally. Or if it even stores data locally. I have it connected to the VRM portal as of last night.

My question is:

Does the GX write much out locally, or is the existing 300 MB more than needed and I shouldn't worry about finding a way to use the other empty partitions the image burn created?

Thanks, Bill



Venus OSRaspberry PiVEConfigure 3
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

4 Answers
ben avatar image
ben answered ·

Speaking from casual observation, I don’t think it persists much. When it’s configured to send data to VRM for the cloud reporting feature, it will log those data locally until it has a network connection and can upload again, but I think that log doesn’t really get too big. And a couple historical software images are cached. Otherwise, I can’t imagine it has anything else of substance to write.

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

alexheylin avatar image alexheylin commented ·

Ben - have you actually seen VenusPi store data when off line and upload later? In another thread two of us are discussing that we expected this but it doesn't seem to work.

Thanks

0 Likes 0 ·
ben avatar image ben ♦ alexheylin commented ·

Funny you mention that.

I personally don't use the VRM logging feature any more, since I am logging to grafana instead on my setup.

But I was working on another system recently, and I noticed that its cached VRM data uploaded, but never showed up in the viewer webapp.

I know it cached, because I can see the record count accumulate in the Venus UI. And I know it uploaded, because I watched it count down quickly when we got reconnected to the internet. But the data never showed up.

I didn't have time to fool with it, and I concluded it might be something wrong with that particular setup. But maybe there is a bigger issue...

0 Likes 0 ·
Bill avatar image
Bill answered ·

Thanks! I wonder why the heck they make the file 1.8 GB then. It appears it could be a LOT smaller and save all the bandwidth when we download it.

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

ben avatar image ben ♦ commented ·

What file do you see that's 1.8GB? Do you mean the disk partitions? (The typical Venus update is something like 100MB.)

I think there is a script you can run to shrink the partitions down after installation, if you want to use the space elsewhere.

0 Likes 0 ·
Bill avatar image Bill ben ♦ commented ·

The image file you download and burn to the microsd card is 1.8 GB. And when you check the space used on any microsd card you burn it to, its only used about 1.8 GB, they rest is unused space. After seeing that I wiped the 32 GB card and burned the imagine to a 2 GB card that i had pulled from a phone long ago. Works just fine. -Bill


PS. I don't need the space back, it just seemed odd it would make 4 partitions consuming 1.8 gb if all it needs is the first 40 MB partion. Maybe they were future proofing it... :)

0 Likes 0 ·
mvader (Victron Energy) avatar image
mvader (Victron Energy) answered ·

Hi all, the four partitions are:

- boot partition

- rootfs1 & rootfs2 (search google or our wikis about “dual rootfs” to learn more)

- data partition

So there are no ‘other empty partitions’.


And indeed the rootfs partitions are larger than strictly necessary; to make then future proof. Also you wont see their real size when looking at the rootfs file system; see our resize2fs.sh for details about that.

The file you download from our website is 95MB. Then unzipping it indeed makes it large; its the image that you write to an sdcard.

If you use Etcher for writing the image; then you don’t even have to unzip and never see its real size.

Improvements are welcome: If anybody has a better idea on how its done; very welcome and please make a PR (or first send a proposal before implementing it). :-). Just ideas are not very helpful; there is not much spare time.


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image
Kevin Windrem answered ·

The boot partition is about 25 MB and contains the little bits responsible for loading the active root partition. Little is stored there and changes very little.

There are two root partitions: the active one and the "backup" one. These flip-flop with each software upgrade. Initially one will be empty. Current usage is about 350 MB each.

The 4th partition is /data where all persistent data is stored: settings, logs, VRM data that hasn't been transferred to the server, etc. The VRM database and the logs are the only things likely to vary in size. With no VRM data, there's about 300 MB stored in this partition.

According to Victron, VRM data amounts to around 20 MB/day. That's about 50 days/1GB, or 8GB to store 1 year. Once the data is transferred to VRM, it is removed from local storage, so you only need storage for the period of time your system is off-line.

The initial image ends up with the root and data partitions of equal size, each being 1/3 the size of the SD card.

I created the initial Venus SD card using Raspberry Pi Imager from the .gz image. Imaging SD cards for backup purposes required some research. I first tried dd from the command line but never got an SD card copy to run.

After looking at several SD cloning apps for Mac OS and unix, I settled on the free "ApplePiBaker" (Mac OS). Not sure about Windows tools.

Cloning tools often complain that the 16 GB image you just made won't fit on a 16 GB card. That's because each card varies in size slightly. Some form of partition size management may be necessary. The problem is that most tools fail to handle the 4-partition Venus image properly when using automatic shrink/expand. The "SD Clone" tool failed to resize at all or damaged the image. ApplePiBaker produced a copy with only two partitions, discarding the other two. I settled on Gparted running on Ubuntu for partition management. There are derivatives that supposedly run on Mac OS but I'd read warnings about flaky EXT drivers for Mac OS. (The derivative I tried wouldn't work with Paragon's EXT driver.)

Hope this helps.

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

larsea-dk avatar image larsea-dk commented ·
So to understand it correctly, there is no tool or a how-to guide for making backup SD cards in case of defects sd cards?

Where is nodered and VRM (custom)settings stored?

0 Likes 0 ·

Related Resources

Additional resources still need to be added for this topic

Raspberry Pi running Victron’s Venus firmware - Blog Post

Venus OS Large image: Signal K and Node-RED - Install

raspberrypi install venus image


Victron Venus OS Open Source intro page

Venus OS GitHub (please do not post to this)

Additional resources still need to be added for this topic