Grafana and VenusOS Large - Experimental project

Question

question

bathnm asked Oct 31, '23

Grafana and VenusOS Large - Experimental project

I have been working with Victron and we have built an experimental Venus OS large image that includes Grafana, InfluxDB and a loader to capture data and feed it into Influx. This is very experimental at this stage and anyone willing to test and feedback would be welcome.

There is a very basic dashboard and anyone wanting to supply some dashboard for including please share what you have and your ideas. No promises, but as a community we can make this as extensive as we see fit.

Venus OS Large Grafana Image is available here

This is based on:

Venus OS v3.20~12
Grafana 9.4.3
InfluxDB 1.8.10

Before starting the Grafana stack though the settings / Venus OS Large features / Grafana Stack option log in with ssh and setup the storage location for influxdb.

Influx DB Setup

Log into VenusOS via ssh
Insert an SDCard or USB storage medium into the system
Locate the medium’s mount point in /run/media
mkdir /data/home/influxdb
ln -s /run/media/DEVICE /data/home/influxdb/storage

Basic Grafana dashboard is available at venus.local:3030 with username of admin and password of admin

The data loader which loads data into the InfluxDB is set to a retention period of 30 days. This on my system generates a DB that is roughly 250Mb. It would be good to get feedback from others on their DB size after a month of running.

NOTE: At this stage the image will not run on a CerboGX, it is limited to RPI only.

Venus OS

1698763258256.png (115.2 KiB)

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mvader (Victron Energy) ♦♦ commented · Oct 31, 2023 at 06:32 PM

Cool @Bathnm !! I hope you get some feedback and enthusiasm here!

2 ·

Modifications Space

Node Red Space

Answer 1 · 2023-10-31T21:04:15Z

Guy Stewart (Victron Community Manager) answered · Oct 31, 2023 at 09:04 PM

Hi @Bathnm,

Very cool project, I’m going to load it up and give it a go.

Any suggestions for the suitable size of micro SD card, and an estimate for how many days per MB of data?

Also can we please get a test image for the new Ekrano?

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

bathnm commented · Nov 02, 2023 at 11:48 AM

@Guy Stewart (Victron Community Manager) Great question. I have updated the main post. The DB is configured with a 30 day retention period and it creates a DB size of roughly 250Mb. In later versions we might open up the influxdb loader to be configurable in which case the retention period and other devices could be configured to collect data from.

0 ·

Answer 2 · 2023-10-31T21:25:07Z

jwfrary answered · Oct 31, 2023 at 09:25 PM

I'm going to have to give this a go as well.

Influx data from signal k should hook in too.

Im guessing updates will be problematic!?

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

bathnm commented · Nov 02, 2023 at 02:40 PM

Depends what you mean by updates being problematic. If you are referring to keeping up with the base OS releases, I shall do my best to keep this image updated.

1 ·

Answer 3 · 2023-11-03T06:41:54Z

juf answered · Nov 03, 2023 at 06:41 AM

Great project

- it would be great if the user and pw for grafana written down ;-)

- in my case I can’t see my battery

- at Venus os I can’t activate the storage permanent

regards

6

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

bathnm commented · Nov 05, 2023 at 07:46 PM

@Juf I have just updated the main text and added the login details, they are admin and admin. Have you added an SDCard into the Cerbo, logged in with ssh and executed the commends to set influxdb to store on the SDCard, if not then that is likely your challenge.

1 ·

juf bathnm commented · Nov 05, 2023 at 08:07 PM

Thx :-) and sure I have add a usb thumb drive and see the links. But will give them a other try

regards

0 ·

juf bathnm commented · Nov 05, 2023 at 08:37 PM

no nothing changed.

Here you see my battery

0 ·

image.jpg (4.0 MiB)

image.jpg (4.5 MiB)

bathnm juf commented · Nov 06, 2023 at 05:07 PM

Can you check what is running, aka is venus-influx-loader and influxdb running?

Also what size is the DB? Issue du -s -k /data/home/influxdb/storage/data/*

Last thing to check is to log into influxdb via the command line and check if there is any data loaded.

0 ·

juf bathnm commented · Nov 06, 2023 at 06:09 PM

I think influx will be running.

but for that „Last thing to check is to log into influxdb via the command line and check if there is any data loaded.“ need help pls.

0 ·

image.jpg (6.6 MiB)

bathnm juf commented · Nov 14, 2023 at 09:22 PM

The client is just influx. Lets start with checking that the database has been created using the show database command. You should see one named venus

0 ·

1699997079932.png (132.6 KiB)

Answer 4 · 2023-11-06T11:09:27Z

craig-westlake answered · Nov 06, 2023 at 11:09 AM

@Bathnm - Great project and thankyou for your efforts. I really want to have a look at this.

I am having a little trouble installing the venus-swu-large-einstein-20231027034329-v3.20~12-Grafana-R1.swu image on a Cerbo GX. I have downloaded to the root of a USB and can see it in the install firmware from SD/USB screen. When I install it seems to work for around 60 seconds and then gives a message that there was a problem installing.

Any thoughts from anyone who has installed on a Cerbo.

5

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

jeroen ♦ commented · Nov 06, 2023 at 01:36 PM

As is, it is simply too large to fit for a Cerbo.

1 ·

greg-lions jeroen ♦ commented · Nov 23, 2023 at 06:29 AM

Awesome effort! Ive just bought a Cerbo GX with the intention of replacing my RPi4 running SignalK/Influxdb/Grafana... and was expecting to retain the Rpi4 to just run Influx/Grafana. so this is fantastic.

Could your install routine have a requirement for an external USB storage device to be connected and the influx binaries, data and Grafana all be installed on the USB storage?

1 ·

craig-westlake jeroen ♦ commented · Nov 06, 2023 at 03:28 PM

Thankyou

0 ·

bathnm commented · Nov 06, 2023 at 05:10 PM

It would seem we have maxed out the available space for the image on the CerboGX. We have some work to do to see if we can bring the size of the image down. It does work on the RPI's

1 ·

jwfrary bathnm commented · Nov 15, 2023 at 09:54 PM

Is the Ekrano any different? I processor is a bit quicker I belive but is the internal storage and ram also?

0 ·

Answer 5 · 2023-11-14T09:26:35Z

ningaloo answered · Nov 14, 2023 at 09:26 AM

Just what I need! I will try it on a RPi4 with SignalK. I see there are size issues on the Cerbo. I'm wondering if loading all this will stress a single RPi? My original plan was to run Venus large with SignalK on one (RPi3) and InfluxDb and Grafana on the Pi4.

3

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

ningaloo commented · Nov 16, 2023 at 08:37 AM

I have not been able to get my Grafana to connect to InfluxDb.

I get the following error when I click the Test button on the data source:
connect: connection refused error performing influxQL query

I am unable to access venus-ip-address:8086

1 ·

ningaloo commented · Nov 16, 2023 at 08:45 AM

Looks like InfluxDb isn't running...

Any idea how this should be started?

1 ·

bathnm commented · Nov 14, 2023 at 08:52 PM

I have been running an RPI4 4Gb all summer on my yacht with no issues. The advantage of the RPI is the large SDCard sizes and therefore ability to have a much large root partition that the Cerbo.

0 ·

Answer 6 · 2023-12-31T08:30:51Z

ningaloo answered · Dec 31, 2023 at 08:30 AM

I am wondering if this project is dead?
As previously advised the basic install did not start Influx on my RPi4.

For now I will continue with the standard (large) install on my RPi3 and use a second RPi4 to run InfluxDb and Grafana, both products that are new to me.
My next dilemma is whether to use OpenPlotter (which has some potentially useful features) which includes InfluxDb v2, or do a native RPi install and add InfluxDb v1.8.
I understand that InfluxDb v2 only supports the Flux query language which seems to have been a short-lived decision by the Influx team as they have abandoned it in InfluxDb v3 and reverted to InfluxQL which is going to be much easier for me (as an SQL guru) to work with.
I appreciate that this is not directly related to Victron so will pursue this on an appropriate FB forum.

3

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

bathnm commented · Dec 31, 2023 at 09:08 AM

I have a build that fixes an issue with influx and the Venus to influx data loader. I can make it available if required. There did not seem to be a lot of interest in the project and so it has taken a back burner.

1 ·

yagu bathnm commented · Jan 03 at 01:59 PM

I am absolutely interested in your project! Would you mind releasing your current version? I am specifically interested in the raspberry 3 build.

0 ·

yagu bathnm commented · Jan 03 at 05:14 PM

Is it possible that your latest build on dropbox for raspberry 2 is missing grafana options?

0 ·

Answer 7 · 2024-03-31T18:26:35Z

kevgermany answered · Mar 31 at 06:26 PM

@Bathnm

Just wondering if this is still alive. I was looking at setting up a dashboard in Node-RED and found that their dashboards are based on Angular 1 which is no longer supported. This seems to be a good option, but I'll have to find a way to make it work while my Cerbo is still in the system.

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 8 · 2024-04-02T07:40:16Z

bathnm answered · Apr 02 at 07:40 AM

@kevgermany Yes it did die. There was not much interest. Also the image was too large for the available flash on a CerboGX so only really ran on a PI as at present there are two flavours of the GUI being stored. Until the GUIv2 project completes and the GUIv1 is removed from the build it is unclear if that would free up enough to fit on a CerboGX flash. There are also some issues as InfluxDB is at release 1 and if the DB gets too large there are index file issues, to resolve this requires InfluxDB v2 or higher and that needs a 64 bit build, which is a massive undertaking.

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

question