Exporting device values to LAN

Question

question

tgc asked Aug 6, '23

Exporting device values to LAN

I'd like to export local device values from CCGX to an integrated monitoring system on LAN.

Either by sh script or redirecting VRM logging.

For the former, what are the local data structures for the devices and the service calls to access them?

For the latter, where is the VRM url defines, and what type of calls are made by the logger?

Thanks !

PS no internet where the system is, so can't use a VRM-based solution, it has to be purely local. integrated monitoring system is very closed, too hard to poll the CCGX through modbus tcp, a push from CCGX much preferred.

Venus OS CCGX Color Control gx device monitoring lan ccgx

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Modifications Space

Node Red Space

Answer 1 · 2023-08-06T20:53:26Z

gerard-van-seventer answered · Aug 06, 2023 at 08:53 PM

I use MQTT for this.

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tgc commented · Aug 07, 2023 at 10:54 AM

Thanks. Sorry I'm not familiar with it. Is it push from CCGX?

Would you by any chance have an example?

0 ·

Answer 2 · 2023-08-07T14:58:53Z

matt1309 answered · Aug 07, 2023 at 02:58 PM

HI @Tgc

If by push you mean you want GX device to actively send data to a server directly. I think you'll need to use Node red running on GX device for this functionality. Or custom python scripts.

Node red would be as easy as choosing the input data note + the http node to send data to your external machine on lan.

However I agree with @Gerard van Seventer, given the rate of data sent from a system like victrons you're probably better off "listening" constantly rather than pushing. either via modbus TCP or MQTT (which is often more convenient nowadays in my opinion). Listening instead of pushing will be easier as there's already devices setup.

Of course depends on the data you're looking to capture but something like Load Watts is constantly updating so "listening" to MQTT/modbus is probably easier.

For a purely lan based setup that you dont have to write from scratch you may be interested in looking into the grafana docker setup victron has. It might not be perfect for your setup but can be tweaked and gives lan access to all data. Here's a link to it:

GitHub - victronenergy/venus-docker-grafana

Some useful links:

DBUS paths are the topic locations for MQTT topics of the victron system:

dbus · victronenergy/venus Wiki · GitHub

Some further MQTT detail in victron system:

GitHub - victronenergy/dbus-mqtt: Venus OS service mapping the D-Bus on Venus OS to MQTT

Modbus TCP register:

https://www.victronenergy.com/upload/documents/CCGX-Modbus-TCP-register-list-3.00.xlsx

7

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tgc commented · Aug 08, 2023 at 08:59 AM

Thanks @matt1309 . Very useful.

The reason I can't set up the listener on the LAN side is that the receiving system is not very open (...), and I can't include an MQTT client for example, or pretty much anything.

So you are right, the solution is to set up a listener on the GX machine that sends the info out to the LAN in a way that they can digest.

You mention node red or phython script. How about a sh script? Surely anyting that a phython script can do can be done in sh. Can you point me to any resource that tells you how to access the right info on the GX machine from simply sh commands?

In the meantime, I'll start delving in the dbus links you sent, see if I can make sense of it. Chinese to me at the moment since I'm lacking the basics, but we'll get there...

Thanks for the help.

0 ·

matt1309 tgc commented · Aug 08, 2023 at 09:22 AM

Which machine is "not open"? And by not open do you mean behind firewalls or as in you dont have access to install/edit programs etc? Is it the GX device or the lan machine that's "not open"?

Assuming it's the LAN machine:

If you can't install external software on the lan machine ie you cant install a mqtt client or modbusTCP client on the machine. Then a built in bash/sh commands I can think of would be to query a http server on the venus device using curl which is often pre-installed on linux machines. (maybe doable with wget also if curl isn't an option)

You could achieve this by using the http node in node red to create a webserver that could output the relevant data each time the lan machine runs the curl command. If that makes sense?

So you would create the http node red node on the GX device: (Here's a useful link):

Node-Red Web Server -HTTP-IN and HTTP Response Nodes (stevesnoderedguide.com)

You'd have all the victron data go into this node so the webserver would output the data you need in a specific format (likely JSON). essentially you could go to the url you define like

https://GX_DEVICE_IP:1881/test and you would see all the data.

Then in you shell script on the locked down machine you'd need something like

curl https://GX_DEVICE_IP:1881/test

Where test is the location you defined in the node. and GX_DEVICE_IP is the ip of your gx device.

And then the rest of the shell script would be extracting the data from the curl command.

If the lan machine has webserver running/php. Then this would be even easier to write in node red. Send the data from node red to the lan machines website with some php code to receive the data.

Another more hacky/less clean solution would be to create a shell script that remotely runs dbus command via ssh to get the data.

If you can share some more details on how the machines are "not open" I'm sure folk will be able to help give some more detailed options.

0 ·

tgc matt1309 commented · Aug 08, 2023 at 07:47 PM

@matt1309 the LAN machine is the not-so-open one, you are correct. It's a Vera home controller. Even a sh script would be hard to integrate. Now to be fair I haven't looked in a while and they may have MQTT-type pseudo devices ready now or even Victron pseudo devices :D (that would be perfect, I'll check), but otherwise it's not very easy. You can send data with wget from somewhere else, though.

I guess it will have to be on the GX machine something like:

While

{ SOC = dbus yadiyada

wget SOC to Vera

}

Three questions, then:

1- Anyone has ideas/tips on Vera integration?

2- Any reaction to the above loop on a GX machine from an update/safety/stability perspective?

3- It's the yadiyada part that puzzles me, I'm getting a bit lost on the Victron device/service/object hierarchy and some unfamilar terminology (VE.bus, VE.direct?). Slowly getting there through the dbus api doc wich is great but I guess as always an example is easier... anyone has a very simple example similar to the above?

Thanks again for the help, I am 200% closer to a solution that I was two days ago!

0 ·

matt1309 tgc commented · Aug 08, 2023 at 07:57 PM

Hi @Tgc

Had quick google of vera devices. This plugin looks promising

The MQTT Status section, you would just need to input the topic of the data you require.

https://github.com/dbochicchio/vera-VirtualDevices

A bit more explanation:

I think your confusion is coming from sending vs receiving.

If you send data to the Lan machine it needs to be ready to receive it. This is probably harder to do if you cant install programs.

What is probably easier is getting the lan machine to "ask" the GX device for data.

So wget https://GC_device:1881/endpoint running in a loop on lan machine

And setup http node in node red so it outputs SoC data at the url /endpoint.

However that plugin seems the easiest option.

0 ·

tgc matt1309 commented · Aug 08, 2023 at 08:56 PM

Thanks @matt1309 , worth looking into the plug-in.

Incidentally, is there a man page somewhere for the dbus command line on venus? I can only see some examples using dbus and dbus -y, but would love to understand it better to test this out. I still haven't even found what my device instances are, surely there is a dbus commend that lists them?

Can I use dbus -h without breaking anything?

0 ·

matt1309 tgc commented · Aug 08, 2023 at 09:00 PM

This wiki page has nearly everything

https://github.com/victronenergy/venus/wiki

The device ids you can get via gx device Web interface. You just select the device and you should see the Id in there.

I think dbus-spy is what you're looking for

1 ·

tgc matt1309 commented · Aug 08, 2023 at 10:26 PM

@matt1309 yes it has!

Thanks for the orientation, Matt, much much appreciated

0 ·

Answer 3 · 2023-08-09T08:06:42Z

In case anyone is interested, as a first step a quick bit of code to extract values from GX onto a LAN Pi (not Vera yet) and store them locally on the LAN. To be clear this runs on Pi, not on GX.

192.168.xxx to be replaced by the GX's local address, and SSH key to be generated on Pi and installed on GX.

#!/bin/bash

# loop started by cron at startup, to grab parameters from victron dbus and set them into enmonloop files

while true; do

MAINPOWERNEW=`ssh -i /home/pi/.ssh/GX_rsa root@192.168.xxx 'dbus -y com.victronenergy.system /Ac/Consumption/L1/Power GetValue'`

#echo $MAINPOWERNEW

BATPOWERNEW=`ssh -i /home/pi/.ssh/GX_rsa root@192.168.xxx 'dbus -y com.victronenergy.system /Dc/Battery/Power GetValue'`

#echo $BATPOWERNEW

BATCURRENTNEW=`ssh -i /home/pi/.ssh/GX_rsa root@192.168.xxx 'dbus -y com.victronenergy.system /Dc/Battery/Current GetValue'`

#echo $BATCURRENTNEW

BATVOLTNEW=`ssh -i /home/pi/.ssh/GX_rsa root@192.168.xxx 'dbus -y com.victronenergy.system /Dc/Battery/Voltage GetValue'`

#echo $BATVOLTNEW

BATSOCNEW=`ssh -i /home/pi/.ssh/GX_rsa root@192.168.xxx 'dbus -y com.victronenergy.system /Dc/Battery/Soc GetValue'`

#echo $BATSOCNEW

sudo echo `date +"%d-%m-%Y %H:%M:%S "; echo $BATPOWERNEW ` >> /srv/voltage/batpowernewhistory;

sudo echo `date +"%d-%m-%Y %H:%M:%S "; echo $BATCURRENTNEW ` >> /srv/voltage/batcurrentnewhistory;

sudo echo `date +"%d-%m-%Y %H:%M:%S "; echo $BATVOLTNEW ` >> /srv/voltage/batvoltnewhistory;

sudo echo `date +"%d-%m-%Y %H:%M:%S "; echo $BATSOCNEW ` >> /srv/voltage/batsocnewhistory;

sudo echo `date +"%d-%m-%Y %H:%M:%S "; echo $MAINPOWERNEW ` >> /srv/voltage/mainpowernewhistory;

sleep 10

done

question