How to implement a script on Venus OS

I guess this question has already been asked a couple of times but I can not get hold of the answer.

Following the issue I had with my LTE/4G router (https://community.victronenergy.com/questions/64911/issue-with-4glte-provider-assigned-non-public-ip-a.html) I have found a script to check and enable the data. I have tweaked with the test to the internet, as it is a part of the problem. Probably not most nice script, but is works.

This script works fine in terminal mode on a connected Mac OS. As I'm a newby in scripting, I want to implement this script on the Venus OS. I was able to connect to the root and run it from the terminal. Now I want to have it run in an unattend mode (when I stop the terminal, my script stops also).

Also I want it also get back running after a reboot or a new release.

Maybe somebody can point me to the right documentation, or explain it.

The cherry on the cake will be: write the messages to a logging and raise an alarm in the VRM.

Thanks in advance because the mobile operator (BASE) put the fault on the router and are not willing to help, and Netgear does not care as it is minor issue.....

Best regards,

Philippe

#!/usr/bin/env python3

"""
This scripts monitors the MR1100 to re-enable Data automatically.
This is useful with operators that use PPP and which systematically disconnect
users after a certain period of time.

Works with firmware 12.06.08.00
"""

import requests
import json
import time
import datetime
import sys
"""

if len(sys.argv) < 2:
    print("Usage: {} <IP> <ADMIN_PASSWORD>".format(sys.argv[0]))
    sys.exit(1) 

ip = sys.argv[1]
pwd = sys.argv[2]
"""

ip = "192.168.1.1"
pwd = " you will get this one :) "

timeout = 2

url_base = "http://{}".format(ip)
url_session = "{}/sess_cd_tmp".format(url_base)
url_js_token = "{}/api/model.json".format(url_base)
url_config = "{}/Forms/config".format(url_base)
url_json = "{}/api/model.json".format(url_base)
url_json_success = "{}/success.json".format(url_base)
url_success = "{}/index.html".format(url_base)

def login(r):
    try:
        api = r.get(url_js_token, timeout=timeout).text
    except:
        print("Failed to connect to router")
        return

    try:
        api = json.loads(api)
        global token
        token = api['session']['secToken']
    except:
        print("Failed to fetch authentification token")
        return

    data = {
        'token': token,
        'err_redirect': '/index.html?loginfailed',
        'ok_redirect': '/index.html',
        'session.password': pwd,
    }

    try:
        redirect_url = r.post(url_config, data=data, timeout=timeout).url
        if redirect_url == url_success:
            return True
        else:
            print("Failed to authenticate, incorrect password?")
            return
    except:
        print("Router failed to respond to authentication challenge")
        return


def get_status(r):
    try:
        json_model = json.loads(r.get(url_json, timeout=10).text)
        status = json_model['wwan']['connection']
        return status

    except:
        print("Cannot retrieve connection status")


def reconnect(r):
    disconnect = {
        'token': token,
        'err_redirect': '/error.json',
        'ok_redirect': '/success.json',
        'wwan.autoconnect': 'Never',
    }
    connect = {
        'token': token,
        'err_redirect': '/error.json',
        'ok_redirect': '/success.json',
        'wwan.autoconnect': 'HomeNetwork',
    }

    try:
        push = r.post(url_config, data=disconnect, timeout=timeout)
        if push.url != url_json_success:
            print("Failed to disconnect data")
    except:
        print("Invalid answer from router")
        return

    try:
        push = r.post(url_config, data=connect, timeout=timeout)
        if push.url != url_json_success:
            print("Failed to reconnect data")
            return
    except:
        print("Invalid answer from router")

def internet():
    url = "http://www.google.com"
    timeout = 5

    try:
        request = requests.get (url, timeout=timeout)
        return True
    except:
        print ("No internet connection")
        return False

def main(dry_run=None):
    data_status = None

    r = requests.Session()
    login_status = login(r)

    if login_status != True:
        return
    else: 
        data_status = get_status(r)
        if dry_run is True:
            return True
    if data_status != "Connected":
        print("Data appears to be disconnected, reconnecting..")

        reconnect(r)

        nb_tries = 0
        while data_status != "Connected" or nb_tries < 3:
            data_status = get_status(r)
            time.sleep(1)
            nb_tries+=1

        if data_status == "Connected":
            timestamp = datetime.datetime.now()
            timestamp = timestamp.strftime("%Y-%m-%d %H:%M:%S")
            print("Reconnection successful {}".format(timestamp))
        else:
            print("Data is still offline, this should not happen")
    else:
 
        
        if internet () != True:
            print("Data appears to be disconnected, reconnecting..")

            reconnect(r)

            nb_tries = 0
            while data_status != "Connected" or nb_tries < 3:
                data_status = get_status(r)
                time.sleep(1)
                nb_tries+=1

            if data_status == "Connected":
                timestamp = datetime.datetime.now()
                timestamp = timestamp.strftime("%Y-%m-%d %H:%M:%S")
                print("Reconnection successful {}".format(timestamp))
            else:
                print("Data is still offline, this should not happen")
        else:
            return
        
if __name__ == '__main__':
    try:
        dry_run = main(dry_run=True)
        if dry_run is None:
            sys.exit(1)
        else:
            print("Connection to router successful. Monitoring disconnections")

        while True:
            main()
            time.sleep(2)

    except KeyboardInterrupt:
        sys.exit(0)


philippe asked
Kevin Windrem answered ·

1 Answer

Grafana refresh rate <10 seconds?

I followed the Victron Grafana w/ Docker instructions, and set the refresh rate in Grafana to 1sec. Even though Grafana seems to be querying every second, the values only update every 10 seconds or so. I want to have a live display like in the Venus remote console. Has anyone gotten this working at a 1 second refresh rate? Any suggestions for me to try? I tried looking in the chrome dev console and it seems like queries are not taking more than 50ms. Is there some other influx or MQTT limit setting I'm unaware of?

vulture asked
marekp commented ·

4 Answers

mqtt Energy Meter for 600/5 CT

Can any one advise if there is a Energy Meter available that accepts the 5A secondary for larger CT ratios 600/5 that could publish real time kw out every 1 or 2 seconds in the form of an mqtt publish.

I would like to read this value into a Venus running Node Red which is 100m away from the Energy Meter.


markus-001 asked

0 Answers

Question regarding MQTT topics - How do I get Total Daily Consumption?

Hi,

I would like to get the "Total Consumption" value from my Victron Setup.

Today it is approximately 20 kWh, but can find this value nowhere in the huge list of MQTT topics.

I found vebus/261/Energy/AcIn1ToAcOut and it has a value of 47 ... increasing a bit every minutes, but I have no idea what it could mean.

Could you please help me ?

BR

Gawan

gawan asked
jbt77 answered ·

1 Answer

Reset log daily in Venus GX

Hello,

i use MQTT from Venus GX to show a lot of datas in Grafana but i have an issue for one of them.

The topic N/serialNo/vebus/#

and for example this one :

N/serialNo/vebus/261/Energy/OutToInverter

There is no reset each day on this value, so it's difficult to get the total consumption of the day.

I found that the restart of the venus reset this values.

But is there a better way maybe in command line to reset this logs daily please ?

Thanks for your advise.

jbt77 asked

0 Answers

Temp sensors for Cerbo GX

I am looking to add temp sensors in my refrigerator & possibly a sensor reading supply air from my AC. What types of sensors do I need?

dfraizer asked
Daniël Boekel (Victron Energy Staff) commented ·

3 Answers

article
Rob Duthie commented Modifications

73 Comments

7Likes

Personal weather station information

I have an Ambient Weather WS-2000 Personal Weather Station co-located with my solar installation. This station has Wind Velocity and Direction, Temperature and Solar Radiation information plus a fairly accurate forecast for the next day available. Is there a way to integrate this data with my installation?

I have noticed that the temperature read-out and weather in the VRM Portal differs quite a bit from the reading I am measuring locally.

Regards,

Oupa Lintvelt.


oupa asked

0 Answers

Multiplus 24/5000/120 won’t switch to inverter mode

Hi there I am wondering anyone could help me, I stupidly purchased a multiplus off eBay, I have the chip version 19 the multiplus has had the transformer replaced and all relays on the boards also, the multiplus output ac the highest it’s got to is 90v ac, I have tested it with ac input also with a load connected to ac1 but it won’t even transfer through, I have connected the laptop and updated with the mk3 usb, I have tried every firmware that is available for the first four digits of the firmware chip number, the people who sold me it are ignoring me and have done ever since I stated it was faulty, I have contacted victron agents in scotland but to no avail as I taken it upon myself to have the components replaced, I know it’s a long shot but just to ask if anyone has had similar problems with this model, the input current sensor I suspect is faulty as it’s reading on ve config as 167v ac input, I have sent the board to have the current input sensors replaced but I am unable to source them, the last time I assembled it all again it given 90vac at ac1 out, but still seen the input voltage as 167ac. Many many thanks in advance,

Tried various cat 5/6 cables, different laptops, updated with ve config and also victron connect, running 8xYuasa EFB batteries.

Thanks david

ddickov asked

0 Answers

Venus PI3 offline cache of data?

I have a Pi3 installed and it collects data great. when I go offline camping I was thinking that it would store this in a DB to upload then the internet returned but it does not. am I missing something? should this work? is there any way to get it to work? can I l log to a file and upload?


p0lar asked

0 Answers

idea
Keith Arnold commented Modifications
5

1 Feedback

0 Comments

VE.Direct diy cable

Hi.

Can I use JST-PH plugs to make VE.Direct to VE.Direct cable? Just simply connect ground to ground, 5v to 5v and Tx to Rx (cross cable)??? Will it work like that???

[image]

This is required to connect BMV712 to VenusGX

Kind Regards

mysik asked
marky01 commented ·

3 Answers

Howto disable BMS control, while keeping CAN-bms connected

Hi everyone,

Below it shows that BMS control is enabled automatically when a BMS is present:


[image]


Is there a way to disable BMS control, so that the MPPT manages charging the battery from its own charge settings, while also having the measurement info (SOC for example) from the BMS displayed via CAN?

ryanb asked
Daniël Boekel (Victron Energy Staff) commented ·

1 Answer

Feature Request: Connect CCGX to hidden WiFi SSID

I know this question has already been asked before (https://community.victronenergy.com/questions/35671/can-the-ccgx-connect-to-a-hidden-wifi-network.html), but the answer was uncertain and still not ideal.

Our network is segregated into multiple VLANs to help isolate devices and allow the firewall to route the traffic. In particular, we have a specific VLAN for IoT devices that require WAN (i.e. Internet) access. I would like to hide that SSID, however the CCGX can only connect to visible SSIDs - I don't have another GX-family device to test, but it may apply to those too.

The answer provided in the previous question was to try making the SSID visible, connecting the CCGX and then hiding the SSID again.

Questions:

  1. Does this actually work?
  2. Are there any impacts of this, for example if the WiFi disappears and reappears, will the CCGX's ability to reconnect be impeded?
  3. Will the CCGX still be able to roam between multiple APs (i.e. BSSIDs) with the same SSID if the SSID is hidden?

The rationale given for not offering connection to hidden SSIDs was that the user experience to enter an SSID manually would be cumbersome. Agreed, but it's no worse than entering a long, complex password which needs doing anyway!

I wasn't sure where best to submit a feature request, hence I'm putting it here. Even if the workaround above does work fine, I still think there is a valid need to manually input a WiFi SSID and password to allow for the use of hidden SSIDs.

Matt Knight asked
Daniël Boekel (Victron Energy Staff) answered ·

1 Answer

Using Cerbo to control a heat pad under a lithium battery

I am going to install a Victron lithium battery and intend to have a heat pad under the battery with a small air space between the two. It will be in a reasonably cold part of the boat and I would like to avoid the possibility of a situation where the smart lithium won't accept a charge.

The question is: can a Cerbo GX be used to automatically start and stop power to a heat mat. I know it has a couple of relays and I don't have a generator so relay one is avaliable and it has four thermometer inputs but I can't find any info as to see if it is possible or not.

markblox asked

0 Answers

Detail instructions for installing Grafana and Docker.

Hi.

I need help to make open source Grafana work on my Mac mini. https://github.com/victronenergy/venus-docker-grafana

The problem is that I struggle to install this correctly. Unfortunately I am not so good at command lines, so hope somebody could give me some tips to get this up and going?

I have cerbo-gx and this is connected with ve.can to nmea2000 cable and mqtt is activated so it should be correct. I have mac os catalina and installed Grafana and docker as described in the venus-docker-grafana project and so far I think this is ok.

-Step 2 with download of docker compose file I only get one page of text and here is my problem with command lines..

-The same goes for step 3, no idea how to place the file..

-Step 4 I do not get access to (http: // localhost: 8088 /) but docker is installed and works fine when I open the app.

-Step 5 and access to Grafane works fine.

Am I far off target or are there minor details missing? Very grateful if somebody can help solve this.

jonnys asked
Peter Buijs - NL answered ·

9 Answers

idea
Mark commented Modifications
3

3 Feedback

0 Comments

For Victron Staff,Is possible to the next upgrade to on Remote console , to see the K1,K2 Aux .. Status?

For Victron Staff,Is possible to the next upgrade to on Remote console , to see the K1,K2 Aux .. Status? At the moment is not possible to se the status of Aux And Relay .Also is possible to see On VRM the Inverters status Aux and Programbile Relay status?

Gabriel Tanasescu asked
Keith Arnold commented ·

3 Answers

How I can run mosquitto_sub command from python script ?

Hi!

I want to run mosquitto sub command in a python script instead of typing it in raspberry pi terminal but I don't know how to do it. I need your help please

katia-khennoune asked
osaether answered ·

1 Answer

Help wanted: testing new Raspberrypi 2,3 and also 4 images

Hi all,

As some of you know, Venus OS for raspberrypi is a hobby project by our developers, worked on in their free time.

And raspberrypi4 support has been wanted and asked for since a while now, as thats the now commonly available raspberrypi model.

We hoped and waited for someone in the community to step up and do the work, but that didn't happen. Perhaps too specialised a set of skills required (openembedded, linux, and-so-forth) - and I must admit that its quite the time investment to work on that.

Anyway, the good news is that we have a new set of images available. One image for the raspberrypi2 and 3, and another for the raspberrypi4.

So now, rather than looking to - or waiting for - the community to do the development work, we're hoping for people to do the testing work. Welcome to comment what work in your system and what does not.

Enjoy, Matthijs


Changes compared to our existing raspberrypi images:

  1. Update linux from 4.9 to 4.19
  2. The whole sdcard is now used and roughly 1/3 is assigned to /data
  3. both root filesystems are also increased to roughly 1/3 each
  4. u-boot settings are now stored redundantly on the sdcard; the FAT

    partition is no longer written to.


Not much has been tested, so the question is how many thing still work.

Note: these images are intended for that, testing, don't use them in anything serious. There are no automatic updates / bug fixes e.g.

This code is open source, and the changes are in this branch: https://github.com/victronenergy/meta-victronenergy/tree/WIP_rpi4


Downloads:


Note that they are full images. Its not possible to install it as a .swu file. They need to be burned to an sdcard, which erases all contents on that sdcard. For detailed instructions, see the Venus OS on raspberrypi instructions.


Things to test

Report results in the comments and answers below. Always include the exact model of RaspberryPi you are using please, and just to make sure there are no confusions, report firmware version & build number of Venus OS that you are using too.



rpi3
rpi4
Serial console (debug)
OK (rpi3B-v1.2 by Matthijs)
OK (rpi4B - Matthijs)
hdmi console
OK (rpi3B-v1.2 by Matthijs)
Not OK (rpi4B - Matthijs)
hdmi gui (remove /etc/venus/headless)
OK (rpi3B-v1.2 by Matthijs)
OK (rpi4B - Matthijs)
USB keyboard
OK (rpi3B-v1.2 by Matthijs)
OK (rpi4B - Matthijs)

GPS Dongles (pls report which one you use)

OK (rpi3B+ by PoloC)

Onboard WiFi

OK (rpi3B+ by PoloC)
OK (by Innar)

Onboard Bluetooth (note that I don't know on which rpis bluetooth is supposed to work)

Not OK (rpi3B-v1.2 by Matthijs)

VE.Direct USB


OK (by Innar)

MK3-USB to a Multi


OK (by Innar)

GX GSM



CAN (see here)

That inserting a USB stick makes VRM Logger recognise it


OK (rpi4B - Matthijs)
Pressing eject & then taking the stick out OK
Not OK (rpi4B - Matthijs)
Digital inputs

Analog inputs

Read through kernel bootlogs

.swu updating online (impossible to test for now)

.swu updating offline (sdcard or usbstick) (impossible to update for now)

booting into backup image (impossible to update for now)

Touchscreens and other add-ons that are being used

Logging to VRM (and thus DNS)
OK (by Innar)


mvader (Victron Energy) asked
Brian Brown answered ·

14 Answers

RPi Digital inputs without analogue?

Hi,

I'm trying to get my pi to accept digital inputs, however, I don't have the ADC chip. I've managed to get the relay to work, however, I can seem to get an I/O menu item on the GUI. I also don't have /sys/bus/iio directory, however, I'm not sure if this is just for the analogue inputs. I've installed dbus-adc to see if that made a difference, but to no avail. Any thoughts or ideas?

I'm running v2.60-22

Cheers

Simon

s-exley asked

0 Answers

2.60 GX Firmware release causing instability with BlueNova batteries

Dear Victron,

With the latest firmware release 2.60 you have caused quite bit of a MESS.

Implementing a forced DVCC functionality for lithium batteries is highly counterproductive and causing sever system instability especial with BlueNova batteries.

THIS MUST BE disabled ASAP.

Erik asked
Daniël Boekel (Victron Energy Staff) answered ·

3 Answers

(unofficial) Venus OS v2.60 for a raspberrypi 4

Since the raspberrypi4 support is not included in Venus OS v2.60, I uploaded the swu and sdcard image manually. Packages are missing at the moment, etc.

A running installation can be updated with the following command:

/opt/victronenergy/swupdate-scripts/check-updates.sh -swu https://updates.victronenergy.com/feeds/tmp/venus-swu-raspberrypi4-20200907083326-v2.60.swu

The sdcard image can be downloaded from https://updates.victronenergy.com/feeds/tmp/venus-image-raspberrypi4-20200907084212-v2.60.rootfs.wic.gz

jeroen asked
Brian Brown answered ·

4 Answers

Venus MQTT Instability - general problem ?

Venus GX, Version 2.57 (and 2.60 too):


I have substantial problems with my installation (see other topic, 3phase multipluss ESS with external control), but my installer will come today and update the multiplusses, so with some luck that might be fixed.

However, while doing extensive monitoring i found very strange problems in the MQTT communication of the Venus.

I am using MQTT to monitor the behavior of ESS (setpoints etc), and the following strange things happen:

a) sometimes values are delivered as nothing (NULL), not JSON formatted, breaking the MQTT protocol.

b) today i was able to capture, by chance, a broken MQTT message completely unrelated to setpoints (so no excuses that data would not be available or the like):

N/xxxxx/temperature/24/Voltage {"value": 4.7182979583740234}

N/xxxxx/vebus/261/Hub4/L2/CurrentLimitedDueToHighTemp {"value": 0}

N/xxxxx/vebus/261/Hub4/L2/OffsetAddedToVoltageSetpoint {"value": 0}

R/xxxxx/sy

stem/0/Serial (null)

N/xxxxx/tank/21/Resistance {"value": 268.96328735351562}

N/xxxxx/tank/22/Resistance {"value": 268.73052978515625}

N/xxxxx/temperature/23/Voltage {"value": 4.7100486755371094}

The problem happens with 2.60 and 2.57 firmware, for me this looks like a memory or stack issue, something "bombing" memory .. the device really must know its serial number ?

This probably contributes to the problems of my installation.

Has anyone seen such a behavior ?
It is quite easy to write a script to subscribe to all topics and detect NULL values of data ..

jacola asked
jacola commented ·

0 Answers

Venus GX Relay 2 output control

Hi,

Why is Victron implementing a relay into the Venus that is only controllable manually. Implementing SOC control for a relay in software should be no big issue. Venus OX 2.60 still does not provide this. Manual relay control is much easier by installing a hardwired switch. No need to open a APP, browse to relay function to activate.

When is SOC relay control coming? This feature would be ideal for load management, etc.

It is also recommended that simple SOC control (not Gen/Start stop) should be an option for relay 1. i.e Switch on at 90% and switch off at 75% ( function should be either NO or NC)

Erik asked
Mark answered ·

1 Answer

Venus GX to turn off charging source from analog input

Is it possible to program an analog input of the Venus GX to turn off any charging source in the Victron components connected to the Venus GX (eg. MPPT or inverter/charger) from an external 2 wire relay? Eg. external BMS/push button/rapid shutdown etc.

I can use the external 2 wire relay source to power a BatteryProtect or a relay to disconnect the battery output of a MPPT, or connect the 2 wire relay into the MultiPlus to stop charge but it would be more elegant to do it via software.

Johnson Lee asked
jon-lowbank commented ·

0 Answers

Unable to boot Venus OS on RBP 3B+

I've been trying to get the Venus OS loaded onto a Raspberry Pi 3 B+, but am just getting frustrated. No attempt to write the img to the card will cause it to boot. I'm using Mac OS for my primary computer (to load the image). I've tried:

  • Loading venus-image-raspberrypi2-20200906135923-v2.60.rootfs.rpi-sdimg.zip
    • Using Raspberry Pi Imager
      • direct from zip
      • unzipped
    • Using balanaEtcher
      • unzipped and renamed extension to .img
    • Using command line dd
  • The same above but with 20200301205316-v2.51
  • EEPROM boot recovery and erasing the SD card before loading
  • Using two different SanDisk Ultra SD cards (8 and 32Gb)
  • Two different Macbook Pro's

In every case, I only get the rainbow screen at startup. As a sanity check, I used the Raspberry Pi Imager to load the latest RBP OS to the card, and it booted the first time, so I'm fairly confident my loading process, cards, and RBP board are fine.


What else can I try?

ckhorne asked
ckhorne answered ·

1 Answer

VRM real time tanks display improvement

I have a SeeLevel tank system connected via NMEA2000 and because of current limitations in Venus, I'm running a program that separates the 3 tanks reported by SeeLevel into separate dBus objects. I hide the original SeeLevel dBus object which reports all three tanks in rotation (modified QML code). On VRM, there isn't a way to hide this constantly changing tank information so there's always a 4th tank that provides incorrect information.

It would be nice to be able to hide this tank on the VRM dashboard also. I can't think of a way to do this automatically so it would need to be a manual configuration item in VRM.

I realize this problem will go away once Venus code is updated to handle multiple tanks from the same NMEA2000 device.

Kevin Windrem asked
mvader (Victron Energy) answered ·

1 Answer

How to connect to VGX ?

Hi,

I am currently working with Venus GX and a development board(LAUNCHXL2-570LC43). I want to know that among the below mentioned communication types which is best suited to get the data received by the VGX to the board:

1. Modbus TCP/IP

2. MQTT

3. LWIP TCP/IP


Regards,

Advaith

advaith-g-n asked

0 Answers

How to establish Venus GX Connectivity using LWIP TCP/IP stack?

Hi,

I am currently working with Venus GX and a development board(LAUNCHXL2-570LC43). I want to know that among the below mentioned communication types which is best suited to get the data received by the VGX to the board:

1. Modbus TCP/IP

2. MQTT

3. LWIP TCP/IP


Regards,

Advaith


advaith-g-n asked

0 Answers