question

philippe avatar image
philippe asked

How to implement a script on Venus OS

I guess this question has already been asked a couple of times but I can not get hold of the answer.

Following the issue I had with my LTE/4G router (https://community.victronenergy.com/questions/64911/issue-with-4glte-provider-assigned-non-public-ip-a.html) I have found a script to check and enable the data. I have tweaked with the test to the internet, as it is a part of the problem. Probably not most nice script, but is works.

This script works fine in terminal mode on a connected Mac OS. As I'm a newby in scripting, I want to implement this script on the Venus OS. I was able to connect to the root and run it from the terminal. Now I want to have it run in an unattend mode (when I stop the terminal, my script stops also).

Also I want it also get back running after a reboot or a new release.

Maybe somebody can point me to the right documentation, or explain it.

The cherry on the cake will be: write the messages to a logging and raise an alarm in the VRM.

Thanks in advance because the mobile operator (BASE) put the fault on the router and are not willing to help, and Netgear does not care as it is minor issue.....

Best regards,

Philippe

#!/usr/bin/env python3

"""
This scripts monitors the MR1100 to re-enable Data automatically.
This is useful with operators that use PPP and which systematically disconnect
users after a certain period of time.

Works with firmware 12.06.08.00
"""

import requests
import json
import time
import datetime
import sys
"""

if len(sys.argv) < 2:
    print("Usage: {} <IP> <ADMIN_PASSWORD>".format(sys.argv[0]))
    sys.exit(1) 

ip = sys.argv[1]
pwd = sys.argv[2]
"""

ip = "192.168.1.1"
pwd = " you will get this one :) "

timeout = 2

url_base = "http://{}".format(ip)
url_session = "{}/sess_cd_tmp".format(url_base)
url_js_token = "{}/api/model.json".format(url_base)
url_config = "{}/Forms/config".format(url_base)
url_json = "{}/api/model.json".format(url_base)
url_json_success = "{}/success.json".format(url_base)
url_success = "{}/index.html".format(url_base)

def login(r):
    try:
        api = r.get(url_js_token, timeout=timeout).text
    except:
        print("Failed to connect to router")
        return

    try:
        api = json.loads(api)
        global token
        token = api['session']['secToken']
    except:
        print("Failed to fetch authentification token")
        return

    data = {
        'token': token,
        'err_redirect': '/index.html?loginfailed',
        'ok_redirect': '/index.html',
        'session.password': pwd,
    }

    try:
        redirect_url = r.post(url_config, data=data, timeout=timeout).url
        if redirect_url == url_success:
            return True
        else:
            print("Failed to authenticate, incorrect password?")
            return
    except:
        print("Router failed to respond to authentication challenge")
        return


def get_status(r):
    try:
        json_model = json.loads(r.get(url_json, timeout=10).text)
        status = json_model['wwan']['connection']
        return status

    except:
        print("Cannot retrieve connection status")


def reconnect(r):
    disconnect = {
        'token': token,
        'err_redirect': '/error.json',
        'ok_redirect': '/success.json',
        'wwan.autoconnect': 'Never',
    }
    connect = {
        'token': token,
        'err_redirect': '/error.json',
        'ok_redirect': '/success.json',
        'wwan.autoconnect': 'HomeNetwork',
    }

    try:
        push = r.post(url_config, data=disconnect, timeout=timeout)
        if push.url != url_json_success:
            print("Failed to disconnect data")
    except:
        print("Invalid answer from router")
        return

    try:
        push = r.post(url_config, data=connect, timeout=timeout)
        if push.url != url_json_success:
            print("Failed to reconnect data")
            return
    except:
        print("Invalid answer from router")

def internet():
    url = "http://www.google.com"
    timeout = 5

    try:
        request = requests.get (url, timeout=timeout)
        return True
    except:
        print ("No internet connection")
        return False

def main(dry_run=None):
    data_status = None

    r = requests.Session()
    login_status = login(r)

    if login_status != True:
        return
    else: 
        data_status = get_status(r)
        if dry_run is True:
            return True
    if data_status != "Connected":
        print("Data appears to be disconnected, reconnecting..")

        reconnect(r)

        nb_tries = 0
        while data_status != "Connected" or nb_tries < 3:
            data_status = get_status(r)
            time.sleep(1)
            nb_tries+=1

        if data_status == "Connected":
            timestamp = datetime.datetime.now()
            timestamp = timestamp.strftime("%Y-%m-%d %H:%M:%S")
            print("Reconnection successful {}".format(timestamp))
        else:
            print("Data is still offline, this should not happen")
    else:
 
        
        if internet () != True:
            print("Data appears to be disconnected, reconnecting..")

            reconnect(r)

            nb_tries = 0
            while data_status != "Connected" or nb_tries < 3:
                data_status = get_status(r)
                time.sleep(1)
                nb_tries+=1

            if data_status == "Connected":
                timestamp = datetime.datetime.now()
                timestamp = timestamp.strftime("%Y-%m-%d %H:%M:%S")
                print("Reconnection successful {}".format(timestamp))
            else:
                print("Data is still offline, this should not happen")
        else:
            return
        
if __name__ == '__main__':
    try:
        dry_run = main(dry_run=True)
        if dry_run is None:
            sys.exit(1)
        else:
            print("Connection to router successful. Monitoring disconnections")

        while True:
            main()
            time.sleep(2)

    except KeyboardInterrupt:
        sys.exit(0)


Venus OSVenus GX - VGX
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

2 Answers
Kevin Windrem avatar image
Kevin Windrem answered ·

Yes, all that is possible. I was not able to find the official documentation on the subject so I'll explain briefly then you can go to one of the projects I've done and extract the relevant code.

Venus uses a startup mechanism that scans the /service directory and attempts to run things it finds there.

A directory named service would contain a run script as well as a log directory with it's own run script. The log script directs stdout and stderr to a log area of your choosing.

Setting this up will cause your script to run at startup and be restarted if it crashes.

Most of the file system is rewritten during a software update. However the /data directory is retained so that is where you want to place your code.

The /service directory is replaced during a software update too, so you need to reinstall your code after a software update. /data/rc.local is a script that's executed at every startup and because of it's location, survives a software update. You can insert script instructions in this file that reinstall your code.

There are more details to these mechanisms. Check out the code here for an example that is probably worth more than 1000 more words here

https://github.com/kwindrem/TankRepeater

The contents of the service directory should illustrate what's needed to run your script from the /service directory.

The setup file is rather complex and handles local as well as remote install from a unix host (like Mac OS). It:

1) sets up a directory for the code in /data

2) copies the files from the host

3) creates the entry in /service

4) adds code to /data/rc.local to reactivate the code after a software update

You can certainly simplify the setup script or split things up into separate scripts that are easier to understand.

After you have looked this over, feel free to come back and ask questions.


3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gnagflow avatar image gnagflow commented ·

Hi Kevin, thank you for this great information. Could you help me with one thing. When I make a new directory inside /service and I reboot (not an update), the directory is gone, so all run scripts inside. Why? What can I do?

Do I have to put my service into the /data directory and put a system link into rc.local for linking my service at every startup into the /service directory ... what is there the common way to get your service survive at a startup?


thank you!


0 Likes 0 ·
ryanb avatar image ryanb gnagflow commented ·

Yes, this is a good question. I ended up making a directory in /opt/victronenergy/service-templates which had the run scripts, and then in rc.local, I copied (you need to do recursive copy) this directory into the /service/ directory. And voila... The service directory picks it up automatically, and runs it...

0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem ryanb commented ·

If you put the service directory in /opt/victronenergy/service, it will get copied to /service at boot. No need to run anything in rc.local every boot.

/opt/victronenergy/service is only copied at boot so if you want the service to run immediately you need to do the copy manually (one time only).

Symbolic links were abandoned a while back. Use copies, not links.

Part of my SetupHelper package is a set of service management functions found in the ServiceResources file. Might give you some ideas for managing your services.

https://github.com/kwindrem/SetupHelper

1 Like 1 ·
philippe avatar image
philippe answered ·

@Kevin Windrem thanks a lot for the explanations.

I got my script implemented, and it kicks in after a reboot.

I have also implement your QML mobile overview with 2 little tweaks (time in 24h format, and added temperature of the battery in stead of the word remaining and replaced "> 10D" the with infinity symbol). For now, I will need to implement something to get it back after an update of the FW ( a script to copy the QML and reboot).screenshot-2020-11-04-at-130137.png

My next challenge is to monitor my SmartBatteryProtect 65A (as Venus does not support the bluetooth and can not interact/read the data of the not directly connected devices such as Orion, Smart Batteryprotect).

It is now the second time that the BP65 goes into alarm and disconnect before the trigger value, and does not get back after the restart value is reached. The idea is somewhere use of the 12v out and convert it with a optocoupler into a digital signal on/off [one of those weird descriptions as bigle alarm...], and a second to switch on/off the Smart Batteryprotect [relay 1].)



2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.