RPi USB port allocation

Question

question

xsilvergs asked Sep 24, '22

RPi USB port allocation

I have been using an RPi for a number of years now and adopted the Large Image some time ago but have occasionally suffered a problem after a reboot. The Pi allocates a different id to the USB-VE.Direct dongles, this then gives a disconnected message in the Edit Nodes for each connected device. I have a BMV-712, 2 X 100/30 MPPT and a Phoenix inverter.

Is there anyway to ensure the RPi identifies the USB dongles correctly to save me have to either a) reboot the Pi multiple times until it selects the dongles in the correct order or b) select the new/alternative device in Node-Red?

VE.Direct

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Modifications Space

Node Red Space

Answer 1 · 2022-09-24T14:14:27Z

bathnm answered · Sep 24, 2022 at 02:14 PM

Maybe have a look at udev and create some rules to identify the USB device and give it a consistent device name.

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xsilvergs commented · Sep 24, 2022 at 04:26 PM

Thanks for tip, I've tried it and will give it a few reboots to see how it goes. The Victron USB - VE.Direct dongles each have the same vendor and product id so included individual dongle serial number.

0 ·

Answer 2 · 2022-09-25T19:02:01Z

Mike Dorsett answered · Sep 25, 2022 at 07:02 PM

I have a Pi with 2 or 3 VE direct connections via USB. Whilst it is possible to list, and iterate through the list to identify the VE direct ports, this does not identify which device is on the end of any particular dongle. For this, you either have to request the device Id, or listen for the Id in the broadcast ASCII (takes longer). The code below is for 2 devices.

from serial.tools import list_ports
from serial.tools.list_ports import comports

msg_get_pid = str.encode(":70001004D"+chr(0x0a)) # com

def Find_VED():
VED_port = ["null"]*4
com_ports_list = list(comports())

for i,port in enumerate(com_ports_list):
if port[1].startswith("VE Direct"):
VED_port[i] = port[0]
return(VED_port)

for port in f:
#print(373,c,port)
if port !="null":
com[c] = serial.Serial( port, baudrate=19200,parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE, bytesize=serial.EIGHTBITS)
c+=1
print(378,len(com), com,c)
time.sleep(1)
msg = msg_get_pid # get PID
f = -1
While f < 0:
f = write_VE(0,msg)
time.sleep(0.2)
print(591,"{:04x}".format(f))
if f == 0x0a46 :
mppt = 0
sms = 1
else:
mppt = 1
sms = 0

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xsilvergs commented · Sep 27, 2022 at 03:33 PM

Prior to the Victron Nodes coming available I looked at VE.Direct data of each USB attached and used the device id. From that I directed data to the right area in Node-Red.

0 ·

xsilvergs commented · Sep 29, 2022 at 11:30 AM

@Mike Dorsett

Could you give a bit of help please?

I have looked at your code (it's a few years since I've written python) and think the line "for port in f:" should have previously had the "f" referenced. Please explain.

Also I have two 100/30 MPPT how will it determine which one without knowing there serial numbers of each MPPT or USB dongle.

And, most importantly, what do you do with mppt = 1, sms = 0 how does this fix a device to a /ttyUSB (how can I ensure that in Node-Red my BMV always connects to ttyUSB0?

Thanks for any help.

0 ·

Answer 3 · 2022-09-29T18:17:19Z

Mike Dorsett answered · Sep 29, 2022 at 06:17 PM

The code is not complete, but the missing line was:

f = Find_VED

Which is the function that enumerates and identifies the ports with ve direct dongles attached.

If you have 2 mppt devices, then if you really need to differentiate between them, you will also need to read the device serial numbers. However, I don't see the need for this. In a 2 mppt system, both devices would need to be sent the same information: Battery voltage, max charge current etc, and the data from the device meters read and totalled.

If you need to track the history from each device separately, then this can be done with the serial.

The last segment of code:

f = write_VE(0,msg)

sends request for PID;

if f == 0x0a46 :
mppt = 0
sms = 1
else:
mppt = 1
sms = 0

this then allocates the correct com port number to variables "mppt" and "sms" so the other routines in the program can then use these port numbers to extract the data and post it to the correct output (Modbus registers in this case).

how can I ensure that in Node-Red my BMV always connects to ttyUSB0?

That is the essence, as after a reboot, the BMV might be on dev/ttyUSB1.

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xsilvergs commented · Sep 29, 2022 at 07:07 PM

@Mike Dorsett

Thanks for some clarification. Without indenting the code isn't so easy to understand for me.

0 ·

Mike Dorsett xsilvergs commented · Sep 29, 2022 at 07:12 PM

One problem. The code looses it's indentation when pasted. This also a python problem as the indentation is the only clue as to the end of if statements.

0 ·

Answer 4 · 2022-10-05T12:52:35Z

vassilis-bourdakis answered · Oct 05, 2022 at 12:52 PM

hello,

anyone managed to get this working somehow and could explain a bit more how to use this code?

I have a BMV700, a MPPT and a Multi II which are swapping between USB0 1 and 2 on each reboot...

cheers

V.

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Mike Dorsett commented · Oct 05, 2022 at 06:20 PM

The multi connects using a Mk3, which is identified as such in the line:

if port[1].startswith("VE Direct"):

port[x] is a list of all of the usb ports.

x will be 0,1,2 3

port[x] will have a value determined by what is plugged into it. this will be a string starting with:

"VE Direct", "MK2" or "MK3" so it is easy to separate out the the Multi from the VE direct connections.

The BMV and the MPPT can only be identified by their PID.

My (PI) system runs 3 programs simultaneously:

BMS monitoring / control program, the VE-Direct program and the Multi/Mk3 program. All of these programs share data through a Modbus map, which I can also use to troubleshoot the system if needed.

0 ·

Answer 5 · 2022-10-05T18:23:24Z

Full code of the VE direct program (without indentation it got lost on posting!!):

#pi@raspberrypi:~ $ sudo date -s"Feb 10 11:25 2018"
#Saturday 10 February 11:25:00 FJT 2018

from struct import *
from time import *
import time
import serial
from pymodbus.client.sync import ModbusTcpClient as ModbusClient
from serial.tools import list_ports
from serial.tools.list_ports import comports

#MPPT / Blue / Smart Solar
msg_get_AbsV =str.encode( ":7F7ED006A"+chr(0x0a))
msg_get_BmaxA =str.encode( ":7F0ED0071"+chr(0x0a))
msg_get_CmaxA =str.encode( ":7DFED0082"+chr(0x0a))
msg_get_tyld =str.encode( ":7DDED0084"+chr(0x0a))
msg_get_tmp =str.encode( ":7DBED0086"+chr(0x0a))
msg_get_err =str.encode( ":7F0ED0087"+chr(0x0a))
msg_get_champ =str.encode( ":7D7ED008A"+chr(0x0a))
msg_get_chvlt =str.encode( ":7D5ED008C"+chr(0x0a))
msg_get_iyld =str.encode( ":7D3ED008E"+chr(0x0a))
msg_get_imxp =str.encode( ":7D2ED008F"+chr(0x0a))
msg_get_pvmp =str.encode( ":7BCED00A5"+chr(0x0a))
msg_get_pviv =str.encode( ":7BBED00A6"+chr(0x0a))
msg_get_pid = str.encode(":70001004D"+chr(0x0a)) # com
msg_get_intt = str.encode(":7DBED0086"+chr(0x0a))
msg_get_devs = str.encode(":70C200022"+chr(0x0a))
msg_get_trcm = str.encode(":7B3ED00AE"+chr(0x0a))
msg_get_dyyd = str.encode(":7D1ED0090"+chr(0x0a))
msg_get_mpyd = str.encode(":7D0ED0091"+chr(0x0a))
msg_ping = str.encode( ':154'+chr(0x0a))
msg_set_ubat = str.encode(":8F1ED00FF70"+chr(0xa))
msg_set_tbatc = str.encode(":8F2ED00006E"+chr(0xa))
msg_set_vbata = str.encode(":8F7ED00C80A97"+chr(0xa))
msg_set_vbatf = str.encode(":8F6ED00C80A98"+chr(0xa))
msg_set_eqoff = str.encode(":8FDED000063"+chr(0xa))
msg_set_tabs = str.encode(":8FBED009001D4"+chr(0xa))
msg_set_vbate = str.encode(":8F4ED00180B49"+chr(0xa))
msg_set_Ibatm = str.encode(":8F0ED00BC02B2"+chr(0xa))
msg_set_nBMS = str.encode(":8E8ED000078"+chr(0xa))
msg_set_extm = str.encode(":80E2000021D"+chr(0xa))
msg_set_extI = str.encode(":8152000DC0537"+chr(0xa))
msg_restart = str.encode(":64F"+chr(0xa))

# Smart Shunt

reg_dict = { # Note all VE registers are byte swapped!!!
0x0001 : [770,32], #PID
0x0C20 : [778, 8], # device state
0x0202 : [779, 32], # remote used
0x0220 : [788,16], #remote v batt
#0xFBED : [777, 16], # Abs time limit
#0xF7ED : [776, 16] , # Abs voltage
#0xF6ED : [775, 16], #float volt
0xDFED : [792, 16], # max current
0xDDED : [783, 32], #System Yield
0xDBED : [775, 15], # internal temp (signed)
0xDAED : [782, 8], #error
0xD7ED : [772, 16], #charge current
0xD5ED : [771, 16], # batt voltage

0xD3ED : [773, 16], # today yeild
0xD2ED : [785, 16], # max power day
0xD1ED : [786, 16], # yeild yesterd
0xD0ED : [787, 16], # max p -24h
0xBBED : [776, 16], #pv volt
0xBCED : [789, 32], # pv power
0xB3ED : [777, 8], #tracker mode
0x1520 : [793, 16], #charge limit
0xF1ED : [800,16],
0xF7ED : [801,16],
0xFDED : [802,16],
0xFBED : [803,16],
0xF4ED : [804,16],
0xE8ED : [805,16],
0xF0ED : [806,16]
#0xD1ED : [786, 16], #Yield -24h
#0xD0ED : [787, 16] #Max P -24h
}

com = ["null"]*4
#VE Direct
def Find_VED():
VED_port = ["null"]*4
com_ports_list = list(comports())

for i,port in enumerate(com_ports_list):
if port[1].startswith("VE Direct"):
VED_port[i] = port[0]
return(VED_port)

def dec2hex_str(dd1):
hi_by = int(dd1/256)
lo_by = dd1-hi_by*256
#print(54,"{:04X}".format(dd1),"{:02X}".format(hi_by),"{:02X}".format(lo_by))
I_str = ("{:02X}".format(lo_by))+("{:02X}".format(hi_by))
msg_set_Imax = "8152000"+I_str
chk = 0x118 -lo_by-hi_by
#print(58,"check","{:02x}".format(chk))
chk = chk%256
st_ch =("{:02X}".format(chk))
msg_set_Imax += st_ch
b_msg_Imax = str.encode(":"+msg_set_Imax+chr(0x0a))
return(b_msg_Imax)

def dec2hex_Vstr(dd1):
hi_by = int(dd1/256)
lo_by = dd1-hi_by*256
#print(54,"{:04X}".format(dd1),"{:02X}".format(hi_by),"{:02X}".format(lo_by))
V_str = ("{:02X}".format(lo_by))+("{:02X}".format(hi_by))
msg_set_v = "8022000"+V_str
chk = 0x12B -lo_by-hi_by
#print(58,"check","{:02x}".format(chk))
chk = chk%256
st_ch =("{:02X}".format(chk))
msg_set_v += st_ch
b_msg_v = str.encode(":"+msg_set_v+chr(0x0a))
return(b_msg_v)

def short_to_long(hi_byte,lo_byte):
long_bytes = hi_byte*65536+lo_byte
return(long_bytes)

def long_to_short(long_byes):
temp = float(long_byes)
hi_byte = int(temp/65536)
lo_byte = int(temp-65536*hi_byte)
#print("152",long_byes,hi_byte,lo_byte)
return(hi_byte,lo_byte)

def long_to_vshort(long_byes):
temp = float(long_byes)
hi_byte = int(temp/256)
lo_byte = int(temp-256*hi_byte)
#print("152",long_byes,hi_byte,lo_byte)
return(hi_byte,lo_byte)

def store_reg(mregisters,address):
global client

slave_id = 0x01
if address == 789:
print(135,mregisters)
try:
rq = client.write_registers(address, mregisters)
except:
rq = -1
return(rq)

def read_reg(nregisters,address):
global client
slave_id = 0x01

#print(106,address,f_registers)
#f_data = context[slave_id].getValues(register,address, f_registers)
f_resp = client.read_holding_registers(address,nregisters)
f_data = f_resp.registers
#print(77,f_data)
return(f_data)

def read_float(addr):
global client
val_lst = [0,0,0,0]
slave_id = 0x01
try:
f_resp = client.read_holding_registers(addr,2)
f_data = f_resp.registers
except:
f_data = [0,0]
val_lst[0],val_lst[1] = long_to_vshort(f_data[0])
val_lst[2],val_lst[3] = long_to_vshort(f_data[1])
val_bytes = bytes(val_lst)
val_t = unpack('f',val_bytes)
val = val_t[0]
return(val)

def read_hex(i):
global com
val = write_VE(i,msg_get_pid)
print(154,"PID","{:04x}".format(val))
store_reg([val],770+i*100)

val = write_VE(i,msg_get_CmaxA)
print(156,"Ch Mx A",val)
store_reg([val],792+i*100)

val = write_VE(i,msg_get_champ)
store_reg([val],772+i*100)
print(158,"bat I",val)

val = write_VE(i,msg_get_chvlt)
store_reg([val],771+i*100)
print(160,"bat V",val)

val = write_VE(i,msg_get_pviv)
store_reg([val],776+i*100)
print(162,"PV V",val)

val = write_VE(i,msg_get_iyld)
store_reg([val],773+i*100)
print(164,"PV Yld",val)

val = write_VE(i,msg_get_pvmp)
hpvp,lpvp = long_to_short(val)
store_reg([lpvp,hpvp],789+i*100)
print(166,"PV Pmx",val)

val = write_VE(i,msg_get_tyld)
htyld,ltyld = long_to_short(val)
store_reg([ltyld,htyld],783+i*100)
print(168,"PV tot yld",val,htyld,ltyld)

val = write_VE(i,msg_get_tmp)
store_reg([val],775+i*100)
print(170,"int temp",val)

val = write_VE(i,msg_get_devs)
store_reg([val],778+i*100)
print(172,"dev state",val)

val = write_VE(i,msg_get_trcm)
val &= 0x03
store_reg([val],777+i*100)
print(174,"track mode",val)

val = write_VE(i,msg_get_imxp)
store_reg([val],785+i*100)
print(176,"PV max P",val)

val = write_VE(i,msg_get_dyyd)
store_reg([val],786+i*100)
print(178,"PV -24h yield",val)

val = write_VE(i,msg_get_mpyd)
store_reg([val],787+i*100)
print(180,"PV mpyd",val)
return()

def readbytes2(i):
global com
rxstring = ""
try:
while int(com[i].inWaiting()) > 0:
byte1 =(ord(com[i].read(1)))
rxstring +=(chr(byte1))

except Exception as e:
print("rxmsg",i,e,com[i].inwaiting())
nl = rxstring.find("PID")
fl = rxstring.find(chr(0x09),nl) #Find the tab after "Pid"
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Pid",i,pvp3)
try:
ppv = int(pvp3,16)
pvp_h,pvp_l = long_to_short(ppv)
store_reg([pvp_l,pvp_h],770+100*i)
except Exception as e:
print(97,i,pvp3,e)

#print(rxstring)
nl = rxstring.find("PPV")
fl = rxstring.find(chr(0x09),nl) #Find the tab after "PPV"
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("PPV",i,pvp3)
#789 sf 10 uint16
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],789+100*i)
except Exception as e:
print(97,i,pvp3,e)

# if fxfield == "V":
nl = rxstring.find(chr(0x0d)+chr(0x0a)+"V")
fl = rxstring.find(chr(0x09),nl) #Find the tab after "V"
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("V",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],771+100*i)
except Exception as e:
print(112,i,pvp3,e)
return()

nl = rxstring.find(chr(0x0d)+chr(0x0a)+"I")
fl = rxstring.find(chr(0x09),nl) #Find the tab after "I"
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
#if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("I",i,pvp3)

try:
ppv = float(pvp3)
if ppv <0:
ppv+=32768
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],772+100*i)
# store_reg([bi_l,bi_h],772)
except Exception as e:
print(131,i,pvp3,len(pvp3),e)
#else:
#fix_i()

nl = rxstring.find("H19")
fl = rxstring.find(chr(0x09),nl) #Find the tab after "V"
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],783+100*i)
except Exception as e:
print(145,i,pvp3,e)

nl = rxstring.find("H20")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Yield",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],784+100*i)
except Exception as e:
print(165,i,pvp3,e)

nl = rxstring.find("H21")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Max power",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],785+100*i)
except Exception as e:
print(174,i,pvp3,e)

nl = rxstring.find("H22")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Yield -24h",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],786+100*i)
except Exception as e:
print(139,i,pvp3,e)

nl = rxstring.find("H23")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Max power -24h",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],787+100*i)
except Exception as e:
print(202,i,pvp3,e)

nl = rxstring.find("ERR")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("Error ",i,pvp3)
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],799+100*i)
except Exception as e:
print(202,i,pvp3,e)

nl = rxstring.find("VPV")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
print("VPV",i,pvp3)
try:
ppv = int(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l,pvp_h],776+100*i)
except Exception as e:
print(156,i,pvp3,e)

nl = rxstring.find("HSDS")
fl = rxstring.find(chr(0x09),nl) #Find the tab after
nl2 = rxstring.find(chr(0x0d),fl) # find the new line after the data
if nl != -1:
pvp3=rxstring[fl+1:nl2+1]
try:
ppv = float(pvp3)
pvp_h,pvp_l = long_to_short(ppv)
# store_reg([pvp_l],791+100*i)
except Exception as e:
print(156,i,pvp3,e)
return()

def read_msg(i):
rxstring = ""
#print(242,com2.in_waiting)
a = False
c = 0
crc = 0
try:
while a==False or c<18: #int(com[i].inWaiting()) > 0:

byte1 =ord(com[i].read(1))
if byte1 == 0x0a:
a = True
rxstring +=(chr(byte1))
crc += byte1
c +=1
except Exception as e:
print(355,e)
com[i].reset_input_buffer()
return(":\n")
print(435,"{:02x}".format(crc))
return(rxstring)

def dec_rx_msg(i,reg,xdata):
global reg_dict
#print(363,i,reg,xdata)
try:
rg = reg_dict[int(reg,16)]
# print(366,i,rg,reg,xdata)
except:
print(368,"dict fail",int(reg,16),xdata)
store_reg([reg],794+100*i)
return(-1)
if rg[1]== 8:
xdata = xdata&0xff
store_reg([xdata],rg[0]+100*i)
if rg[1] == 16 or rg[1] == 15:
store_reg([xdata],rg[0]+100*i)
if rg[1] == 32 and xdata <= 0xffff:
store_reg([xdata&0xffff],rg[0]+100*i)
if rg[1] == 32 and xdata > 0xffff:
# should be store float..
store_reg([xdata&0xffff],rg[0]+100*i)
store_reg([xdata>>16],rg[0]+1+100*i)

return(1)

def write_VE(i,msg):
global com
time.sleep(0.1)
rx_str = ""
val_str = ""
#print(330,i,msg)
com[i].reset_input_buffer()
f = com[i].in_waiting
if msg == msg_restart:
return
if f > 0:
time.sleep(0.5)
com[i].reset_input_buffer()
b = com[i].write(msg)
#print(256,msg, b)
f=0
while f < 6:
f = com[i].in_waiting
#print(362,f)
if f > 0 :
rx_str = read_msg(i)
nl = rx_str.find(":")
el = rx_str.find("\n")
rs_str = rx_str[nl:el]
if nl != 0:
print(477,"rxstring short",nl)
return(-2)
#print("\n","\n",382,nl,len(rx_str),len(rs_str),ord(rs_str[0:1]))
flags = rx_str[6:8]
reg = rx_str[2:6]
val_str = rx_str[8:-3]
if len(val_str) ==7:
val_str = val_str[0:4]
if str(flags) !="00":
print(372,reg,flags,len(val_str),val_str,rs_str)
if len(val_str) == 4:
big_val_str = val_str[2:4]+val_str[0:2] #swap bytes
try:
val = int(big_val_str,16)
#
except:
print(493,i,reg,big_val_str,msg)
time.sleep(0.5)
return(-3)
dec_rx_msg(i,reg,val)
elif len(val_str) == 2:
val = int(val_str)
#print(420,i,reg,val)
dec_rx_msg(i,reg,val)
elif len(val_str) == 8 and reg == "0001":
big_val_str = val_str[4:6]+val_str[2:4] #swap bytes
val = int(big_val_str,16)
#print(425,i,reg,val)
dec_rx_msg(i,reg,val)
elif len(val_str) == 8 and reg != "0001":
big_val_str = val_str[6:8]+val_str[4:6]+val_str[2:4]+val_str[0:2] #swap bytes
val = int(big_val_str,16)
#print(430,i,reg,val)
dec_rx_msg(i,reg,val)
else:
print(377,i,msg,rs_str,val_str)
val = -1

return(val)

def Set_MPPT(i):
write_VE(i,msg_restart)
time.sleep(2)
f = write_VE(i,msg_set_ubat)
print(375,f)
f = write_VE(i,msg_set_tbatc)
print(377,f)
f = write_VE(i,msg_set_vbata)
print(379,f)
f = write_VE(i,msg_set_vbatf)
print(381,f)
f = write_VE(i,msg_set_eqoff)
print(383,f)
f = write_VE(i,msg_set_tabs)
print(385,f)
f = write_VE(i,msg_set_vbate)
print(387,f)
f = write_VE(i,msg_set_Ibatm)
print(389,f)
f = write_VE(i,msg_set_nBMS)
print(391,f)
f = write_VE(i,msg_set_extm)
print(393,f)
f = write_VE(i,msg_set_extI)
print(395,f)
return()

def getImax(i):
global com
val = write_VE(i,msg_get_BmaxA)
print(554,"Bmax A",val)

if val < 70:
Val = 70
return(val)

def get_params(i):

# Float Voltage
# Bat max amps
write_VE(i,msg_ping) # shut the ascii mode up!!
f = write_VE(i,msg_set_Ibatm)
print(566,f)
val = getImax(i)
I_max_set = val
store_reg([val],792+100*i)
store_reg([val],780+100*i)
# Bulk Voltage
val = write_VE(i,msg_get_AbsV)
print(573,"param Abs V",val)
store_reg([val],774+100*i)
return(I_max_set)

f = Find_VED()
print(f)
c = 0
for port in f:
#print(373,c,port)
if port !="null":
com[c] = serial.Serial( port, baudrate=19200,parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE, bytesize=serial.EIGHTBITS)
c+=1
print(378,len(com), com,c)
msg = str.encode(":451"+chr(0x0a))
f = write_VE(0,msg)
print(f)
I_set_max = [70]*c
time.sleep(1.0) #wait for other program to setup and run

client = ModbusClient('localhost', port=502)
client.connect()

for i in range (0, c):
Set_MPPT(i)
I_set_max[i] = get_params(i)
store_reg([I_set_max[i]],792+100*i)
I_set = [800]*c
I_get = [800]*c
v = True

while True:

time.sleep(0.5)
for i in range(0,c):
read_hex(i)

f = int(read_float(11) *10)
v_bat = int(read_float(3)*100)
store_reg([v_bat],788)

# This loop sets the mac pv current for battery charge limiting
#print(459,f,I_set_max[i])
if f > 700:
f=700
I_set[i] = f
print(560,"I set",i,I_set[i],v_bat)
store_reg([I_set[i]],793+100*i)

ve_str = dec2hex_str(I_set[i])
val = write_VE(i,ve_str)
print(579,"I set",val)

ve_str = dec2hex_Vstr(v_bat)
val = write_VE(i,ve_str)
print(583,"V set",val)

store_reg([val],781+100*i)
#v = False

Answer 6 · 2022-10-10T13:59:57Z

pcurtis answered · Oct 10, 2022 at 01:59 PM

This is an issue which has been raised several times before including by myself, unfortunately searches of the Modifications area do not seem to go very deep into comments so they are difficult to find. The last time was 9th September in https://community.victronenergy.com/questions/155118/venus-os-v29027-available-for-testing.html and to quote the end of a long interchange:

pcurtis  commented · Sep 13 2022 at 7:07 AM
I echo that it is a problem which needs to be resolved. It happens frequently to 
me on a Raspberry Pi and I had assumed it was a problem which would therefore not merit much attention but if it is also occurring on Victron hardware it needs to 
be understood. I often find many of my devices show as disconnected in Node-RED 
after a Restart even if all the cables remain connected and have be set up again 
in the Victron nodes.
.......
Dirk-Jan commented · Sep 13 2022 at 11:26 AM

I am fairly sure that this can also happen on Victron hardware. And I have created an issue: Change node binding from full service name to VRM device instance for 
this to get this fixed in the future.

@Dirk-Jan Faber is assigned to the issue and it looks as if there is real progress and this has been resolved subject to suitable testing. We should give thanks to @Dirk-Jan Faber for picking this up and I must apologise for not noticing this post earlier and commenting.

18

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Dirk-Jan Faber (Victron Energy) ♦♦ commented · Oct 10, 2022 at 02:13 PM

I am still looking for some extra systems to further test this on. Would it be possible for you to test the code of the https://github.com/victronenergy/node-red-contrib-victron/tree/faberd/issue-144 branch?

0 ·

pcurtis Dirk-Jan Faber (Victron Energy) ♦♦ commented · Oct 10, 2022 at 04:15 PM

@Dirk-Jan Faber I only have one system with Victron devices connected which is running everything on my boat and I have been loath to play with once it was all working It is a RPI 3b+ which is still running 2.80 mainly because of the problems you are addressing. When I checked it has been over 3000 hrs since the last reboot! I also have an RPI 4 where I have been doing a lot of development with 2.90x ready for when I update my operational system.

I would like assist but realistically it will be at least a week before I can start to look at it and I will need a period (or several) on the boat even if I get it all set up on my development Pi 4b ready.

However the system on the boat has 4 ve.direct connections from 2 Smart Shunts, a Phoenix 1600 inverter and Smart Solar 100|20 and usually only the Phoenix connection survives a reboot so it would be a good test.

0 ·

Dirk-Jan Faber (Victron Energy) ♦♦ pcurtis commented · Oct 10, 2022 at 09:13 PM

No worries. I did some testing with a single smartshunt myself on a Cerbo, which worked as expected. And I do have a second smartshunt, so I can test it with multiple devices too. I'll see if I can test it on a Raspberry Pi next Wednesday.

0 ·

pcurtis Dirk-Jan Faber (Victron Energy) ♦♦ commented · Oct 13, 2022 at 02:57 AM

@Dirk-Jan Faber I have never used npm or looked at the way the Node ecosystem works so I will need some initial guidance if I am to help testing quickly.

I had a quick look at how npm install is used and I see it has many options for the package source including a git repository url so do I just need to do something as simple as:

npm install https://github.com/victronenergy/node-red-contrib-victron/tree/faberd/issue-144

in the .node-red folder having halted the node-red-venus service and do I need options, for example, to overwrite the existing package.

I am happy to try something simple like that on my development system, I already have some backup systems on CD cards for the Pi 4B and if I get a working system I may be able to take it to Corinna and plug in several of the Victron devices for a few hours next week

0 ·

Dirk-Jan Faber (Victron Energy) ♦♦ pcurtis commented · Oct 13, 2022 at 08:40 AM

What I usually do when testing is replacing `src/*` files with the new ones (in the `/usr/lib/node_modules/@victronenergy/node-red-contrib-victron/` directory), without using npm for it.
I did test already on some other systems and have discovered that there are still several issues with the branch. So no need for you to test it.

0 ·

pcurtis Dirk-Jan Faber (Victron Energy) ♦♦ commented · Oct 13, 2022 at 01:56 PM

@Dirk-Jan Faber I should be able to manage that if ever required! I guess that means that the filesystem has to be changed to r/w by /opt/victronenergy/swupdate-scripts/remount-rw.sh first.

Let me know if/when you ever need a test.

0 ·

natteverf Dirk-Jan Faber (Victron Energy) ♦♦ commented · Oct 31, 2022 at 09:15 PM

@Dirk-Jan Faber I am suffering from this node-red issue on my Raspberry Pi, every update or restart means i have to go and check to see what is or isn't connected in node-red and it is causing me some real headaches. I am happy to test for you if there is a chance that it will help resolve the issue

0 ·

Dirk-Jan Faber (Victron Energy) ♦♦ natteverf commented · Nov 01, 2022 at 08:57 AM

Yesterday I've made some good progress on this issue. I am still testing more thoroughly to make sure it won't break existing systems. I do have enough systems to test with, but it takes some time to get everything checked.

If needed you can always test the mentioned branch.

2 ·

natteverf Dirk-Jan Faber (Victron Energy) ♦♦ commented · Nov 01, 2022 at 08:12 PM

I am impressed, I have tried the branch and for me at least, it seems to be working really well, i see the transition from using the usb number to the VRM instance number, very nice indeed. I have tried a few different methods of restarting and so far node-red has always come up without having to make any changes. Dank u wel Dirk-Jan!

0 ·

natteverf natteverf commented · Nov 02, 2022 at 03:58 PM

Other things that i have noted:

I have a usb GPS dongle plugged in also and since i have used the new branch files it seems to always appear to be on ttyUSB0 after reboots, is this coincidental do you think or a result of somehow "pinning" the Victron devices? Would be nice if it was, i have been looking at udev to try and pin this device but it is a bit daunting.
When i initially changed to using the new branch it was not seamless, i had to manually reconfigure the nodes and change some message topics etc. as the payloads had slightly different topics now based on the VRM instance number rather than the ttyUSB number. To me that is not an issue, but maybe not what @Dirk-Jan Faber is aiming for.

0 ·

Dirk-Jan Faber (Victron Energy) ♦♦ natteverf commented · Nov 02, 2022 at 04:09 PM

I believe it is coincidental as I did nothing on how the devices appear on Venus or on the dbus.
My aim is that users do need to update their flows, while they remain functional if they don't update. I just pushed another update to the branch that fixes an issue when there are multiple devices that both have DeviceInstance 0. Mainly output nodes where affected with that.

0 ·

pcurtis Dirk-Jan Faber (Victron Energy) ♦♦ commented · Nov 02, 2022 at 09:35 AM

@Dirk-Jan Faber I have put the latest onto my Pi 4B 'development system' and nothing has broken as I was confident would be the case. It would however need to be taken to the boat to plug in all the USB devices to be a useful check as it currently only has Ruuvi sensors and relays. It has however proved my procedure so let me know if/when you have version which you really need checking.

UPDATE Checked again with commit 340518c , all works very well on my limited test system and displays all the right messages.

It will be a great relief when this is fixed, I have only risked updating/rebooting my operational system on the boat twice in the last six months. Thank you for all your efforts to fix this issue.

0 ·

bathnm commented · Oct 10, 2022 at 03:05 PM

@pcurtis I think there are two issues going on here. The node-red issue will be fixed by your ticket, however as things are based on VRM device instance. The underlying VenusOS system already ensure devices irrespective of the USB device map to a consistent VRM device instance. This is why no data is lost in VRM when a reboot occurs.

What people with a PI are concerned about is that their own additions to VenusOS have problems when a device is reported under a different USB device ID. This is something that their own code needs to handle, just like Victron does to ensure a device can be uniquely identified.

0 ·

pcurtis bathnm commented · Oct 10, 2022 at 03:46 PM

@Bathnm I thought the original post was primarily referring to issues in Node-RED which is what is being addressed by @Dirk-Jan Faber . I can see that there may be an addition layer of problems for some other users.

I must also thank you for the tremendous work you have put in on the RPI

0 ·

xsilvergs bathnm commented · Oct 10, 2022 at 03:48 PM

I agree with your last paragraph, it's just finding a reliable method with Node-Red. I don't see that Node-Red can read the serial numbers allocated to Victron's Ve.Direct-USB dongles, am I right?

0 ·

Mike Dorsett xsilvergs commented · Oct 10, 2022 at 05:18 PM

It is not the serial number of the dongle that needs to be read, but the serial number of the attached device - which is a hex code register request - therefore Node Red (which I don't use) should be able to request this data as a message.

0 ·

Dirk-Jan Faber (Victron Energy) ♦♦ bathnm commented · Oct 10, 2022 at 06:12 PM

That is correct. The way node-red-contrib-victron handles it now is, just like other Venus software, by looking at the 'DeviceInstance' on the dbus.

0 ·

pcurtis bathnm commented · Oct 25, 2022 at 09:23 AM

@Bathnm ( @Dirk-Jan Faber ) Unfortunately the assertion thet "The underlying Venus OS system already ensure devices irrespective of the USB device map to a consistent VRM device instance. This is why no data is lost in VRM when a reboot occurs" may not always be true. I have recently changed from v2.80 large to v2.91 large on my Pi 3B+ and the instances have changed. This was carried out with a new microSD card plugged into the (unpowered) Pi without breaking any of the USB or other connections. Worse still one of the instance numbers was reused by a different device so I can see no way to rescue my longer term data sets on VRM. Unusually the node-RED allocations did not change!

See below a screenshot of part of my VRM device list which shows 289 being reused as the solar charger changes to 291from 289. Prior to the update the instances had been stable for 4000 hrs under OS v2.80.

Screenshot_20221025_095512.jpg

0 ·

screenshot-20221025-095512.jpg (209.2 KiB)

Answer 7 · 2022-10-25T09:49:09Z

bathnm answered · Oct 25, 2022 at 09:49 AM

@pcurtis There is a simple reason that they changed. The configuration on the v2.80 which mapped devices to VRM instance ID's would not have been taken into the new 2.91 system. That means that all devices are once again detected, and mapped to VRM instance ID's. This is all stored in the dBus and configuration files, so needs to be moved between systems to stop this from happening.

Victron at present does not have a means of handling this, but setup helper by @Kevin Windrem does.

Depending on when you did this, you could take the file /data/conf/settings.xml from the old 2.80 to the new 2.91

5

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pcurtis commented · Oct 25, 2022 at 10:52 AM

@Bathnm Thanks for the near instant answer. So I understand this only occurs with a fresh install, not a firmware update or reboot.

This was done about a week ago but I did not notice immediately as I was concentrating on Node-RED. I must have another look at setup helper by @Kevin Windrem or at adding saving it to my rc.local actions which set up my relays, restricted use of sudo etc.

1 ·

bathnm pcurtis commented · Oct 25, 2022 at 11:18 AM

@pcurtis Correct, only a fresh install. Pleas be careful about the file I referenced. If it was that simple then I am sure that Victron would have had a backup and restore by now! Not even setup helper seems to copy the file in it's entirety!

This link provides the set of data which setup helper will backup, so it is not all dBus data paths.

Some good background reading, if you are into customisation is this dbus-api reference. There is a section on "VRM Device instances" which is how the system ensures each device is unique and gets the same value irrespective of the tty used.

1 ·

pcurtis bathnm commented · Oct 25, 2022 at 12:25 PM

@Bathnm You are correct I have not been accessing dBus directly so far but you have started me thinking. Setup helper does not seem to backup any of the VE-Direct dBus paths at present so does not help. The chances are that Victron will have a backup and restore by the time the next release that requires a reinstall comes along and the Node-RED/VE Direct issue which is my big problem as even a reboot can break my system is being sorted by @Dirk-Jan Faber

0 ·

Kevin Windrem pcurtis commented · Oct 25, 2022 at 04:06 PM

Unfortunately, SetupHelper does not back up any parameters that are indexed in a way that may change between systems or from firmware version to version or based on device discovery order. So as Mark says, this won’t help in this situation.

Victron’s backup mechanism was tested then removed because of some unspecified problems. We will have to see if the Victron mechanism works for those settings.

1 ·

pcurtis Kevin Windrem commented · Oct 26, 2022 at 03:51 PM

@Kevin Windrem do you ever use rc.local as well as or instead of rcS.local in SetupHelper and associated packages - I would like to try it but I already use rc.local to give persistence to my Node-RED based system through reboots and simple firmware updates (or will do once the VE-direct devices stay in the same place!)

0 ·

Answer 8 · 2022-10-26T16:25:06Z

Kevin Windrem answered · Oct 26, 2022 at 04:25 PM

My code ADDS the reinstall… call to rcS.local so as long as SetupHelper setup runs after your additions to rcS.local then it should all work fine.

There is one line in EssentialResources that controls which rc file is used but you would need to change it after every SetupHelper update.

You can always use rc.local for more than one thing but I suggest launching any time consuming tasks in the background:

nohup foo &

That way the rc script continues to execute and returns to the caller rapidly.

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pcurtis commented · Oct 27, 2022 at 01:26 AM

@Kevin Windrem Yes, I had seen the option in EssentialResources hence the question as to whether it ever got changed. I already do many things including GPIO setup and enabling limited use of sudo and a few more permission changes because Node-RED no longer runs as root. Only one needs to execute in the background but I do also use background execution in some Exec Nodes.

I may need to rewrite my rc.local slightly to improve compatibility as it has two exit points depending on whether it is a firmware update or just a reboot.

0 ·

Answer 9 · 2022-11-07T11:53:04Z

xsilvergs answered · Nov 07, 2022 at 11:53 AM

Today I rebooted my RPi4 and once again Node-Red lost contact with my MPPT's with the message "disconnected", this is with Large Image 2.92.

If the Venus OS can detect devices, why can't the Victron Nodes, even if there was manual input for the VRM instance number (or what ever is required).

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 10 · 2022-11-21T09:01:50Z

pcurtis answered · Nov 21, 2022 at 09:01 AM

@Dirk-Jan Faber I have been using the various versions from your issue-144 branch for several weeks and have done dozens of test reboots and a few complete power off/on cycles and have never lost contact with the 4 Victron VE-Direct devices except after I have done 'updates' by copying in new versions of the `src/*` files to the `/usr/lib/node_modules/@victronenergy/node-red-contrib-victron/` directory, the mechanism you suggested. This is on a RPI 3B+ with 2.92 .Thank you for an excellent fix.

I note that your branch has now been merged into master - does that mean it is now in the new OS version 3.00~7 ? EDIT Changes are in v3.00~8

Do you know if there is a reason why the node-red-contrib-victron have lost the Load Current option for the Solar chargers ? I have asked the question in the modifications space and put up a crude work round using dbus as I need it but have since wondered if it was removed for a good reason and you seem to be the expert!

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

question