Cerbo keeps losing connection with CG EM340 Grid Meter

My systems suffers from losing connection with EM340 intermittently, going into passthru mode multiple times per hour. It happens mostly during charging (98% of the time) my 48kWh battery. Never when discharging. It started a few months ago (probably before v3.50?) The behavior is identical on the latest official release and the latest beta (3.60~24).

The meter is connected via an official Victron USB-485-converter.

I have tried:

  • Other USB-port
  • Taking cabling far apart from AC-cabling
  • Setting high value for Grid Set-Point
  • I checked: There are no other instances with VRM id 30
  • I double checked the wiring. Especially terminator wire on terminal 7/9
  • I checked if no defects were reported with CG configuration software, latest version

From the logs it looks like losing the grid meter is the result of another error, not caused by the gridmeter or the USB/485 connection if I am nog mistaken.

Does anybody have a clue?

root@victron:/data/log/dbus-modbus-client.ttyUSB0# tail -8 /data/log/dbus-modbus-client.ttyUSB0/current |  tai64nlocal 
2025-02-10 12:26:30.018023500 /opt/victronenergy/serial-starter/run-service.sh: line 5: shift: 2: shift count out of range
2025-02-10 12:26:32.506648500 INFO     dbus-modbus-client.py v1.62
2025-02-10 12:26:32.522456500 INFO     Waiting for localsettings
2025-02-10 12:26:32.562553500 INFO     Starting background scan
2025-02-10 12:26:32.565001500 INFO     Scanning ttyUSB0 @ 115200 bps (quick)
2025-02-10 12:26:39.267153500 INFO     Scanning ttyUSB0 @ 19200 bps (quick)
2025-02-10 12:26:45.974211500 INFO     Scanning ttyUSB0 @ 38400 bps (quick)
2025-02-10 12:26:52.689458500 INFO     Scan completed in 20 seconds
root@victron:/data/log/dbus-modbustcp# tail -7 /data/log/dbus-modbustcp/current | tai64nlocal 
2025-02-10 12:26:01.943783500 INFO  "[DBusService] Service offline: com.victronenergy.grid.cgwacs_ttyUSB0_mb1 (0)" 
2025-02-10 12:26:15.395556500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:26:30.412974500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:26:45.441220500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:27:00.383334500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:27:15.378395500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:27:15.385447500 INFO  "[DBusService] Service online: com.victronenergy.grid.cgwacs_ttyUSB0_mb1 (30)" 
root@victron:/data/log# tail -12 /data/log/dbus-cgwacs.ttyUSB0/current | tai64nlocal 
2025-02-10 12:00:52.080531500 WARN  Incorrect number of registers received 6 2 1 
2025-02-10 12:27:08.307656500 *** starting dbus-cgwacs ***
2025-02-10 12:27:08.370181500 INFO  dbus-cgwacs v2.0.24 started 
2025-02-10 12:27:08.370352500 INFO  Built with Qt 6.6.3 running on 6.6.3 
2025-02-10 12:27:08.370390500 INFO  Built on Feb  5 2025 at 16:31:58 
2025-02-10 12:27:08.370841500 INFO  Connecting to "/dev/ttyUSB0" 
2025-02-10 12:27:08.385086500 INFO  Wait for local settings on DBus...  
2025-02-10 12:27:08.391805500 INFO  Local settings found 
2025-02-10 12:27:08.479305500 INFO  Device ID: 341 
2025-02-10 12:27:09.727393500 INFO  Device found: "298156V" @ "/dev/ttyUSB0" 
2025-02-10 12:27:12.516758500 INFO  Registering service "com.victronenergy.grid.cgwacs_ttyUSB0_mb1" 
2025-02-10 12:27:12.550104500 [VeQItemDbusPublisher] Registered service "com.victronenergy.grid.cgwacs_ttyUSB0_mb1"
root@victron:/data/log# tail -9 /data/log/serial-starter/current | tai64nlocal 
2025-02-10 12:26:15.850649500 INFO: Start service vedirect-interface.ttyS5
2025-02-10 12:26:16.989450500 INFO: Start service vedirect-interface.ttyS6
2025-02-10 12:26:18.100858500 INFO: Start service vedirect-interface.ttyS7
2025-02-10 12:26:19.386717500 INFO: Start service dbus-fzsonick-48tl.ttyUSB0 once
2025-02-10 12:26:24.660911500 INFO: Start service dbus-imt-si-rs485tc.ttyUSB0 once
2025-02-10 12:26:29.998607500 INFO: Start service dbus-modbus-client.serial.ttyUSB0 once
2025-02-10 12:26:53.633139500 INFO: Start service gps-dbus.ttyUSB0 once
2025-02-10 12:27:03.010246500 INFO: Start service vedirect-interface.ttyUSB0 once
2025-02-10 12:27:08.296246500 INFO: Start service dbus-cgwacs.ttyUSB0 once
root@victron:/data/log# tail -3 /data/log/dbus-systemcalc-py/current | tai64nlocal 
2025-02-10 12:26:02.032315500 INFO:dbusmonitor:com.victronenergy.grid.cgwacs_ttyUSB0_mb1 disappeared from the dbus. Removing it from our lists
2025-02-10 12:27:12.565569500 INFO:dbusmonitor:Found: com.victronenergy.grid.cgwacs_ttyUSB0_mb1, scanning and storing items
2025-02-10 12:27:12.660022500 INFO:dbusmonitor:       com.victronenergy.grid.cgwacs_ttyUSB0_mb1 has device instance 30

All log lines from all logs sorted in the right order during loss of connection and shortly before the loss. The question is, what process is calling the serial stopper and why. No clue in the logs as far as I can see.

root@victron:/data/log# grep 4000000067a9f0 -rh * | sort | tai64nlocal | tail -136 | head -n 128
2025-02-10 12:25:58.774813500 INFO:__main__:Ping received, sending pong back
2025-02-10 12:25:58.775633500 INFO:__main__:Starting ping response
2025-02-10 12:25:58.778047500 INFO:__main__:Finished ping response after 1 messages
2025-02-10 12:26:01.069208500 warning No functional TLS backend was found
2025-02-10 12:26:01.071131500 warning No functional TLS backend was found
2025-02-10 12:26:01.072628500 warning No functional TLS backend was found
2025-02-10 12:26:01.074205500 warning No functional TLS backend was found
2025-02-10 12:26:01.075724500 warning No functional TLS backend was found
2025-02-10 12:26:01.779097500 INFO:__main__:device-list-2 received, starting scan for devices
2025-02-10 12:26:01.780163500 INFO:__main__:Starting device-list-2 response
2025-02-10 12:26:01.792589500 INFO:serialstopper:Calling /opt/victronenergy/serial-starter/stop-tty.sh ttyUSB0 ttyS7 ttyS6 ttyS5
2025-02-10 12:26:01.943783500 INFO  "[DBusService] Service offline: com.victronenergy.grid.cgwacs_ttyUSB0_mb1 (0)" 
2025-02-10 12:26:01.944035500 Grid meter @ "com.victronenergy.grid.cgwacs_ttyUSB0_mb1" disappeared.
2025-02-10 12:26:01.947953500 MainThread-dbusmonitor: com.victronenergy.grid.cgwacs_ttyUSB0_mb1 disappeared from the dbus. Removing it from our lists
2025-02-10 12:26:02.032315500 INFO:dbusmonitor:com.victronenergy.grid.cgwacs_ttyUSB0_mb1 disappeared from the dbus. Removing it from our lists
2025-02-10 12:26:02.338841500 INFO:swupdate:calling ['/opt/victronenergy/swupdate-scripts/check-updates.sh', '-check']
2025-02-10 12:26:02.497410500 *** Checking for updates ***
2025-02-10 12:26:02.497418500 arguments: -check
2025-02-10 12:26:02.497598500 *** Checking for updates ***
2025-02-10 12:26:02.497601500 arguments: -check
2025-02-10 12:26:02.619528500 Retrieving latest version... (from https://updates.victronenergy.com/feeds/venus/candidate/images/einstein/venus-swu-large-einstein.swu)
2025-02-10 12:26:02.619705500 Retrieving latest version... (from https://updates.victronenergy.com/feeds/venus/candidate/images/einstein/venus-swu-large-einstein.swu)
2025-02-10 12:26:02.933147500 installed: 20250205160352 v3.60~24
2025-02-10 12:26:02.933156500 available: 20250205160352 v3.60~24
2025-02-10 12:26:02.933335500 installed: 20250205160352 v3.60~24
2025-02-10 12:26:02.933340500 available: 20250205160352 v3.60~24
2025-02-10 12:26:02.934665500 No newer version available, exit.
2025-02-10 12:26:02.949270500 No newer version available, exit.
2025-02-10 12:26:05.966060500 INFO:__main__:dispatching to plugin: zyc.device_list()
2025-02-10 12:26:06.399946500 INFO:streamcommand:Command ('vup', '--xml', '--no-local', '--canbus', 'socketcan:can0') terminated with status 0
2025-02-10 12:26:08.419413500 INFO:streamcommand:Command ('xupt', '--xml') terminated with status 89
2025-02-10 12:26:12.566679500 INFO:streamcommand:Command ('dup', '--xml', '-s', '/dev/ttyUSB0 /dev/ttyS7 /dev/ttyS6 /dev/ttyS5') terminated with status 3
2025-02-10 12:26:15.084923500 WARNING:thirdpartyupdater:"/opt/victronenergy/mqtt-rpc/thirdparty/zyc/bms-zyc -l" returned abnormal exit code 255
2025-02-10 12:26:15.124183500 INFO:serialstopper:Calling /opt/victronenergy/serial-starter/start-tty.sh ttyUSB0 ttyS7 ttyS6 ttyS5
2025-02-10 12:26:15.366390500 INFO:__main__:Finished device-list-2 response after 24 messages
2025-02-10 12:26:15.395556500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:26:15.850649500 INFO: Start service vedirect-interface.ttyS5
2025-02-10 12:26:15.888856500 *** starting vedirect-interface ***
2025-02-10 12:26:16.052260500 vedirect-dbus 3.84
2025-02-10 12:26:16.989450500 INFO: Start service vedirect-interface.ttyS6
2025-02-10 12:26:17.001276500 *** starting vedirect-interface ***
2025-02-10 12:26:17.047651500 vedirect-dbus 3.84
2025-02-10 12:26:18.100858500 INFO: Start service vedirect-interface.ttyS7
2025-02-10 12:26:18.131519500 *** starting vedirect-interface ***
2025-02-10 12:26:18.197364500 vedirect-dbus 3.84
2025-02-10 12:26:19.386717500 INFO: Start service dbus-fzsonick-48tl.ttyUSB0 once
2025-02-10 12:26:19.397867500 *** starting dbus-fzsonick-48tl ***
2025-02-10 12:26:21.528617500 INFO:root:starting /opt/victronenergy/dbus-fzsonick-48tl/dbus-fzsonick-48tl.py
2025-02-10 12:26:21.530685500 INFO:root:identifying battery...
2025-02-10 12:26:21.658011500 INFO:root:failed to identify battery at 1 : failed to get slave id from 1 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:21.658964500 INFO:root:identifying battery...
2025-02-10 12:26:21.791434500 INFO:root:failed to identify battery at 2 : failed to get slave id from 2 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:21.792305500 INFO:root:identifying battery...
2025-02-10 12:26:21.916387500 INFO:root:failed to identify battery at 3 : failed to get slave id from 3 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:21.917152500 INFO:root:identifying battery...
2025-02-10 12:26:22.040382500 INFO:root:failed to identify battery at 4 : failed to get slave id from 4 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.041824500 INFO:root:identifying battery...
2025-02-10 12:26:22.167515500 INFO:root:failed to identify battery at 5 : failed to get slave id from 5 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.168262500 INFO:root:identifying battery...
2025-02-10 12:26:22.292564500 INFO:root:failed to identify battery at 6 : failed to get slave id from 6 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.293288500 INFO:root:identifying battery...
2025-02-10 12:26:22.419218500 INFO:root:failed to identify battery at 7 : failed to get slave id from 7 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.420202500 INFO:root:identifying battery...
2025-02-10 12:26:22.544514500 INFO:root:failed to identify battery at 8 : failed to get slave id from 8 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.545223500 INFO:root:identifying battery...
2025-02-10 12:26:22.679649500 INFO:root:failed to identify battery at 9 : failed to get slave id from 9 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.680372500 INFO:root:identifying battery...
2025-02-10 12:26:22.804619500 INFO:root:failed to identify battery at 10 : failed to get slave id from 10 : Modbus Error: [Input/Output] Modbus Error: [Invalid Message] Incomplete message received, expected at least 2 bytes (0 received)
2025-02-10 12:26:22.805446500 INFO:root:found 0 batteries
2025-02-10 12:26:24.660911500 INFO: Start service dbus-imt-si-rs485tc.ttyUSB0 once
2025-02-10 12:26:24.682901500 *** starting dbus-imt-si-rs485tc ***
2025-02-10 12:26:26.792734500 INFO:main:starting dbus-imt-si-rs485tc.py
2025-02-10 12:26:26.794733500 INFO:imt_si_rs485_sensor:searching for IMT SI-RS485 sensor on /dev/ttyUSB0
2025-02-10 12:26:27.822861500 ERROR:main:no IMT SI-RS485 sensor found
2025-02-10 12:26:27.823919500 INFO:main:dbus-imt-si-rs485tc.py has shut down
2025-02-10 12:26:29.998607500 INFO: Start service dbus-modbus-client.serial.ttyUSB0 once
2025-02-10 12:26:30.018023500 /opt/victronenergy/serial-starter/run-service.sh: line 5: shift: 2: shift count out of range
2025-02-10 12:26:30.412974500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:26:32.506648500 INFO     dbus-modbus-client.py v1.62
2025-02-10 12:26:32.522456500 INFO     Waiting for localsettings
2025-02-10 12:26:32.562553500 INFO     Starting background scan
2025-02-10 12:26:32.565001500 INFO     Scanning ttyUSB0 @ 115200 bps (quick)
2025-02-10 12:26:39.267153500 INFO     Scanning ttyUSB0 @ 19200 bps (quick)
2025-02-10 12:26:45.441220500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:26:45.974211500 INFO     Scanning ttyUSB0 @ 38400 bps (quick)
2025-02-10 12:26:52.689458500 INFO     Scan completed in 20 seconds
2025-02-10 12:26:53.633139500 INFO: Start service gps-dbus.ttyUSB0 once
2025-02-10 12:26:53.643752500 *** starting gps-dbus ***
2025-02-10 12:26:53.697615500 gps_dbus 1.07
2025-02-10 12:26:55.695881500 001.998 ERR.VALUES: timeout connecting device
2025-02-10 12:26:55.708867500 gps_dbus 1.07
2025-02-10 12:26:57.709671500 002.001 ERR.VALUES: timeout connecting device
2025-02-10 12:26:57.722897500 gps_dbus 1.07
2025-02-10 12:26:59.725766500 002.003 ERR.VALUES: timeout connecting device
2025-02-10 12:26:59.737074500 gps_dbus 1.07
2025-02-10 12:27:00.383334500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:27:01.739887500 002.003 ERR.VALUES: timeout connecting device
2025-02-10 12:27:03.010246500 INFO: Start service vedirect-interface.ttyUSB0 once
2025-02-10 12:27:03.022286500 *** starting vedirect-interface ***
2025-02-10 12:27:03.069285500 vedirect-dbus 3.84
2025-02-10 12:27:06.066762500 
2025-02-10 12:27:06.066769500 -------- dumping backlog -------
2025-02-10 12:27:06.066772500 000.045 INF.task: using ttyUSB0 as unique service part
2025-02-10 12:27:06.066775500 000.045 INF.VE.Direct: Setting status to VdUnknown
2025-02-10 12:27:06.066777500 000.060 INF.settings: connected to settings on dbus (com.victronenergy.settings)
2025-02-10 12:27:06.066781500 002.997 ERR.values: timeout connecting device
2025-02-10 12:27:06.066947500 002.997 INF.task: done in 002.997
2025-02-10 12:27:08.296246500 INFO: Start service dbus-cgwacs.ttyUSB0 once
2025-02-10 12:27:08.307656500 *** starting dbus-cgwacs ***
2025-02-10 12:27:08.370181500 INFO  dbus-cgwacs v2.0.24 started 
2025-02-10 12:27:08.370352500 INFO  Built with Qt 6.6.3 running on 6.6.3 
2025-02-10 12:27:08.370390500 INFO  Built on Feb  5 2025 at 16:31:58 
2025-02-10 12:27:08.370841500 INFO  Connecting to "/dev/ttyUSB0" 
2025-02-10 12:27:08.385086500 INFO  Wait for local settings on DBus...  
2025-02-10 12:27:08.391805500 INFO  Local settings found 
2025-02-10 12:27:08.479305500 INFO  Device ID: 341 
2025-02-10 12:27:09.727393500 INFO  Device found: "298156V" @ "/dev/ttyUSB0" 
2025-02-10 12:27:12.516758500 INFO  Registering service "com.victronenergy.grid.cgwacs_ttyUSB0_mb1" 
2025-02-10 12:27:12.538143500 Grid meter found @ "com.victronenergy.grid.cgwacs_ttyUSB0_mb1"
2025-02-10 12:27:12.543537500 MainThread-dbusmonitor: Found: com.victronenergy.grid.cgwacs_ttyUSB0_mb1, scanning and storing items
2025-02-10 12:27:12.550104500 [VeQItemDbusPublisher] Registered service "com.victronenergy.grid.cgwacs_ttyUSB0_mb1"
2025-02-10 12:27:12.565569500 INFO:dbusmonitor:Found: com.victronenergy.grid.cgwacs_ttyUSB0_mb1, scanning and storing items
2025-02-10 12:27:12.626732500 MainThread-dbusmonitor:        com.victronenergy.grid.cgwacs_ttyUSB0_mb1 has device instance 30
2025-02-10 12:27:12.660022500 INFO:dbusmonitor:       com.victronenergy.grid.cgwacs_ttyUSB0_mb1 has device instance 30
2025-02-10 12:27:15.378395500 ERROR "Error processing function code 3, unit id 30, start address 2600, quantity 38, src ::ffff:192.168.178.20: Error finding service with device type grid at device instance 30" 
2025-02-10 12:27:15.385447500 INFO  "[DBusService] Service online: com.victronenergy.grid.cgwacs_ttyUSB0_mb1 (30)" 
2025-02-10 12:27:17.067000500 warning No functional TLS backend was found
2025-02-10 12:27:17.068025500 warning No functional TLS backend was found

In the settings.xml I find device 30 two times. One time as grid meter (correct) and one time as pv-inverter-meter (incorrect) I don“t know if this could be a problem?

    <cgwacs_298156V>
      <ClassAndVrmInstance type="s" default="grid:30" silent="False">grid:30</ClassAndVrmInstance>
      <CustomName type="s" default="" silent="False"></CustomName>
      <IsMultiphase type="i" default="1" silent="False">1</IsMultiphase>
      <L1ReverseEnergy type="f" min="0.0" max="1000000.0" default="0.0" silent="True">5368.861970761242</L1ReverseEnergy>
      <L2ReverseEnergy type="f" min="0.0" max="1000000.0" default="0.0" silent="True">5349.338392120267</L2ReverseEnergy>
      <L3ReverseEnergy type="f" min="0.0" max="1000000.0" default="0.0" silent="True">5398.405303954375</L3ReverseEnergy>
      <Position type="i" default="0" silent="False">0</Position>
      <SupportMultiphase type="i" default="1" silent="False">1</SupportMultiphase>
    </cgwacs_298156V>
    <cgwacs_298156V_S>
      <ClassAndVrmInstance type="s" default="pvinverter:30" silent="False">pvinverter:30</ClassAndVrmInstance>
      <CustomName type="s" default="" silent="False"></CustomName>
      <Enabled type="i" default="0" silent="False">0</Enabled>
      <Position type="i" default="0" silent="False">0</Position>
    </cgwacs_298156V_S>