Why do Cerbo CX devices fail to get Internet connectivity from Peplink Routers?

Question

question

Martin Silbernagl asked Nov 29, '23

Why do Cerbo CX devices fail to get Internet connectivity from Peplink Routers?

There is a problem I have been having for months now and I cannot figure out how to fix it:

I have a Cerbo GX that is running a clean install of VenusOS Large v3.20~30 (but same problem on v3.12 small). It works as expected on my home network (both via wired Ethernet or WiFi) or via my mobile phone: it connects to the Internet for the VRM online portal or firmware updates without fail.

But when connected to a Peplink MAX BR1 Pro 5G or Peplink MAX BR2 Pro (with Firmware 8.3.0 or 8.4.0), Internet connectivity fails in strange ways:

As expected, the Cerbo successfully obtains a DHCP lease from the Peplink and will then ask it to resolve ipv4.connman.net for connman's connectivity check. But after the Peplink returns the correct result, the Cerbo makes no attempt at sending an HTTP request to that address or any other remote IP:

Here is the full package capture in a Wireshark compatible format: lan.pcap.zip

What sticks out in the log is this message:

daemon.warn connmand[857]: Online check failed for 0x118900 Wired

So, does connman tears down the connection, because it couldn't verify Internet connectivity?

But why don't we see an HTTP GET request from the Cerbo GX?

Full log: messages-log.txt

In the display, Ethernet shows as "Connected" with expected IP address, netmask, gateway and DNS server.

In the VRM online portal page, I get an #153 Connection error:

<urllib3.connection.VerifiedHTTPSConnection object at 0xb5a04688> Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

This also happens without HTTPS (with a slightly different error message) or two-way communications.

This will also happen when using WiFi instead of wired Ethernet to connect the two.

The Simrad Chartplotters and other will happily find the Cerbo via Bonjour and connect to it via the link-local address, but on the assigned LAN address, the Cerbo is silent (including SSH).

The Peplinks work great for all other devices connected to it (Laptops, Phones, Starlink, NMEA2k Brige, Chartplotter, etc.) and are gernerally absolutely awsome devices (I get well over 1Gbps throughput on the boat when the 5G Modems and Starlink are at full bandwith). I would not even know what to replace these devices with.

Others may have this problem, too.

To escalate this, I have ordered a USB-to-TTL debug header, so I can connect to the Cerbos GX's console while it is wired to the Peplink.

In the meantime: any ideas anyone? Pretty please?

TCP IP Networking

screenshot-package-capture.png (409.4 KiB)

img-3430.jpeg (1.2 MiB)

lanpcap.zip (2.7 KiB)

messages-log.txt (3.5 KiB)

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Modifications Space

Node Red Space

Answer 1 · 2023-11-29T13:46:27Z

djsmiley answered · Nov 29, 2023 at 01:46 PM

I would try setting a fixed IP adress and an external DNS server (Eg google 8.8.8.8 or Cloudflares 1.1.1.1) or assign those DNS servers using dhcp.
Does the DHCP server provide any other options which might interfere? (thats why I would start with a fixed IP).
I do see a DHCP option 138 assigned, altough I would expect the Cerbo to just ignore this.

Which MTU? MTU issues can also cause similar problems.

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Martin Silbernagl commented · Nov 30, 2023 at 10:23 AM

Thank you for your advice DJ Smiley!

Nice catch with the DHCP Option 138. Peplink routers use it to adopt access points. I will try turning it off.

I had already tried with a fixed IP and known good DNS servers (and adding pertinent lookups to /etc/hosts). I am sorry I didn't mention this.

On the 5G connection, the MTU is set to 1428, which is just below fragmentation on the mobile networks I use (default on Peplink devices is 1440). But if MTU was a problem, the packet capture should still show the Cerbo's attempt to connect to ipv4.connman.net, no? And, if MTU was a problem, wouldn't the other devices also be unable to communicate?

0 ·

Answer 2 · 2023-12-10T18:29:10Z

Martin Silbernagl answered · Dec 10, 2023 at 06:29 PM

So, I bit the bullet and got a USB to TTL serial cable...

...which, after a couple of hours ultimately led me to the answer:

The build-in WiFi access point of the Cerbo GX uses the 172.24.24.0/24 range, which, as luck would have it, was the same sub-net that I had selected for the LAN

It would have been nice to see this documented somewhere in the manual. Or for someone from support to look at this post here (since the only way to get support is through retailers, who mostly have no idea of TCP/IP networking at depth).

For those who come after me and also are on MacOS: beyond the info at https://www.victronenergy.com/live/ccgx:root_access you need to know that to connect from a MacBook to the serial console, you run...

screen /dev/cu.usbserial-xxxREPLACExxx 115200,cs8

Beforehand, you find the correct serial device by grep-ing over the list of all devices, like so:

ls /dev/ | grep cu.usbserial

img-3503.jpeg (6.7 MiB)

screenshot-2023-12-10-at-191129.png (15.2 KiB)

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

question