Network bridge eth0-wifi0 on Venus OS - Cerbo GX / Einstein

Question

question

huit asked Apr 7, '21

Network bridge eth0-wifi0 on Venus OS - Cerbo GX / Einstein

I have a cerbo gx connected to a router over ethernet and want to allow devices connected to the wireless AP on the cerbo to access the internet.

I attempted to create a bridge intending to add each of the eth0 and wifi0 network devices to it, e.g as root on cerbo:

ip link add br0 type bridge
ip link set wifi0 master br0
ip link set eth0 master br0
ip link set br0 up

But adding the bridge in the first step raises an error:

Error: Unknown device type

The bridge type is listed in the help page and this same command works on most linux systems that I am familiar with

ip link help

.. so I might guess that this is some particularity with the way that ip link is compiled for Venus OS. I thought it might be customised to support CAN/ve.bus etc.

My other thought is to check if selinux might be blocking.

Does anybody have advice or cautions about going in this direction?

wifi ethernet linux

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Warwick Bruce Chapman commented · Apr 08, 2021 at 08:11 PM

Agree this would be useful.

0 ·

mvader (Victron Energy) answered · Apr 08, 2021 at 04:45 AM

Hi, we dont do anything Victron specific to the ip command.

the thing I’d check first is if maybe some kernel config option is disabled.

The sunxi recipe here defines location of sources as well as version of the kernel for the cerbo:

https://github.com/victronenergy/meta-victronenergy/tree/master/meta-bsp/recipes-kernel/linux

Make sure to select the v2.65 tag if you are working with venus os v2.65.

If thats the issue, you’ll have to recompile the kernel

17

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

huit commented · Apr 08, 2021 at 01:45 PM

I am running 2.7 large which isn't tagged in your link so I took a look at the kernel config directly:

zcat /proc/config.gz

shows that indeed

#CONFIG_BRIDGE is not set

I think this is required so a recompile is probably necessary, which will take a while for me to figure out. I will accept this answer and post back if I make any progress.

Thanks for your help.

0 ·

mvader (Victron Energy) ♦♦ huit commented · Apr 08, 2021 at 01:47 PM

hi, ok then see the venus os large change notes which kernel is being used, I'm pretty sure its a 5.10 version.

0 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 08, 2021 at 11:07 PM

Thanks again, I found your post for the change notes. For my current v2.70~4-large-16 it's kernel version 5.10.21.

I pulled the source for 5.10.26 from github and found config files in /linux/arch/arm/configs, including one recently added, sunxi_victron_defconfig that I guess is the one used. I also found references in some of the other ARM config files where CONFIG_BRIDGE is enabled as a module.

I will next add the line for CONFIG_BRIDGE=m to sunxi_victron_defconfig and compile. I just need to read a bit about how to set up for this.

0 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 09, 2021 at 11:36 AM

I did the following on an ubuntu 18.04 system to recompile 5.10.26. Some of the first line was already installed but just for documentation I have listed them here. This is all run from the root of the source directory (linux).

sudo apt-get install fakeroot build-essential ncurses-dev xz-utils libssl-dev bc flex libelf-dev bison

sudo apt install crossbuild-essential-armhf

export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabihf-

make sunxi_victron_defconfig
make menuconfig

-> Networking support (e)
  -> Networking options (e)
     <*> 802.1d Ethernet Bridging (d)

save as .config and exit

make -j6 zImage modules dtbs
make modules_install INSTALL_MOD_PATH=arch/arm/boot/modules

That all compiled without any warnings or errors and created files in arch/arm/boot.

The next step is copying this over to the cerbo.

I am a little wary not having replaced the kernel on this device before. Any advice on safe procedures or ideally a way to test booting with this modified kernel would be appreciated.

0 ·

huit huit commented · Apr 09, 2021 at 01:55 PM

oh no...

I copied over the new kernel and now can't access the cerbo

The first time I just overwrote the zImage file and put the headers and modules too. It failed to set up the hotspot but I could still connect over the vrm portal and reloaded the swu.

I thought I would try again with the original (5.10.21) headers and modules (because I clearly don't know what I am doing) and just copied the zImage to boot. This time though I called it zImage.custom and just created a symlink to it.

Now on reboot I am locked out completely.

Can I connect an hdmi monitor and keyboard to access this thing now?

0 ·

mvader (Victron Energy) ♦♦ huit commented · Apr 09, 2021 at 02:09 PM

Hi Huit, an accident waiting to happen a little bit :).

But don't worry, its fixable, but might be a bit different than you're expecting. Its not like a rpi where you can connect a display and a keyboard.

I think your best solution is to connect with a serial console cable. See the root access document: https://www.victronenergy.com/live/ccgx:root_access. It explains what cable and how to connect.

Once connected, boot the unit up, press a key to stop the autoboot; And then you'll need to change the active partition, so make it boot in the backup partition.

There are two rootfs partitions, each with their own kernel copy.

The alternative is to run the installer image, https://updates.victronenergy.com/feeds/venus/release/images/einstein, but, then you'll probably loose your wifi password as well as other things, and it might not even boot normally; I can't remember. So, if you're going to work on kernels and such, you'll need the serial console cable anyway, so I recommend to go that.

1 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 09, 2021 at 02:26 PM

Thanks for the fast reply! Yeh.. you are right about the accident waiting to happen. I will order a cable online and have a look into the installer image.

0 ·

mvader (Victron Energy) ♦♦ huit commented · Apr 09, 2021 at 03:06 PM

Hi, beware: running the installer image, ie booting with that inserted, wipes the data partition without any warning.

0 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 09, 2021 at 03:10 PM

At this stage I will be happy to recover the system. I only really have the ssh keys and some signalk stuff set up on the data partition that are easily replaced. Thanks for the warning though.

0 ·

huit huit commented · Apr 15, 2021 at 02:40 PM

With the serial cable and tty console I was able to boot into a functioning system, restore the original kernel and reboot. This recovered the system without wiping anything so I didn't have to reconfigure everything.

As such, I didn't need to switch to the backup partition, although I was wondering how to go about that. When I pressed a key to interrupt the boot it wasn't immediately obvious what commands I could run. Was that like a grub environment or something?

At least now with tty I can try to work out what is breaking when I boot into the modified kernel. Thanks for the guidance.

0 ·

mvader (Victron Energy) ♦♦ huit commented · Apr 15, 2021 at 09:27 PM

Hurray! Nice that its restored.

the environment you’re in after booting is uboot.

It has a list of variables. Called env or the environment.

the one you’re after is called version.

and after changing it you need to do save env or something. And then I normally just reboot the unit since I never remember what command to use to make it start the kernel. :-)

0 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 19, 2021 at 08:58 AM

Progress update...

I moved over to venus OS 2.66 to work this out and recompiled 4.19.123 as above, although this time I needed to install more gcc compilation stuff

sudo apt-get install gcc-arm*

I then copied over the kernel into /boot (renamed 4.19.123.custom) and replaced the kernel symlink in /boot to link to the modified kernel.

I also replaced the modules folder /lib/modules/4.19.123 with the newly built folder. On a diff of find in these directories we now have the bridge module as expected but are missing a few wifi drivers and a video driver.

--- original_modules.txt
+++ modified_modules.txt
-./rtl8723du 
-./rtl8723du/8723du.ko 
-./8192eu.ko  
-./video-./video/mali.ko 
-./kernel/net/wireless/8723bu.ko
+./kernel/net/bridge
+./kernel/net/bridge/br_netfilter.ko

I copied these over, and looking at the boot log of the original 4.19.123 i noticed that 8723bu.ko is being used

rtl8723bu 5-1.1:1.2 ap0: renamed from wlan1
rtl8723bu 5-1.1:1.2 disabled-wlan0: renamed from wlan0So

So I enabled this one and rebooted

depmod
modprobe 8723bu
[  147.268367] 8723bu: loading out-of-tree module taints kernel.
[  147.351445] usbcore: registered new interface driver rtl8723bu
[  147.416480] rtl8723bu 5-1.1:1.2 ap0: renamed from wlan1
[  147.451111] rtl8723bu 5-1.1:1.2 disabled-wlan0: renamed from wlan0
reboot

Now ap0 is up and I can see the wifi, but the ssid has changed. So I went onto vrm online and set the password (ethernet connection to internet is still up), but I still can't connect to the wireless access point... I will have to try to get some logs to see why the connection is failing.

----

Plans of attack:

- I wonder if I need to compile the 8723bu driver in the context of the bridge module somehow.. Where are the wifi drivers and video driver coming from in the image install?

- I found hostapd-run that seems to set up the access point, but it depends on ap0 already being configured. I am wondering where that is set up. Where is the ap0 configuration, ssid, password coming from?

- I read something about the access point configuration using dnsmasq and hostapd (after abandoning connmanctl) so might have to dig into the dnsmasq configuration to understand what is going on. Is this just failing because of some dhcp issue? Why is this failing on my system?

- I had tried with connmanctl before I found the missing wifi driver. I was able to power on the wifi but tethering didn't work. Maybe I should revisit this now that the driver is installed. Need to understand rfkill as that was raised as blocking the device when activating using connmanctl. Managed to unblock but not sure why it is raised in the first place.

- I haven't enabled the video driver but doubt this is a concern here.

- I might try the other network drivers but doubt that these are the issue here since the 8723bu is running in the original installation. I wonder if multiple might be enabled, but this seems unlikely and there is no mention of the others in the boot log on the original 2.66.

0 ·

huit huit commented · Apr 19, 2021 at 07:53 PM

So, the ssid and password (wpa_psk) are stored in /data/venus.

The wifi driver is working somewhat, I can connect to another wifi network. I just can't authenticate into the access point.

The driver does crash also in a kernel panic when I attempt to log in to it a few times. Thinking more that this is now just a problem with the wifi driver that I copied. Not sure if kernel modules are considered in compiling the driver?

0 ·

mvader (Victron Energy) ♦♦ huit commented · Apr 19, 2021 at 08:16 PM

hey, if I was doing what you are, I would try and build the kernel the exact same way we do, but then with that flag enabled. Now, I don't know if it works, I'm just pasting here what I'd try first if I were you:

To do that, try:

git clone https://github.com/victronenergy/venus
cd ./venus
git checkout matthijs/fixes
make fetch
make einstein-bb

now you're in a new shell, ready to build the kernel according to v2.70~5, ie not venus os large, and thus not version 5.10. But that should't matter anyway and lets keep it as simple as possible first time around

Now in that shell, you need to run:

bitbake linux-sunxi

At least, thats what I think you should run :).

Now everything will take a while. The OE (=OpenEmbedded) way is to build everything from source, including cross compilers and so forth.

Could be hours.

Once that first build is succeeded, next you need to figure out how to change that config file. Which ehm, is a bit hard to explain. The best I can think of is to clone our linux repo (which you've already done), make the change you need; and thereafter change the kernel recipe in ./venus/sources/meta-victronenergy/meta-bsp/recipes-kernel/linux/linux-sunxi_4.19.123-sunxi5.bb, to have your repo on disk as the src-uri, rather than the repo online.

The manual for what you're then working in is here: https://www.yoctoproject.org/docs/3.0.4/mega-manual/mega-manual.html.

Hope that helps. Could also be that you're already close going your route; and as usual there are tons of different ways to achieve the same. :).

0 ·

mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented · Apr 19, 2021 at 08:16 PM

ps build results will be in ./venus/deploy/ x y z

0 ·

huit mvader (Victron Energy) ♦♦ commented · Apr 19, 2021 at 11:33 PM

Wow thanks! I will give it a try tomorrow and report back.

0 ·

huit huit commented · Apr 20, 2021 at 10:51 PM

After following your guidance to set up and as suggested I modified the bitbake recipe, just the SRC_URI line

SRC_URI = "file:///path/to/custom/tarball/v${PV}.tar.gz"

then back in bitbake terminal ran

bitbake linux-sunxi

And it compiled happily (not on my first try, see below for how I prepared the modified kernel tar ball).

I then started with a fresh venus 2.7 install and copied the kernel image and dtb file into /boot. (linking them to make sure they are used).

For the modules i just copied (merged) over the contents from the bitbake modules tar ball into the /lib/modules/4.19.123 path. Leaving all the existing modules there.

I then rebooted and was able to add a bridge device!! but then got a kernel panic after a few attempts to set br0 up and down... still work to do..

If there is a simpler/better way to use the contents of deploy it would be good to know. I am half hoping it is an issue with this that is causing the kernel panic and not a lack of driver support or something.

-----

So, just to be thorough in documenting things, here is modifying the kernel config from scratch to prepare the tarball:

git clone https://github.com/victronenergy/linux
cd linux
git checkout sunxi_b4.19.123
git fetch
export ARCH=arm
export CROSS_COMPILE=arm-linux-gnueabihf-  make sunxi_victron_defconfig
make menuconfig

-> Networking support (e) 
  -> Networking options (e)        
   <*> 802.1d Ethernet Bridging (d)

save as .config and exit

make savedefconfig
cp defconfig arch/arm/configs/sunxi_victron_defconfig
make mrproper

the mr proper step was needed as bitbake compiler complained later if not done. I probably should have just manually edited the defconfig file. It would have been much simpler.

Anyway, next I needed to create a tar ball for bitbake to import. I tried just linking the folder but it seems to expect the tarball. also needed to rename the dir before archiving so that pathing worked.

cd ..
mv linux linux-4.19.123-sunxi5
tar -czf v4.19.123-sunxi5.tar.gz --exclude-vcs linux-4.19.123-sunxi5

exclude-vcs option was to avoid including all the git tracking and I guess the other branches

0 ·

Modifications Space

Node Red Space

Answer 1 · 2021-09-28T04:09:32Z

dakoal answered · Sep 28, 2021 at 04:09 AM

Hello.

I am wondering if someone got the bridge working, because I am trying the same.

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 2 · 2022-03-29T13:57:49Z

mawi answered · Mar 29, 2022 at 01:57 PM

Hi,

actually I will be not able to make such a change myself so I am wondering if that could become a official firmware feature. I believe many user would appreciate it since installing an other HotSpot where there is such a nice device which is capable to do is much smarter.
In my case the Victron is in the garage connected with ethernet where my wall box has Wi-Fi and my router is too far - so the cerbo GX would be a smart solution for me to bridge the wall box to the internet.

Thanks

Matthias

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 3 · 2022-08-17T17:43:39Z

ted answered · Aug 17, 2022 at 05:43 PM

Bridge would be useful. Wifi Connect and EM24 to the Cerbo Ethernet Port should be possible

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 4 · 2022-08-18T04:28:19Z

johanndo answered · Aug 18, 2022 at 04:28 AM

You can use iptables and nat to route the traffic from LAN to WiFi or just route the traffic through. You must run the WiFi in AP as hotspot (install and configure hostapd)

1

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Petr Pospíšil commented · Sep 12, 2022 at 09:28 AM

Have you tested it? Does it really works?

0 ·

Answer 5 · 2023-01-26T19:36:16Z

blacksun answered · Jan 26, 2023 at 07:36 PM

Hello, is it already possible to create a bridge?

Installed a Waveshare RJ45 HAT on my Raspberry PI to get another RJ45 interface.

ll-eth0   Link encap:Ethernet  HWaddr 82:B7:22:E7:67:EB
          inet addr:169.254.10.228  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::80b7:22ff:fee7:67eb/64 Scope:Link
          inet6 addr: 2003:ca:af16:b800:80b7:22ff:fee7:67eb/64 Scope:Global
          UP BROADCAST RUNNING MULTICAST  MTU:1450  Metric:1
          RX packets:6215 errors:0 dropped:64 overruns:0 frame:0
          TX packets:102 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:968838 (946.1 KiB)  TX bytes:24637 (24.0 KiB)

But as soon as I want to add eth0 to the bridge I get this error message.

Using username "root".
root@192.168.178.40's password:
Last login: Thu Jan 26 18:25:33 2023
root@raspberrypi2:~# ip link add br0 type bridge
root@raspberrypi2:~# ip link set ll-eth0 master br0
root@raspberrypi2:~# ip link set eth0 master br0
RTNETLINK answers: Device or resource busy
root@raspberrypi2:~#

Is the bridge not supported by VenusOS?

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 6 · 2024-01-31T18:07:31Z

I got NAT working on cerbo. All is done on roots filesystem due to strict requirement between .ko files and kernel version, so it won't preserve during system update. If you will put this on /data partition, most likely it will brick cerbo, so you will have to perform recovery

If victron will add NAT support to their kernel defconfig, iptables and forwarding setup could be moved into script on /data partition, and will preserve during system update

On your host machine

git clone https://github.com/victronenergy/linux
cd linux
git checkout BRANCH(issue command "uname -a" on cerbo to get branch version name, for me "Linux cerbo 5.10.109-venus-13" branch_tag was v5.10.109-venus-13)

ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make sunxi_victron_defconfig
ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make menuconfig
-> General setup
 ->Local version
  -> "-venus-13"

->Networking support 
 -> Networking options 
  -> Network packet filtering framework (Netfilter) 
   -> IP: Netfilter Configuration
    -> set iptables NAT support
    -> MASQUERADE target support
    -> NETMAP target support
    -> REDIRECT target support

save and exit

ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- make modules

On cerbo

copy .ko files in "net/netfilter/", "net/ipv4/netfilter/", "net/ipv6/netfilter/" from your build directory to respective directories on cerbo "/lib/modules/5.10.109-venus-13/kernel/net/"

run depmod -a

append /etc/sysctl.conf with
net.ipv4.ip_forward = 1
net.ipv4.conf.all.forwarding = 1

reboot cerbo

setup NAT in iptables
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i ap0 -o eth0 -j ACCEPT
iptables-save > /etc/iptables/rules

comment "no-resolv" in /etc/dnsmasq.ap.conf, without this you will be able to ping ip addresses, but dns on ap clients won't work 
# If you don't want dnsmasq to read /etc/resolv.conf or any other
# file, getting its servers for this file instead (see below), then
# uncomment this
no-resolv


restart dnsmasq /etc/init.d/dnsmasq.ap restart

do on your own risk

question