[Kernel Fix] Fix for "Unexpected SMP Command 0x0b" (Bluetooth LE Connection Failures)

Venus OS, Kernel, Bluetooth, BLE, Development

Hi all,

I’ve been troubleshooting issues with certain Bluetooth LE peripherals (specifically some BMS units) failing to maintain a connection with the Cerbo GX / Venus OS.

The Symptoms:

The device connects via L2CAP, but almost immediately disconnects. dmesg shows the following error repeatedly: Bluetooth: hci1: unexpected SMP command 0x0b

The Root Cause:

This is caused by strict state handling in the Linux kernel’s BlueZ SMP implementation (net/bluetooth/smp.c). When a peripheral device initiates a Security Request (0x0b) after the connection is already established, the kernel rejects it if an SMP context exists, unless that specific command is already flagged as allowed.

The Fix:

I have successfully patched and built the Venus OS kernel (venus-6.12 branch) with a small modification to explicitly allow SMP_CMD_SECURITY_REQ even when an SMP context is active. This behavior aligns with other Bluetooth stacks (like Android’s Fluoride) and fixes the connection dropouts.

Code Change in net/bluetooth/smp.c:


// Function: smp_sig_channel

// Old:

if (smp && !test_and_clear_bit(code, &smp->allow_cmd))

goto drop;

// New:

if (smp && code != SMP_CMD_SECURITY_REQ && !test_and_clear_bit(code, &smp->allow_cmd))

goto drop;

Result:

With this kernel patch, my BLE devices now connect, pair, and communicate reliable without the “unexpected command” error.

I have generated a patch and can submit a Pull Request to the victronenergy/linux repo if this is of interest to the team. Has anyone else encountered this with newer BLE hardware?

Thanks!

1 Like