question

So happy you decided to publish this. Thanks!!!

I know it was a good choice to move all my solar chargers and battery monitors to Victron products :-)

…and just in case someone is interested…yes, there is already some code available on GitHub to read the data:

https://github.com/keshavdv/victron-ble

With the new documentation, I‘m sure keshavdv (the project initiator) and other contributors (like myself) can work on a complete integration.

Thank you I have updated my ESPHome (ESP32) integration to use this:

https://github.com/Fabian-Schmidt/esphome-victron_ble

Hoping the updated Victron App is soon available and will display encryption key and mac address. Was a bit painful to fetch them as the Windows app does not support Bluetooth connections.

Cheers,

Fabian

Hi mvader,

thanks for publishing the BLE format information! I'm waiting for this still a long time to read out my SmartSolar charger 100/50 via BLE. Is there any update on the VictronConnect app to get the advertisement keys and MAC directly from the app?

Best regards

oldEdDy

Hi all,

I got now my advertisment key from my solar charger with a Linux system. For all who want to now what's to do here a small instruction:

I used an old laptop (it has to be at least 64bit capable!) plus Asus USB-BT400 Bluetooth 4.0 stick, other sticks will do the job as well if they are BLE capable. Installed the current Linux Mint 21.1 with Xfce (Source: https://linuxmint.com/edition.php?id=304). Downloaded the VictronConnect Linux AppImage v5.83 (Released officially) from the Victron page https://www.victronenergy.com/live/victronconnect:beta. Made it executable with

 chmod +x *.AppImage

in the terminal and started the app with

./*.AppImage

Thanks Victron team for your description at https://community.victronenergy.com/questions/43667/victronconnect-for-linux-download-instructions.html.

After the first connection with the solar charger I followed the instruction from https://github.com/keshavdv/victron-ble. Installed sqlite3 and used the following command in terminal:

sqlite3 ~/.local/share/Victron\ Energy/Victron\ Connect/d25b6546b47ebb21a04ff86a2c4fbb76.sqlite 'select address,advertisementKey from advertisementKeys inner join macAddresses on advertisementKeys.macAddress == macAddresses.macAddress'

The result was the MAC and the advertisement key ;-).

Best regards

oldEdDy

Wow! I'm so happy Victron is trying to actively work with the DYI/Developer community. It's no surprise I have only seen Victron controllers in camper van conversions.

I'd like to provide some additional context as the original document on how to decrypt the BLE advertisement data leaves quite a bit to be desired: https://community.victronenergy.com/storage/attachments/48745-extra-manufacturer-data-2022-12-14.pdf

Here is a practical example with some pitfalls I encountered:

1) Advertisement Key Retrieval: As far as I could tell there is no way to get the advertisement keys from the Windows App, iPhone app, or Android app. I was able to get the key from my friends Mac and I've read you can also get it from Linux but haven't tried it myself.

2) Raspberry Pi: I wasn't able to obtain the key from a Raspberry Pi either. The current Raspberry Pi OS BT driver won't pair or connect to the Victron Controller so you can't get the keys. The Victron maintained Raspberry Pi image, VenusOS, doesn't allow connecting to their devices via BT, only serial, which validates the issues I was having; even Victron couldn't figure it out!

However, the BT driver in Raspberry Pi OS can listen for the the BLE advertisement messages which is great if you've already obtained the keys.

3) BLE Advertisement Data: This was particularly unclear from the provided documentation. Here are 6 BLE advertisement data packets I got from one of my SmartSolar MPPT 100/50 charge controllers:

10 02 57 a0 01 00 02 20 b0 08 09 0f 73 44 a4 e6 6f 7d 00 24
10 02 57 a0 01 10 02 20 e4 c5 b2 08 2a b8 8d 83 33 7e 4f cc
10 02 57 a0 01 20 02 20 17 a5 b2 d8 02 bf ca 8c 4a c3 ad 39
10 02 57 a0 01 30 02 20 d2 de 94 01 9e 56 6e 03 d7 23 f1 b2
10 02 57 a0 01 40 02 20 58 bd 6b 2c ea ab 78 6d 7a 5b f1 4f
10 02 57 a0 01 50 02 20 b9 40 bc c1 66 e5 38 97 66 dc 4a d8

Byte [0] is the Manufacturer Data Record type and is always 0x10.
Byte [1] and [2] are the model id. In my case the 0x02 0x57 means I have the MPPT 100/50 (I forget where I found this info but it's always 2 bytes and it's not really needed for decryption)
Byte [3] is the "read out type" which was always 0xA0 in my case but i didn't use this byte at all

The first 4 bytes aren't mentioned in the provided documentation so it was difficult to figure out where the "extra data" started. Now we get into the bytes documented:
Byte [4] is the record type. In my case it was always 0x01 because I have a "Solar Charger"
Byte [5] and [6] are the Nonce/Data Counter used for decryption (more on this later)
Byte [7] should match the first byte of your devices encryption key. In my case this was 0x20.

The rest of the bytes are the encrypted data of which there are 12 bytes for my Victron device.

4) Decryption: If you have the Encryption Key and the Nonce/Data Counter (from bytes [5] and [6]) in the BLE advertisement data you can decrypt the packet. The description in the provided documentation is not the best. I had to use reverse engineer the Python library referenced above in this thread because I'm using C#. Here is the Python library function in question:

https://github.com/keshavdv/victron-ble/blob/841f5601800a70b35df910526f174427a5e39c12/victron_ble/devices/base.py#L388

Now there are a few issues here.
1) The Python library pads the data bytes to get a full AES block of 16 bytes, and the documentation refers to using "1 AES operation" to decrypt the data if less than or equal to 16-bytes. This suggests a block cipher algorithm. However, the AES-CTR is not a block cipher algorithm, it's a stream cipher algorithm. This means no block extension should be necessary because the data is decrypted byte by byte, that's what a stream cipher means. In my case 12-bytes in gives me 12-bytes out.

2) The "Nonce/Data Counter" referred to in the documentation is often referred to as the "Salt" or the "Initialization Vector" in encryption lexicon which made a google search difficult

3) Now this was the worst part! The "Nonce/Data Counter" (bytes [5] and [6]) are not only transmitted in the advertisement data LSB first (little endian), but must also be used in the decryption as a little endian value! The provided documentation doesn't mention this fact at all!!! I only figured this out from looking at the referenced github Python library. Now what does this actually mean in practice if you aren't using Python? Because if you run the python code in a debugger it sets up a new counter object in little endian format but it puts it in an obscure object where you don't have access to the underlying byte array. Take this BLE advertisement packet as an example:

10 02 57 a0 01 40 02 20 58 bd 6b 2c ea ab 78 6d 7a 5b f1 4f

Byte [5] and [6] are the Nonce/Data Counter, in this case it's 0x40 0x02. Put this Nonce/Counter/Salt/Initialization Vector a 16 byte array in little endian format it should look like this:

{ 0x40 0x02 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x00 }

4) AES counter mode isn't supported in C# but you can see this stack overflow answer for an algorithm that works: https://stackoverflow.com/a/51188472. Just use the counter as described previously.

Hi @mvader (Victron Energy) , I looked through the docs for alarm codes for the battery monitor (0x02), but can't find them.. maybe I'm overlooking them? The integration is otherwise perfect. I'd just like to give some context to these alarms. The remark in the docs for the 16-bit alarm field is shown as "VE_REG_ALARM_REASON".

Any news on App update to be able to get advertisement Key easy

Hi all, it took a while, but per VictronConnect v5.93, released last Friday, you can see the keys. Here two screenshots:

Has anyone made any effort to try to get the encryption key without using the official apps (e.g. reverse engineer how the key is fetched from the device?)
Since the official apps can do it, it should be possible for other as well.

Getting the key using for instance a RPI Zero is impossible today, as they only release the appimage for x86_64 not ARM.

Hi all

I've been following this thread with interest for some time, and finally, it seems all the pieces have come together. So I'm embarking on developing an Arduino/ESP32-based solution to access realtime Advertisement data from my two 100/30 MPPT controllers.

Apologies for the length of what follows....

I have started with the "BLE-Scan" example in the Arduino IDE, and running it in my ESP32 recognises my two MPPT controllers:

Example output from BLE-Scan:

Advertised Device: Name: MPPT Port, Address: ff:01:6f:13:e7:3b, manufacturer data: e10210024aa001870b9009f6f5e5cecf272b95552e6b, rssi: -93

Advertised Device: Name: MPPT Stbd, Address: cc:18:68:b6:4f:d4, manufacturer data: e102100276a001bd073651a0cc59bb4d94d966b29049, rssi: -95

Some successive "manufacturer data" records for the Stbd device (from advertisedDevice.getManufacturerData()), eg:

MPPT Stbd: e102100276a001ca6e363ae821389810e0ee0315568c

MPPT Stbd: e102100276a001d16e367a16705dbfc2fd72b1cfedde

MPPT Stbd: e102100276a001ec6e365a7d0892e2b974afe10cb98c

MPPT Stbd: e102100276a001f76e36e2f52ad7bd23912f5f4c0d4e

MPPT Stbd: e102100276a001fa6e365bd3fb3c6cd3a343d225048d

MPPT Stbd: e102100276a001326f367a286066dc43595b827d15b2

MPPT Stbd: e102100276a0013b6f365e9d850ad6964623559abfb8

MPPT Stbd: e102100276a001416f36963d7871dd70f1e5c320733d

MPPT Stbd: e102100276a001476f36f5543d0966d0a36647bafa54

MPPT Stbd: e102100276a0014f6f368f5ab68b399ed92e0c66dd2f

MPPT Stbd: e102100276a0015d6f3633615c8132fa6c7eb2e6bc86

My hope was that the "manufacturer data" above would contain the data I want in encrypted form, plus a 7-byte preamble as described by @Jake Baldwin above. But I can't reconcile my preamble with what @Jake Baldwin describes, ie:

Byte [0], e1, expecting 10 (record type)

Byte [1-2], 0210, feasible (device ID; mine is 100/30, 100/50 is 0257)

Byte [3], 02, expecting a0 (readout type)

Byte [4], 76, expecting 01 (record type, Solar Charger is 01)

Byte [5-6], a001, expecting an incrementing value (nonce)

Byte [7], variable, expecting 33 (the first byte of my Encryption Key "36b...")

I've also had a look at the Payload, which similarly doesn't seem to correspond with expectations.

For info, here are the public properties of the class I'm using:

class BLEAdvertisedDevice {

public:

BLEAdvertisedDevice();

BLEAddress getAddress();

uint16_t getAppearance();

std::string getManufacturerData();

std::string getName();

int getRSSI();

BLEScan* getScan();

std::string getServiceData();

std::string getServiceData(int i);

BLEUUID getServiceDataUUID();

BLEUUID getServiceDataUUID(int i);

BLEUUID getServiceUUID();

BLEUUID getServiceUUID(int i);

int getServiceDataCount();

int getServiceDataUUIDCount();

int getServiceUUIDCount();

int8_t getTXPower();

uint8_t* getPayload();

size_t getPayloadLength();

esp_ble_addr_type_t getAddressType();

void setAddressType(esp_ble_addr_type_t type);

bool isAdvertisingService(BLEUUID uuid);

bool haveAppearance();

bool haveManufacturerData();

bool haveName();

bool haveRSSI();

bool haveServiceData();

bool haveServiceUUID();

bool haveTXPower();

std::string toString();

I have a suspicion that what I'm seeing labelled as "manufacturer data" is not the same as "Extra Manufacturer Data", which is apparently what I need. Can anyone please shed some light on what might be going on? TIA!!

If anyone is interested in a basic standalone application to read this data using an ESP32 module, I've put together a simple (well... kinda) example that receives, decrypts, decodes, and outputs the Bluetooth Low Energy advertising beacons from Victron SmartSolar charge controllers.

Credit where credit is due: Thanks, Victron for enabling and documenting these beacons, and to @FabianS for his ESPhome code that helped me figure out how to do the AES decryption.

Feel free to give it a try to see if it works for you. I've tried this on several different ESP32 boards, including some basic three-for-$20 ESP32 dev modules from Amazon.

https://github.com/hoberman/Victron_BLE_Advertising_example

And extending on this I've also written code for the very nice M5StickC and M5StickCPlus modules that uses their integrated graphics hardware to display data from the BLE beacons.

https://github.com/hoberman/Victron_BLE_Scanner_Display

I am very happy about getting this data from my battery monitor and solar chargers into my home assistant instance aboard nowadays

Are there any plans to extend this feature to the smart chargers as well? It would be a great way to quickly see if it has been disconnected from shore power

Thanks @Jake Baldwin for your detailed post which helped me make sense of what I was seeing using command line and online tools.

This post is for a 500A SmartShunt operating on the test bench, powered by a mains power supply connect to the VBatt and Aux inputs. No battery and zero amps. A Raspberry Pi 5 is communicating with the shunt using the pi's onboard bluetooth.

Open 2 pi terminals. On the first terminal execute "sudo hcitool lescan --duplicate".

On the second terminal execute "sudo hcidump --raw". You should see something resembling the image below.

Look at the second long line. Find "02 81 04 42". From the Victron "Extra manufacturer Data", "02" indicates battery monitor. "81 04" is Nonce/Data counter in LSB order. "42" is the first byte of the encryption key. The rest of the block is encrypted data.

This post https://devzone.nordicsemi.com/f/nordic-q-a/86096/decrypting-secret-hex-output-from-aes-ctr-example-in-3rd-party-application-python from a Nordic Semi forum led me to this https://cryptii.com/pipes/aes-encryption super useful online utility.

So, looking below, if we copy and paste from the pi terminal window into the cryptii decryption utility we can start to see some sensible looking decrypted data.

Into the left dialogue box is entered the encrypted data. In the middle dialogue is entered the key from the Victron Connect app on my phone. IV (initialization vector) is "81 04" padded with zeroes out to 16 bytes. Thank-you @Jake Baldwin.

And for the image below, the encrypted data is completely different, key the same, Nonce is now "82 04" padded and the data is the same except for the last byte. I haven't investigated that last byte yet.

Looking at the decrypted data and referring to Victron "Extra manufacturer Data", the first 2 bytes (16 bits) is TTG (time to go?) and NA value is 0xFFFF.

Next 2 bytes (16 bits) is battery voltage. A signed 16 bit integer in little-endian format (low byte first, reads back to front). So 0x04e0 is decimal 1248 which is 12.48 volts.

Next 2 bytes (16 bits) is alarm reason. No alarms so I guess 0x0000.

Next 2 bytes is aux voltage. So 0x04e1 is decimal 1249 being 12.49 volts. The two inputs are reading differently by 0.01 volts.

The next 3 bytes (24 bits) consists of 2 bits representing how the aux input is configured. For voltage in this case. The remaining 22 bits is the current. Zero in this case because nothing is connected to the shunt.

Thank-you fellow contributors for your help. I hope this post will help others.

Regards.

Although documentation specifies a value of 0x10 for the field "Manufacturer Data Record type", I received frames with a different value.

Sources:

- https://community.victronenergy.com/questions/187303/victron-bluetooth-advertising-protocol.html

- pdf doc "Byte [0] is the Manufacturer Data Record type and is always 0x10".

I'm thus expecting the third manufacturer data byte to be 0x10. Here are 3 frames received from Victron product, I see in my logs:

Orion DC/DC

e1 02 10 00 d1 a3 04 60 5a f7 11 18 14 62 e3 e9 f9 d8 0e 5c

> manufactuter bytes are (0x10, 0xe1) > Victron

> fine we, have 0x10

SmartSolar

e1 02 02 41 ff 5e 79 97 0c d6 75 b0 8a 87 fc f1 d6 70 0c 79 bf a5 3a 93 f2 a0 49 fc 7b

> manufactuter bytes are (0x10, 0xe1) > Victron

> what is this 0x02

Phenix AC charger

e102 02 41 b7 dd 26 99 3e ed be 81 94 81 9c 29 40 fc d8 c3 17 2b 24 3a

> same 0x02 type I don't understand

Any idea why I get 0x02 and not 0x10?

Could be an error on my receiving code, but I suspect a different or older protocol for these devices, too.

Thanks !

Bonjour,

I am working on retrieving data from a SmartBMV 712 using an M5STACK Core 2 in C++. I have used the extra-manufacturer-data document, which allowed me to extract certain values such as the State of Charge (SOC) and the voltage. However, I am unable to extract the 'Battery Current.' Has anyone succeeded in decoding the 22 bits of this data? I am getting huge, completely erroneous values.

Merci !

Thank you very much for this information. I haven't fully grasped all of your explanations yet, but I understand that the encoding of the "current" value I've used isn't correct despite the valuable help from @theterminalman. The entire code is too lengthy and isn't on Github, so here are some portions to try to explain better. Here's the structure I'm using:

// Structure des données envoyees par le controleur Smart BMV 712
typedef struct {
  uint16_t bmvTTG;
  uint16_t bmvBat1Voltage;
  uint16_t bmvAlarmReason;
  uint16_t bmvBat2Voltage;
  uint8_t byte0;    // byte 0-2 , 3 octets qui, selon de Extra_manufacturer_data de Victron, contient: Aux input, Battery current
  uint8_t byte1;    // on n'utilisera que battery current.
  uint8_t byte2;
  uint8_t byte3;    // byte3, 1 octet contenant le MSB du Consumed Ah. Cet octet sera extrait mais pas affiché, car nécessiterait aussi le LSB
  uint32_t bmvData2;// 32 derniers bits des données contenant entre autre la valeur SOC et le LSB de Consumed Ah.
} __attribute__((packed)) victronBMVData;

The 24 bits of "Aux input" and "Battery current" are the variables byte0, byte1, and byte 2.

Here's the data extraction: (Hoberman will recognize is own code for SmartSolar, modified. Thank you)

          uint16_t bmvTTG = uint16_t (victronData->bmvTTG);
          float bmvBat1Voltage = float(victronData->bmvBat1Voltage) * 0.01;
          uint16_t bmvAlarmReason = uint16_t(victronData->bmvAlarmReason);
          float bmvBat2Voltage = float(victronData->bmvBat2Voltage) * 0.01;
          uint8_t byte0 = (victronData->byte0);
          uint8_t byte1 = (victronData->byte1);
          uint8_t byte2 = (victronData->byte2);
          uint8_t byte3 = (victronData->byte3);
          uint32_t bmvData2 = uint32_t(victronData->bmvData2);

This extraction works well to obtain bmvTTG, bmvBat1Voltage, bmvAlarmReason, bmvBat2Voltage, and the SOC (Stored in bmvData2) whose values are accurate.

And finally, here are the raw datas I obtain for the 3 bytes. Zeros at left are not displayed.

11:32:51.472 -> byte0=10100
11:32:51.472 -> byte1=100101
11:32:51.472 -> byte2=0

11:33:02.184 -> byte0=1000100
11:33:02.184 -> byte1=100101
11:33:02.184 -> byte2=0

11:33:06.335 -> byte0=1110100
11:33:06.335 -> byte1=100101
11:33:06.335 -> byte2=0

11:33:14.350 -> byte0=10111100
11:33:14.350 -> byte1=100010
11:33:14.350 -> byte2=0

11:33:17.022 -> byte0=1001100
11:33:17.022 -> byte1=100000
11:33:17.022 -> byte2=0

11:33:25.037 -> byte0=1000000
11:33:25.037 -> byte1=100101
11:33:25.037 -> byte2=0

11:33:34.505 -> byte0=10110100
11:33:34.505 -> byte1=100011
11:33:34.505 -> byte2=0

11:33:38.442 -> byte0=11000100
11:33:38.442 -> byte1=100110
11:33:38.442 -> byte2=0

11:33:39.895 -> byte0=111000
11:33:39.895 -> byte1=100011
11:33:39.895 -> byte2=0

11:33:41.114 -> byte0=11001100
11:33:41.114 -> byte1=100101
11:33:41.114 -> byte2=0

If I understand correctly, I should start by concatenating Byte0, 1, 2 to obtain a 24-bit word, then shift 2 bits to the right to remove the "Aux Input" value, which is actually on the 2 bits at the far right and not on the left. Then, this is where I encounter a problem in reversing the remaining 22 bits, then identifying and interpreting the sign bit, and finally decoding the absolute value in milliamps.