question

kevlat avatar image
kevlat asked

Victron Bluetooth BLE protocol Parsing?

Hi All,

I have been hacking on the Smart Shunt BLE protocol using Python and now am stuck hoping there might another python person out here whom might be able to lend some insights on where I am going wrong.

I started off looking at victron_ble app code code base and while it has some wonderful things happening in there. It is an overkill for my needs right now. Also I like to keep all 3rd party modules to a minimum in my personal coding projects.

I was able to pull some parts from the victron_ble code base like the decrypt calls into my code.


Where I am stuck is taking the binary string coming back from Bleak scanner and parsing that down into the smart shunt data points.

I hacked the v_ble code to show me output of the raw, and decrypt data points so I know that my code is giving me the same answers as it and the Victron Connect app.


I take those raw data strings and run it through my code but end coming up different numbers than victron_ble's while parsing the bits


The Victron_ble app uses a Python Module called "construct" to parse that raw advertising data from the shunt. Which is great but support for the code base appears to slacking off or it's very stable and not needing much work these days.


My point being is the "construct" code hurts my head and how it works is so far off my radar that is not something I want depend on for this app that I am working on.

Which is why I am trying to parse the the raw bit string based upon the published Victron white paper protocol start stop bit data points.


I've done lots of parsing but never at the bit level like the white paper defines.

I am taking the long bit string and using python string slicing calls to grab those bits and then using int(bit_string[start:stop], 2) to convert that to an int.

Given all the other parts of my code seems to be working it's obvious my final approach is not working yet.

Anyone out here have any insights on what I might be missing?



To sum it up

I am grabbing the raw data using Bleak's scanner calls to grab the adverting data from the smart shunt,

Running that through the Victron_ble app decrypt code to get that raw binary data decrypted

I then convert that into a hex string

and then into a bit string

Using Python string splicing I am grabbing a series of bits using this snippet doing

int(bit_string[start:stop], 2) to get an int back.

But it's not getting the results back like it should be.


Since I've not worked direct with a bit strings like this before

I have no idea if some other kind bit banging tricks are required to get back the correct bits for the correct results back.


Sorry this has post got a bit long but the whole topic of using Python to hack on Victron BLe based items is pretty thin on the internet.


Thanks for any suggestions or ideas of what I might try to do to get this fully working.

To sum it I feel like code alpha code is about 90 to 95% working using some simple bleak calls to grab the smart shunt data points using Python

It's having to parse that encrypted binary data points into bits that has proven to be a bit of challenge.


Thanks

-Kevin






Bluetooth Low Energy - BLE
1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kevgermany avatar image kevgermany ♦♦ commented ·
@kevlat

Moved to modifications space. Hope you get help there.

0 Likes 0 ·
4 Answers
kevlat avatar image
kevlat answered ·

After more research I am thinking my first attempt at parsing the binary data into bits and then using Python's string slicing either is not the correct or the best way to solve the problem with.

Given the Victron_ble app (https://github.com/keshavdv/victron-ble) is using the "Construct" module, It appears using a C style 'struct' is how this done or a better way to solve it.

If any one is using Python's std lib 'struct' calls to parse a Victron's smart shunt data points.

If you might point me towards some example of how this or you are doing that, I would greatly appreciate those ideas and or coding insights.


Thanks

Kevin


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Ole Saether avatar image
Ole Saether answered ·
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

kevlat avatar image
kevlat answered ·

Yes I have read the protocol paper.

Ideally I don’t want to depend on using a complicated 3rd party modules like construct to parse this data

Because Victron is using variable length data fields Python stdlib struct calls does not play well with them out of the box.

I’ve come across some example ideas of how Python can do this, I just have not been able to get all of that Python code working correctly yet.

My hope is to reduce a couple thousand lines of 3rd party app code down to about 100 lines or less of code using as much of stdlib calls as I can.


Getting there but for now I have run out time that I can spend on the getting this idea working how I want it to be


I ended up with some bleak module scanner calls with a few calls I pulled from the Victron_ble app to parse the c_struct to get a proof of concept running.

Thanks

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

luka-renko avatar image
luka-renko answered ·

Any progress on this? Can you share your python code so that I can look into it?
I would also like to have simplified python code, as victron-ble has too many dependancies to be run on Cerbo.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Related Resources

Additional resources still need to be added for this topic