question

mitchese avatar image
mitchese asked

Alternative to ET340 - SMA Home Manager 2.0

See my first comment below... mqtt doesn't work for this, but dbus does


tl;dr: How can I emulate a et340 meter on the MQTT bus? What do I need to publish to make Venus believe there is a meter there?


I already have an SMA system installed which includes a SMA Home Manager (SMA's version of the ET340: a meter at the grid connection point). I want to extend my system with a Multiplus-ii and add some storage.

Let's get this out of the way: I know the SMA meter is not officially supported.

SMA's meter outputs the values it sees at the grid connection point once per second, and it does this via multicast. I have written a little python script which subscribes to the multicast, and provides the output like this:

A:  buy: 0.0 A: sell: 821.2 -------------- L1  buy: 0.0 L1 sell: 381.6 L2  buy: 0.0 L2 sell: 81.6 L3  buy: 0.0 L3 sell: 358.0 

The above table is my current buy and sell across all phases ("A"), where I am currently selling 821.2W and buying 0. The values are broken into each phase (so L1 is selling 381, L2 81, and L3 358). There is much more available in the data from the meter, I am only focusing on what is necessary to emulate the 340.

The script is available here: https://gist.github.com/mitchese/afd823c3c5036c5b0e5394625f1a81e4

As I understand the ET340, it connects to the Venus via rs485, and then a cgwacs process is responsible for taking data from RS485 and putting it on dbus. I've also read that MQTT can be used for writing things into Venus as well. My idea is to write a simple python script which connects to the SMA broadcasts, massages the data to whatever venus would expect, and publishes this via MQTT. This should be darn close to what cgwacs is doing.


I have done some publishing to the topics (*** removed my device id) with some bogus-but-sane values to see if I can get my Venus GX to think I have a meter.

W/**********/grid/33/Ac/Energy/Forward {"value": 5000.0} W/**********/grid/33/Ac/Energy/Reverse {"value": 5000.0} W/**********/grid/33/Ac/Power W/**********/grid/33/Ac/L1/Current '{"value": 123.0}' W/**********/grid/33/Ac/L1/Energy/Forward '{"value": 30.0}' W/**********/grid/33/Ac/L1/Energy/Reverse '{"value": 500.0}' W/**********/grid/33/Ac/L1/Power '{"value": 300.0}' W/**********/grid/33/Ac/L1/Voltage '{"value": 230.0}' ...

but this is just a bit of publishing in the dark. I don't actually have a device id, 33 was just a guess. Anyone have some tips on how I can use MQTT?


I also tried a publish to W/***/system/0/Ac/Grid/DeviceType {"value": 340} to see if that would trigger an adoption of the 340 meter.


Currently I have an ET340 for testing and a Venus GX - there is nothing hooked up yet and I'll be ordering the multiplus as soon as I can get this setup.



ESSVenus GX - VGXMQTTpower meter
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

20 Answers
mitchese avatar image
mitchese answered ·

I've made some progress and I believe I have this working. It did not work as I had initially wanted with MQTT, so the code below needs to run on your GX device for it to work (it can't run on an external service)


If anyone wants to test this, please see my code at https://github.com/mitchese/sma_home_manager_printer and https://github.com/mitchese/shm-et340 . The first repo should be safe to execute on your device, the second one registers on the DBUS and publishes messages, so it has the potential to break things in unpleasant ways.

It has been working for me, but use these at your own risk, and I would welcome someone to help test this.

I don't have a MultiPlus-ii yet, so I've had to rely on the replay of demos for doing my testing. I've been running this for about a week now and the results are correctly showing both in the local UI, as well as in the VRM portal. It would appear like I have an ET340 meter, but I only have the SMA Home Manager 2.

Hopefully someone finds this useful :)



2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

PeterM avatar image
PeterM answered ·

@mitchese , have you got the full details of the SMA HM Interface ? Is the HM1 and HM 2 the same I wonder . I have about 90% of the details for HM1 but for 1 or two parameters I have not figured out yet.


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mitchese avatar image
mitchese answered ·

This applies to the SMA Home Manager 2.0 (not 1.0). I don't have a 1.0 to test with, but does it work the same as the 2.0, just in two separate devices? Does it do a broadcast every second of energy flow at the revenue meter?


The details I have is a description of each bitfield in the udp packet sent from the home manager 2.


I've also updated the topic title to reflect the HM "2.0".

6 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

munga avatar image munga commented ·

Thanks for this! I have used your GIT hub code to hack my SMA EM v1 to work on this. There is essentially a 4 byte offset on the decodePhaseChunks.

Also, you clearly live in a country with reliable power as one issue with your forever loop Listener is that the values are only updated when a message is received from the UDP multicast from the SMA (about 1Hz). But if the power goes out (as it frequently does in South Africa) the SMA EM stops broadcasting and the last values are used by the CCGX. I have added a timeout after 2 seconds to send a 0 so that you don't have an offset from the last message when the power goes out. Again, not an issue in "real countries".

0 Likes 0 ·
munga avatar image munga munga commented ·

Is there a way to access "critical loads view" as you have:


Mine shows as follows:

0 Likes 0 ·
1599557780308.png (41.0 KiB)
1599557762100.png (43.0 KiB)
nerchimond avatar image nerchimond munga commented ·

Could you please share your changes? I want to use a Energy Meter too.

Thanks

0 Likes 0 ·
Holger Schultheiß avatar image Holger Schultheiß nerchimond commented ·

see here: https://github.com/mitchese/shm-et340/issues/7

0 Likes 0 ·
graydeath avatar image graydeath munga commented ·
Hello munga, can you share your code for the SMA EM 1.0?
0 Likes 0 ·
Holger Schultheiß avatar image Holger Schultheiß graydeath commented ·

see here: https://github.com/mitchese/shm-et340/issues/7

0 Likes 0 ·
schnema avatar image
schnema answered ·

Hi all,


Thank you mitchese to publish your code and describe your progess. I did a fork of your code and can confirm it running on an Multiplus 2 GX together with SMA products. If someone needs help you are welcome to ask.

https://github.com/Schnema1/shm-et340


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

PeterM avatar image
PeterM answered ·

Will the code here described that works on Venus GX device work on the Cerebo GX?


3 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

schnema avatar image schnema commented ·

Hi Peter,


I have it running on an Multiplux 2 GX. As the venus is included I confirm it works. But I do not know about Cerebo GX


Kind regards

Marc

0 Likes 0 ·
mitchese avatar image mitchese commented ·

I don't have a Cerebo to test on, but I don't see why it wouldn't. In the readme, there are two actual programs, one test program and one actual. Running the test makes no changes and it's safe to give it a try and see if it works. This will confirm that the architecture (ARM) is the same and working, and that there is network connectivity between the SHM and the Cerebo (but do nothing else).

0 Likes 0 ·
andrewb avatar image andrewb mitchese commented ·

How can I contact you? there is a similar question

0 Likes 0 ·
weeman avatar image
weeman answered ·

Is anyone using this code having any issues with the online VRM portal?

All the data and functionality with ESS is working perfectly on the local device and the local web interface is showing all the correct data, however the online VRM portal stops receiving all data whenever this script is running. As soon as I stop the script, the online VRM portal instantly starts working and collecting data from my device.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

dominik-feser avatar image
dominik-feser answered ·

Hi!

Is https://github.com/mitchese/shm-et340 publishing values to "AC Input" or only to "AC Loads"? I can only see values on "AC Loads" and "AC Input" is empty...

Any help is appreciated!


2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

mitchese avatar image mitchese commented ·

The code only provides values for the grid meter (the red box in the picture), AC Loads is inferred from other areas in the system along with the grid meter - so the values here have an effect on AC loads but don't define them outright.

0 Likes 0 ·
timmi avatar image timmi commented ·
I have an similiar issue. At daylight AC-Input displays negativ values (PV generated energy) and no AC Loads.


After Sunset AC Loads are displayed correctly.


Any Ideas?


0 Likes 0 ·
panamax avatar image
panamax answered ·

Good day,


I have a similar setup and would need the SHM faker. i followed the Instruction and all works fine.

However i was not able to make the script boot automatically, it only runs when i start it over SSH and stops once i close the terminal....


I would highly appreciate a more detailed description how to make it bootable.


Thank you!

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

akabza avatar image
akabza answered ·

Hello,

try to use this command to start the program permanent:

# nohup ./shm-et340 &

Alex

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Christian Butterweck avatar image
Christian Butterweck answered ·

Hello, i added a little script to restart the shm-et340 service in case of power failure or network problems.

https://github.com/mitchese/shm-et340/pull/6

Christian

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Christian Butterweck avatar image
Christian Butterweck answered ·

@mvader (Victron Energy) Is it possible to integrate this into Venus OS so that the Sunny Home Manager 2.0 energy meter gets supported? 70% soft infeed by the SMA inverter is working parallel and not affected.

There are already two solutions to support this smartmeter:

https://github.com/mitchese/shm-et340 (written in Go)

https://github.com/akabza/VE.bus-SMA-Sunny-Home-Manager-service (python)

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

alexalex avatar image
alexalex answered ·

Hi, i have used the et340 emulator, works fine..but i have short shown values of 0, then the normal values again...is that only a display error, or are there values missing ??

I have the feeling that in python Script no values are missing...

Is there any advantage between the 2 solutions go or python ?

How to get they Python Script started automatically at boot and have a watchdog if its running ?

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

dominik-feser avatar image
dominik-feser answered ·

Hi @mitchese!

Thank you for your SHM2.0 driver, it is working perfectly since a year now.

However, now I tried to make use of mqtt to read out and configure things on VenusOS. What I found out is, that dbus-mqtt of VenusOS is dying as soon as your driver registers to dbus and writes messages to it.

I already filed an issue on your git repo but wanted to document it here, too.

Do you think you can have a look at it?

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

hundefreund02 avatar image
hundefreund02 answered ·

I installed a SMA Homemanager 2.0 phyton script to the Venus OS on the Raspberry PI4 and the new firmware 3.08. The two python files vedbus.py and ve_utils.py are located in the root directory of the PI4.

However, I don't see a SMA-Homemanager in Venus Os! What could be the cause?

But with the shm-et340 driver it works perfect ?????

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

sma-victron avatar image
sma-victron answered ·

Hi, I have the SMA Energy Meter and without the Faker it looks like:


1675940731269.png

When the Faker is running I can see:

1675940600879.png

The problem that I have is that the whole production just goes to the AC load and no real load will be calculated.

1675940675279.png

So the system does not recalculating anything. And even if I have more load than production

1675940972100.png

the behavior does not change.

Online the Grid is correct as far I can see:

1675941462668.png

Does somebody know where my issue is?

Thx


1675940600879.png (19.2 KiB)
1675940675279.png (49.3 KiB)
1675940731269.png (46.3 KiB)
1675940972100.png (9.9 KiB)
1675941462668.png (39.3 KiB)
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gereic avatar image
gereic answered ·

Hi all,

I tried also the SMA-faker, but it doesn't do the job as well as I want it.

Problem is, that the cpu-load on the cerbo goes up, so that real-time in VRM-portal is not working anymore.

However I decided to do a small trick.

I Build a program for an ESP32-Board, to collect the SMA-Data and emulate a real ET340 on the USB-Port.

You can also decide, which Energy-Meter you want to take, in case you have more than one Energy-Meter from SMA installed.

If somebody is interessted in the project, write me a PM.


1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

siartec avatar image siartec commented ·

Hello, I am interested in your project with Esp32. Could you share it with me? Thank you

0 Likes 0 ·
becker avatar image
becker answered ·

I´ve build the SMA HM by MQTT (freakent):

1682415522019.png

works great, CPU Load on RPi2 with VenusOS large ~25%. On RPi4 ~11%.

1682415613910.png


1682415522019.png (112.1 KiB)
1682415613910.png (31.7 KiB)
6 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tom-68 avatar image tom-68 commented ·
Hi becker! Looks great! Is the Home Manager then recognised as an ordinary and fully functional grid meter in VenusOS?

Cheers
Thomas

0 Likes 0 ·
becker avatar image becker tom-68 commented ·
of course
0 Likes 0 ·
maxpd avatar image maxpd becker commented ·

So you don't need the SMA Faker?

Can you share the script, respecively give a more detailed step by step guide for newbies? :)

Thanks in advance.

PS: I have the data from HomeManager in iobroker if this somehow helps for some kind of workaround.

0 Likes 0 ·
xzv avatar image xzv commented ·
Hi @Becker I also have a SMA HM. Could you share your Node-RED flow? Would really appreciate this as I have tried for a long time and didn't succeed so far!
0 Likes 0 ·
jgeraerts avatar image jgeraerts xzv commented ·
did you receive the flow and would you be able to share it?
0 Likes 0 ·
jgeraerts avatar image jgeraerts commented ·
Hi @Becker , can you per any chance share this script?

I'm also running a Home Manager 2 and don't have physical room to put a EM24 anymore, this would be a lifesaver.

Thanks!

0 Likes 0 ·
tom-68 avatar image
tom-68 answered ·

I used this solution for a while now, and it worked perfectly in the 1-phase environment.

I recently upgraded to a 2-phase system; unfortunately, this solution stopped working.

The values from the Sunny Home Manager are still displayed correctly in the VenusOS remote console, but the two Multiplus II and the ESS cannot use the values anymore. The two Multiplus stay in "inverter" mode and no longer charge or discharge the battery.

Any idea? Thanks in advance and best regards.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

maxpd avatar image
maxpd answered ·

Hi,

using a Sunny Home Energy Meter 2.0 with SMA Sunny Tripower 20000 inverter.

As initial test as recommended by mitchese I put the file sma_home_manager_printer.armv7l into the folder data/home/root

If I execute ./sma_home_manager_printer.armv7l I am receiving

"Implausible serial, rejecting"

Serial looks like 1924586782

Am I doing something wrong?

Wanted to use shm_et340 faker to use Sunny Home Manager in Victron ESS environment.


PS: I have the data from HomeManager in iobroker if this somehow helps for some kind of workaround.


Regards
maxpd

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

merlin5 avatar image
merlin5 answered ·

@maxpd, the problem could be a "wrong" switch between SMA homemanager and Victron device, SMA homemanager send the data in Multicast and some switches do igmp spoofing which do not work with multicast, so replace that switch or configure direkt adresses in SMA

1715357733267.png


1715357733267.png (19.2 KiB)
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.