question

ks4z avatar image
ks4z asked

Frequent write to dbus settings

Hi,

I am currently developing a service that uses a PZEM-017 as a smart shunt alternative.

In order not to lose the current SOC during a restart of the GX device I'd like to store it in the settings object.

Now I wonder what my Multiplus II GX (and other GXs) uses for permanent storage and whether or not it's a problem to frequently update the data on that device.

Is there a recommendation regarding frequent write to settings objects?

Thanks

ks4z


Multiplus-IIcerbo gxSmartShunt
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

13 Answers
Louis van der Walt avatar image
Louis van der Walt answered ·
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

ks4z avatar image
ks4z answered ·

Hi Louis,

thanks for the quick answer.

I know about the localsettings. The question that came to my mind is, if it hurts the storage if I write to it quite frequently. Some permanent storage devices like eeproms etc. don't like frequent writes. Any insight on that?

Thanks

ks4z


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Louis van der Walt avatar image
Louis van der Walt answered ·

Yes that can be an issue.

The localsettings object will optimise the disk usage for you as best it can. But it will still be writing to the disk. It is stores it's data in a XML file under the /data/ partition which is persisted, but I don't have more details without going through the code in detail.


2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image
Kevin Windrem answered ·

One way to handle this is to publish the SOC value in the dbus service for the device and update this say once every 1-10 seconds. This parameter is volatile and exists only in RAM.

Also create a non-volatile version in settings and update this parameter say every minute.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image
Kevin Windrem answered ·

I'm interested in this work as a low-cost alternative to SmartShunt for DC Meter situations. Will you support this as well?

SmartShunt configuration is handled with VictronConnect which your alternate would not be able to use so you'd need to implement configuration in the GUI Menus.

Thanks for your work.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

ks4z avatar image
ks4z answered ·

Hi,

I guess I will go for some reduced updates on the settings as you propose. Maybe depending on the change of the value itself. It will probably be (slightly) wrong anyway after a restart.

I didn't look into the DC meter yet, but I see no reason why this could not be supported.

The config in the GUI isn't a huge problem, I did that before. However, I don't like that it requires changes on system files that will be overwritten by updates and might interfere with other extensions. Is there some best practice for that?

Thanks

ks4z



5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image Kevin Windrem commented ·

Venus OS firmware updates overwriting mods to the system files is a concern. Many suggest adding code to /data/rc.local to reinstall updates. Even services you create to add drivers to the system need to be reinstalled after a firmware update. Some place code in /data/rc.local that runs every boot to reinstall things but that can slow boot-up.

I've taken this a huge step forward with SetupHelper. Packages developed with the SetupHelper hooks are able to reinstall after a firmware update automatically.

https://github.com/kwindrem/SetupHelper

Developing packages for SetupHelper is fairly involved but I have help available should you choose to go that way.


0 Likes 0 ·
ks4z avatar image ks4z Kevin Windrem commented ·
/data/ rc.local is the way I currently use for my extensions. I try to just create links to /service and maybe the GUI directory.

Using sed to patch system files is a possible way to go, but probably a nightmare in detail. Maybe I'll try patch, which could be robust and simple.

However, the issue I see is not only with updates but with other custom stuff that might want to modify the same files. This can't be solved by having different versions for different firmwares and especially not with replacing the whole file with my version.

I'll check your Setup helper. Sounds like a useful thing.

0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem ks4z commented ·
Tried patch but it didn't work across multiple original versions. If the original didn't match the one patch used to create diff/edit commands it either did the wrong thing or failed. It's been so long I forget.
0 Likes 0 ·
ks4z avatar image ks4z Kevin Windrem commented ·
Yes, that can happen if if't too different.

I looked into you SetupHelper project. Godd workm really like it.

What I swas that you simply replace the original file with the version of the extention. This would destroy all changes made by other extentions, or am I mistaken? Especially in the GUI I can imagine that some files are quite often modified.

0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem ks4z commented ·
The way SetupHelper works is it renames the stock file to a file of the same name but with a .orig extension. It will do this only once. That is, if the .orig file exists it skips the rename. This way the stock file is preserved if the setup script is run again.

But as you say, if two packages modify the same file then the file is the replacement from the last setup script run. For sure this is a conflict and one I've carefully avoided with my packages. But as others create packages the risk of conflicts increases.

1 Like 1 ·
Louis van der Walt avatar image
Louis van der Walt answered ·

I also do not like to make GUI changes, but there does not seem to be a way around it. The best way I see is to make a backup of any files you change. The issue is that as the firmware updates so does the GUI also and if there is a mismatch the UI will not display.

From what I saw Kevin' GUIMods has a version that match each firmware version, and his setup tool then install the matching GUI changes.

This does seem like a lot of work to keep up with firmware changes, so my driver tries to only have one or two entry points and have it's own pages under those.
I would also love to know if there is a better option.

2 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image Kevin Windrem commented ·
I forgot to mention above that SetupHelper does back up the stock version of any files it modifies so the system can be restored to stock by moving the backups into the active location.

Yes, there is some work associated with support of new Venus OS firmware versions but it's not too bad.

There are other ways to modify files than replacing them with a version-specific file. You can also use tools like sed to modify the file. I did this for a while with SetupHelper but it has it's own difficulties in some cases. The latest version of SetupHelper does modify the stock copy of PageSettings.qml. This makes it version-independent. Adding a device's config menu tree to this file could be done the same way and eliminate conflicts with other packages.

1 Like 1 ·
ks4z avatar image ks4z Kevin Windrem commented ·
Sounds interesting. At least I will learn something from it.
0 Likes 0 ·
Kevin Windrem avatar image
Kevin Windrem answered ·

Thinking about the PZEM-17 as a low-cost alternative to a SmartShunt. For the PZEM to function as a DC meter or even a battery monitor, it would need to be powered all the time and in a system that doesn't always have shore power or even sufficient solar energy to power things the PZEM's power consumption might be an issue. The spec for the SmartShunt is "under 1 mA".

Has the PZEM-17's current draw been measured?

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

olafd avatar image
olafd answered ·

According to this Link

https://open-boat-projects.org/en/wifi-batteriemonitor/#Stromreduzierung

the PZEM-17 is in the same 1mA range.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image
Kevin Windrem answered ·

This is good news and increases my interest in using the PZEM-17 as a lower cost alternative to SmartShunt.

Venus OS includes Modbus TCP but not sure if it includes Modbus RDU (RS-485 serial).

The PZEM series of power monitors are Modbus and therefore support multiple devices on the same RS-485 serial connection. I did a project a while back using 4 of the AC versions. Multiple devices on the bus requires configuring the PZEM's slave address before connecting to the bus. For my project, I ignored the default address of 0 to avoid issues with unconfigured units. I wrote a small C program to scan Modbus for addresses and allowed any (including 0) to be changed.

Another issue is a way to configure the use of the shunt: battery monitor / DC Meter, and if the latter which measurement type: alternator, DC load, DC source, etc. This is done on the SmartShunt using VictronConnect via bluetooth. Once setup in VictronConnect, the shunt then appears on dbus with a service name including the measurement type. If this method of configuration is used for PZEM devices, it's appearance on dbus will change after configuration. You might want to consider configuring the device type when setting the Modbus slave address. Options are limited but the PZEM includes a high and low voltage alarm threshold that aren't likely to be used for anything in this application so the measurement type could be placed in this register. The other could maybe hold SOC so it doesn't need to go into dbus Settings.

Another configuration item is the shunt range: 50, 100, 200, 300 amps. This would need either a menu item or be configured with a setup app also used to set the slave address and possibly the measurement type.

I'm willing to help with development and testing.

5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

johnny-brusevold avatar image johnny-brusevold commented ·

From what I have read, this is not a bipolar current meter, so it does not show whether current is flowing from or to the battery.


0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem johnny-brusevold commented ·
Very good point. I just watched a video that confirms the PZEM-17 doesn't report current direction.


So ... the PZEM would not work at all for a battery monitor or a DC system monitor if there were loads and charging sources, but might for other applications like a DC load monitor or monitoring alternator power. Both if these would be one way.
0 Likes 0 ·
johnny-brusevold avatar image johnny-brusevold Kevin Windrem commented ·

@Kevin Windrem

I just finished a bipolar "smart shunt" with parts from ali-x which is simple and works fine. using python, rpi_GPIO library and node-red in venus-os. and costs around 12$ in total. you can also use smbus for communication.

All the modules together are so small that they fit nicely in half the size of a matchbox

It is based on

MCP3424 Module I2C (4 channel 18 bit bipolar adc )

ISO1540 Module bidirectional I2C isolator

5V to 5V dc isolated power suply module

Shunt

0 Likes 0 ·
Kevin Windrem avatar image Kevin Windrem johnny-brusevold commented ·
I'd be interested in details: parts list, the code, any configuration requirements.

Is it just as a battery monitor or DC meter also? Would Venus support multiple units? You mention I2C. Would this only be on the PI? If not, what's the interface (USB, ModBus RDU, ???)

0 Likes 0 ·
johnny-brusevold avatar image johnny-brusevold Kevin Windrem commented ·

I use this AD converter connected directly to a 50A/75mv shunt.

I have set the gain to x8 (+-250mv) and 16bit,

and there are still 2 channels available for other measurements.

If you look at the example on page 33 of the data sheet

"Battery Voltage and Charging/Discharging Current Measurement".

MCP3424 ADC-4 Data sheet

I have tested with this node-red node, which worked fine

https://flows.nodered.org/node/node-red-contrib-redplc-rpi-mcp342x


I2c use 2 gpio pins. I have rtclock, 16 channel led light controller and 8 channel relay controller on the 2 same gpio pins 3 and 5 (SDA and SCL)


I don't know if you have access to i2c/gpio on the GX devices

0 Likes 0 ·
olafd avatar image
olafd answered ·

On Github there is already a VenusOS-Driver available:

https://github.com/mildred/venusos-peacefair-pzem

But this driver is maybe only for python 2.7 not 3.0.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Michael Schmidt avatar image
Michael Schmidt answered ·

Is it possible to change the Syetem Setup/Ac Input 1/ (Grid to Generator and vice versa)

with a dbus command?

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem avatar image Kevin Windrem commented ·

Yes. How it is done depends on the language you are using. For shell scripts, use dbus -y. There's also a python library for the dbus interface. Look in any of the stock services for the files in .../ext

More info here:

https://github.com/victronenergy/venus/wiki/commandline-introduction#working-with-d-bus

and here:

https://github.com/victronenergy/venus/wiki/dbus

The specific parameter is com.victronenergy.settings /Settings/SystemSetup/AcInput1

If you are looking for a way to switch the system from grid/shore to generator, check out my package that handles everything switching the input type and the input current limit and Do not run generator when on AC 1 as well

https://github.com/kwindrem/ExtTransferSwitch

0 Likes 0 ·
weeman avatar image
weeman answered ·

Any update on this?

I'm keen to setup a cheaper smartshunt alternative for measuring alternator output and this would be perfect. Otherise I will probably play around with the INA226 work @ks4z has done.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.