Collective action / help needed! lets fix the logging of all popular mods out there

Modifications
1

Hello all,

There are still various mods/scripts out there that write their log files to the GX memory without thinking of the consequences. So:

Help needed: please find mods that don’t do their logging in the best way, and report them here. Best way is to write to stdout and stderr. So seeing a filehandler for logging in source code is a red flag.

And worst case is filehandler + no log rotation at all.

A detailed discussion on how we want logging to be done is here:

Which was solved quickly - nice work by the opendtu team.

Why am I so worked up on this?
Its a major waste of time, and loss of time that could have been spent on progress. On our side.

And similarly, major waste of time and point of frustration of all the people globally that are in the line of support.

What happens to people that brick their GX device by a mod?
after at some point your GX has stopped working, you might be able to get it going again yourself, with or without asking for precious time of other people for help.

And, even though our global support network will often hand out a new Cerbo under warranty just because they’re told to be easy, actually its of course a waste and in the end everyone will pay.

To conclude
I know above might be a bit harsh and gut response - but thats how it is.

I’m 100% in favor of openness and mods, its what I’ve been building at Victron for many years, and I am super keen to improve and build it further. Various nice initiatives for that have been started recently, around gui-v2 custom qml, rpi5, rpi gui-v2 and more.

But:

Providing mods also comes with a responsibility, and people trying out mods and installing them also have responsibility but some can’t carry that.

End of rant.

!

5 Likes
2
3

Here is a real popular one that bricks your GX:

4

SetupHelper, and any shell script that utilizes the utilities provided by it are guilty of writing to a log file and not providing any log file rotation. It would be difficult to do it any other way and still provide a log file that can help debug packages.

SetupHelper now writes to the PackageManager log file which is run as a service and does employ multilog. It seems PackageManager’s multilog is providing the necessary log rotation for SetupHelper also. If not, or this is unacceptable somehow, let me know ASAP.

mpvader writes: No in-app writing to log files. Instead write to stdout and stderr.

This would imply that SetupHelper could not write to any log file so a script written to use it would have no persistent log.

What is the correct thing to do here ???

At one point, SetupHelper had its own log file: /data/log/SetupHelper. This was abandoned long ago and I’ll modify SetupHelper to simply delete the file.

5

That sounds perfect to me. And its easy to check on a system: make sure there are no logfiles other than the multilog ones.

6

Thanks!

7

Had a thought:

A script that runs at boot time to inspect all log files and trim them if they are “too big”. eg:

tail -1000

This of course requires the log file to be in /data/log or other known places (signal K and node red, etc.)

8

dbus-mqtt-devices uses the python logging framework and runs as a service under daemontools. Daemontools handles log rotation. Shell scripts just echo to stdout. I basically flowed what was used in the Victron code so I hope it’s as it should be! If there is anything wrong with how logging is implemented I am happy to change it.

1 Like
9

hey @freakent , dbus-mqtt-devices is doing it in a good way, thanks for checking in!

1 Like
10

Thanks, yes that is an option. But conflicts with another project: speeding up boot time. Of course this trimming can then be done delayed etc etc. But I prefer to fix the root cause and keep the work where its supposed to be.

11

Yes, I see your concern. You could do this only if the /data partition was close to full since that’s when a problem could occur.