Feature request: Standardized “Mod Manifest” for Improved Diagnostics on Modified Systems

As the ecosystem of third-party drivers (SerialBattery, custom BMS integrations, Aggregators) grows, the generic “Modified” banner on Venus OS is becoming less useful for diagnostics. Currently, it tells us that the system is modified, but not what is running, if that code is up-to-date, or if the user has broken the script logic while trying to change a setting.

I propose a lightweight standard where installers drop a JSON manifest into a specific directory (e.g., /data/etc/venus-mods/). This would allow the system, support staff, or UI to easily list installed extensions without digging through the filesystem.

My Proposal

  1. Standard Location: /data/etc/venus-mods/
  2. Standard File: unique-package-id.json containing version, source, and a smart checksum map of the active files.

Benefits

  • Granular Visibility: Instead of just “Modified”, we can see “Modified by: Samsung-SDI-Driver v1.1.0”.
  • Smart Integrity Check (The “Seal”): The system hashes all executable code (.py, .sh) but intentionally ignores configuration files (.ini, .conf, .json).
    • Result: If a user changes config.ini to set their battery capacity, the hash remains valid (Green).
    • Result: If a user edits driver.py and breaks the logic, the hash mismatches (Red/Tampered).
  • Conflict Resolution: Easier to see if two installed mods are trying to control the same hardware.

Reference Implementation (Shell/Python Polyglot)

I have implemented this in my drivers. Since Venus OS ships with Python 3, we can use it to perform robust, recursive hashing without complex shell dependencies.

Here is the register_mod function that any script developer can include in their install.sh:

#!/bin/bash
# Venus OS Mod Registration Standard
# Usage: register_mod <id> <name> <version> <repo_url> <install_directory>

register_mod() {
    local MOD_ID=$1
    local MOD_NAME=$2
    local MOD_VERSION=$3
    local MOD_REPO=$4
    local MOD_PATH=$5

    # Use embedded Python for robust JSON handling and recursive hashing
    python3 -c "
import json
import os
import hashlib
import datetime

def get_hash(path):
    with open(path, 'rb') as f:
        return hashlib.md5(f.read()).hexdigest()

data = {
    'id': '$MOD_ID',
    'name': '$MOD_NAME',
    'version': '$MOD_VERSION',
    'repository': '$MOD_REPO',
    'installed_at': datetime.datetime.utcnow().isoformat() + 'Z',
    'files': {}
}

root = '$MOD_PATH'
# Recursive walk that audits code but ignores config
if os.path.exists(root):
    for dirpath, dirnames, files in os.walk(root):
        if '__pycache__' in dirnames: dirnames.remove('__pycache__')
        
        for f in files:
            if f.startswith('.'): continue
            if f.endswith(('.pyc', '.log', '.txt', '.md')): continue
            # IGNORE CONFIGS: Allow users to edit settings without breaking integrity check
            if f.endswith(('.ini', '.yaml', '.json', '.conf', '.xml')): continue
            
            full_path = os.path.join(dirpath, f)
            rel_path = os.path.relpath(full_path, root)
            try:
                data['files'][rel_path] = get_hash(full_path)
            except: pass

manifest_dir = '/data/etc/venus-mods'
if not os.path.exists(manifest_dir):
    os.makedirs(manifest_dir)

with open(f'{manifest_dir}/{data['id']}.json', 'w') as f:
    json.dump(data, f, indent=2)
"
    echo "Module '${MOD_ID}' registered to Venus OS manifest."
}

Example Usage in install.sh

# During installation...
register_mod "samsung-sdi-driver" \
             "Samsung SDI Integration" \
             "v1.1.0" \
             "https://github.com/drurew/samsung-sdi-victron-integration" \
             "/data/samsung-sdi" 

Resulting Manifest (/data/etc/venus-mods/samsung-sdi-driver.json)

Notice how config.ini is absent, allowing for user customization, while the core logic is fingerprinted.

{
  "id": "samsung-sdi-driver",
  "name": "Samsung SDI Integration",
  "version": "v1.1.0",
  "repository": "https://github.com/drurew/samsung-sdi-victron-integration",
  "installed_at": "2024-06-15T10:30:00Z",
  "files": {
    "samsung_sdi_bms_service.py": "5d41402abc4b2a76b9719d911017c592",
    "samsung_sdi_can_client.py": "7d793037a0760186574b0282f2f435e7",
    "install.sh": "1a79a4d60de6718e8e5b326e338ae533"
  }
}

Extension: Request for Flexible “Blind Install” Filenames

In addition to the Manifest system, I would like to propose an enhancement to the Automatic Install (USB/SD Card) mechanism found in /etc/rcS.d/S90setup.

Current Limitation:
The system strictly looks for a file named venus-data.tar.gz. This creates confusion for users and prevents different drivers from sharing a USB stick installer.

Proposed Feature:
Allow the system to scan for *.venus-data.tar.gz.

Example Workflow:

  1. User places samsung-sdi.venus-data.tar.gz on USB.
  2. User places mqtt-driver.venus-data.tar.gz on USB.
  3. Venus OS iterates through all matches → Extracts → Runs setup → Renames to .done.

Why this is the “Endgame” for Mods

I’ve seen discussion about using opkg or custom repos. While opkg exists, Venus OS wipes the rootfs on every update, making standard package management “volatile”.

The real solution is a First-Party Frontend for modifications. But for Victron to ever build that, they need a standardized way to know what a mod is.

Why the Manifest is the missing piece:

  1. Metadata Equality: Whether a mod is installed via opkg, a shell script, or SetupHelper, it writes a .json manifest. This gives the OS a unified way to “audit” the system.
  2. VRM Visibility: If this metadata is standardized, it becomes trivial for the VRM portal to report installed mods remotely.
  3. Transition to Repos: If we eventually move to a “Custom Repo” model, the transition is easy because we’ve already standardized the metadata schema.

I have updated all my repositories to auto-generate this JSON. Even if Victron doesn’t use it today, having a directory of “Installed Mods” on every system makes it much easier for the community to help troubleshoot.

1 Like

FYI, SetupHelper/PackageManager does some of this already but in a different way.

When SetupHelper installs a package:

It writes a file /etc/venus/installedVersion. (e.g., installedVersion.GuiMods. This file includes the version that was installed. PackageManager uses this to report which packages are installed and the installed version, but it could also be used by the OS to identify which packages have been installed.

It writes files named /etc/venus/installedModificaitons/installedFiles- and …/installed-Services- indicating which files have been modified by the package.

Each file that is modified also adds the package name in a file named …package. This indicates which package or packages modified the file.

Each file that is modified is backed up (e.g., /opt/victronenergy/gui/qml/main.qml.orig) before it is modified. For files that are modified by more than one package, a reverse patch is used to remove the modifications rather than restoring the .orig

Uninstalling a package involves restoring each file with the .orig backup (or reverse patching the file) and removing the /etc/venus/installed… files and removing the package name from each .package file.

Any information that needs to persist a firmware update is stored in /data/setupOptions/. This is used for reinstalls when command-line options are necessary.

A document included with SetupHelper provides more details of what SetupHelper and PackageManager do.

I am not saying SetupHelper’s mechanism is better, just that it is similar and already exists for those packages managed by SetupHelper.

Hey Kevin, thanks. I think this highlights exactly where the current ‘community’ method reaches its limit and where we need to head for a more ‘Linux-native’ experience.
The core issue I’m trying to solve is that the current method, while effective for persistence, is essentially a firmware update workaround. It requires users to be terminal-proficient and forces Victron support into a ‘forensics’ role every time a system fails. They have to dig through .orig files and /etc/venus/ logs just to see who to blame for a crash.

A JSON manifest is easily picked up by the VRM agent. Victron shouldn’t have to SSH into a Cerbo to see what’s running; the VRM dashboard should show a list of third-party maintainers and their support links automatically.
By including a support field and an integrity hash, we point the manufacturer (and the user) directly to the source of the issue.

We can’t have a ‘Custom Repo’ or a ‘Third-Party App Store’ in the Venus OS UI until we have a standardized metadata format. The manifest is the first step toward a UI where a user can simply click ‘Install’ or ‘Update’ without ever opening a terminal.

This saves user, installers, distributors and victron staff from having to stop work every time a new cerbo is born.

You and I wouldn’t have to have this discussion, and half the posts on this forum could be avoided, if there were just a frontend installer with third-party repo support.

​To frame it in an analogy: the person who decided the frontend won’t have an installer seems to be an “Arch dude.” That person is forcing their misery on all new users by essentially saying, “If you can’t install a package manually, you shouldn’t be running my OS.” That is just mean to a new user who just wants to charge their batteries and see a cool display.

​As I’ve said in my other rants, that is the impression the system leaves on newcomers. There is a whole ecosystem of complexity surrounding that single choice. Victron has made steps to improve, and you’ve provided brilliant workarounds—but they are still workarounds. Not solutions

There would appear to be a reason why every single nix derivative uses a variation of a frontend installer, why an open source is simply refuses on what essentially is an Arduino is beyond what I can comprehend.