I am trying to store the data from the Cerbo Gx and keep this in an SQL database. (I am not a programmer or tech nerd but also not completely stupid.)
The first thought was to try and capture the data log from the Cerbo and transferred via packets to VRM. Unfortunately the packets are encrypted. The VRM upload data would be ideal because it should be a snapshot at 1 minute intervals.
The second thought is to use MQTT. This will capture the data each time a new record is added, but each broker message can be multiple times per minute and will need quite a deal of data adjustment to get an average 1 minute data point.
The third thought was to download CSV data (presently done with each CSV file period set for 1 week). But this has some major issues. The first is the multiple variations in column headings between weeks. For example one CSV file covering 2026/02/10-2026/05/16 records 184 column. Another covering 2026/06/21-2026/06/27 has a total of 154 columns. So a deal of data manipulation needs to occur if the CSV file is to be imported into the database. (Of the column heading differences between the two files 50% relate to DESS fields (DESS is not enabled in my system as I am Off Grid), 40% relate to Battery Monitor fields and the remaining 10% relate to 2 fields in each of the Meterological Sensors of which I run 3 Solar Sense units.) Another major issue is that in the downloaded CSV data files, data is not confined to a 1 minute interval. Two column fields (Cerbo Relay State and SolarSense power output ‘Watts’) are the culprits. For the CSV file 2026/6/21-2026/06/27 a total of 58911 timestamp records are included, a 1 week with 1 minute interval times should produce 10080 timestamp records. The resulting display of data with an SQL query is therefore compromised as I have set up a 1 table database with timestamp as the primary key.
I was hoping that a SolarSense firmware update and a Cerbo firmware update done today may have addressed these issues but it has not. It reminds me of Donkey from Shrek jumping up and down screaming “pick me, pick me”.
Ideally I would really like if Victron would make available the 1 minute data logs ( a simple query can isolate the the 2 overzealous sensors and a mass delete will then correct the data). An example of such an approach is in my Meteobridge router that handles Davis Vantage Pro weather data. The Meteobridge maps all the weather station sensors and I am able to write an INSERT INTO statement for the sensor data that I want to store and this is then sent to a Raspberry Pi SQL server and recorded. This works very well and if I add a new sensor Meteobridge displays the new mapping structure and the new sensor is then available to be added to the SQL query for the data collection. So for instance is I added a soil moisture sensor, I would find the sensors mapping reference and then add this to the Sql insert query, add the required field to the database table and unless I buggered up my syntax all should run smoothly. Meteobridge gives quite a number of different timestamp intervals but these are not important in the instance of Cerbo data.
How are others handling data storage?