question

mvader (Victron Energy) avatar image
mvader (Victron Energy) asked

Node-RED config files

Hi all, in one of the recent v2.90 betas we changed the setup of config files a bit. Goal of that change is to be better prepared for any future requirements


In this post I’m explaining how it now works, which is in v2.90 official release and onwards.


Warning: the vast majority of systems using (the amazing power of-) Node-RED will not and (should not !!) have to modify anything described here. Venus OS itself, including Venus OS large and Node-RED, is made such that its not necessary to dive into the command line.

Anyone that does go in and change them needs to be prepared for things going wrong, for example after a firmware update. The information here provided is for the benefit of the handful of users that do need (or want) to change them; and to put it bluntly: know what they are doing and are prepared to spent hours and hours fixing issues without getting any help.


On startup of Node-RED on Venus reads three settings files:

1. /usr/lib/node_modules/node-red/venus-settings.js is read first. Never change this file as it will be overwritten by a Venus OS update.

2. /data/home/nodered/.node-red/settings-venus.js is used to override and/or extend the first file and will survive Venus OS updates. Do not edit this file as this is for Victron to use.

3. /data/home/nodered/.node-red/settings-user.js is used to override and/or extend the second file and will survive Venus OS updates. You are allowed to edit this file if you please, but don't be too surprised if that causes Node-RED to break after an upgrade.


There are more configuration files in /data/home/nodered/.node-red/. You might not have all of them on your system or even some more if you installed one or more of our candidate releases.

- flows.json - This is the flowFile, which is used to store your flows. You can also export this file from the Node-RED menu.

- flows_cred.json - If your flows contain credentials/login information, these get stored in this file.

- .config.* - These files are for Node-RED to store its settings.


For more information on configuring Node-RED, you might want to take a look at https://nodered.org/docs/user-guide/runtime/configuration.



Above information will be added to the Venus OS large manual. In case of any things or issues, perhaps you can then help each other here.


Have a good weekend, and fingers crossed for v2.90 official release in coming week! Matthijs

Venus OSNode-RED
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

8 Answers
pcurtis avatar image
pcurtis answered ·

@mvader (Victron Energy) I make one important change in what was data/home/nodered/.node-red/settings.js, which is covered fully in the Node-RED documentation, to give a choice to make global context variables persistent - ie hold their values through a machine reboot which I exploit for many purposes such as switch states and long term averages. It is an addition of 5 lines in the middle of the file.

I am currently away for several weeks so I can not check myself so can I confirm that data/home/nodered/.node-red/settings-user.js is now the equivalent file and such changes can still be made. It was still working without changes after update to ~24

4 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

thomasw-1 avatar image thomasw-1 commented ·

Can be still used:

    contextStorage: {
         default: { module:"memory" },
         storeInFile: { module:"localfilesystem" }
     },

Implemented in "settings-user.js" of my installation and works.

1 Like 1 ·
pcurtis avatar image pcurtis thomasw-1 commented ·

@ThomasW. Mine is effectively the same and, I think, came directly from the Node-RED document on use of 'context'.

 contextStorage: {
    default: "memoryOnly",
    memoryOnly: { module: 'memory' },
    file: { module: 'localfilesystem' }
},

I have always been surprised this has not become the default configuration by Node-RED as it seems to have all the advantages.

0 Likes 0 ·
thomasw-1 avatar image thomasw-1 commented ·

in /home/nodered/.node-red/settings-user.js you should only include those changes, which are different to standard config (i.e. previous changes).
For me it works perfectly fine, especially also after firmware updates.

0 Likes 0 ·
pcurtis avatar image pcurtis thomasw-1 commented ·

@ThomasW. Thanks, That's basically what I have been doing and everything is working well. It has survived a couple of updates as I would expect.

The only difference is that I also copy the comments associated with the section alongside the change so I can understand at a latter stage and also copy the file contents directly onto my web development diary. I find it easy to forget the details of changes as time passes.

On version updates - I also make extensive use of rc.local for GPIOs, permissions of some diagnostics and now for configuring for safe[r] use of sudo by Node-RED and they have all survived so far.


0 Likes 0 ·
tdv avatar image
tdv answered ·

Hi I modified my settings file in a previous version to set contextstorage to localfilesystem.

After the update to 28 I see my settings were backed up. I added the same modification to settings-user.js but my context storage to disk isn't working. It kind of looks like my settings-user.js file is not being read. Also there is no settings-venus.js in the .node-red folder. Did something go wrong with my update? I see the venus-settings.js in the folder where it should be. Any suggestions?


I updated to 29 to see if it solves it but still the same problem.

2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tdv avatar image tdv commented ·
@mvader (Victron Energy) I probably should have tagged you in the question... or let me know if I should rather open a new thread
0 Likes 0 ·
pcurtis avatar image pcurtis tdv commented ·

@tdv I can now confirm the changes I gave above to implement persistent context storage work when transferred to settings-user.js as does the addition of uiHost: "", as suggested by @Jeroen to restore http: access on port 1880 for android users on a local network. It is definitely being read after an update direct from v2.90~26 to v2.91~1 but note a reboot was required before the changes were implemented.

I also do not have a settings-venus.js in the .node-red folder. I assume it will be used in future releases

0 Likes 0 ·
zeron avatar image
zeron answered ·

I have neither settings.js or settings-user.js in my /data/home/nodered/.nodered/ folder.

Shoud I copy the settings.js from /usr/lib/... ? And rename it to settings-user.js once in the nodered home directory ?

Should I leave everything as is except the things I want to change ? Or should that local file only contain my edits/additions ?

1 comment
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pcurtis avatar image pcurtis commented ·

@Zeron Just having the changes in settings-user.js means you do not unnecessarily overwriting any changes made by Victron in a new version.

I also copy the comments associated with the section alongside the change so I can understand at a latter stage and also copy directly onto my web development diary.

I also copy @mvader (Victron Energy)'s warning at the top alongside any changes I document!

0 Likes 0 ·
pcurtis avatar image
pcurtis answered ·

@mvader (Victron Energy) I have just done a completely fresh install of OS v2.91 and I now have the essentially same problem as @tdv in that I initially had neither settings-venus.js or settings-user.js in my /data/home/nodered/.nodered/ folder. I then added my standard additions to a new settings-user.js file but it is not being read.

The only difference between my working system which had been updated to v2.90~28 and to 2.90 release and on to 2.91 thus having an automatically generated the settings-user.js file and the fresh install is that the working auto-generated one has an extra entry:

credentialSecret: "********************************************",

Do I need to add that as there does not seem to be one set in /usr/lib/node_modules/node-red/venus-settings.js ?

I have also added execute permissions to settings-user.js just in case and have been rebooting between changes.

@ThomasW. Did you do a fresh install or did you just update from a working system like I did.

My fresh system is RPI 4B v1.5 with 2 Gbytes. Still connected via Ethernet as well as Wifi, SSH and SFTP working. Node- Red only has a few test flows.

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tdv avatar image
tdv answered ·

@pcurtis mine was an upgrade that gave the same problem. I gave up and ended up editing the settings file in /usr/lib/node_modules/node-red/venus-settings.js - which is exactly what you shouldnt do. But I couldnt get it to reed my settings in the /data/home/nodered/.nodered/ folder. If you can figure out how to get it to read the right settings it would be great to know. I dont want to re flash my cerbo and lose all my other settings - so a factory reset isnt an option for me.


For now I have turned firmware updates off since I will of course lose my settings....

2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pcurtis avatar image
pcurtis answered ·

@tdv I think I have cracked it. The settings-user.js file needs the additions to be wrapped in a module.exports = { .......

}

and does not need the credentialSecret: "c7f9598040465c76c60cc767e7cc76653de473774eba3d6ec110cfef2f4d55e3", line

So my actual working settings-user.js file with all its comments is now:

module.exports = {
//    credentialSecret: "c7f9598040465c76c60cc767e7cc76653de473774eba3d6ec110cfef2f4d55e3",
/*    jeroen  commented · Sep 11 2022 at 2:21 PM
      if you add
          uiHost: "",
      to /data/home/nodered/.node-red/settings-user.js
      the http server will be available again on port 1880
      and that will survive a firmware update.
*/
    uiHost: "",

/*    * Context Storage
      * The following property can be used to enable context storage. The configuration
      * provided here will enable file-based context that flushes to disk every 30 seconds.
      * Refer to the documentation for further options: https://nodered.org/docs/api/context/
      */
     //contextStorage: {
     //    default: {
     //        module:"localfilesystem"
     //    },
     //},
    contextStorage: {
       default: "memoryOnly",
       memoryOnly: { module: 'memory' },
       file: { module: 'localfilesystem' }
    },
}
6 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

tdv avatar image tdv commented ·
Cool when I get a chance I will try that. Would be great to have it functioning like its supposed to.
0 Likes 0 ·
tdv avatar image tdv tdv commented ·

@pcurtis is the settings-venus.js file empty by default?

0 Likes 0 ·
pcurtis avatar image pcurtis tdv commented ·
@tdv It is currently not present at all.


I have been 'exploring'. The venus-settings.js file has been modified slightly from the one provided by Node-RED and has calls to function they have added at top to read and include the settings-venus.js and settings.user.js files IF present at the very end. All very sensible.

There is a Linux program called meld which enables you to compare two files very easily and you can see the other changes - useful to avoid overwriting any of them by settings-user.js.

0 Likes 0 ·
tdv avatar image tdv pcurtis commented ·
Thanks @pcurtis I got the settings-user.js working and could update my install without breaking my settings....
0 Likes 0 ·
pcurtis avatar image pcurtis tdv commented ·
@tdv I have also been right through from flashing a new MicroSD again as a final check and it now works first time as expected with file above.
0 Likes 0 ·
antrac1t avatar image antrac1t commented ·
Hi its still workable even for actual versions of VenusOS?
0 Likes 0 ·
zeron avatar image
zeron answered ·

That's perfect to keep it future-proof against Victron changes in the rest of the settings file. Also is there any reason context storage isn't set like this by the Victron devs?

2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

pcurtis avatar image pcurtis commented ·

@Zeron A very good question why Node-RED themselves and Victron do not include such a change to allow persistent context. I have asked before! I think Victron must prefer to keep to the default Node-RED settings but perhaps one of the devs will see this post and look at it again. It is however very easy to implement now there is a separate user settings file and we, hopefully, understand the format.

I also append the .json file which contains the value pairs to a 'log' file every 2 hours so I have a continuous audit trail.

I have been wondering for while whether we need some Articles in the Modifications area on how to use context, implement persistence and other such enhancements. Use of sudo is another possible as is simple ways to add GPIOs using Node-RED for extra relays. I tend to write everything up as I go on my web site but it would still need a lot of polishing.

1 Like 1 ·
neuer-user avatar image neuer-user pcurtis commented ·
I am wondering if Victron is afraid that the frequent writes (every 30 sec) will slowly wear out the flash memory and thus lead to a too big number of warranty cases ?
0 Likes 0 ·
buddhafragt avatar image
buddhafragt answered ·

I have a problem with a Node-Red node for the i2c bus... I always get the error message "Error: EACCES: permission denied, open '/dev/i2c-1'"

So I think the problem is the node dont have permissions for the i2c. How can I solve this problem?

Which user name the nodes use?

2 comments
2 |3000
Toggle Comment visibility. Current Visibility: Viewable by all users

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

johnny-brusevold avatar image johnny-brusevold commented ·

@buddhafragt

Maybe this can solve the problem for you

https://community.victronenergy.com/questions/196633/how-to-activate-i2c-bus-and-use-it-in-node-red.html?

1 Like 1 ·
buddhafragt avatar image buddhafragt johnny-brusevold commented ·
hey... this was working.....

thanks a lot!!! :-)


0 Likes 0 ·