question

First of all, don't use power supply mode when connected to any battery bank. There's no control over charging current in this mode so you could damage your batteries with either too much current or too high a voltage.

With no load, the output voltage will fall slowly because of capacitors on the output. Your best bet for adjusting power supply mode voltage is to connect a small load. This will allow the voltage to fall more rapidly when the voltage is turned down. A couple of automotive 12 volt bulbs connected in series for 24 volt adjustments should be fine.

I discovered that you need to disconnect both voltage sense leads. Just disconnecting one so it's open circuit isn't enough.

So the procedure, which hopefully someday will be in the manual, is to disconnect both voltage sense wires, and disconnect the battery output. I'm guessing you probably need to disconnect all the battery outputs, not just the main one, but I haven't tried that. Presumably Victron has and know the answer. Then you can adjust the output voltage, with a note that any decrease in output voltage will take a while to be reflected in the output, so it's best to start low and creep up on the desired voltage, avoiding any overshoot. If you happen to carry around some incandescent 24V light bulbs and clip leads, then they can provide a small load to help with down-adjustment. But those aren't normally in my bag. Can't even think when I last saw or used an incandescent bulk.