[Feature Request] - Remote SSH ability

Question

question

5teve asked Jun 22, '21

[Feature Request] - Remote SSH ability

Hi Guys

For those of us running Venus OS devices on board boats and in remote locations that are not manned, it strikes me that a key missing part of the maintenance of these devices is SSH remotely. For those of us not blessed with a clever brain, I cant seem to find a way of accessing my PI via an LTE (4g) connection that resides on the boat due to CGnat. The LTE router that I have only does OPENVPN server, not client (which is useless behind CGnat) so unless i'm on the boat with the laptop I have no access.

Now the like of Freenas have a built in 'terminal' or Shell.. It strikes me that would be a mightily useful thing to include in either the remote console (via VRM) or via VRM itself? It would certainly make life a little easier for doing the things that require SSH..

I know I could just get a router that supports a VPN client, hook it up to my home VPN server and access that way.. but it just seems like a nice touch having remote SSH...

I'm sure being a heathen.. I am ignoring security concerns and a million other things.. but I live in a simplistic bubble.. my brain hurts less that way ;O)

Steve

Venus OS remote console

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Kevin Windrem commented · Mar 30 at 05:54 PM

I just released a beta version of a wrapper to install and setup tailscale on GX devices:

https://github.com/kwindrem/TailscaleGX

The ReadMe.md hopefully provides all that you need to know. Including how to add the package to Package manager so you can install it from the main GUI (gui-v1).

Please have a look but ONLY if you have local ssh access to uninstall the package should something go wrong.

Please post comments and bug reports to the issues section of the GitHub repo.

@mvader (Victron Energy) I would appreciate your feedback too.

1 ·

Kevin Windrem commented · Apr 06 at 05:48 PM

Following on the work by @mvader (Victron Energy) I have released TailscaleGX, a wrapper for tailscale (described in another answer here).

TailscaleGX includes a menu for controlling tailscale and provides the necessary status in order to add he GX device to your tailscale account and establish a connection.

https://github.com/kwindrem/TailscaleGX

TailscaleGX is a SetupHelper package and can be added from Inactive packages / new:

Package name: TailscaleGX

GitHub user: kwindrem

GitHub branch or tag: latest

You also need to set up a tailscale account at tailscale.com.

The ReadMe will hopefully provide enough information to set up a connection.

tailscale provides ssh and http access, and probably many more I haven't tested.

Please post comments, questions, bugs to the issues section of TailscaleGX repo.

Thanks, enjoy

0 ·

mvader (Victron Energy) answered · Feb 16 at 02:48 PM

Hey @XZv I looked into that a bit further, and tailscale looks like the best solution.

Here is why:

to make all of it work well, also when installed behind various types of firewalls, then you need some cloud hosted service / server. Where all devices connect to and where the user can connect to as well.

And security is critical.

To host & offer such service for free, support it, maintain it, have it pentested and so forth by ourselves is a step to far.

The other route, totally open source / developer would be a rc.Local script that enables Wireguard on GX device + a (dockerised?) server that is the pivot. Nice but nerdy and complex.

Enter tailscale. All of it done and they offer a free service.

So, hereby the invitation to you and the rest of Community: can someone work on a script to get tailscale up and running on a GX device + instructions?

And in the interim do it with a one time self compiled tailscale binary - use our SDK to do that. Should be pretty simple.

If that works well, one of our guys can take care of having the binary included as a standard plus having it as enableable daemon which is watched over by Daemontools like all other services on the GX.

And the configuration/provisioning will need to be solved. If all that is needed is some long key, then perhaps gui-v2 allows cut & paste so solves that problem as well.

Note to myself for when including this into the normal Venus OS builds: https://github.com/ChristophHandschuh/meta-tailscale/

And wrt the other solutions:

- wireguard is a protocol; with implementations; tailscale is built in top of that

- zerotier is closed source solution (so I prefer wireguard, but in the end its jusr a choice

- setting up port forwarding, or having to setup and maintain anything at all, is too complex for most of intended user group.

- ngrok is an alternative solution; tailscale seems to be the more popular one after reading around a bit.

8

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

xzv commented · Feb 16 at 07:03 PM

Thanks @mvader (Victron Energy) for the investigation and your thoughts. Today I spent half a day on comparing different solutions and approaches as well. I totally agree, Tailscale looks like the most promising solution right now when the priorities are security, ease of commissioning and reliability.

I expect to have a large fleet of devices that I can access for maintenance remotely once in a while via SSH, so I would highly value a package that will always ship with Venus OS and just needs some initial "machine key" to join your Tailscale network.

I will do some initial tests and report back. Would be good if you or your team could comment on the feasibility of integration into Venus and the GUI then.

0 ·

mvader (Victron Energy) ♦♦ xzv commented · Feb 16 at 09:48 PM

Ok I couldn't help myself, and had a play :)

Here is more or less what I did:

1. Go to tailscale.com and create an account; and installed it somewhere. Either on my computer or iphone or both.

2 Logged on on a GX device, downloaded the latest linux binary for arm https://pkgs.tailscale.com/stable/tailscale_1.60.0_arm.tgz

3. Un tarred the two files and put them in /data/tailscale/

4. Created a run-tailscale file, /data/tailscale/run-tailscale, with these contents:

#!/bin/sh

# Start tailscale daemon and use a dir on data partition
# to keep state (certificates and such). tailscaled will auto-
# create that dir if it doesn't exist.
#
# Logs are written to /var/lib/tailscale, which -on Venus OS-
# is volatile.
#
# To see further commandline options for the daemon, run
#
#     tailscaled -?
#
# I didn't find any docs for its options.

/data/tailscale/tailscaled -no-logs-no-support -statedir /data/tailscale/stat

4. Added the /data/rc.local file, that starts above, with these contents:

#!/bin/sh

/data/tailscale/run-tailscale

5. made both files executable

6. rebooted the GX, and checked that it runs:

root@einstein:~# uptime
 21:32:07 up 1 min,  load average: 2.82, 1.43, 0.54
root@einstein:~# ps | grep tailscale
 1187 root      2952 S    {run-tailscale} /bin/sh /data/tailscale/run-tailscal
 1189 root      541m S    /data/tailscale/tailscaled -no-logs-no-support -stat
 1933 root      2688 S    grep tailscale

7. started up tailscale for the first time, triggers this autoconf:

8. copied and pasted that url into a browser, logged in. It then simply says success and goes back to commandline. Quite nice how they did that! Logging in on that URL auto-provisions that device with proper keys and everything.

9. rebooted the GX again, to see if and how it initialised:

root@einstein:~# /data/tailscale/tailscale status
100.89.189.104  einstein-2           matthijsvader@ linux   -
100.111.71.54   mva-ubuntu2204       matthijsvader@ linux   -

# Health check:
#     - dns-os: writing to "/etc/resolv.pre-tailscale-backup.conf" in rename of "/etc/resolv.conf": open /etc/resolv.pre-tailscale-backup.conf: read-only file system
#     - router: setting up nat/ts-postrouting: running [/usr/sbin/iptables -t nat -N ts-postrouting --wait]: exit status 3: iptables v1.8.4 (legacy): can't initialize iptables table `nat': Table does not exist (do you need to insmod?)
Perhaps iptables or your kernel needs to be upgraded.

root@einstein:~#

10: so that worked nicely, that GX is now connected to my tailscale network, and the connection will survive firmware updates and reboots; until ofcourse something changes that makes the binary incompatible with Venus OS.

Next steps would be:

1. add including tailscale into Venus OS builds, and look into tailscale lite: https://tailscale.com/kb/1207/small-tailscale

2. Catch that login url, and show it in the gui somewhere, extra nice is as a QR code, so users can open it to login that device easily.

3. Document the whole thing.

2 ·

1708119257664.png (12.3 KiB)

mvader (Victron Energy) ♦♦ mvader (Victron Energy) ♦♦ commented · Feb 16 at 09:52 PM

ps, after above process, I could connect to the console on the GX from (for example) my iphone super easily:

1. install the tailscale app, login.

2. once thats done, you'll see a list of online devices, which includes the GX.

3. install a ssh app, for example termius, and login to the ip address of the GX device, and voila.

1 ·

Kevin Windrem mvader (Victron Energy) ♦♦ commented · Feb 17 at 06:57 AM

The archive you linked apparently uses systemd to start tailscaled. Is there a reason you didn't start this as a daemontools service?

Thinking I might want to roll this into SetupHelper until Venus OS includes it.

Would the same arm binary run on all platforms (including RPI)?

0 ·

mvader (Victron Energy) ♦♦ Kevin Windrem commented · Feb 17 at 07:40 AM

Hi @Kevin Windrem , I only took the binaries from the archive. There is no systemd on venus os.

Its not a daemontools service because I didn’t get to that.

Yes, will work on pi’s as well I’d think!

Nice!

0 ·

Kevin Windrem mvader (Victron Energy) ♦♦ commented · Feb 17 at 06:33 PM

Carrying on with @mvader (Victron Energy)'s work:

The linked archive above does run on RPI also. Yea!

Executable size is unfortunately large:

The size of the two executables in the archive link above is about 47 MB.

I built one with the --extra-small flag and it came in at 22 MB with a combined executable for both commands as suggested. (.tar.gz is 8.5 MB).

To build the small tailscale executable:

install 'go' environment per their instructions

clone the tailscale archive

GOOS=linux; export GOOS
GOARCH=arm; export GOARCH
cd <tailscale clone>
./build_dist.sh --extra-small -o tailscale.combined -tags ts_include_cli ./cmd/tailscaled
ln -s tailscale.combined tailscale
ln -s tailscale.combined tailscaled

I then used the commands mvader provided to start tailsale on the GX device.

The login URL appears stable across a reboot but haven't tried a Venus OS firmware update yet. I think mvader did that already though. I did delete the GX from the tailscale console and had to do

./tailscale login

and at that point got a new login URL

Note that a tailscale up or tailscale login will wait for the connection from the tailscale console to complete!!!

So to establish the connection, the GX device needs to set tailscale up or tailscale login and report the login URL on the GUI. The GX device then needs to be connected to the tailscale console from another host.

Next I'll work on a daemontools service to manage all this and put tailscale status and info on dbus and a GUI page.

1 ·

Kevin Windrem mvader (Victron Energy) ♦♦ commented · Feb 17 at 03:40 PM

Thanks for the info. I'll give it a go.

0 ·

xzv mvader (Victron Energy) ♦♦ commented · Feb 23 at 11:12 AM

Great work! Thanks for sharing! Appreciate it!

0 ·

Modifications Space

Node Red Space

Answer 1 · 2021-06-22T12:43:11Z

mvader (Victron Energy) answered · Jun 22, 2021 at 12:43 PM

Hi, one available solution is zero tier:

https://community.victronenergy.com/questions/81273/remote-access-to-venusos-zerotier-one-installation.html

I will need some self compiling and such. It shouldn't take you more than a few hours of work.

And, for sure it would be great to have a solution natively in Venus OS; but its not something we'll be focussing on anytime soon I'm afraid.

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Answer 2 · 2021-06-22T16:09:43Z

Warwick Bruce Chapman answered · Jun 22, 2021 at 04:09 PM

Put a Mikrotik router in-front of the GX-device and use one of its tunnelling features. I setup a Wireguard server on a Hetzner cloud container that cost a couple Euro and tunnel from the Mikrotik to that server. I can then port forward the Mikrotik to the GX-device and SSH to the GX-device from the cloud container.

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

Warwick Bruce Chapman commented · Jun 22, 2021 at 04:23 PM

Actually, something like this looks nicer using Zerotier and a Raspberry Pi: https://iamkelv.in/blog/2017/06/zerotier.html

0 ·

nebulight commented · Jan 07, 2022 at 07:17 PM

Mikrotik now natively supports ZeroTier right in their router OS 7. It's so easy to get up an running, it's amazing that it's free (for personal use). Zero firewall rules needed.

0 ·

Answer 3 · 2021-06-22T16:37:31Z

Kevin Windrem answered · Jun 22, 2021 at 04:37 PM

A huge security hole, but you can forward port 22 to your Venus device to allow SSH from the outside world.

The VPN solution is much more secure and the one I'd recommend.

2

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

gone-sailing commented · Jun 05, 2023 at 06:21 AM

The issue here is CGNAT, your router’s IP address is not addressable from the internet behind CGNAT. VPN is the only solution. Unfortunately I have to deal with CGNAT on the boat and at home.

0 ·

mvader (Victron Energy) ♦♦ gone-sailing commented · Jun 05, 2023 at 07:37 AM

have you looked at this?

https://community.victronenergy.com/questions/81273/remote-access-to-venusos-zerotier-one-installation.html

0 ·

Answer 4 · 2023-06-01T23:05:55Z

mvader (Victron Energy) answered · Jun 01, 2023 at 11:05 PM

This looks like a really promising solution to me /

https://community.victronenergy.com/questions/205323/venus-os-ngrok.html

3

2 |3000

Attachments: Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

iv4n commented · Jun 06, 2023 at 05:20 PM

Pretty good indeed! I still hope that one day Victron adds the package wireguard-tools so we can setup wireguard on the gx device.

0 ·

iv4n iv4n commented · Jun 07, 2023 at 05:45 PM

I was able to compile it from source! I will post these as an reply on the main thread.

0 ·

mvader (Victron Energy) ♦♦ iv4n commented · Feb 16 at 03:12 PM

- Per Venus OS v3.10 (or something) wireguard kernel module is included.

- Per Venus OS v3.20, wireguard tools are standard in our online repository of installable packages, and installable with opkg

And for anyone looking for the most simple solution, see Tailscale. I’m hoping for some volunteers to help document and do the leg work for that; perhaps start with simple venus-data-tailscale.tgz package.

question