Idea

laurenceh avatar image
laurenceh suggested

Setting time without internet connection on Rpi Venus OS

As I will be using my Venus OS Rpi3B+ on my boat (Lady's Smock) I will not always have an internet connection when I switch the Raspberry Pi on, and the Raspberry Pi does not have an RTC.

I wonder if others have this problem in remote or mobile (motorhome/boat) installations?

In my case I will always be using GPS and should be more likely to have a GPS data fix than an internet connection.

The script here sets the data and time from the GPS data. It waits around 100 seconds for the GPS device to be found (or start up) and collects up to a hundred lines from the GPS device looking to check that

(1) It has a satellite fix

(2) It can provide the data and time

Once decoded it automatically sets the Rpi date and time according to the GPS data.

It has been tested with Venus OS 2.57 in July 2020.

I will look at adding it to /data/rc.local so that it runs at start up. I would not at this stage suggest running it continuously to provide updates as it might conflict with the NTP server system used by default in Venus OS.

The location of the GPS device "/run/serial-start/gps/ttyACM0" is hard wired at present and may need to be changed if you are using an NMEA GPS interface.

It relies on GPGGA and GPRMC data packets from my GPS device - I presume these are pretty standard across devices.

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;

if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi

echo
echo "Device attached waiting for data"

input=$FILE
N=0

# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GPRMC'

FIX=""

# extract date and time from GPS packet

function extract {
[[ $1 =~ ^\$GPRMC,([0-9]{4})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}), ]]
  COMMAND="${BASH_REMATCH[5]}${BASH_REMATCH[4]}${BASH_REMATCH[3]}${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
  date -s $COMMAND
exit 0
}

while IFS= read -r line && [ $N -lt 100 ]
do
#  echo "$line"
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 1 ]] && extract $line
  ((N++))
done < "$input"


Venus OSVenus GX - VGXRaspberry Pi
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

laurenceh avatar image
laurenceh commented

This now works from /data /rc.local on my Raspberry Pi.

Add the following lines to /dat/rc.local

if test -f /data/venus-rpi-setup/gps.sh; then
 /data/venus-rpi-setup/gps.sh &>/var/volatile/log/gps-time &
fi

Clearly you can change the location where you store the script and where you want the log to go.

Notes:

1) Don't forget the & at the end of the line it is important to make sure it runs as a background process otherwise it stops the rest of the initialisation.

2) You can't rely on standard output to log the output information. The standard boot output logging which goes to /var/log/boot can be shut down before the script terminates so you lose output messages.

3) I changed this line in the original script to read 200 lines as sometimes the script terminated before finding the data it needed in the GPS data stream.

while IFS= read -r line && [ $N -lt 100 ]

Change to:

while IFS= read -r line && [ $N -lt 200 ]

It would be nice if Victron added something like this into the start up sequence as they already detect cases where there in no hardware clock and should be able to detect if there is no route to an NTP server.

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

natteverf avatar image
natteverf commented

Hi
does this script still work ok?
I get multiple "GPS device has a fix" followed by "date: invalid date '.'"

5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

johnny-brusevold avatar image johnny-brusevold commented ·

Tested it and got it working with some changes due to my usb-gps (ttyUSB3) using the $GNRMC string.

I didn't get "date -s $COMMAND" to work either, so I changed the code a bit.

Pi3b+ venus-os 2.90

#!/bin/bash
N=0;
FILE=/run/serial-starter/gps/ttyUSB3
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 100 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 1;
 
if [ $N -ge 99 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi
 
echo
echo "Device attached waiting for data"
 
input=$FILE
N=0
 
# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GNRMC'
 
FIX=""
 
# extract date and time from GPS packet
function extract {
[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]
  COMMAND="date -s '20${BASH_REMATCH[6]}-${BASH_REMATCH[5]}-${BASH_REMATCH[4]} ${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]}'"
  eval $COMMAND
exit 0
}
 
while IFS= read -r line && [ $N -lt 200 ]
do
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 2 ]] && extract $line
  ((N++))
done < "$input"
0 Likes 0 ·
natteverf avatar image natteverf johnny-brusevold commented ·

Thanks for your help

I am not getting an error now, but i have no idea if it is really working, i created the rc.local entry as you said and there is an output in /var/volatile/log/gps-time of multiple "GPS device has a fix" after a restart

0 Likes 0 ·
johnny-brusevold avatar image johnny-brusevold natteverf commented ·

If everything is ok, you get this when you run the bash script from terminal, and the same in /var/volatile/log/gps-time

root@raspberrypi2:~# /data/GpsClock/gps-clock.sh
Waiting for GPS data

Device attached waiting for data
GPS device has a fix
Thu Sep 22 17:28:00 UTC 2022


You probably have to edit line 33 $GxRMC to match your string from gps

Mine is $GNRMC for multi (GPS, Galileo and GLONASS)

$GNRMC,174721.000,A,5993.123983,N,01074.878083,E,0.01,148.82,220922,,,D*7C

And this code picks data from gps string

[[ $1 =~ ^\$GNRMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{3},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]


You get $RxRMC data from your GPS.

cat /dev/your-GPS |grep RMC

0 Likes 0 ·
natteverf avatar image natteverf johnny-brusevold commented ·

ok, the output of /var/volatile/log/gps-time is this:

Waiting for GPS data

Device attached waiting for data
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix
GPS device has a fix

which i guess looks like it is working.
one of my $RxRMC data lines looks like this:

$GPRMC,201015.000,A,5222.7424,N,00458.1954,E,0.00,356.24,230922,,,D*60

do i need to change lines 27 and 33 to match?

0 Likes 0 ·
natteverf avatar image natteverf natteverf commented ·

I changed line 33 but got the same results, so i changed line 27 also and it is working properly now :-)

Thanks @Johnny Brusevold for your help with this.
I am not a scripting guru, but would it not be possible to make the script a bit more agnostic by somehow saying G*RMC? Or are there more options that can appear there? I am also thinking about maybe detecting the usb port because mine has changed for some reason sometimes on a restart. Also a Pi3 w venus-os 2.90 large

0 Likes 0 ·
laurenceh avatar image
laurenceh commented

Hi

I wrote the original code and guess I can fix it.

I have a GitHub repository and my new services package of services there has the fix for the date command format in the line that starts COMMAND=

Basically I wrote the code in 2019, at that time a date format yymmdd was sufficient but somewhere after 1/1/2020 the format was changed to yyyymmdd so the code needed to “20” to be added to the command ( it should now work until 1/1/2100 by which time non of us will care ;-). I’m Not sure if the format change was just as the year changed or because of a change in the base code of Venus OS.

Either way new code on GitHub published a few months ago fixed that.

The code looping saying GPS has a fix, is NOT an indication of working. The code is just looping saying it has a fix but still can’t find the line with the time/date information. This is probably due to the noted GxRMC problem.

I’ll fix the code on GitHub to recognise GxRMC but others will need to test it.

5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

natteverf avatar image natteverf commented ·
I am happy to test of course :-)

What is the GitHub repository?


1 Like 1 ·
laurenceh avatar image laurenceh natteverf commented ·

I have updated the script here

https://github.com/LHardwick-git/Victron-Dbus-Services/blob/main/Rpi-GPS-time/gps-time.sh

To accept different GPS speakers so it should accept xxRMC.

The NMEA Spec is here

https://gpsd.gitlab.io/gpsd/NMEA.html#_rmc_recommended_minimum_navigation_information

Currently untested but the change was trivial.

0 Likes 0 ·
natteverf avatar image natteverf laurenceh commented ·

Hi

I know it's been a long while, i was suffering from terrible interference issues with my GPS that caused my Pi3 to randomly freeze up, finally resolved using some ferrite cores around the usb cable, amazing what a difference that makes!

I have tested your "agnostic" script, unfortunately it doesn't work and produces the constant "GPS device has a fix after x lines" until it times out.

0 Likes 0 ·
natteverf avatar image natteverf natteverf commented ·

It looks like the data lines are a slightly different format, I have no idea in the world how to fix that

GNRMC,165306.00,A,5222.74129,N,00458.18549,E,0.020,,300523,,,D*6F


0 Likes 0 ·
natteverf avatar image natteverf natteverf commented ·

I am proud of myself! I used bits of both of your scripts and one minor change to the REMATCH and it works!

                                                                                                    
#!/bin/bash
#add this line to /data/rc.local
#/data/gps-time.sh > /var/log/gps-date &
# This script read the GPS data from a USB connected dongle and sets the Rpi
# time and date from the GPS data received.

N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 50 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 5;

if [ $N -ge 17 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi

echo
echo "Device attached waiting for data"

input=$FILE
N=0

# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$..RMC'

FIX=""

# extract date and time from GPS packet

function extract {
[[ $1 =~ ^\$..RMC,([0-9]{2})([0-9]{2})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}),.* ]]
  COMMAND="date -s '20${BASH_REMATCH[6]}-${BASH_REMATCH[5]}-${BASH_REMATCH[4]} ${BASH_REMATCH[1]}:${BASH_REMATCH[2]}:${BASH_REMATCH[3]}'"
  eval $COMMAND
exit 0
}

while IFS= read -r line && [ $N -lt 200 ]
do
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 2 ]] && extract $line
  ((N++))
done < "$input"
echo "Timed out no date/time information in 2000 lines from GPS reciver"


0 Likes 0 ·

Your Opinion Counts

Share your great idea, or help out by voting for other people's ideas.