question

benc-ipind avatar image
benc-ipind asked

Gaps in logging on RPi Venus OS

I have an RPi 3B+ running v2.87 with GuiMods on a 32GB SD card.


The partition size is the default, so there is ~11GB of available space for the OS to write to.


I am finding that when Venus cannot see the internet for a day or so, it stops logging to the card intermittently.


Rebooting, full reinstallation and even using a different SD card does not resolve the issue.


Logging is set at 1min intervals.


Any assistance is greatly appreciated.


Ben.

Here is a month worth of data to show the random gaps:


1658094452899.png

VRMVenus OSRaspberry Pi
1658094452899.png (81.9 KiB)
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

4 Answers
Louis van der Walt avatar image
Louis van der Walt answered ·

Check that you have an large enough power supply for the Pi. If you use a few USB connections or perhaps a hub you need at least 2.5A PSU.

1 comment
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

benc-ipind avatar image benc-ipind commented ·
Thanks.


It's on a 5A 12V to 5V PSU.

0 Likes 0 ·
johanndo avatar image
johanndo answered ·

Did you auto-reboot the pi when connection to vrm is lost or for any other reason?

The pi has no RTC, so the timestamp of your logs are then 1970, and your data transferred to the VRM has this time stamps (epoch data). You may add a gps receiver or a rtc as time source when the internet is unavailable and you need to reboot your device. Without getting time from a ntp server over the internet or another local source you will loose the logs.

The logs will be fine as long as the pi remains running and is not rebooted , it counts the time internally during the offline phases.

5 comments
2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

benc-ipind avatar image benc-ipind commented ·
Hi @JohannDo,


Thank-you for coming back to me.

The 'reboot on lost connect' option is turned off.

I suspect you may be on the money about the RTC thing though. The RPi is probably only getting it's time via NTP when I have the boat close enough to my home WiFi or when I turn on mobile hotspot on my phone.

If it gets rebooted without either of these internet sources available, then that would explain the 'gaps'.

Are you able to assist with getting a DS1307 or DS3231 RTC module working with Venus OS?

1 Like 1 ·
johanndo avatar image johanndo benc-ipind commented ·

Sorry, we are sailing in Colombia and later to San Blas with no access to parts and very little Internet access. But it shouldn't be a big issue to add a RTC to the Pi I think, there are lots of examples on how to do it.


I have solved the issue the other way around (i hope), I have a U-blox GPS mouse connected to my RPi / Venus that is supported out of the box, it provides location and speed data to the Venus OS. A great benefit / side effect is, GPS is a very precise time source too. I installed a ntpd deamon on the pi and configured it to gather the time from the GPS receiver and propagate it to the local network (and use it itself of course). So I should have a reliable rtc this way, but I still avoid rebooting my pi, it hangs / halts sometimes instead of rebooting and I have to temporarily unplug the sensors to get it back running. Have not found a workaround for this yet

To install any missing services (dnsmasq, ntpd, hostapd or whatever) you can use the opkg package manager, most essentials are there.

.

1 Like 1 ·
laurenceh avatar image laurenceh johanndo commented ·
I have a shell script running from rc.local that collects the time and date from the connected GPS module and sets the Rpi time and date on every reboot. A simple fix and easy to install an no extra hardware or tinkering required. I’ll post the script tomorrow.
1 Like 1 ·
johanndo avatar image johanndo laurenceh commented ·
I integrated it into the ntpd as time source, it is reboot safe and also supplies time to the wireless sensors.
0 Likes 0 ·
benc-ipind avatar image benc-ipind johanndo commented ·

Thanks.


I just went and picked up a DS3231 RTC module and have got it working. You can also use a DS1307, but the DS3231 is more accurate.


Now to see if it resolves the issue.


Here are the instructions on how to make the RTC module work on Venus OS. The DS3231 modules work with the DS1307 kernel module.


Download the required overlays:

ds1307 rtc overlay download: https://github.com/PiNet/PiNet-Boot/raw/master/boot/overlays/ds1307-rtc-overlay.dtb

i2c-rtc overlay download: https://github.com/PiNet/PiNet-Boot/raw/master/boot/overlays/i2c-rtc-overlay.dtb


Copy the .dtb files to /u-boot/overlays

Add the following to [all] section of /u-boot/config.txt

#ds1307 rtc interface - lives on the i2c bus
dtoverlay=i2c-rtc,ds1307-rtc

Save the changes

Go to /data and create and rc.local file then edit it and add the following line:


echo ds1307 0x68 > /sys/class/i2c-adapter/i2c-1/new_device
hwclock -s

Save and exit

Make the file executable

chmod 755 rc.local


Install the RTC kernel module

opkg install kernel-module-rtc-ds1307


Reboot and run the following command to check the RTC time

hwclock -r







0 Likes 0 ·
johanndo avatar image
johanndo answered ·

Well done, easy peasy...

2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.

laurenceh avatar image
laurenceh answered ·

So this is an answer of sorts:

If you have a connected USB GPS module, and most do seem to work. One of the features of GPS is that is has to know the time quite accurately to work, and it get the time from the GPS satellites.

And this data is available in the messages being transmitted to the Venus device serially over the USB. You can see the data as it arrives:

cat /run/serial-starter/gps/ttyACM0

The data format for each line is well documented if you search online.

So here is what I have done.

/data/rc.local is easy, add this line:

/data/gps.sh > /var/log/gps-date &

Now the bit that took time to write and debug. Don't ask me how it works I did it a couple of years ago and shell scripting is arcane.

The script gps.sh is:

#!/bin/bash
#add this line to /data/rc.local
#/data/gps.sh > /var/log/gps-date &
N=0;
FILE=/run/serial-starter/gps/ttyACM0
echo "Waiting for GPS data"
while [ ! -e $FILE ] && [ $N -lt 50 ];
do
    sleep 1;
    ((N++))
    echo -n "."
done;
sleep 5;

if [ $N -ge 17 ]; then
   echo " GPS data not available is the device plugged in"
   exit 0;
fi

echo
echo "Device attached waiting for data"

input=$FILE
N=0

# check if GPS has FIX
LINE1='^\$..GGA,[0-9\.]+,[0-9\.]+,[NS],[0-9\.]+,[EW],([0-2])'
# Catch lines with date and time info
LINE2='^\$GPRMC'

FIX=""

# extract date and time from GPS packet

function extract {
[[ $1 =~ ^\$GPRMC,([0-9]{4})([0-9]{2})\.[0-9]{2},.,[0-9\.]+,[NS],[0-9\.]+,[EW],[0-9\.]*,[0-9\.]*,([0-9]{2})([0-9]{2})([0-9]{2}), ]]
  echo "found this"
  COMMAND="20${BASH_REMATCH[5]}${BASH_REMATCH[4]}${BASH_REMATCH[3]}${BASH_REMATCH[1]}.${BASH_REMATCH[2]}"
  echo command is
  date -s $COMMAND
exit 0
}

while IFS= read -r line && [ $N -lt 2000 ] 
do
#  echo "$line"
  [[ $line =~ $LINE1 ]] && FIX=${BASH_REMATCH[1]} && echo "GPS device has a fix after $N lines"
  [[ $line =~ $LINE2 ]] && [[ $FIX -eq 1 ]] && extract $line
  ((N++))
done < "$input"
echo "Timed out no date/time information in 2000 lines from GPS receiver" 

The log output looks like this:

Waiting for GPS data


Device attached waiting for data
GPS device has a fix after 1 lines
found this
command is
Sun Jul 24 19:25:48 UTC 2022

The fix after n lines will take longer (than 1) on a system that has just been switched on.





2 |3000

Up to 8 attachments (including images) can be used with a maximum of 190.8 MiB each and 286.6 MiB total.