Author Topic: create webcam images & time lapse video with Ecowitt HP10  (Read 2254 times)

0 Members and 1 Guest are viewing this topic.

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
create webcam images & time lapse video with Ecowitt HP10
« on: January 01, 2023, 05:44:13 PM »
Hi!

For the HP10, Ecowitt offers an upload service for webcam images and time lapse videos at Ecowitt.net.
However, these pictures as well as the time-lapse videos are difficult to integrate into your own web pages and the holding time of the videos @Ecowitt.net is unfortunately only 7 days.
In addition, I personally miss the essential weather data of the weather station:

 [ You are not allowed to view attachments ]

Fortunately, the HP10 offers an easy-to-use interface to generate your own images and videos. I had already written this in my review of the HP10.
And that too, completely locally and without the aid of any cloud services!
All that is required for this solution is a 24/7 Linux system (possibly also under Windows with WSL - untested).

Since the script generates local files, the possibilities for further processing and integration into existing web pages are almost unlimited.
So I use the single images and the time-lapse videos (after a few minor changes in the original PWSDashboard files) on my PWSDashboard page.
I can give more information about the necessary changes in the PWSDashboard files here.
However, the pictures and videos can be integrated everywhere in this way, because the names are always the same and the files are overwritten when they are transferred to the target server.

If you already use FOSHKplugin, you can adopt this script without major changes. However, a few more tools may have to be installed.
Required are ffmpeg, curl, bc and possibly recode - these are to be installed with the respective package manager.
For Debian-like distributions this is done with
Code: [Select]
sudo apt-get install ffmpeg curl bc recode
The weather data to be embedded is taken from FOSHKplugin via http/GET and embedded in the individual images.
Users who have not yet used FOSHKplugin - but instead use a "more modern" console with http query option - can still embed the current weather data in the webcam images. I have already written more about this nice feature here.
For processing the JSON responses of the console, another package has to be installed: jq (sudo apt-get install jq).

Basically, this script works with ANY other camera that can output a current image via a software trigger. However, depending on the camera model and possibilities, further adjustments are necessary.
The actual procedure is always:
  • fetch the current image from the camera and save it as YYMMDDHHMM-capture.jpg
  • if necessary, fetch weather data; integrate as overlay via ffmpeg and save as new image YYMMDDHHMM-snapshot.jpg
  • Copy the last file as capture.jpg or snapshot.jpg and transfer it via FTP to the target server
  • at midnight, create a time-lapse video as YYMMDD-snapshot.mp4 (or YYMMDD-capture.mp4) from all locally available YYMMDDHHMM*.jpg files using ffmpeg
  • copy YYMMDD-snapshot.mp4 (or YYMMDD-capture.mp4) to snapshot.mp4 (or capture.mp4) and transfer to the target server via FTP
  • when called with parameter -timelapse, create a tlpartial.mp4 from the existing YYMMDDHHMM*.jpg possible up to this point and transfer it to the target server
It is advisable to start this script via cron every n minutes - this will then fetch a new image from the camera every n minutes, store it locally and transfer it to the server as a single image.
In addition, there is the possibility of partial time-lapse videos. These can also be started at set times via cron. The script has to be started with the additional parameter -timelapse.
In my case, an image is generated for the webcam every minute and uploaded to the web server. In addition, a time-lapse video is generated at regular intervals from the images available up to that time:
Code: [Select]
*/1 * * engel /home/engel/getHP10/getHP10dm.sh
0 10,12,14,16,18 * * * engel /home/engel/getHP10/getHP10dm.sh -timelapse

About the configuration:
The main changes are made in the front part of the script between the two "##################################################################" lines.
NAM defines the name, which is displayed somewhat larger at the bottom left of the screen. CAM defines the IP address of the HP10 camera.
For the query of the current weather data (here via FOSHKplugin), the source of the data is defined with ADR.
If you only want pictures or time-lapse videos during daylight - as is the case with Ecowitt.net - use daymode=true. In this case, only pictures between sunrise (minus offset in seconds) and sunset (plus offset in seconds) are triggered.
To calculate sunrise and sunset¹, the geographic coordinates must be specified with lat/lon. The offset can be used to specify how many seconds before sunrise or after sunset the pictures are to be fetched from the camera.
If you want to do without the weather data in the picture and video, simply set EMB to false in the script. This means that the weather data is not displayed in the image.
TIM defines the format for displaying the time in the image (lower right).
To make the values of the weather station easier to read in the picture, they are displayed in a block with a slightly different background. BLK can be used to determine whether this block should have a dark (black) or light (white) background or no background (none) at all.
JPGDIR AND TLVDIR define the local directories for pictures and videos. These directories are created as needed - according to the rights of the user context. The names specified in this configuration block (captname, snapname, partname and tlvname) specify the output name of the respective file on the remote server.
Of course, it is also important to specify the correct server name and credentials for access via ftp so that the upload to the target server works.

Important:
So far, no images have been deleted - so the storage requirement grows daily! At least you have the possibility to create a new time-lapse video afterwards.
For productive operation, however, a solution for automatically deleting the pictures (and videos if necessary) should be built in!
Depending on the selected resolution of the individual images and the frequency of the image recording, quite a few MByte per day can accumulate - in this case approx. 120KByte per minute per individual image and with currently approx. 10 hours between sunrise and sunset, 70MByte per day for the individual images as well as approx. double the amount for the time-lapse videos.

If you have any questions, problems or suggestions for improvement, please reply here in the thread.

¹The calculation of sunrise and sunset within a bash script is based on a solution found in the Web.

A few words about the picture quality:
The camera is currently placed indoors behind a window with a fly screen. I hope to find a solution for mounting and power supply outside soon. Then the pictures should look a bit better - here it is only meant to demonstrate the possibilities. Sorry!

Regards, Oliver

Code: [Select]
#!/bin/bash
# Script for gathering the current snapshot of a local HP10 camera with
# integration of current real time data via FOSHKplugin
# creates a capture.jpg & snapshot.jpg (& YYMMDDHHMMSS-snapshot.jpg)
# sunrise/sunset based on https://www.anginf.de/?p=138
#
# needs: ffmpeg, curl, bc, foshkplugin & recode or jq & http-aware station
# put the script in an interval of your choice in the crontab
# 2do: lightning
# Phantasoft, Oliver Engel 02.02.2023, 30.09.2023
# v0.1d better ffmpeg compatibility; don't try to send files without specified FTPLOC
PRG=getHP10dm
VER=0.1d
##################################################################
# name the cam (bottom left)
NAM="Hohen Neuendorf, Brandenburg"                             # you may specify any name here

# set the camera ip
CAM=192.168.15.182                                             # ip address of the HP10 camera

# ip address of FOSHKplugin host to gather weather data from
ADR="http://192.168.15.236:8080"                               # for embedded values

# daymode: only capture from sunrise-offset till sunset+offset
daymode=true                                                   # do not grab snapshots at night
offset=3600                                                    # before/after offset 3600 = +/-1h
lat="52.6695"                                                  # latitude
lon="13.2665"                                                  # longitude

# embedd some weather station values
EMB=true                                                       # use false to skip

# time format (bottom right)
now=`date +%s`                                                 # current local time
TIM=`date -d @${now} '+%d.%m.%Y %H:%M:%S'|sed 's/:/\\\:/g'`    # TIM=`date -d @${now} '+%Y/%m/%d %H:%M:%S'|sed 's/:/\\\:/g'`

# define block style
BLK="black"                                                    # choose block style: none, black, grey, white

# local output destination dir
JPGDIR="/home/engel/getHP10/jpg/"                              # define a local output path here
captname=capture.jpg                                           # name for raw input file
snapname=snapshot.jpg                                          # name for file with embedded weather data

# enable time lapse video
TLV=true                                                       # uncomment to enable time lapse video
TLVDIR="/home/engel/getHP10/mp4/"                              # define a local output path here
partname=tlpartial.mp4                                         # partial time lapse video triggered via parameter -timelapse
tlvname=timelapse.mp4                                          # complete time lapse file generated on new day

# remote destination for output file
#DST="https://yourdomain.tld/some/where/specialscript.php"     # uncomment to transfer via http/POST

# remote destination and credentials for ftp(s) access
#FTPLOC=ftp://yourdomain.tld/path/to/img-dir/                  # uncomment and adjust to use FTP
FTPUSER=username
FTPPASS=password
###################################################################

calcSunStamps() {
  zoneinfo=$(date -d @${now} +%z)                              # time zone
  T=$(date -d @${now} +%j)                                     # day of year
  pi="3.14159265358979323844"                                  # pi=`echo "4*a(1)" | bc -l`
  rad=$(echo "${pi}/180" | bc -l)
  h=$(echo "-(5/6)*(${rad})" | bc -l)                          # Höhe des Sonnenmittelpunkts bei sunrise: Radius+Refraktion
  BreiteRAD=$(echo "${lat}*${rad}" | bc -l)
  sonnendekl=$(echo "0.409526325277017*s(0.0169060504029192*(${T}-80.0856919827619))" | bc -l)
  sonnendeklDEG=$(echo "${sonnendekl} / ${rad}" | bc -l)
  arccosint=$(echo "(s(${h})-s(${BreiteRAD})*s(${sonnendekl}))/(c(${BreiteRAD})*c(${sonnendekl}))" | bc -l)
  arccosintsign=${arccosint:0:1}
  if [ ${arccosintsign} == "-" ]; then usesign="+"; else usesign="-"; fi
  arc2cosint=$(echo "(${arccosint}) * (${arccosint})" | bc -l)
  acoszeit=$(echo "${pi}/2 ${usesign} a(sqrt(${arc2cosint} / (1 - (${arc2cosint}) ) ) ) " | bc -l)
  zeitdiff=$(echo "12*${acoszeit}/${pi}" | bc -l)              # KORREKT!
  zeitgleich=$(echo "-0.170869921174742*s(0.0336997028793971 * ${T} + 0.465419984181394) - 0.129890681040717*s(0.0178674832556871*${T} - 0.167936777524864)" | bc -l)
  sunrise=$(echo "12-(${zeitdiff})-(${zeitgleich})-(${lon}/15)${zoneinfo:0:3}" | bc -l)
  sunset=$(echo "12+(${zeitdiff})-(${zeitgleich})-(${lon}/15)${zoneinfo:0:3}" | bc -l)
  today=$(date -d 0 +%s)
  sunrisestamp=$(echo "scale=0;$today+$sunrise*3600/1-$offset"|bc -l)  # sunrise time stamp with offset!
  sunsetstamp=$(echo "scale=0;$today+$sunset*3600/1+$offset"|bc -l)    # sunset time stamp with offset!
}

# save the current time
datestr=`date -d @${now} +%y%m%d%H%M%S`

if [ "$daymode" == true ]; then                                # calculate sunrise/sunset
  calcSunStamps
fi

# make sure dirs exist
[ ! -d ${JPGDIR} ] && mkdir "${JPGDIR}"
[ ! -d ${TLVDIR} ] && mkdir "${TLVDIR}"

# main
curDate=`date -d @${now} +%y%m%d`
if [ "$1" = "-timelapse" ]; then paramTL=true; else paramTL=false; fi
if [ "$paramTL" == true ]; then                                # check whether param set - if so, just create a TLV
  find ${JPGDIR}${curDate}*.jpg -size 0 -delete
  ffmpeg -y -hide_banner -loglevel error -framerate 25 -pattern_type glob -i "${JPGDIR}${curDate}*.jpg" -pix_fmt yuv420p -c:v libx264 -crf 17 "${TLVDIR}${curDate}-${partname}"
  # make a local copy
  cp -f ${TLVDIR}${curDate}-${partname} ${TLVDIR}${partname}
  # send video to webspace
  if [ $FTPLOC ] && [ $FTPLOC != "" ]; then
    curl --silent -p --insecure --ssl "${FTPLOC}" --user "${FTPUSER}:${FTPPASS}" -T ${TLVDIR}${partname}
  fi
else                                                           # regular image creation
  if ([ "$daymode" == true ] && [ ${now} -ge ${sunrisestamp} ] && [ ${now} -le ${sunsetstamp} ]) || [ "$daymode" == false ]; then

    # get the current picture
    curl --silent -o ${JPGDIR}${captname} "http://${CAM}/capture"

    # for additional weather data in the picture
    # if you don't use FOSHKplugin but a current GW1100/GW2000 or HP25xxC you may use it's realtime data instead:
    # ADR=192.168.15.229
    # temp=`curl -s http://${ADR}/get_livedata_info?|jq ".common_list[0].val"|sed 's/\"//g'`
    # humidity=`curl -s http://${ADR}/get_livedata_info?|jq ".common_list[1].val"|sed 's/\"//g'|sed 's/\%//g'`
    # ... - see: https://www.wxforum.net/index.php?topic=43811.msg453151#msg453151
    if [ "$EMB" == true ]; then
      # you may add Logging\IGNORE_LOG = crondaemon in FOSHKplugin config file and "\&crondaemon" to the getvalue-URL (backslash to escape the &) to prevent the output of these lines
      tempc=`curl ${ADR}/getvalue?key=tempc 2>/dev/null`
      humidity=`curl ${ADR}/getvalue?key=humidity 2>/dev/null`
      baromrelhpa=`curl ${ADR}/getvalue?key=baromrelhpa 2>/dev/null`
      windspeedkmh=`curl ${ADR}/getvalue?key=windspeedkmh 2>/dev/null`
      dailyrainmm=`curl ${ADR}/getvalue?key=dailyrainmm 2>/dev/null`
      leafwetness_ch1=`curl ${ADR}/getvalue?key=leafwetness_ch1 2>/dev/null`
      solarradiation=`curl ${ADR}/getvalue?key=solarradiation 2>/dev/null`
      uv=`curl ${ADR}/getvalue?key=uv 2>/dev/null`
      sunhours=`curl ${ADR}/getvalue?key=sunhours 2>/dev/null`
      wnowtxt=`curl ${ADR}/getvalue?key=wnowtxt 2>/dev/null|recode l1..utf8`
      wprogtxt=`curl ${ADR}/getvalue?key=wprogtxt 2>/dev/null|recode l1..utf8`
      dateutc=`curl ${ADR}/getvalue?key=dateutc 2>/dev/null|sed 's/:/\\\:/g'`

      # modify block definiton
      if [ $BLK == "black" ]; then
        BLOCK="drawbox=y=1075:color=black@0.5:width=iw:height=125:t=h,"
      elif [ $BLK == "grey" ]; then
        BLOCK="drawbox=y=1075:color=black@0.4:width=iw:height=125:t=h,"
      elif [ $BLK == "white" ]; then
        BLOCK="drawbox=y=1075:color=white@0.4:width=iw:height=125:t=h,"
      else
        BLOCK=""
      fi

      # stamp weather data into the picture
      ffmpeg -y -hide_banner -loglevel error -i ${JPGDIR}${captname} -vf "[in] \
      $BLOCK \
      drawtext=font=monospace:fontsize=25:fontcolor=white:text='${NAM}':x=10:y=(h)-30, \
      drawtext=font=monospace:fontsize=25:fontcolor=white:text='${TIM}':x=(w)-295:y=(h)-30, \
      \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Temperatur\:\   ${tempc}\°C':x=10:y=(h)-120, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Luftfeuchte\:\  ${humidity}\\\\\%':x=10:y=(h)-95, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Luftdruck\:\    ${baromrelhpa}hPa':x=10:y=(h)-70, \
      \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Wind\:\         ${windspeedkmh}kmh':x=400:y=(h)-120, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Regen/Tag\:\    ${dailyrainmm}mm':x=400:y=(h)-95, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Blattfeuchte\:\ ${leafwetness_ch1}\\\\\%':x=400:y=(h)-70, \
      \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Solar\:\        ${solarradiation}W/m²':x=790:y=(h)-120, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='UVI\:\          ${uv}':x=790:y=(h)-95, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Sonnenstd\:\    ${sunhours}h':x=790:y=(h)-70, \
      \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Wetterlage\:\   ${wnowtxt}':x=1190:y=(h)-120, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='Vorhersage\:\   ${wprogtxt}':x=1190:y=(h)-95, \
      drawtext=font=monospace:fontsize=20:fontcolor=white:text='UTC-Zeit\:\     ${dateutc}':x=1190:y=(h)-70 \
      \
      [out]" ${JPGDIR}${snapname} 2>/dev/null
    fi

    # define filename to upload & create frame for time lapse video
    if [ "$EMB" == true ]; then outname=${snapname}; else outname=${captname}; fi
    if [ "$TLV" == true ]; then cp ${JPGDIR}${outname} ${JPGDIR}${datestr}-${outname}; fi

    # upload via http/POST
    if [ $DST ] && [ $DST != "" ]; then
      curl -k -F "image=@${JPG}" -F "user=user" -F "password=password" -F "filename=${JPGDIR}${outname}" -F "fwd_type=SAVE" ${DST}
    fi

    # upload via ftp(s)
    if [ $FTPLOC ] && [ $FTPLOC != "" ]; then
      # unsecured ftp
      #curl --silent -p --insecure "${FTPLOC}" --user "${FTPUSER}:${FTPPASS}" -T ${JPGDIR}${outname}
      # ftps
      curl --silent -p --insecure --ssl "${FTPLOC}" --user "${FTPUSER}:${FTPPASS}" -T ${JPGDIR}${outname}
    fi
  fi
  # check new day - if so create a time lapse video
  if [ -f ${JPGDIR}${PRG}last.txt ]; then wasDate=$(<${JPGDIR}${PRG}last.txt); else wasDate=${curDate}; fi
  if [ "$TLV" == true ] && [ "$curDate" != "$wasDate" ]; then
    echo ${curDate}>${JPGDIR}${PRG}last.txt                    # do this first to prevent another start
    find ${JPGDIR}${wasDate}*.jpg -size 0 -delete
    ffmpeg -y -hide_banner -loglevel error -framerate 25 -pattern_type glob -i "${JPGDIR}${wasDate}*.jpg" -pix_fmt yuv420p -c:v libx264 -crf 17 "${TLVDIR}${wasDate}-${tlvname}"
    # make a local copy
    cp -f ${TLVDIR}${wasDate}-${tlvname} ${TLVDIR}${tlvname}
    # send video to webspace
    if [ $FTPLOC ] && [ $FTPLOC != "" ]; then
      curl --silent -p --insecure --ssl "${FTPLOC}" --user "${FTPUSER}:${FTPPASS}" -T ${TLVDIR}${tlvname}
    fi
    # delete the local partial time lapse video
    #rm -f ${TLVDIR}${wasDate}-${partname}
    # delete all local already processed images
    #rm -f ${JPGDIR}${wasDate}*.jpg
  fi
  echo ${curDate}>${JPGDIR}${PRG}last.txt
fi


Online kheller2

  • Forecaster
  • *****
  • Posts: 518
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #2 on: January 02, 2023, 03:20:09 PM »
I just want to say that I enjoy reading a fellow BASHer's code.

So much can be done in BASH, including functions, global and local variables etc..
Ambient Consoles: WS-2000, WS-1900, WS-1200, WS-2902C, WS-3000-X3, WS-0900-IP(observerIP), WS-1001-WIFI
Ambient Arrays: WH65B
Ambient Sensors: WH31E(3), WH31B(2), WH32B, WH31SM(2), WH31PGW, AQIN, WH31LA(3)
Ambient Spares: WH24B(2), WH25B.
Ecowitt: HP2551BU, GW1000B(dead), GW1100B(2), GW2000B
Ecowitt Sensors: WH51, WN34BL, WN34(2), WH31, WH41, WH40

Offline box

  • Forecaster
  • *****
  • Posts: 384
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #3 on: January 02, 2023, 05:31:38 PM »
Wow, simply wow =D&gt;

This post is now stashed away for future reference!

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #4 on: January 07, 2023, 04:02:02 AM »
Hi!

I have revised the script in the first post a little - it is now available in version v0.1b.
For example, the lines for deleting the partial time-lapse videos and JPGs are now included (albeit commented out).
In addition, a problem has been fixed in which the creation of the daily time-lapse video (00:00 hrs) could occur several times because the first job was not yet finished and the corresponding status was not yet set.

For experimental purposes, I have now integrated the webcam into Awekas, WOW, WC and Ambient Weather. So you can display your webcam (as a single image) on several services. I have not yet checked whether and how to insert the time-lapse video. With PWSWeather or WU, I have not found a way to integrate a webcam.
Overall, however, I am pleasantly surprised at how many services also support a webcam at all.

One finding:
The sunrise and sunset times differ by 1-2 minutes from the times calculated by CMX or the weather station itself. However, I have not yet been able to find the place to correct this.

Just out of interest:
Does anyone (besides me) actually use the script?

Oliver

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 1989
    • Mini Wind and Solar Data project
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #5 on: January 11, 2023, 12:05:25 PM »
Quick Question on the HP10, if doing a straight FTP upload of single images (timed jpg snapshot for instance), does it support FIFO (First In First Out), with a time stamp on the files and can control the contents of the remote directory, by checking the  number existing  before it starts deleting on a FIFO basis?
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #6 on: January 11, 2023, 12:47:04 PM »
Hi!

You have everything in your own hands. Because the HP10 itself cannot currently use FTP.
It only provides the images that you can fetch via httplGET.

Oliver

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 1989
    • Mini Wind and Solar Data project
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #7 on: January 11, 2023, 01:09:36 PM »
Ahh, OK, I thought they had put out an FTP option , or had planned to. This explains my reading of the manual and seeing nothing.

Oliver, your script is cool, and I would use it but went a different route with a standard camera. Actually your presentation looks a lot like mine, but you are embedding data into the image, I just float a DIV with the current data, so yes, there is a data upload and an image upload, also why there are 2 time stamps (image / data)

I may take mine to a time lapse presentation, but it will not be as nice as yours that actually compiles an mpeg, I just shoot down the jpegs to the browser (ok js requests) .

I am trying to find a reason to get the HP10 (for me , for others without cams, I think it is a good deal)

This is just the image, this is the image with the overlay https://rovr1.com/wind/weather.html (yes this will change as it is live)

 [ You are not allowed to view attachments ]
« Last Edit: January 11, 2023, 01:12:28 PM by Rover1822 »
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

Offline BoDuke

  • Global Moderator
  • Contributor
  • *****
  • Posts: 132
  • Ambient Weather, Ecowitt, WeeWX, Custom PHP
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #8 on: January 11, 2023, 02:33:35 PM »
Oliver,

I don't yet have an HP10 so I don't use your script yet.  I wholeheartedly thank you for sharing it in case I do buy one or want to do the same type of thing with a similar camera.
 UU

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #9 on: January 11, 2023, 04:39:03 PM »
Hi!

@BoDuke: You're welcome!

@Rover: An FTP option should come with a new firmware version at some point. From a purely image quality point of view, I like your pictures better than those of the HP10.
Personally, I would also prefer a real overlay of the values. But that complicates the use for most users. The images or even time-lapse videos with stamped values are much easier to integrate into existing solutions such as Ambient Weather, Awekas, Weathercloud, PWSDashboard or CMX.

Oliver

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 1989
    • Mini Wind and Solar Data project
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #10 on: January 12, 2023, 12:47:06 PM »
You are obviously correct that mine is more complex and requires one to have their own external site, and would never work for an upload image web site weather service.

I am curious , as you do compile the images into a mp4, and maybe I missed it, but is that server side or are you uploading an mp4?

I don't work in PHP so, I am curious on things like server load

If I am completely off base, feel free to thrash me .
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #11 on: January 12, 2023, 01:06:51 PM »
Hi!

No, this solution is local based.
So the images are fetched locally in the network from the HP10 and temporarily stored on a local computer. Only one image at a time (the current one) is sent to the web server via FTP - the existing image there is simply overwritten.
A time-lapse video (the mp4)  is then generated on the local server from the large number of images, which is also sent to the remote web server via FTP:

Code: [Select]
ffmpeg -y -hide_banner -loglevel error -framerate 25 -pattern_type glob -i "${JPGDIR}${wasDate}*.jpg" -pix_fmt yuv420p -c:v libx264 -crf 17 "${TLVDIR}${wasDate}-${tlvname}"

While processing the mp4 the cpu usage is getting high (very high!). And currently it takes about 60 seconds to create the video from the individual images - although at the moment there are only 8 hours between sunrise and sunset. In the summer, there are then significantly more images and thus also a significantly longer processing time, where the CPU load is so high. I am curious.

So most of the processing is done locally - because not all web hosts offer these possibilities.

Oliver

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 1989
    • Mini Wind and Solar Data project
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #12 on: January 12, 2023, 01:15:19 PM »
Ahh OK.

Yeah we have limitations here as well , well anywhere on a hosted site.
What is the frequency of the MP4 upload, once a minute?, hour? day?

My host , gets weird if we are doing something that looks like a video hosting site.

But very cool
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #13 on: January 12, 2023, 01:22:02 PM »
Hi!

Quote
What is the frequency of the MP4 upload, once a minute?, hour? day?
This depends on the frequency set in the crontab. Currently, one video has about 60MB (for 10 hours). The complete daily video is sent to the server once at 00:00.
Before that, there is an (additional) partial file with all the images available at the time of creation at 2-hour intervals - i.e. at 8:00, 10:00, 12:00, 14:00 and 18:00. The one on the server is always overwritten.

Oliver

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #14 on: February 02, 2023, 12:33:22 PM »
Hi!

Daily use here revealed a problem with ffmpeg processing corrupt frames (such as when the camera was rebooted due to firmware update or a reboot of the host computer).
The creation of the MP4 stopped at exactly this broken frame instead of simply skipping it.
Unfortunately, I can't find a way to catch this within ffmpeg.
Therefore, all 0-byte JPGs are now searched for and deleted before ffmpeg is started.
The updated script with now v0.1c can be found in the first post.

Oliver

Offline beskidus

  • Member
  • *
  • Posts: 3
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #15 on: September 28, 2023, 05:29:57 AM »
Good morning.
I have an HP10 camera and I'm trying to integrate it with the above script.
I am downloading the image from the camera but an overlay with weather data is not created. An error occurs:
" cp: cannot stat '/home/pi/getHP10/jpg/snapshot.jpg': No such file or directory
curl: Can't open '/home/pi/getHP10/jpg/snapshot.jpg'!
curl: try 'curl --help' or 'curl --manual' for more information
./getHP10dm.sh: line 198: /home/pi/getHP10/jpg/getHP10dmlast.txt: Permission denied"


Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #16 on: September 28, 2023, 06:45:56 AM »
Hi!

It's nice that another user wants to use this script. Since then, it has been working here completely inconspicuously and without any problems.

Quote
Permission denied
Apparently the permissions to the directory /home/pi/getHP10/jpg/ specified under $JPGDIR are missing/wrong.
You should make sure that all files and directories used in this script belong to the user pi (in your case) and that he has write access there.
For example with
Code: [Select]
sudo chown -R pi:pi /home/pi/getHP10as long as the script runs in the user context pi.

Regards, Oliver

Offline beskidus

  • Member
  • *
  • Posts: 3
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #17 on: September 28, 2023, 07:01:26 PM »
Hi.
Thanks for your reply. I tried your advice but there is still a problem:
"cp: cannot stat '/home/pi/getHP10/jpg/snapshot.jpg': No such file or directory
curl: Can't open '/home/pi/getHP10/jpg/snapshot.jpg'!"
Is there any advice for this?

Edit:
When EMB=false is set, the files capture.jpg, 230929113401-capture.jpg, etc. are created.
When EMB=true
everything stops...
Snapshot.jpg missing.
Not connected to foshkplugin?
« Last Edit: September 29, 2023, 05:46:17 AM by beskidus »

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #18 on: September 29, 2023, 11:25:06 AM »
Hi!

For debugging, I would insert an "echo sequence number" before and after each line containing a cp to find the location of the problem.
Then you should be able to find the faulty line very precisely when you start the script manually:
Quote
1
2
3
"cp: cannot stat '/home/pi/getHP10/jpg/snapshot.jpg': No such file or directory
4
There are exactly 3 cp lines - so you should be able to find the location quite quickly.
 
Once the exact problem line has been found, I would output the command line via echo before the actual cp command. For example
Code: [Select]
echo cp -f ${TLVDIR}${curDate}-${partname} ${TLVDIR}${partname}
I suspect that one of the required variables was not specified or not specified correctly.
You could also send me your script by PM - then I'll take a look at it here.

Oliver

Offline olicat

  • Forecaster
  • *****
  • Posts: 1515
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3800C
    • FOSHKplugin
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #19 on: October 02, 2023, 04:07:41 PM »
Hi!

There is a new version of the script - I have included it in the first post.
In the new v1.0d there is better compatibility with modern ffmpeg versions and an FTP upload is only triggered if FTPLOC was actually defined.
This script has now been tested with Raspbian 10 (buster) and 11 (bullseye) as well as Ubuntu.

Oliver

Offline beskidus

  • Member
  • *
  • Posts: 3
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #20 on: October 10, 2023, 11:49:55 AM »
Hello,
I would like to thank Oliver very much for the help he gave me while implementing this script.
It took me some time but I think it's worth it ;)
Good job.
Thanks again!
Paul
« Last Edit: October 10, 2023, 11:58:17 AM by beskidus »

Offline Airwavemagic

  • Member
  • *
  • Posts: 3
    • Weather
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #21 on: January 15, 2024, 07:53:50 AM »
Hello everyone.
My language is not English so I use a translator I hope it will be ok.

Now I use a laptop (sw Weather Display) to save the weather record, which has to run 24/7 (oregon station and hikvision webcam). The station is already old and I will have to renew it with a newer one, so I am thinking about an Ecowitt device.
Since I also have a Synology NAS server running at home (24/7), I wanted to use this NAS server for storing weather and time-lapse camera images.
I have now purchased a HP10 camera for testing and am trying to download an image using the server (I'm not a programmer, I only know basic HTML and home LANs).

I am able to generate the current image via the web browser (http://192.168.0.102/capture).
But I don't know how to use a NAS server for that (I have Python, PhP...).
Does anyone have experience with this, or should I continue with the current configuration of data storage via the laptop.

Thank you for your opinion or help.

(https://nemcicky.net/WD/wx3.html)

Offline SamiS

  • Member
  • *
  • Posts: 28
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #22 on: January 15, 2024, 11:00:57 AM »
Since I also have a Synology NAS server running at home (24/7), I wanted to use this NAS server for storing weather and time-lapse camera images.
I have now purchased a HP10 camera for testing and am trying to download an image using the server (I'm not a programmer, I only know basic HTML and home LANs).

I am able to generate the current image via the web browser (http://192.168.0.102/capture).
But I don't know how to use a NAS server for that (I have Python, PhP...).

Just a random idea that probably does not require programming skills. Since you have a Synology, maybe you could use the Download Station app to download the current picture from the above url at specified time interval?

Edit. This of course does not help with timelapse videos, but might be an easy way to collect a series of still images.
« Last Edit: January 15, 2024, 11:03:39 AM by SamiS »

Offline Airwavemagic

  • Member
  • *
  • Posts: 3
    • Weather
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #23 on: January 15, 2024, 12:14:57 PM »
Since I also have a Synology NAS server running at home (24/7), I wanted to use this NAS server for storing weather and time-lapse camera images.
I have now purchased a HP10 camera for testing and am trying to download an image using the server (I'm not a programmer, I only know basic HTML and home LANs).

I am able to generate the current image via the web browser (http://192.168.0.102/capture).
But I don't know how to use a NAS server for that (I have Python, PhP...).

Just a random idea that probably does not require programming skills. Since you have a Synology, maybe you could use the Download Station app to download the current picture from the above url at specified time interval?

Edit. This of course does not help with timelapse videos, but might be an easy way to collect a series of still images.

You can't set the time recording there, but only one-time download. If I set it in task scheduling, it is possible to start after 1 hour at the earliest, and also the entire program, not an individual task.

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 1989
    • Mini Wind and Solar Data project
Re: create webcam images & time lapse video with Ecowitt HP10
« Reply #24 on: January 15, 2024, 12:26:41 PM »
I don't have an HP10, but I do similar things with security cameras at my house.

Does the HP10 always have the same filename, or is it one that changes with the date/time?


If I remember correctly the HP10 was supposed to have FTP capabilities, other words can use FTP  protocol to upload to a site (Oliver will know more)

There are multiple parts if the goal is a DIY time lapse. If it is just show the last , much simpler
1) transport , FTP, HTTP etc
2) Storage of the images , you probably don't want to store all images in perpetuity , so store a certain time duration, days, weeks, etc, but delete the old ones (FIFO rotation)
3) Access to the images , something like HTTP, which will need the intelligence server side to provide the images in sequence , or compile a video  (MP4) . Then have something client side in a browser , JS will do for some of it.

There are other items to consider, but it is a complicated task if you want to roll your own and not be dependent on a site not managed by you. I have done it, but not with the HP10 but with regular security cameras. And frankly, if that is your goal, not sure the HP10 is an appropriate decision for a weather cam. The HP10 does get you some benifits, like posting to Ecowitt.net, but you have to decide if that is what you need.




Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

 

anything