Author Topic: Buffer data during outage and forward to ecowitt.net after?  (Read 462 times)

0 Members and 1 Guest are viewing this topic.

Offline HaraldS

  • Senior Member
  • **
  • Posts: 87
Buffer data during outage and forward to ecowitt.net after?
« on: January 10, 2023, 02:36:33 PM »
Is there a way to buffer the data and then later forward to ecowitt.net? It looks like while the 2561 might store it locally, it does not "catch up," once reconnected.

I live the SF Bay Area and we've been hit with several waves of "atmospheric rivers" featuring heavy rainfall and winds, even thunderstorms! Our power has been out up to 5 days. We run a generator during the daytime, but not during sleep hours.

My panel is on a LiIon battery, but it does not have enough to run the full network with WAN, LAN router, etc. I could run a raspberry pi. I also have a GW2000 that could be integrated instead.

However, I do not want to spend the time for a major software project.

Offline wardie

  • Senior Contributor
  • ****
  • Posts: 212
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #1 on: January 10, 2023, 06:20:08 PM »
Maybe a guru like [mention]olicat [/mention] will have some ideas from FOSHKplugin work? E.g. you’d need the ability to load historic records with non current time stamps via whatever their upload API is. But that fairly normal for writing to time series databases. My solar panel inverter does exactly that with its web monitoring platform - caches data until the server is available. Anyway, not trivial and would need a software project unless anyone’s cracked it already.


Sent from my iPhone using Tapatalk
Froggit HP1000SE Pro-C console (HP2551-C)
Froggit HP1000SE Pro ultrasonic multi sensor with Ecowitt EC0002 heater (WS80)
Ecowitt Anemometer 5-in-1 array (WS68)
Froggit DP80 rain gauge (WH40) with spikes
Froggit indoor temp/humidity/pressure (WH32B)
Froggit DP50 Internal temp/humidity x2 (WH31)
Ecowitt Outdoor temp/humidity & RS-00001 shield (WH32)
Froggit DP200 PM2.5 outdoor (WH41)
Ecowitt indoor CO2 PM2.5 PM10 (WH45)
Froggit DP100 soil moisture (WH51)
Froggit DP60 Lightning detector (WH57)
Froggit DP1500 server dongle (GW1000A) x2
Raspberry Pi 4 / WeeWx-GW1000 API interface
WU: IKNEBW2

Offline HaraldS

  • Senior Member
  • **
  • Posts: 87
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #2 on: January 10, 2023, 07:40:11 PM »
It looks like setting up WeeWX on a Pi would be the way to go.

Offline vinceskahan

  • Senior Contributor
  • ****
  • Posts: 182
    • home site
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #3 on: January 10, 2023, 08:45:45 PM »
I'd start with the question of whether the gateway has any storage and catchup capability at all.  I'd expect it to go into a reboot loop anyway once your Internet went down for long enough due to the watchdog timers in the firmware, but if it has no storage/catchup that part doesn't even matter.

The '...and upload to ecowitt.net' might be a showstopper too.  Do they support third-party software doing so ?
WeeWX sites:
  Davis VP2+DFARS to a pi4
  EcoWitt GW1000, WH32 outdoor T+H, multiple WH31 indoor T+H, WH51 soilMoisture, WH34 soilTemp (pi4)
  Davis AirLink (inside)
  PurpleAir (outside)
Home site:        https://www.skahan.net/
Wunderground: KWAFEDER15
PWS:                KWFEDER15
CWOP:              CW6881

Offline Gyvate

  • Forecaster
  • *****
  • Posts: 3329
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #4 on: January 11, 2023, 10:45:26 AM »
I think you are all +/- missing the point.
Of course weewx and other software, also FOSHKplugin can store data in their local database - but (so far) you CANNOT retroactively update data on ecowitt.net. Missed is missed, lost is lost. Afaik you can't do this with WU either, while with AWEKAS it seems possible.
It's not a console restriction of application software restriction.
The restriction is at the cloud server data storage end.
WS2350 1.6.7, GW1000(3) 1.7.7,WH2650 WiFi (2) 1.7.7 (test/backup), GW1100 2.3.1, GW2000(3) 3.1.1, HP2551 1.9.5,5.1.5;HP3500 1.7.2,WS3800 1.2.8, WN1910 1.2.3,WN1980 1.2.3;
Ecowitt WS90(2)1.3.5/1.4.0, WS80(2)1.2.5, WS68, WS69, WH40, WH31, WH31-EP, WN30, WN34L, WN35, WH32, WH32-EP, WH32B, WH57 [Lightning], WH41 [PM2.5], WH51, WH45, WH55
MeteobridgePro(2)[test,prod] 5.8 Mar 01 2024, 15185 - Blake-Larsen Sun Recorder - RPi4/weewx 4.8.0/4.10.2/CumulusMX 3283/Meteobridge RPi4B-2GB(3169)
Barani Meteoshield Pro, MetSpec Rad02 - Ecowitt 5763,34418;WU ISAARB3(WH4000SE),ISAARB22(HP2553), http://meshka.eu

Offline olicat

  • Forecaster
  • *****
  • Posts: 1522
  • GWxx00, HPx5x1C, WN1900C, WN1980C & WS3xx0C
    • FOSHKplugin
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #5 on: January 11, 2023, 11:04:10 AM »
Hi!

That is correct.
I have researched the possibilities of uploading afterwards with various services and programmes.
So far, I have only found InfluxDB (v1 and v2) that accepts the subsequent automatic upload without any problems.
FOSHKplugin will support this with the upcoming version v0.10.

Of the weather services I know, only Awekas offers a manual subsequent upload via Excel file. This actually works - although I would like to see an automatic function (although I was able to minimise the handling in the FOSHKplugin).

All other services discard the time that may be included in the data set and take the receipt of the data at the server as the time of the measured values.
I had explicitly asked Ecowitt about this - answer: subsequent sending is not possible - missed is missed.

We will have to do a lot of convincing if we want to change this.

Oliver

Offline mcrossley

  • Forecaster
  • *****
  • Posts: 1140
    • Wilmslow Astro
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #6 on: January 11, 2023, 11:47:37 AM »
Slightly irrelevant, but Davis do support it in their cloud, but only via a private API used by their WeatherLink software, or the WLL device (consumer devices).

My point is Davis thought about it and included the feature from the start.
Mark

Offline vinceskahan

  • Senior Contributor
  • ****
  • Posts: 182
    • home site
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #7 on: January 11, 2023, 12:21:07 PM »
I think the operative answer here to the original question is 'no - ecowitt does not support backfill'....

So the original poster would need to keep the gateway and full internet connectivity working to not miss readings.
WeeWX sites:
  Davis VP2+DFARS to a pi4
  EcoWitt GW1000, WH32 outdoor T+H, multiple WH31 indoor T+H, WH51 soilMoisture, WH34 soilTemp (pi4)
  Davis AirLink (inside)
  PurpleAir (outside)
Home site:        https://www.skahan.net/
Wunderground: KWAFEDER15
PWS:                KWFEDER15
CWOP:              CW6881

Offline Rover1822

  • Forecaster
  • *****
  • Posts: 2017
    • Mini Wind and Solar Data project
Re: Buffer data during outage and forward to ecowitt.net after?
« Reply #8 on: January 11, 2023, 12:57:34 PM »
Sadly , you can't always keep your internet connectivity up and running.
In my area , the internet connection is powered by the grid (so even if you have backup power, you still lose internet), unless one goes cellular, and even that can be sketchy.
I understand the need to backfill missing data to sites, but that is entirely up to them. Seems simple , but I guarantee, it is not.
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam