Author Topic: Missing Data using lates WeeWx and wu-client  (Read 764 times)

0 Members and 1 Guest are viewing this topic.

Offline kq6up

  • Senior Member
  • **
  • Posts: 56
Missing Data using lates WeeWx and wu-client
« on: February 26, 2019, 06:54:19 PM »
I see that I am not getting a steady stream of posts on findu.  My server is sending every 5minutes and I don't see clues in the syslog files.  I do see some timeout notices, but within the same minute there is a succesful post notice in the syslog file.  Any suggestions as to what is causing the missing data in my data table?

Syslog example:

Feb 26 06:30:35 raspberrypi weewx[29048]: restx: CWOP: Exception <class 'socket.timeout'> (timed out) when looking for response to packet packet
Feb 26 06:30:35 raspberrypi weewx[29048]: restx: CWOP: Published record 2019-02-26 06:30:00 PST (1551191400)

Here is my station:

http://www.findu.com/cgi-bin/wx.cgi?call=kq6up

Thanks,
Chris Maness

Offline UberEclectic

  • Member
  • *
  • Posts: 5
Re: Missing Data using lates WeeWx and wu-client
« Reply #1 on: May 01, 2019, 07:52:00 PM »
WU has been getting slower.

Have a look in the restx.py and you will find numerous places where there are 10 second timeouts governing various connections to different services.
I have had to increase WU-related timeouts from 10 seconds to 20 seconds.
In addition I use wunderfixer regularly to catch any misses.

I did just today discover an interesting case of several false positive-missing records.
Data wasn't really missing, but was intentionally ignored by WU, because it was in between a 5-minute boundary.
In the case that I drilled into deeply a missing record was from 9:57:00 a.m., but WU already had records for 9:50, 9:54.59, and 10:00 a.m.

The wunderfixer utility defaults to 120 second time offsets, so called "epsilon" when deciding whether a record is missing.  My local record from 9:57.00 is exactly 121 seconds after WU's 9:54.59 record, e.g. 1 second more than the default 120 second "epsilon," so wunderfixer deemed the record missing.  I increased the epsilon to 125 seconds to compensate.

I had several such records close to the 5-minute boundaries, but just barely outside, like that.  But using 125 seconds to align the buckets seems to be an effective workaround for WU showing the fractional minutes like 9:54.59, instead of just calling it 9:55.

HTH

 

anything