Author Topic: WxDisplayHistory, data pts going missing  (Read 1532 times)

0 Members and 1 Guest are viewing this topic.

Offline NanoWx

  • Member
  • *
  • Posts: 35
WxDisplayHistory, data pts going missing
« on: December 01, 2017, 09:20:51 AM »
Wonder at times if wu is being or has been hacked. Recently seeing data go missing over course of a day, using WXDailyHistory.asp interface. For many months (to years), one station has had data every 5 minutes, another alternated 5 and 6 min. Both recently have 10 - 20 min skips, apparently at random.
In one instance noticed yesterday (for the latter stn), there had been a point:
2017-11-30 03:32:00,32.0,...
But over the course of the day it was lost to a 17 min skip. That happened to have been TLow; which after skip was:
2017-11-30 03:21:00,32.2,...

NWS volunteer spotter, and coop observer

Offline NanoWx

  • Member
  • *
  • Posts: 35
Re: WxDisplayHistory, data pts going missing
« Reply #1 on: December 03, 2017, 09:07:54 AM »
Looking like either a hack, or defective DB server being used by WxDisplayHistory.asp
Queries at different times (sometimes within minutes) return different sets of data points.
Eg: https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KORCONDO5&year=2017&month=11&day=30&graphspan=day&format=1
has returned 208, 206 and 207 lines (header + data), all queries made after 30Nov.
« Last Edit: December 03, 2017, 09:10:55 AM by NanoWx »

Offline NanoWx

  • Member
  • *
  • Posts: 35
Re: WxDisplayHistory, data pts going missing
« Reply #2 on: December 03, 2017, 09:17:51 AM »
Queries of stn KORCONDO4 also have data skips. Have not checked it, to see whether queries at different times return different data (which could be contributing to 'skips').

Offline Maumelle Weather

  • Forecaster
  • *****
  • Posts: 1824
    • Maumelle Weather
Re: WxDisplayHistory, data pts going missing
« Reply #3 on: December 03, 2017, 09:22:56 AM »
Hacked???? Screwed DB server???? Not WU!!!! Believe me it will get worse before it disappears, just like everything they USED to have (NOAA Weather Radio, photos, Webcams are ending on December 15th, and the list goes on).

If you really want to see data online, start sending your data to PWS Weather, or Weather Cloud

John
GR2AE, GR3, Cumulus

Offline NanoWx

  • Member
  • *
  • Posts: 35
Re: WxDisplayHistory, data pts going missing
« Reply #4 on: December 03, 2017, 09:59:53 AM »
@  ... PWS Weather or Weather Cloud
Good idea, but some low-end stations (such as from FineOffset (HK), or re-branded such as Tycon) may not have software available to do that.

Offline NanoWx

  • Member
  • *
  • Posts: 35
Re: WxDisplayHistory, data pts going missing
« Reply #5 on: December 03, 2017, 10:01:35 AM »
Looking like either a hack, or defective DB server being used by WxDisplayHistory.asp
Queries at different times ...
has returned 208, 206 and 207 lines (header + data), all queries made after 30Nov.
Compared two of those sets of results. A subset of diff of results of two of those queries:

20c18  [ actually two different data samples; diff goes by overall similarity]
< 2017-11-30 01:58:00,33.1,25.3,30.15,SW ...
---
> 2017-11-30 01:53:00,33.1,25.3,30.15,West ...
 ...
148d144
< 2017-11-30 17:09:00,37.4 ...
155a152
> 2017-11-30 18:04:00,36.1 ...
173a171
> 2017-11-30 20:09:00,36.5 ...

Looks like the queries are fetching a scatter of the DB data that actually matches the query.

Offline NanoWx

  • Member
  • *
  • Posts: 35
Re: WxDisplayHistory, data pts going missing
« Reply #6 on: January 09, 2018, 08:55:51 AM »
   ...
Looks like the queries are fetching a scatter of the DB data that actually matches the query.
Situation is actually worse. Instead of data for local time day, WxDisplayHistory.asp too often fetches day data for EasternStandardTime. For summary data (graphspan default), that sometimes seems to result in no summary data being returned for some days.
« Last Edit: January 09, 2018, 08:58:06 AM by NanoWx »

Offline ConligWX

  • Forecaster
  • *****
  • Posts: 836
  • #conligwx
    • conligwx.org
Re: WxDisplayHistory, data pts going missing
« Reply #7 on: January 09, 2018, 09:36:14 AM »
loooking at my own site today I have had random issues with WU's website and how pages are displayed. 1 minute site looks ok, next my own site is missing, next its back again, then I have no webcams attached.  I think they having severe issues again. tried various web browsers and different forms of hardware/net connections, they all have issues.

the two images are less than 1 minute apart!
« Last Edit: January 09, 2018, 09:41:21 AM by Toxic »
Regards Simon
Davis Vantage Pro2 Plus (6162UK) • Daytime FARS • WeatherLink Live • AirLink • PurpleAir PA-II-SD • CumulusMX •