Topic: Weather underground historical data is not available now

[pimohdaimaoh]

Only parts are working.  Fetching data found

<!-- curl fetching 'https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCASARAT1&month=05&day=21&year=2019&format=1&graphspan=day' -->
<!-- HTTP stats:  RC=404 dest=173.223.144.97 port=443 (from sce=74.208.252.86)
      Times: dns=0.028 conn=0.059 pxfer=0.357 get=0.165 total=0.523 secs -->
<!-- headers:
HTTP/1.1 404 Not Found
Server: Apache/2.2.15 (CentOS)
Pragma: no-cache
Access-Control-Allow-Origin: http://www.wunderground.com
Access-Control-Allow-Credentials: true
Content-Type: text/html

Yet the graph link shows just fine   

Hmm. . . Me too I think they restore it after I try to viewing it on my page, as I thought they already remove it permanently?
or something conscience kicking their heads? becasue imagine if you are using PWS data coming from their sensors and in return, you manage to RESTRICT sharing parsed data from them. . .   

[wunderground-PWS]

Folks, there's no conspiracy, no one here is mad at IBM. 
Every single thing needs to be re-written. 
Every single part of weather underground. 
What is currently running was built with spit and bailing wire, when something stops working, 90% of the time its because it broke.  When a feature on the new pages is missing, its because we didn't have time make it work efficiently.

I think we have a pretty good general idea of the complaints you all have, we're working on getting some of them addressed.  Some probably won't be addressed. 

[AboutAndAround]

Some probably won't be addressed. 

Are you able to tell us a few of those that aren't planned to be addressed at least? Thanks.

[mldenison]

Folks, there's no conspiracy, no one here is mad at IBM. 
Every single thing needs to be re-written. 
Every single part of weather underground. 
What is currently running was built with spit and bailing wire, when something stops working, 90% of the time its because it broke.  When a feature on the new pages is missing, its because we didn't have time make it work efficiently.

I think we have a pretty good general idea of the complaints you all have, we're working on getting some of them addressed.  Some probably won't be addressed.

I'm mystified.  How do things break and quit working unless someone is tinkering with the code?  And why are new pages going into production when these features do not work?  Do you folks not have the time to test these new features before releasing them?  Just curious...

[txweather.org]

Folks, there's no conspiracy, no one here is mad at IBM. 
Every single thing needs to be re-written. 
Every single part of weather underground. 
What is currently running was built with spit and bailing wire, when something stops working, 90% of the time its because it broke.  When a feature on the new pages is missing, its because we didn't have time make it work efficiently.

I think we have a pretty good general idea of the complaints you all have, we're working on getting some of them addressed.  Some probably won't be addressed. 

You just confirmed my suspicion. This migration was not planned... But winged...
Also it was confirmed that the yearly data was pulled and was not coming back... From the FB group, Please look here:
https://www.wxforum.net/index.php?topic=36954.msg380400#msg380400

[galfert]

Folks, there's no conspiracy, no one here is mad at IBM. 
Every single thing needs to be re-written. 
Every single part of weather underground. 
What is currently running was built with spit and bailing wire, when something stops working, 90% of the time its because it broke.  When a feature on the new pages is missing, its because we didn't have time make it work efficiently.

I think we have a pretty good general idea of the complaints you all have, we're working on getting some of them addressed.  Some probably won't be addressed.

Why does it take you over 48 hours to put a message out on the situation?

Some probably won't be addressed

Why not? You just keep removing features. Soon what will be left?

Are you planning on charging for premium services to restore functionalities to your data contributors?

Are decimals coming back? For everything: live dashboad, summary, data tables, and graphs?

Can you implement a reporting system so that station owners can continue to name their stations as they have for almost 2 decades?

Can WU come up with a core framework of functionalities that they promise to uphold to their contributors and alow at least 10 months advanced notice before retiring any more functionalities? At least 6 months would seem fair.

Can you create some delegated volunteer program/process where some data contributors are identified and made ambasadors and have some policing authority to help out with the limited resources that apparently WU has? Perhaps they gain some back end tools to help administer WU. This would alow stations to be custom named. Maybe station status returns and goes through an approval process checkpoint. Maybe you can utilize these ambassadors to do station data quality assurance. Not one ambassador should be given 100% autonomy but maybe actions require approval of more than one ambassador. Maybe these ambassadors can be tagged as experts in particular hardware and software to help out users with station issues. We want to help you do more with less so that you can continue to serve your users and clients.

[waiukuweather]

and now its stopped working again
e.g
https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KCASARAT1&month=05&day=19&year=2019&format=1&graphspan=day

returns a 404 error
hopefully being worked on and it will come back again

[wunderground-PWS]

Folks, if something doesn't come from wu pm or me, its speculation and panic.  Please take a deep breath and realize we're trying to help get you what you want.  I've said this other places, but i'll re-iterate here.  Be reasonable and i'll hear you out, throw a fit, say the sky is falling and wu is evil and we should be ashamed of ourselves and I'll quite happily ignore you.

Names  - working on it
decimals - working on it

csv download - investigating a way for owners to get their data via csv
custom date range - maybe, will take a long time to implement this


data access - still there, you can access through the API easily.

[galfert]

Folks, if something doesn't come from wu pm or me, its speculation and panic. 

<deep breath> Thank you for the update. Better late than never. Panic and speculation ensued because of lack of communication. You know what they say; actions speak louder than words. What were we to think? We have high expectations from you because of IBM. A little more professionalism would gain you a good bit more supporters.

[txweather.org]

and we are getting somewhere.
Thanks wunderground-PWS...

[AboutAndAround]

BTW, the Horizon subdomain with the old site seems to have been restored as well, seems to be working here at least.

[catdon]

BTW, the Horizon subdomain with the old site seems to have been restored as well, seems to be working here at least.

Yeah here as well... Ten minutes ago it was not. Lol spoke too soon, seems to be hit or miss for now.

[AboutAndAround]

BTW, the Horizon subdomain with the old site seems to have been restored as well, seems to be working here at least.

Yeah here as well... Ten minutes ago it was not. Lol spoke too soon, seems to be hit or miss for now.

Yeah, earlier I could access it on my phone but not on my PC. Does seem hit and miss.

[scudfest]

decimals - working on it

You don't know how happy I am to read this. 
I was half way through collating data of the ~10min interval history pages.  This was to study adjacent sites and the progression of sea breeze fronts and storm fronts in my area. Most of the data I have is in decimals and I thought the rest of my project was screwed.

Can't wait to see decimals come back for temperature and dew points.  Then my data science studies can continue with data that is actually interesting. Not freaking petal sizes and nonsense.   

[dunc1ca]

Looking at the page now shows this:

An error occurred while processing your request.
Reference #30.4032417.1558639466.3429ce4f

[saratogaWX]

I've got a fixup for the issue .. see this topic for more info.

Note that the graph image also seems to be 'down' now too, in addition to the WU WXDailyHistory.asp page for the CSV data.

[dunc1ca]

I have an Excel spreadsheet that uses VBA to automatically retrieve data from WU csv page to fill in all my columns.  I contacted WU about the problem and got a response:

The URL you are trying to use has been deprecated. It may be up and down before it’s gone for good, but it is on its way out.
On your dashboard, using the table view, you should be able to copy and paste into a spreadsheet.
https://www.wunderground.com/dashboard/pws/ICAVANMO7/table/2019-05-23/2019-05-23/daily
From there you should be able to convert to csv format, if that’s your preference.
Thank you for using Weather Underground.
—Victoria

Unfortunately, if I reprogram and use their API, the data is incorrect.  The API removes the decimals from the temperature (not even rounding properly).  My temperature was 17.9°C and WU was reporting 17°C, a difference of almost a full degree.  I can not corrupt my own historical data with this.  I really just want them to stop removing useful features.

[saratogaWX]

You can use my new WXDailyHistory.php script to retrieve data via the API and format to CSV .. but... the JSON doesn't contain decimal points for temperatures, so neither will the CSV.

As WU said earlier, they're working on getting decimals back into the temperatures.

[mkutche]

i updated and now I'm getting this.. "Sorry... the WU data for this date is not currently available. Please try again later." http://www.gosportwx.com/wxwuhistory.php

[mkutche]

Update it seems to be working now .. but i get this /home/hj2ke8lfpx43/public_html/WU-History-inc.php on line 396

[saratogaWX]

Looks like it's working again.. when it happens, do a view-source on the page.  A success for 'daily' display should show something like

<!-- WU-History-inc.php - Version 3.4d - 23-May-2019  -->
<!-- final units='E' -->
<!-- curl fetching 'http://www.gosportwx.com/WXDailyHistory.php?ID=KINGOSPO2&month=05&day=25&year=2019&format=1&graphspan=day' -->
<!-- HTTP stats:  RC=200 dest=160.153.94.162 port=80 (from sce=160.153.94.162)
      Times: dns=0.004 conn=0.005 pxfer=0.005 get=0.149 total=0.154 secs -->
<!-- headers:
HTTP/1.1 200 OK
Date: Sun, 26 May 2019 00:20:13 GMT
Server: Apache
X-Powered-By: PHP/7.2.16
Expires: Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control: no-store, no-cache, must-revalidate
Pragma: no-cache
Set-Cookie: PHPSESSID=e0cf938cc25a8138b11a4120cfa21dbc; path=/
Upgrade: h2,h2c
Connection: Upgrade
Vary: Accept-Encoding,User-Agent
Transfer-Encoding: chunked
Content-Type: text/plain;charset=ISO-8859-1
 -->
<!-- Size of the array is 289 -->
<!-- Size of the final array is 289 -->

Otherwise, there'd be HTML comments in that area highlighting the problem with data retrieval.

Re the notice errata.. the WU-History-inc.php may still have $PHP_SELF variables in it.. replace all with $_SERVER['PHP_SELF'] as the old global variable has been deprecated in later PHP versions.

[djroby19]

Good afternoon the month wunderground data string has no data
this https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=ILAZIOP4&graph=month&year=2019&month=07&format=1&units=metric happens to you too?
I sent an email to wunderground but no answer 

[dunc1ca]

They are no longer supporting this page.  When I contacted them in May, I got this reply:

The URL you are trying to use has been deprecated. It may be up and down before it’s gone for good, but it is on its way out.
On your dashboard, using the table view, you should be able to copy and paste into a spreadsheet.
https://www.wunderground.com/dashboard/pws/ICAVANMO7/table/2019-05-23/2019-05-23/daily
From there you should be able to convert to csv format, if that’s your preference.
Thank you for using Weather Underground.
—Victoria

[djroby19]

but I don't understand why some people work and others don't if it is no longer supported it should not work to anyone I guess
Thanks a lot for the answer

[wvdkuil]

Today multiple stations (not all) can retrieve their data again. They were shut off from  end May or mid June, but they all can get their data with the original .asp
https://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=IDIDIDID&graphspan=year&year&format=1

Also, the file contents are slightly changed. The extra  linefeed at the start and all extra "<br>" are gone.

Code: [Select]


Date,TemperatureHighC,TemperatureAvgC,TemperatureLowC,DewpointHighC,DewpointAvgC,DewpointLowC,HumidityHigh,HumidityAvg,HumidityLow,PressureMaxhPa,PressureMinhPa,WindSpeedMaxKMH,WindSpeedAvgKMH,GustSpeedMaxKMH,PrecipitationSumCM<br>
2019-1-1,8.7,6.9,5.2,7.9,5.7,2.2,96,89,79,1034,1030,23,2,23,0.03
<br>
2019-1-2,6.5,5.4,4.2,2.8,1.7,0.5,83,77,69,1040,1034,16,1,21,0.00
<br>
2019-1-3,5.9,4.4,3.0,3.8,2.7,1.8,93,88,83,1041,1040,8,0,13,0.05
<br>

Code: [Select]

Date,TemperatureHighC,TemperatureAvgC,TemperatureLowC,DewpointHighC,DewpointAvgC,DewpointLowC,HumidityHigh,HumidityAvg,HumidityLow,PressureMaxhPa,PressureMinhPa,WindSpeedMaxKMH,WindSpeedAvgKMH,GustSpeedMaxKMH,PrecipitationSumCM
2019-1-1,8.7,6.9,5.2,7.9,5.7,2.2,96,89,79,1034,1030,23,2,23,0.03
2019-1-2,6.5,5.4,4.2,2.8,1.7,0.5,83,77,69,1040,1034,16,1,21,0.00
2019-1-3,5.9,4.4,3.0,3.8,2.7,1.8,93,88,83,1041,1040,8,0,13,0.05
BIG QUESTION:  Is WU/TWC/IBM still actively working on that part?

Wim