Author Topic: advforecast2.php issue  (Read 5095 times)

0 Members and 1 Guest are viewing this topic.

Offline Intheswamp

  • Early on one frosty morn'...
  • Forecaster
  • *****
  • Posts: 1485
  • Sure...I know enough to be dangerous, now.
    • Beeweather
Re: advforecast2.php issue
« Reply #50 on: October 08, 2018, 03:54:27 PM »
Uh, er, uh....well, the point forecast is gone again and the zone has reverted back to yesterday, 10-7-2018.  I'm outta here!!!!!  ](*,) ](*,)

Davis VP2, v3.12
Cumulus v1.9.4  Build 1079
Today's weather beats no weather at all!
www.beeweather.com
www.beausandbonnets.com

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: advforecast2.php issue
« Reply #51 on: October 08, 2018, 04:15:08 PM »
Same here, I just checked and the "new" version is working again so I again changed back to version 5.09.  Will just keep an eye on it.
I have my NWS scripts still in "test" with different lat-lon settings every few days. Currently for 3 Miles ENE of Chandler OK
In my day-time (your night-time) the v3-API is fully functional about  40-60% of the time.  In your day-time it functions for 75% or more of the time.  The most troublesome part is alerts-api https://api.weather.gov/alerts/active?point=35.7236,-96.8205
I included an automatic fall-back to v2-json  so that at least a normal forecast is shown when the new API is not available.
For this demo-site I use relative long timeouts (20 seconds) for the NWS scripts.  Also error-reporting is set to maximum to catch all errors.

Feel free to request a download  link to test the scripts yourself.  All but NWS are full production versions.

Menu page for all forecasts:  https://www.weerstation-herent.be/wsfct4/

Demo links to my  "National Weather Service (v3) Weather forecast " :
Stand-alone version:  https://www.weerstation-herent.be/wsfct4/wsFctNoaaPPage4.php
In Saratoga-template:  https://www.weerstation-herent.be/saratoga/wx_lvn_noaaPP.php

ALL feedback and suggestions for improvements are welcome. I live on another continent and need local expertise to improve on these scripts. For me the metno,  darksky or WU forecasts are far easier to test compared to NWS.

Wim


Offline Jasiu

  • Forecaster
  • *****
  • Posts: 947
    • LexMAWeather
Re: advforecast2.php issue
« Reply #52 on: October 08, 2018, 04:33:58 PM »
I've been looking into the new API problems, and timeouts seem to be the biggest issue for me. Essentially, the https connection gets set up (or an existing connection is reused), the GET for the API endpoint is issued, the GET is ack-ed at the TCP level, and then.... crickets.

After some experimentation, I now have every fetch in a for loop with an increasing CURL timeout value for each retry, starting at one second and maxing out at nine.  It often takes multiple calls to get the data, but I have not yet seen a case where I don't eventually get the data.

For example, running the forecast script right now:

1) Point forecast fetch: Failed three times, worked on 4th. On the successful fetch, the total time spent was only 0.122 seconds.

2) Alert "point" fetch: Failed six times, worked on 7th.  0.124 seconds for the successful fetch.

EDIT: I was able to verify the actual traffic sent to the NWS server by running the script from my laptop. Nice that Mac OS has good-old Unix underneath.  :grin:
https://lexmaweather.info
On Mastodon: @LexMAWeather@toot.community

Online saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2.php issue
« Reply #53 on: October 08, 2018, 04:42:23 PM »
I think it may be a caching problem on the NWS end.

The query for /point/<lat>,<long>/forecast is supposed to return a 301 to /gridpoint/<g1>,<g2>/forecast and a subsequent query to /gridpoint/<g1>,<g2>/forecast then either yields data (200 OK) or (404 Not Found) if the gridpoint data is not current/present.

In my 5.09 script, I cache the successful point->gridpoint URL and just use the gridpoint subsequently for the query.  Rebuilt if a force=1 (or equivalent) is done.

Given their issues with keeping the gridpoint forecasts loaded/current, and perhaps bugs with their caching service, it's not a big wonder to see why things aren't working reliably for api.weather.gov/forecast-v3.weather.gov right now.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline DaculaWeather

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3206
    • North Georgia Weather
Re: advforecast2.php issue
« Reply #54 on: October 08, 2018, 07:40:44 PM »
SENIOR DUTY METEOROLOGIST NWS ADMINISTRATIVE MESSAGE
NWS NCEP CENTRAL OPERATIONS COLLEGE PARK MD
2323Z MON OCT 08 2018


NCO is aware of issues with products not updating on weather.gov,
specifically the WAWA map and WFO sites. NCO is fully engaged and
working to restore the products.

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: advforecast2.php issue
« Reply #55 on: October 09, 2018, 03:34:15 AM »
This is an example of the 07:00 UTC  test of my  NWS scripts.
Extremely  long response times / time outs for the first URL of the 4 needed => results in no-data retrieved

https://api.weather.gov/
URL         time/seconds
1=points    5.1   20.0  4.7  4.3  20.0
2=offices   2.2   x     0.3  1.4  x
3=forecast  8.2   x    19.4  0.1  x
4=alerts   14.0   x     1.4  1.4  x


No idea why the access times vary so much, even for the same location in subsequent requests.
Wim

Detailed list of CURL access times:
Code: [Select]
At 09:00 CEST  /  07:00 UTC

======= test for 35.7236,-96.8205

[url] => https://api.weather.gov/points/35.7236,-96.8205 
    [total_time] => 5.163184
    [namelookup_time] => 0.008313
    [connect_time] => 0.010489
    [pretransfer_time] => 0.021661
    [starttransfer_time] => 5.163055

[url] => https://api.weather.gov/offices/OUN
    [total_time] => 2.203137
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.001654
    [pretransfer_time] => 0.011606
    [starttransfer_time] => 2.203021

[url] => https://api.weather.gov/gridpoints/OUN/122,104/forecast 
    [total_time] => 8.208564
    [namelookup_time] => 2.2E-5
    [connect_time] => 0.001735
    [pretransfer_time] => 0.014604
    [starttransfer_time] => 8.208464

[url] => https://api.weather.gov/alerts/active?point=35.7236,-96.8205
    [total_time] => 14.004311
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.002216
    [pretransfer_time] => 0.017703
    [starttransfer_time] => 14.004184

======= test for 41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 20.011297
    [namelookup_time] => 0.023046
    [connect_time] => 0.043851
    [pretransfer_time] => 0.157907
Error  codes:Operation timed out after 20011 milliseconds with 0 bytes received

==> alternative v2 forecast trieved

[url] => https://forecast.weather.gov/MapClick.php?lat=41.1511&lon=-74.3511&FcstType=json
    [total_time] => 1.201338
    [namelookup_time] => 0.018547
    [connect_time] => 0.040375
    [pretransfer_time] => 0.229122
    [starttransfer_time] => 1.2013

=======  repeat same test for 41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 4.740465
    [namelookup_time] => 0.024589
    [connect_time] => 0.045082
    [pretransfer_time] => 0.15167
    [starttransfer_time] => 4.7329

[url] => https://api.weather.gov/offices/OKX
    [total_time] => 0.361493
    [namelookup_time] => 2.0E-5
    [connect_time] => 0.021295
    [pretransfer_time] => 0.125193
    [starttransfer_time] => 0.361398

[url] => https://api.weather.gov/gridpoints/OKX/17,52/forecast
    [total_time] => 19.49374
    [namelookup_time] => 4.4E-5
    [connect_time] => 0.02147
    [pretransfer_time] => 0.127005
    [starttransfer_time] => 19.493604

[url] => https://api.weather.gov/alerts/active?point=41.1511,-74.3511
    [total_time] => 1.431452
    [namelookup_time] => 2.1E-5
    [connect_time] => 0.021789
    [pretransfer_time] => 0.129773
    [starttransfer_time] => 1.431374

=====  force RELOAD after 2 minutes after all data OK  for  41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 4.351943
    [namelookup_time] => 0.008756
    [connect_time] => 0.028074
    [pretransfer_time] => 0.15786
    [starttransfer_time] => 4.351842

[url] => https://api.weather.gov/offices/OKX
    [total_time] => 1.431917
    [namelookup_time] => 3.3E-5
    [connect_time] => 0.023675
    [pretransfer_time] => 0.159705
    [starttransfer_time] => 1.43182

[url] => https://api.weather.gov/gridpoints/OKX/17,52/forecast
    [total_time] => 0.131973
    [namelookup_time] => 2.2E-5
    [connect_time] => 0.020575
    [pretransfer_time] => 0.109796
    [starttransfer_time] => 0.131874

[url] => https://api.weather.gov/alerts/active?point=41.1511,-74.3511
    [total_time] => 1.420042
    [namelookup_time] => 2.3E-5
    [connect_time] => 0.020782
    [pretransfer_time] => 0.099507
    [starttransfer_time] => 1.41994

=====  force RELOAD after 2 minutes after all data OK  for  41.1511,-74.3511

[url] => https://api.weather.gov/points/41.1511,-74.3511
    [total_time] => 20.026838
    [namelookup_time] => 0.021461
    [connect_time] => 0.042464
    [pretransfer_time] => 0.158886
Error  codes:Operation timed out after 20026 milliseconds

Online saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2.php issue
« Reply #56 on: October 09, 2018, 01:08:20 PM »
I did a bit of 'dig' for api.weather.gov

Quote
10/09/18 09:58:46 dig api.weather.gov @ 208.67.222.222
Dig api.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for api.weather.gov type=255 class=1
  api.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net


10/09/18 09:59:05 dig sancert.weather.gov.edgekey.net @ 208.67.222.222
Dig sancert.weather.gov.edgekey.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for sancert.weather.gov.edgekey.net type=255 class=1
  sancert.weather.gov.edgekey.net CNAME (Canonical Name) e278.dscg.akamaiedge.net


10/09/18 09:59:20 dig e278.dscg.akamaiedge.net @ 208.67.222.222
Dig e278.dscg.akamaiedge.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for e278.dscg.akamaiedge.net type=255 class=1
  e278.dscg.akamaiedge.net A (Address) 23.6.200.7
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:2096:0:0:0:116
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:208a:0:0:0:116

It looks like they're trying to use Akamai cloud for their load balancing/primary serving for requests and it's likely not set up quite correctly.  I got 26s for the same query.. all of it wait for response.  Not exactly a speedy response for a cloud caching service.
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Online saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: advforecast2.php issue
« Reply #57 on: October 10, 2018, 03:46:13 PM »
Sigh... looks like BOTH api.weather.gov (the Beta/non-production service used by V5.09) AND forecast.weather.gov (used by V4.05) are using the same Akamai IP address, so ... no safe-harbor while they get the Akamai hosting issues fixed.  I'm seeing 25 second responses from forecast.weather.gov now too..\

Grumble.

Quote
10/10/18 12:38:32 dig api.weather.gov @ 208.67.222.222
Dig api.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for api.weather.gov type=255 class=1
  api.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net

10/10/18 12:37:59 dig forecast.weather.gov @ 208.67.222.222
Dig forecast.weather.gov@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for forecast.weather.gov type=255 class=1
  forecast.weather.gov CNAME (Canonical Name) sancert.weather.gov.edgekey.net

10/10/18 12:38:42 dig sancert.weather.gov.edgekey.net @ 208.67.222.222
Dig sancert.weather.gov.edgekey.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for sancert.weather.gov.edgekey.net type=255 class=1
  sancert.weather.gov.edgekey.net CNAME (Canonical Name) e278.dscg.akamaiedge.net


10/10/18 12:38:51 dig e278.dscg.akamaiedge.net @ 208.67.222.222
Dig e278.dscg.akamaiedge.net@208.67.222.222 ...
Non-authoritative answer
Recursive queries supported by this server
 Query for e278.dscg.akamaiedge.net type=255 class=1
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:6081:0:0:0:116
  e278.dscg.akamaiedge.net AAAA (IPv6 Address) 2001:559:19:6083:0:0:0:116
  e278.dscg.akamaiedge.net A (Address) 23.197.96.136

Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Intheswamp

  • Early on one frosty morn'...
  • Forecaster
  • *****
  • Posts: 1485
  • Sure...I know enough to be dangerous, now.
    • Beeweather
Re: advforecast2.php issue
« Reply #58 on: October 12, 2018, 08:19:52 AM »
This morning...

Cities and towns reported by the Mobile/Pensacola office seems to be working well and is faster in loading than I've observed lately.

Birmingham/Calera office seems about the same, working well but might be a tad slower.

Tallahassee office....only zone forecast.  Understandable.

But, Mobile/Pens and B'ham/Calera seem to be doing good.

Davis VP2, v3.12
Cumulus v1.9.4  Build 1079
Today's weather beats no weather at all!
www.beeweather.com
www.beausandbonnets.com

Offline DaculaWeather

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3206
    • North Georgia Weather
Re: advforecast2.php issue
« Reply #59 on: October 15, 2018, 02:17:56 PM »
For me here in Atlanta, today is the first day that I've gotten the point forecast using this script, it has only displayed the zone forecast for months and months and months.