WXforum.net

Weather Station Hardware => Blitzortung => Topic started by: miraculon on March 20, 2022, 04:00:34 PM

Title: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 20, 2022, 04:00:34 PM
I am getting the Subject error in a red band on my lightning maps.

https://www.rogerscityweather.com/NorthAm-blitz.php (https://www.rogerscityweather.com/NorthAm-blitz.php)

Saratoga-Weather has the same message on the map at https://saratoga-weather.org/NA-lightning.php (https://saratoga-weather.org/NA-lightning.php)

The main Blitzortung (https://www.blitzortung.org/en/live_lightning_maps.php?map=30) site and Lightningmaps.org seem to be OK though.

Any ideas on this?

Greg H.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 20, 2022, 05:59:42 PM
The sites seem to be up, but the data site is getting 404s (Not Found) for the files needed:
Code: [Select]
fetching new strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/40.json
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/40.json
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:03 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new GZ strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/40.json.gz
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/40.json.gz
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:04 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/50.json
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/50.json
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:04 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new GZ strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/50.json.gz
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/19/50.json.gz
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:05 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/00.json
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/00.json
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:05 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new GZ strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/00.json.gz
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/00.json.gz
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:06 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/10.json
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/10.json
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:06 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new GZ strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/10.json.gz
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/10.json.gz
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:07 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/20.json
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/20.json
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:07 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

fetching new GZ strikes file at http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/20.json.gz
Headers returned from http://user:pass-omitted@data.blitzortung.org/Data_3/Protected/Strikes/2022/03/20/20/20.json.gz
Array
(
    [0] => HTTP/1.1 404 Not Found
    [Server] => nginx/1.14.0 (Ubuntu)
    [Date] => Sun, 20 Mar 2022 21:40:08 GMT
    [Content-Type] => text/html
    [Content-Length] => 178
    [Connection] => close
)

...
Unknown why this is happening.  The Service Monitor (https://monitor.lightningmaps.org/) shows all is well with data.blitzortung.org

I'll poke around a bit...
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 20, 2022, 06:45:07 PM
Looks like the data server stopped posting updates after 2022-03-19 18:20 GMT .. the 2022-03-20 set is completely empty.  So it is (at this time) a Blitzortung data issue (they're not publishing it).
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: W0BTU on March 20, 2022, 10:09:23 PM
This page stopped updating about the same time.
https://www.lightningmaps.org/blitzortung/america/index.php?bo_page=statistics&bo_show=station&lang=en&bo_station_id=1174
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: Cutty Sark Sailor on March 21, 2022, 07:24:10 AM
If this isn't simply a 'glitch', I suspect this will prove to be related to stroke/station "regional assignment" realignments and upgrades. Example, since last year stations in 'North America' count activity to two "servers": 3 and 10,  which relate to'geographic areas' for stroke location. I have 'personally' (While awaiting Egon's 'final' designations and processing) have been referring to them (https://frankfortweather.us/fwxBLPublic/index.html#rassign) as
"North America 1 (NA Global) — Updates @ 1-5 min — All GLOBAL stations localizing Atlantic, Pacific, Americas.
(old 'region 3)
and

North America 2 (NA  Local) — Updates @ 1 min — NORTH AMERICA stations ONLY localized.
(although not reflecting the 'actual' intent and activity of the 'name' I'm using...that is 'quick and dirty')
Something similar for the other 'original' regions, as the refinements continue...
... and I would expect a bit of further confusion as the season begins in Northern Hemisphere and Egon pulls a bit more together...
(https://frankfortweather.us/misc/DaffyHuh120.png)
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 21, 2022, 08:39:48 AM
Thanks, Ken, for posting on the Blitzortung forum. I was just going to do that and I saw your post. No response yet as of now.

I was going to check the strikes on GRLevel3, but the strikes out in the Atlantic might be too far offshore to show up.  I suspect that the placefiles on GRLevel3 will have the same issue. UPDATE: Now there is some lightning in Oklahoma, no "B" strikes showing from the Placefile, but ENTLN data is showing strikes.

I do hope that this is fixed before lightning season (around here) gets underway.

Greg H.

Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 21, 2022, 01:52:30 PM
All the /Data/Protected/Strikes_{n}/2022/03/21 directories are also empty.. some 'fix' was installed after 2022-03-19 18:20 UTC as that's the last data for any of them.

Hope Toby/Egon sees the post on the BO forum and fixes it soon...
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 23, 2022, 12:26:06 PM
So I got impatient waiting for the 10-minute JSON files to begin populating and rewrote a bunch of BOmapgen-inc.php V1.07 to use the new last_strikes.php query instead.  It's now running on Saratoga-weather.org.

You can download the full package (https://saratoga-weather.org/BOmap-V1.07.zip) or just use the attached replacement for BOmapgen-inc.php.

The new version will scan your $Maplist entries for the maximum N,W to S,E lat/long pairs and use that extent in the query to last_strikes.php on data.blitzortung.org.

Code: [Select]
Parms: using last_strikes.php query method for strikes
$Maplist used:
  'NorthAmerica.png|BONorthAmerica.png|62.0,-145.0,10.0,-50.0|bottom,left||320|'
  'USA.png|BOUSA.png|52.0,-127.0,16.0,-65.0|bottom,right|placefile.txt|120|'
  'SWN.png|BOSWN.png|43.0,-125.0,30.0,-108.0|bottom,left||120|'
  'SFBayArea.png|BOSFBay.png|39,-124,36,-120|bottom,left||120|'
  'NorthWest.png|BONorthWest.png|52.0,-130.0,39.0,-100.0|bottom,left||320|'
  'MidWest.png|BOMidWest.png|50.4,-98.5,35.3,-79.2|bottom,left||320|'
  'MidSouth.png|BOMidSouth.png|42.0,-107.0,25.0,-82.0|bottom,left||320|'
  'Hawaii.png|BOHawaii.png|23,-161,18,-154|bottom,left|HI-placefile.txt|120|'
  'Alaska.png|BOAlaska.png|72,-172,54,-129|top,right|AK-placefile.txt|120|'
  'Ontario.png|BOOntario.png|58.0,-98.0,40.0,-72.0|top,right||120|'
  'SouthernOntario.png|BOSouthernOntario.png|50.0,-98.0,40.0,-72.0|top,right||120|'
  'EasternCanada.png|BOEasternCanada.png|62.0,-100.0,35.0,-50.0|top,right||'
  'WesternCanada.png|BOWesternCanada.png|62.0,-150.0,35.0,-90.0|bottom,left||'
$MapList scan has N=72,W=-172 and S=10.0,E=-50.0 for overall map coordinates.
Query='https://user:pass-omitted@data.blitzortung.org/Data/Protected/last_strikes.php?number=999999&north=72&west=-172&south=10.0&east=-50.0&sig=0&time=1648045200'
Query took 1.816 seconds
Query returned 28895 strikes .. saved to BO/strikes.txt
Query processing took 0.148 seconds.
Strikes from Wed, 23 Mar 2022 05:38:04 to Wed, 23 Mar 2022 09:20:00 (13315.477874756 seconds)

Enjoy!
Best regards,
Ken
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: satcop on March 23, 2022, 02:46:39 PM
Hi Ken'

Thanks, that got the maps working again. 

Bob
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 23, 2022, 04:20:49 PM
I just noticed that the "B" strikes were showing again on GRLevel3. Also, the embedded maps are working as well as the thumbnail lightningmaps image.

I saw this posted by Egon on the Blitzortung forum:

Hi,

I found the bug. The missing dates from March 19 to March 23 are copied soon.

/Egon


I didn't try the new patch from Ken, but since it's working again, I'll leave it as-is.

Greg H.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 23, 2022, 05:08:53 PM
You might try the patch.. it runs faster than the old method and draws much less on the data.blitzortung.org server (since it omits the 'which stations were involved' list which cuts down greatly on the traffic).
My new one runs in <15 seconds .. former one (using the 10-minute JSON files) would run a minute or more with heavy strike data.

I just checked, and Egon has started the flow of the old format data and repopulated the missing ones from 2022-03-19 also.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: pimohdaimaoh on March 23, 2022, 09:59:09 PM
Hello guys,

I was about to post the same problem here, but I came here already has a solution. Thanks ken
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: gwwilk on March 24, 2022, 06:15:43 AM
Thanks Ken!  The patch is working here, too.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 24, 2022, 12:40:01 PM
You might try the patch.. it runs faster than the old method and draws much less on the data.blitzortung.org server (since it omits the 'which stations were involved' list which cuts down greatly on the traffic).
My new one runs in <15 seconds .. former one (using the 10-minute JSON files) would run a minute or more with heavy strike data.

I just checked, and Egon has started the flow of the old format data and repopulated the missing ones from 2022-03-19 also.

OK, I did it and it seems to be working. Thanks Ken.

Greg H.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: pimohdaimaoh on March 25, 2022, 05:44:57 AM
I wonder if theres a script that the coordinates written on the strikes.txt can be plotted at google maps automatically so that we can reviewed it where the EXACT location these strokes are. I try to sample the txt I saw on stackoverflow but it didnt work, I just want to try plotting these strokes coordinates in google maps well if Ken have a time to create something like this in the future.

<!DOCTYPE html>
<html>
<head>
<style>
  html, body, #map-canvas {
    height: 100%;
    margin: 0px;
    padding: 0px
  }
</style>
<script src="https://maps.googleapis.com/maps/api/js?key=AIzaSyD_RofMXCFSlcxaiq7X32E3-Jybmuqgl-8&callback=initMap"></script>
<script>

function initialize() {
    loadFile('strikes.txt'); // text file to read coordinates
}

function loadFile(uri) {
    var r = new XMLHttpRequest();
    r.open('GET', uri, true);
    r.onreadystatechange = function() {
        if (r.readyState == 4) {
            var lines = r.responseText.split('\n');
            loadMap(lines);
        }
    }
    r.send(null);
}

function loadMap(lines) {
    // Create the map.
    var mapOptions = {
        zoom: 9,
        center: new google.maps.LatLng(14.5040,121),
        mapTypeId: google.maps.MapTypeId.ROADMAP
    };

    var map = new google.maps.Map(
        document.getElementById('map-canvas'),
        mapOptions
    );
    for( var i = 0;  i < lines.length; i++ ) {
        var segments = lines.split('\t');
        var dbm = segments[0];
        var lat = segments[2];
        var lng = segments[1];

        // Construct the circle for each value in citymap.
        // Note: We scale the population by a factor of 20.
        var populationOptions = {
            strokeColor: '#FF0000',
            strokeOpacity: 0.8,
            strokeWeight: 2,
            fillColor: '#FF0000',
            fillOpacity: 0.35,
            map: map,
            center: new google.maps.LatLng(lat, lng),
            radius: 280
        };
        // Add the circle for this city to the map.
        var cityCircle = new google.maps.Circle(populationOptions);
    }
}

google.maps.event.addDomListener(window, 'load', initialize);

</script>
</head>
<body>
    <div id="map-canvas"></div>
</body>
</html>
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: PaulMy on March 25, 2022, 01:03:25 PM
Hi Ken,
I have updated with the full package http://www.komokaweather.com/komokaweather-ca2/ (http://www.komokaweather.com/komokaweather-ca2/)

I had an issue with my GoDaddy cron jobs but have now fixed that and these are executed every 5 minutes and files being updated in the /cache folder.


Before the update and fixing the cron job I found the maps' top header to be way out of date - North America shows 2015-05-22 and the Ontario and Great Lakes maps as 2020-06-18 and that is still the case after the update.  Clicking on a station location opens the details and that seems to be current.  I don't think it is a computer cache issue as the wrong date shows on all my computers.    Have I missed something to have the map header show current, or should I wait a while and see if it updates?

For a couple of other scrips I use cron-job.org and can I use that for the animate.txt execution?  And if so, would there need to be any special parameters added to the full url of the animate file?

Thanks,
Paul
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 25, 2022, 02:29:23 PM
Hi Paul,
What is the URL to the Blitzortung map(s) having the issue.  Also, the URL to the gen-BO-maps-log.txt log of the action would be very helpful.
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: PaulMy on March 25, 2022, 05:41:55 PM
Hi Ken,
The issue shows on the map in the home page http://www.komokaweather.com/komokaweather-ca2/ (http://www.komokaweather.com/komokaweather-ca2/)
As well as the links I have there for the Ontario and Great Lakes
http://www.komokaweather.com/BOmaps/index.php (http://www.komokaweather.com/BOmaps/index.php)
http://www.komokaweather.com/BOmaps/indexON.php (http://www.komokaweather.com/BOmaps/indexON.php)
http://www.komokaweather.com/BOmaps/indexSON.php (http://www.komokaweather.com/BOmaps/indexSON.php)

I don't see a gen-BO-maps-log.txt.  Where should that be?
All the script files are in www.komokaweather.com/BOmaps folder including the cron job file.

Thanks,
Paul

Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: saratogaWX on March 25, 2022, 06:39:34 PM
Hi Ken,
The issue shows on the map in the home page http://www.komokaweather.com/komokaweather-ca2/ (http://www.komokaweather.com/komokaweather-ca2/)
As well as the links I have there for the Ontario and Great Lakes
http://www.komokaweather.com/BOmaps/index.php (http://www.komokaweather.com/BOmaps/index.php)
http://www.komokaweather.com/BOmaps/indexON.php (http://www.komokaweather.com/BOmaps/indexON.php)
http://www.komokaweather.com/BOmaps/indexSON.php (http://www.komokaweather.com/BOmaps/indexSON.php)

I don't see a gen-BO-maps-log.txt.  Where should that be?
All the script files are in www.komokaweather.com/BOmaps folder including the cron job file.

Thanks,
Paul


So, your gen-BO-maps-log.txt is in http://www.komokaweather.com/BOmaps/cache/gen-BO-maps-log.txt

It shows you are generating 3 maps:

/BOmaps/cache/BONorthAmerica-ani.gif (which is old.. likely a permissions issue preventing the file from being overwritten)
[img]
/BOmaps/cache/BONorthAmerica-sm-ani.gif (which is current and updating)
/BOmaps/cache/BOOntario-ani.gif (which is current)
/BOmaps/cache/BOSouthernOntario-ani.gif (which is current)

The real issue may be unwanted caching of the image.   Looking at http://www.komokaweather.com/BOmaps/cache/BONorthAmerica-ani.gif

gives headers of
Quote
    HTTP/1.1 200 OK
    Server: Sucuri/Cloudproxy
    Date: Fri, 25 Mar 2022 22:31:52 GMT
    Content-Type: image/gif
    Content-Length: 776217
    Connection: keep-alive
    X-Sucuri-ID: 11020
    X-XSS-Protection: 1; mode=block
    X-Frame-Options: SAMEORIGIN
    X-Content-Type-Options: nosniff
    Last-Modified: Wed, 23 Mar 2022 17:07:27 GMT
    ETag: "398199a-bd819-5dae5c397a524"
    Cache-Control: max-age=315360000
    Pragma: no-cache
    Expires: Thu, 31 Dec 2037 23:55:55 GMT
    X-Sucuri-Cache: HIT
so the file is updating, but the Sucuri cache is hanging on to it for dear life.  I'd recommend turning off the automatic caching on the /BOmaps/ directory (and subdirectories).

Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: PaulMy on March 26, 2022, 11:30:16 AM
Thanks Ken =D&gt;
Quote
I'd recommend turning off the automatic caching on the /BOmaps/ directory (and subdirectories).

I've " Boldly Gone Where no man (from Komoka) has gone before " and found GoDaddy's Caching Level settings \:D/

All good now http://www.komokaweather.com/komokaweather-ca2/index.php (http://www.komokaweather.com/komokaweather-ca2/index.php)
Including updates to my webcam image \:D/


Enjoy,
Paul

Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 26, 2022, 03:12:54 PM
Paul,

When I just landed on your page, the lightning only went until 11:00 AM (today). I tried directly viewing the map image and got the same map timings.

I double checked mine and it was current.

I took your direct image URL and appended a random number to it, and it updated. This is a good "cache-busting" measure and I use it a lot across my website.

http://www.komokaweather.com/BOmaps/cache/BONorthAmerica-ani.gif?rand=69 BUT don't literally use this because it will still cache this same one after a while. Use something similar to what I have in the iframe line below.

To generate a new random number at each page visit, I have the random function contained in my page:

Code: [Select]
<iframe src="http://rogerscityweather.com/BOmaps/cache/BOUSA-ani.gif?rand=<?=rand(1,1000);?>" width="800" height="480" scrolling="no" frameborder="no"></iframe>
This appends a "random" number between 1 and 1000 to the URL, which makes caching browsers (and I assume the likes of CloudFlare) pull a fresh copy.

?rand=<?=rand(1,1000);?

Unfortunately, I don't remember where I found this trick. I have been using it for several years.

Greg H.


Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: PaulMy on March 27, 2022, 01:52:00 PM
Thanks Greg,
I have added that "?rand=" to the 3 BOmaps index files and that seems to be updating them much better.

Enjoy,
Paul
Title: Re: "Map is not current. Data from Blitzortung is not available." message
Post by: miraculon on March 27, 2022, 02:34:48 PM
Thanks Greg,
I have added that "?rand=" to the 3 BOmaps index files and that seems to be updating them much better.

Enjoy,
Paul

Glad to have helped.

Greg H.