Author Topic: Lightning placefile code (GR-series programs)  (Read 7633 times)

0 Members and 1 Guest are viewing this topic.

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Lightning placefile code (GR-series programs)
« on: June 05, 2014, 08:40:48 AM »
So, as I think I mentioned before, one of my goals was to incorporate lightning data into Gibson Ridge Software's GR2Analyst Nexrad display program.  That's normally done using "placefiles" (text files that instruct the program to overlay additional data on screen).  I now have a first draft of a PHP program to do that, using icons that vary according to age of the strike and kA intensity (strike current is also shown with hover text).

NOTE: you will need your Blitzortung participant username/password to access the raw data downloaded and displayed by these placefiles.  This information needs to be inserted in the "lightningloader.php" file.

As a reminder, the Blitzortung raw data that this placefile uses is available only to system participants.  The placefile won't work without such data access.


In order to keep the load on the server and bandwidth usage light, while at the same time update nearly as soon as new data is available, the program polls the most recent raw data log file headers every 10 seconds, checks whether the log file has been modified since the last poll, and if so, only then downloads it and adds any new lightning log entries to a running list of strikes stored locally.  It trims the local list to 1/2 hour, but when Gibson Ridge updates their software to include time ranges for placefile display data, I'll probably up that to an hour so it can animate the lightning data.

Right now, it throws the last 1/2 hour of lightning data on the screen all at once, with color coding to distinguish the age of the strike.

Here are some pics from the intense thunderstorms over Kansas this morning, and a copy of the code.  The code is run locally and served up using a locally-running Apache web server.  Icons are by Joel Wright (posted at the Gibson Ridge forum).









UPDATE 6/6/2014 17:00 GMT: I've modified the placefile into "loader" and "display" placefiles.  Both should be added to the placefile list.  This allows for ongoing lightning data gathering in the background without necessarily displaying the data (the loader should always be enabled).

I've also added some comments and simplified some of the code.  It will now proceed to download data, for the time frame set by the variable $MaxAge, until it finds data that's already in the local lightning log.  This allows the placefile to "catch up" when the GR-series program is first run.

UPDATE 6/13/2014: The loader file has been modified to accept the latest revision of GR programs, which have changed the way they identify themselves.

UPDATE 6/25/2014: Unfortunately, the "strike strength" data value hasn't been included in the log files for several days now - I think the Blitzortung developers are working on a number of enhancements, including faster data access.  So, all lightning strikes are appearing only as dot icons.  I may modify the placefile generator to calculate strike strength based on the distance to the farthest station that detects an individual strike.
« Last Edit: June 25, 2014, 04:55:15 PM by n0ym »

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #1 on: June 06, 2014, 02:30:53 PM »
Just a note to say I updated the code attached to the first post (6/6/2014, 17:00 GMT).

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #2 on: June 07, 2014, 02:33:52 PM »
I'd like to try this out, but I assume I have to have something that can run a PHP script? Is it possible you could give step-by-step directions on what needs to be done to get this working?

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #3 on: June 07, 2014, 03:18:41 PM »
You'll need to install a webserver with PHP - I have Apache 2.4 running on the same machine as the one I use to run GR2Analyst (lowest bandwidth overhead).  I run Apache as a manual service (though usually I keep it running - it has low overhead).  It's actually pretty straightforward to install, but if you haven't done it before there are a couple of things you have to configure.

The placefile should also work if it's installed to be served up by a remote web server, BUT at this point it's really designed to be served to one computer at a time.  I'd need to rewrite the code if it was intended for more general placefile generation for multiple computers from a single server.

I'm running Windows 7 64 bit.

1) I manually downloaded Apache 2.4.9 64 bit from here:
http://www.apachelounge.com/download/

...and unzipped it into the c:\ directory (so you end up with c:\apache24\...). 

2) and PHP 5.5 thread-safe 64 bit from here:
http://windows.php.net/download/

...unzipped into the c:\php directory.
Which version to download depends on whether you're running a 32 bit or 64 bit version of Windows.

3) I added this to the httpd.conf file in c:\apache24\conf:

Code: [Select]
LoadModule php5_module "c:/php/php5apache2_4.dll"
AddHandler application/x-httpd-php .php

# configure the path to php.ini
PHPIniDir "C:/php"

4) The only other thing you might need to do is to copy the "php5apache2_4.dll" from the c:\php\ext directory to c:\php, if it's not there already.  But I'm not positive that's necessary.

5) Click Start, and run "c:\apache24\bin\httpd.exe -k install".  That installs Apache as a service.  There's no need to open up your local firewall, as you're not going to be running a webserver for anyone but yourself.

6) If you unzip the lightning placefile in c:\apache24\htdocs, you should then be able to add the placefile in the GR program of your choice by using "http://127.0.0.1/lightningloader.php" for the data loading placefile (not displayed) and "http://127.0.0.1/lightningpf.php" for the display placefile.

Does this work for you?  It's been a bit since I installed Apache/PHP, so I'm going from memory.

Otherwise, if you want a pre-configured package that comes with a few additional things, you can install WAMP:

http://www.wampserver.com/en/
« Last Edit: June 07, 2014, 03:34:11 PM by n0ym »

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #4 on: June 07, 2014, 04:03:28 PM »
I'm also running W7 64 bit.

Followed the steps, but GR is still not reading the two links "http://127.0.0.1/lightningpf.php" and "http://127.0.0.1/lightningloader.php"

I also added the code below to the httpd.conf file in c:\apache24\conf: but does it need to go in any specific spot? I placed the code at the very bottom of the file.

Code: [Select]
LoadModule php5_module "c:/php/php5apache2_4.dll"
AddHandler application/x-httpd-php .php

# configure the path to php.ini
PHPIniDir "C:/php"

"php5apache2_4.dll" is already in c:\php and I installed Apache as a service just fine too. Lightning placefiles are in htdocs as well.



Thanks for the step-by-step by the way. I'll get it working eventually.  :oops:

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #5 on: June 07, 2014, 04:05:33 PM »
No problem.  As a first test, if the apache service is up and running, you should be able to open any browser, go to "http://127.0.0.1/index.html" and have it open a welcome page.

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #6 on: June 07, 2014, 04:18:09 PM »
"http://127.0.0.1/index.html" is giving me a 404 Not Found error. So apparently I messed up somewhere, but not sure where.

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #7 on: June 07, 2014, 04:21:02 PM »
It's possible after installing Apache as a service that you need to reboot.  I've sometimes had problems with that.  But I think it's installed to be run automatically otherwise (unless you change the service to manual start).

Oh, and one more thing -- in the "lightningloader.php" file, you'll need to insert your own Blitzortung username/password for the file to be able to download data.  Embarrassed I forgot to mention that  :oops:
« Last Edit: June 07, 2014, 04:42:25 PM by n0ym »

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #8 on: June 07, 2014, 04:39:07 PM »
Did a reboot and still won't connect to the index page. I also deleted everything and started from scratch, but no luck with that either.

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #9 on: June 07, 2014, 04:50:47 PM »
Did a reboot and still won't connect to the index page. I also deleted everything and started from scratch, but no luck with that either.

Hmmm...is the service running?  The only thing I can think of is there might be some sort of error preventing startup.  You can check in C:\Apache24\logs\error.log to see if it tried to start, but had a problem...

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #10 on: June 07, 2014, 09:17:25 PM »
I'm thinking it's not even starting up. Checked the logs folder and there isn't an error.log file. Also looked at the services in the task manager and I don't see anything named Apache running. I'll keep tinkering with it I suppose and see what I can come up with.

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #11 on: June 07, 2014, 09:21:25 PM »
Was finally able to catch the error in a screenshot. (Barely shows up for half a second)


Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #12 on: June 07, 2014, 09:22:34 PM »
You may need to open a command window, navigate to c:\apache24\bin and then run "httpd.exe -k install" from the command line.

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #13 on: June 07, 2014, 09:25:05 PM »
Here's the other error for when I try to install it. The other was unistalling.


Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #14 on: June 07, 2014, 09:27:32 PM »
Ah, OK.  I think if you run the command line window as an administrator (search for "command", right click on the "command prompt" search result that comes up and select "run as administrator") before running the install command, I think that may solve the problem.

Offline VaWx

  • Member
  • *
  • Posts: 29
Re: Lightning placefile code (GR-series programs)
« Reply #15 on: June 07, 2014, 09:40:30 PM »
Well that got me a little closer, but it's still giving me a fit.


Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #16 on: June 07, 2014, 09:46:45 PM »
I haven't run across this error before, but a suggestion I've seen online is to directly run httpd.exe from the command line, so if your command-line window is still open to c:\apache24\bin, just run "httpd.exe".  One common problem is Skype running at the same time (if Skype's ports haven't been changed).

Offline sacreyweather

  • Echo Valley Weather
  • Forecaster
  • *****
  • Posts: 1778
    • Echo Valley Weather
Re: Lightning placefile code (GR-series programs)
« Reply #17 on: June 08, 2014, 10:48:56 AM »
Hi,

First off, thank you for making this. I've been contemplating making one, but haven't had time. I am attempting to use this on the server my website is hosted.  When I run lightningloader.php I get:

;Input is invalid
RefreshSeconds:10

I have tried every known combination of directory structures with no luck.  I have also created both the lightning.log and lightningplacefile.txt files, set them to 666 to be writable to with no luck. I am open to ideas/suggestions, etc.

Thanks,

John
CWOP: D2073, GR2AE, GR3, Cumulus, PWSweather,  CoCoRaHS: AR-SL-23  

Saline Weather on Twitter
Blitzortung Station 1387

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #18 on: June 08, 2014, 10:57:24 AM »
On this version, that message shows up when the calling program doesn't include the latitude and longitude of the radar.

GR-series programs, when they call a placefile, include the latitude and longitude as part of the URL, e.g.:

http://127.0.0.1/lightningloader.php?lat=41.958&lon=-100.576

You DON'T have to include the latitude and longitude explicitly if the placefile is called from the GR-series program--the program adds it automatically (along with a "version" variable).  But you will need to do so if you call it from a browser like Firefox.

I'm working on an updated version that adds additional validation that requires the placefile to be called from one of the GR-type programs (user agent check), to try to prevent people from obtaining raw text data.  Nothing's perfect, of course.

EDIT: the current version has added both a user agent check and validation of the radar lat/lon (using radar parameters from GR2Analyst).
« Last Edit: June 08, 2014, 11:10:13 AM by n0ym »

Offline sacreyweather

  • Echo Valley Weather
  • Forecaster
  • *****
  • Posts: 1778
    • Echo Valley Weather
Re: Lightning placefile code (GR-series programs)
« Reply #19 on: June 08, 2014, 11:15:11 AM »
I have lightningloader.php running in the placefile manager, but both the lightningpf.php and lightningplacefile.txt file are showing "could not load". Plus when I run the lightningpf.php file from the browser, it shows as a blank screen.
CWOP: D2073, GR2AE, GR3, Cumulus, PWSweather,  CoCoRaHS: AR-SL-23  

Saline Weather on Twitter
Blitzortung Station 1387

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #20 on: June 08, 2014, 11:18:52 AM »
I can think of one thing that might be wrong - you said you're running this on a different server, right?  The code changes directory to a "lightning" subdirectory based on the idea that the webserver is running on the same computer as the GR-series program, and it's apache in c:\apache24\htdocs.

Let me see if I can modify it to make it a bit more general, and upload a new version.

Offline sacreyweather

  • Echo Valley Weather
  • Forecaster
  • *****
  • Posts: 1778
    • Echo Valley Weather
Re: Lightning placefile code (GR-series programs)
« Reply #21 on: June 08, 2014, 11:28:44 AM »
Hi,

I now have it running. I had inadverdantly placed the:

include   "lightningplacefile.txt"; into parenthesis and had taken out the ( ,"c+") in the lightningloader.php thinking it was for a local machine.

Thanks for the help.


CWOP: D2073, GR2AE, GR3, Cumulus, PWSweather,  CoCoRaHS: AR-SL-23  

Saline Weather on Twitter
Blitzortung Station 1387

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #22 on: June 08, 2014, 11:39:40 AM »
Hi,

I now have it running. I had inadverdantly placed the:

include   "lightningplacefile.txt"; into parenthesis and had taken out the ( ,"c+") in the lightningloader.php thinking it was for a local machine.

Thanks for the help.

Glad you have it working!

I've uploaded a new version that fixed the directory issues, as well as reduced a little redundancy.

Offline Dr Obbins

  • Forecaster
  • *****
  • Posts: 1148
Re: Lightning placefile code (GR-series programs)
« Reply #23 on: June 09, 2014, 08:45:55 PM »
Hi,

I now have it running. I had inadverdantly placed the:

include   "lightningplacefile.txt"; into parenthesis and had taken out the ( ,"c+") in the lightningloader.php thinking it was for a local machine.

Thanks for the help.
So if I understand correctly, you have this script running on a remote web server and the GRL3 running on your home computer and it is working for you? I have been trying that with out any luck. If you don't mind can you post your settings and/or explain what you did to get it running?

Offline JonathanW

  • Engineer
  • Forecaster
  • *****
  • Posts: 323
Re: Lightning placefile code (GR-series programs)
« Reply #24 on: June 13, 2014, 08:58:24 PM »
Updated to fix a minor issue that arose with the latest versions of GR programs.

Doc Obbins, did you get your issue resolved?