Author Topic: Generate NOAA reports using Meteobridge uploaded data?  (Read 15622 times)

0 Members and 1 Guest are viewing this topic.

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Generate NOAA reports using Meteobridge uploaded data?
« on: June 06, 2014, 12:44:56 PM »
I'd like to generate NOAA reports, or any text report that shows daily high/low/average data and a monthly /yearly summary using the data that my Meteobridge can upload to my website and display on my website. I looked at Display .TXT NOAA style forecasts PHP, but that appears to use the WeatherLink software to upload the data (not WL.com data - which would be okay too). Or am I wrong on that? I also looked at Meteoplug tables, but they don't appear to offer monthly high/low/average data.

Thanks for any guidance on this.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline Bushman

  • Forecaster
  • *****
  • Posts: 7549
    • Eagle Bay Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #1 on: June 06, 2014, 01:18:11 PM »
I'm interested in eventually doing this so it will be interesting to see if there is an easy way.  Of course you could  use MB Push Services to send the data to a MySQL database and write a simple report from there but it would be nice if you did not have to do that.
Need low cost IP monitoring?  http://wirelesstag.net/wta.aspx?link=NisJxz6FhUa4V67/cwCRWA or PM me for 50% off Wirelesstags!!

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #2 on: June 06, 2014, 02:55:40 PM »
I'd like to generate NOAA reports, or any text report that shows daily high/low/average data and a monthly /yearly summary using the data that my Meteobridge can upload to my website and display on my website. I looked at Display .TXT NOAA style forecasts PHP, but that appears to use the WeatherLink software to upload the data (not WL.com data - which would be okay too). Or am I wrong on that? I also looked at Meteoplug tables, but they don't appear to offer monthly high/low/average data.

Thanks for any guidance on this.
If you are uploading your data to WeatherUnderground those high-low_avg values are available.
And you can download them as a CSV file also.

There are multiple scripts which download and use that data already.
Jim McMurry - jmcmurry @ mwt.net for the tabular / graph data
Radomir Luza with its wuGraphs
The WeatherUnderground CSV files are a "poor mans SQL" way of getting at your history.
I think you can then generate your text reports the way you like them.

Wim

P.S. I am using the WU CSV files for a new script shortly available.
As there are no NOAA reports available with some weather programs as you discovered, the Detail/Summary/Season Files from Murry Conarroe of Wildwood Weather needed a "clone" as my users wanted those files also.
So I wrote that clone which is beta-tested now:
http://www.weerstation-wilsele.be/index.php?p=11-2&wp=MH&lang=en


Offline txweather.org

  • Forecaster
  • *****
  • Posts: 1597
    • Texas Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #3 on: June 06, 2014, 03:53:25 PM »
I'd like to generate NOAA reports, or any text report that shows daily high/low/average data and a monthly /yearly summary using the data that my Meteobridge can upload to my website and display on my website. I looked at Display .TXT NOAA style forecasts PHP, but that appears to use the WeatherLink software to upload the data (not WL.com data - which would be okay too). Or am I wrong on that? I also looked at Meteoplug tables, but they don't appear to offer monthly high/low/average data.

Thanks for any guidance on this.
If you are uploading your data to WeatherUnderground those high-low_avg values are available.
And you can download them as a CSV file also.

There are multiple scripts which download and use that data already.
Jim McMurry - jmcmurry @ mwt.net for the tabular / graph data
Radomir Luza with its wuGraphs
The WeatherUnderground CSV files are a "poor mans SQL" way of getting at your history.
I think you can then generate your text reports the way you like them.

Wim

P.S. I am using the WU CSV files for a new script shortly available.
As there are no NOAA reports available with some weather programs as you discovered, the Detail/Summary/Season Files from Murry Conarroe of Wildwood Weather needed a "clone" as my users wanted those files also.
So I wrote that clone which is beta-tested now:
http://www.weerstation-wilsele.be/index.php?p=11-2&wp=MH&lang=en



I would like to be part of your beta test! :D
I am very interested on this script.

Thanks!

----
Davis Vantage Pro2 Plus +FARS|Meteobridge Nano SD|Meteohub|Meteobridge MR-3020|WU KTXSPRIN75/PWS JRARGWX75/CWOP EW2972/WBB TXWDVUE75/Blitzortung ID: 1142|AWEKAS: 12095
Donations are welcome: https://paypal.me/ffuentesb

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #4 on: June 06, 2014, 03:57:51 PM »

. . . shortened . . .

Wim

P.S. I am using the WU CSV files for a new script shortly available.
As there are no NOAA reports available with some weather programs as you discovered, the Detail/Summary/Season Files from Murry Conarroe of Wildwood Weather needed a "clone" as my users wanted those files also.
So I wrote that clone which is beta-tested now:
http://www.weerstation-wilsele.be/index.php?p=11-2&wp=MH&lang=en

I would like to be part of your beta test! :D
I am very interested on this script.

Thanks!
No problem, next week I will send you the final version with the english language.
Wim

Offline txweather.org

  • Forecaster
  • *****
  • Posts: 1597
    • Texas Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #5 on: June 06, 2014, 03:59:08 PM »

. . . shortened . . .

Wim

P.S. I am using the WU CSV files for a new script shortly available.
As there are no NOAA reports available with some weather programs as you discovered, the Detail/Summary/Season Files from Murry Conarroe of Wildwood Weather needed a "clone" as my users wanted those files also.
So I wrote that clone which is beta-tested now:
http://www.weerstation-wilsele.be/index.php?p=11-2&wp=MH&lang=en

I would like to be part of your beta test! :D
I am very interested on this script.

Thanks!
No problem, next week I will send you the final version with the english language.
Wim

Awesome!!! :D

Thanks! :)

----
Davis Vantage Pro2 Plus +FARS|Meteobridge Nano SD|Meteohub|Meteobridge MR-3020|WU KTXSPRIN75/PWS JRARGWX75/CWOP EW2972/WBB TXWDVUE75/Blitzortung ID: 1142|AWEKAS: 12095
Donations are welcome: https://paypal.me/ffuentesb

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #6 on: June 07, 2014, 01:52:21 AM »
Thanks for the info. I would like to be a part of your beta test too. Your page looks exactly like the data I'm interested in. Does the script automatically grab and process the data from the CSV file daily from WU, or is there some manually downloading/uploading that needs to be performed each day?

There must be a lot of interest in my weather data because I've been getting about 5000 page views per month on my WU PWS page
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #7 on: June 07, 2014, 01:59:03 AM »
Thanks for the info. I would like to be a part of your beta test too. Your page looks exactly like the data I'm interested in. Does the script automatically grab and process the data from the CSV file daily from WU, or is there some manually downloading/uploading that needs to be performed each day?

There must be a lot of interest in my weather data because I've been getting about 5000 page views per month on my WU PWS page
The script grabs the year CSV every day two times. First time when the script is executed for the first time that day. And a second time in the afternoon. The "problem" is that in europa the yesterday data is not in the csv before 12:00 UTC time.  After a second grabbing of the CSV after that time the yesterday data is there also. All provious years are fetched once only.  All data is stored in  a folder on the webserver.


Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #8 on: June 07, 2014, 03:01:39 AM »
@mjc775
@xcom
The download location is www.weerstation-leuven.be/wsreports_beta.zip
There is a more-or-less English readme after you unzip the download.

You should first test the install with the stand-alone version with my weather-station settings.
So first test: yourwebsite.xx/wsreports/startReports.php  (if you put the wsreports folder in the root)
That should work always.

Than step 2: adapt the settings and use your version of the start script (Saratoga / stand-alone / Leuven)
Please be aware that the setting for the start-date for WU is dd-mm-yyyy day - month -year.

===

It will be over 30 degrees C this Whitsunday/monday weekend here in Leuven-Belgium, so I probably will not be to often close to a computer.

Succes, Wim

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #9 on: June 07, 2014, 09:20:12 AM »
Those instructions were simple. Took me about 10 minutes to make the reports live: http://wx.ctreno.com/wsreports/startReports.php

Thank you!

After a quick look the only weird thing I saw was a day with a "-2" Wind Average. Probably bad WU data.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #10 on: June 07, 2014, 08:27:11 PM »
How can I change or add to the script to show Hi GustSpeedMax rather than or in addition to Hi WindSpeedMax ? I'm more interested in the highest/peak wind gust.

FYI, in composing this reply I noticed major discrepancies in the WU Weather History chart data on the website vs. the CSV file. For example:

Date      WU - Wind Max / Gust      WU CSV - Wind Max / Gust      wsReports Script
6/1      16 / 23                       21 / 23                              21
6/2      24 / 26                       25 / 27                              25
6/3      20 / 25                       27 / 27                              27
6/4      12 / 16                       17 / 21                              17
6/5      17 / 23                       20 / 23                              20
6/6      20 / 21                       20 / 21                              20


The WU CSV file matches exactly what my weather station's peak wind gust displays. I don't know what WU is doing to mess up their website charts and graphs.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #11 on: June 08, 2014, 03:12:03 AM »
How can I change or add to the script to show Hi GustSpeedMax rather than or in addition to Hi WindSpeedMax ? I'm more interested in the highest/peak wind gust.

FYI, in composing this reply I noticed major discrepancies in the WU Weather History chart data on the website vs. the CSV file. For example:

Date      WU - Wind Max / Gust      WU CSV - Wind Max / Gust      wsReports Script
6/1      16 / 23                       21 / 23                              21
6/2      24 / 26                       25 / 27                              25
6/3      20 / 25                       27 / 27                              27
6/4      12 / 16                       17 / 21                              17
6/5      17 / 23                       20 / 23                              20
6/6      20 / 21                       20 / 21                              20


The WU CSV file matches exactly what my weather station's peak wind gust displays. I don't know what WU is doing to mess up their website charts and graphs.
Good morning,
English is not my native language, so sometimes it is not clear to me what kind of question one is asking. Is there an error in the scripts or an discrepancy in the CSV data?

Be aware that there are always two versions of the CSV. An "english" one and a "metric" one. Both versions have no decimals. So there can be differences of 1 unit (degree or mph a.s.o.)  between reports/csv files/wu websites. As far as I know there is no way to select the units when downloading the data from within a program. So sometimes one gets the CSV in "metric", the next day in "english" units.
As a programmer I have to take the contents of the CSV as is,  and convert the values as to how the user wants them to be displayed.

As I understand the first part of your question correctly, you would prefer to have a page with highest windgust compared to highest windspeed?  I tried to add such a page. The first column is the reported windgust (label high). As there exist no average windgust, I added the highest windspeed for that day in the second column (label avg) to compare the windgust with.

Make a back-up of the current wsReports1part.php, unzip the attachement and upload the new version of wsReports1part.php to your wsreports folder.

Succes, Wim

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #12 on: June 08, 2014, 09:24:40 AM »
Thank you for putting so much time into the script and correspondence. Access to the Wind Gust data is exactly what I wanted. I have uploaded the changed file and it is now live on my website.

My other comments were not an error in the script, rather observations of incorrect data show on the WU website. For example this screenshot from June 4 shows the High Wind Speed / Wind Gust as 12.1 mph / 16.1 mph. But the CSV file shows the data on that date as 17 / 21. The CSV data is always correct and matches what my weather station shows for high gust.

Let me know if you need me to test anything else. So far it seems perfect.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #13 on: June 09, 2014, 11:27:47 AM »
UPDATE: I do see an error in the script... Barometric Pressure does not display a hi / lo value. Is it displaying an average value? Hi / Lo would be preferred.

Thank you.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #14 on: June 09, 2014, 11:57:34 AM »
UPDATE: I do see an error in the script... Barometric Pressure does not display a hi / lo value. Is it displaying an average value? Hi / Lo would be preferred.

Thank you.
@ALL
ERRORS !
I am sorry, there are some errors popping up because of adding features and finding solutions. It is still a beta.Tomorrow I will have a spare day testing.
I updated the download and attached three scripts to replace.
Barometric Pressure does not display a hi / lo
I did not see much variation at my site, but those reports can be very easily modified.
So after you replace the three scripts in the download, the barometric pressure has high-low  columns.

Wim

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #15 on: June 09, 2014, 12:11:31 PM »
I understand it's in beta. No complaints from me. I'm glad that you understand programming more than I do...

I'm now getting Hi / Lo Barometric Pressure, but NOT getting monthly High / Mean / Low at the bottom of the page.

Thank you.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #16 on: June 09, 2014, 02:00:20 PM »
I understand it's in beta. No complaints from me. I'm glad that you understand programming more than I do...

I'm now getting Hi / Lo Barometric Pressure, but NOT getting monthly High / Mean / Low at the bottom of the page.

Thank you.
When I check the url you gave a few post ago
http://wx.ctreno.com/wsreports/startReports.php?lang=en
I see, for Barometric pressure, hree total lines at the bottom with High / Mean /Low for every month.

Please post an url with the incomplete data.

Wim

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #17 on: June 09, 2014, 10:15:20 PM »
My mistake. I was expecting to see 2 columns per month along the bottom, but after looking :shock: at it again, the information is all there.

Sorry & thank you.
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #18 on: June 11, 2014, 07:17:07 AM »
There's an "Error: No data..." today on both of our sites.  :?:
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #19 on: June 11, 2014, 08:08:41 AM »
There's an "Error: No data..." today on both of our sites.  :?:
Yes, the response times on retieving the CSV on our WU-dashboard page are also extremely long. The same with the WU history scripts from  Jim McMurry and Radomir Luza.

So the only solution is to set the CURL time longer.
It is now 10 seconds. I tried 60 seconds, which seems adequate.
Change the wsreports/wsReportsFunctions.php script.
The line to change is 217 from
Code: [Select]
curl_setopt ($ch, CURLOPT_TIMEOUT, 10); to
Code: [Select]
curl_setopt ($ch, CURLOPT_TIMEOUT, 60);That should take care of for now.
Wim


Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #20 on: June 11, 2014, 09:36:28 AM »
Yep, that fixed the error.

Interestingly, yesterday's data is appearing now (6:30am), where it usually doesn't appear until around noon. I wonder if that script change added that additional benefit?
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline wvdkuil

  • Wim van der kuil
  • Forecaster
  • *****
  • Posts: 1986
    • My PWS at Leuven Belgium Europe
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #21 on: June 11, 2014, 09:40:54 AM »
Yep, that fixed the error.

Interestingly, yesterday's data is appearing now (6:30am), where it usually doesn't appear until around noon. I wonder if that script change added that additional benefit?
Not so. The script is started and it reads the data. It reads the data a second time after 12:00.  If yesterdays data is there now already, it is maybe because you were reading it slightly later than the other days.
Or most likely, because WU is very busy updating / upgrading their systems.

Wim

Offline mjc775

  • Senior Member
  • **
  • Posts: 58
    • Reno NW Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #22 on: June 12, 2014, 12:58:52 AM »
Today's data was again displayed early - before 6am. I like that WU is making the data available earlier.

I noticed an minor error in the Color key. Currently the numbers overlap. For example in the temperature color key it shows "50-60", "60-70", "70-80", etc. It should show "51-60", "61-70", "71-80", etc. The overlap in numbers is present for all weather values. I searched the script files for where those numbers are generated, but couldn't find it.
« Last Edit: June 12, 2014, 09:20:22 AM by mjc775 »
I had a Heathkit anemometer and barometer in the 1980s.  After a hiatus, in 2014 I purchased a cabled Davis Vantage Pro2.
WeatherLink IP, MeteoBridge, Meteoplug Cloud.
WU: KNVRENO97, CWOP: N7FEQ, MESOWEST/MADIS ID: AU849
Reno NW Weather

Offline txweather.org

  • Forecaster
  • *****
  • Posts: 1597
    • Texas Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #23 on: June 12, 2014, 11:40:23 AM »
@mjc775
@xcom
The download location is www.weerstation-leuven.be/wsreports_beta.zip
There is a more-or-less English readme after you unzip the download.

You should first test the install with the stand-alone version with my weather-station settings.
So first test: yourwebsite.xx/wsreports/startReports.php  (if you put the wsreports folder in the root)
That should work always.

Than step 2: adapt the settings and use your version of the start script (Saratoga / stand-alone / Leuven)
Please be aware that the setting for the start-date for WU is dd-mm-yyyy day - month -year.

===

It will be over 30 degrees C this Whitsunday/monday weekend here in Leuven-Belgium, so I probably will not be to often close to a computer.

Succes, Wim

Wim,

Thanks for the post.
I am out on vacation. Once I get back I will try it out.

Thanks!

----
Davis Vantage Pro2 Plus +FARS|Meteobridge Nano SD|Meteohub|Meteobridge MR-3020|WU KTXSPRIN75/PWS JRARGWX75/CWOP EW2972/WBB TXWDVUE75/Blitzortung ID: 1142|AWEKAS: 12095
Donations are welcome: https://paypal.me/ffuentesb

Offline txweather.org

  • Forecaster
  • *****
  • Posts: 1597
    • Texas Weather
Re: Generate NOAA reports using Meteobridge uploaded data?
« Reply #24 on: June 15, 2014, 11:39:50 PM »
Ok I got it installed on my test site.
But on my header my banner seems to show incorrect when I go to the page.
It shows ok on all of my other pages but with this one the size is not kicking in.
IE:  <a href="index.php" title="Browse to homepage"><img src="banner.jpg" title="www.txweather.org" height="108" hspace="0" vspace=
"3" border="0"</img><?php echo langtransstr($SITE['organ']); ?></a>

Any ideas?

Regards.

----
Davis Vantage Pro2 Plus +FARS|Meteobridge Nano SD|Meteohub|Meteobridge MR-3020|WU KTXSPRIN75/PWS JRARGWX75/CWOP EW2972/WBB TXWDVUE75/Blitzortung ID: 1142|AWEKAS: 12095
Donations are welcome: https://paypal.me/ffuentesb