Topic: Help with creating "warmest since" or "coolest since" data

[WX1N]

Good afternoon all,

My name is Rob, from East Unity, N.H.  My weather website is http://www.eastunitynhweather.com.  I am using a Davis weather station, running Cumulus software, and using Saratoga Weather Templates for the website.

I would like to have certain data automatically calculated and either stored in a local file, or displayed in Cumulus, or displayed on my website - I don't really have a preference, if one of these is more do-able than the others.

What I am looking for is a way to automatically determine that today is the warmest or coolest day since (the last time it got this warm or cold).  For example, today we reached 47.5 degrees.  This is the warmest we have been since November 25th of last year. In order to determine this, I had to manual search my dayfile.  I'd like to have this data calculated automatically.

Any help or pointers would be greatly appreciated.  Thank you for taking the time to read this.  If this is better posted in a different forum (here or elsewhere), please let me know.

Rob

[DW7240]

Hi Rob,

Have you tried Wildwoodnaturist.com weather site ? you will find on there downloads for Cumulus setup, check the link below.  The "Detail and Summary Files" option would be just what your looking for, try out the example.  Also there is a file called "Daily Records" again, check out the example.  You could just download both files, then display both on your site like I have, again check it out and check out my site to see the finished result.

http://weather.wildwoodnaturist.com/downloadsCVW.php

Best Regards....

Nick. DW7240.com.

[WX1N]

Good morning Nick,

Thanks for the reply.  I checked out the example, but it just shows charts.  How would I make those pages do what I want done (specifically state that today is the warmest/coolest since <whatever date this temperature was last reached>)?

Rob

[Jáchym]

Hmm I guess I have something similar to that on my page and it would not be a problem to program it, but I am not sure how your data is structured. In other words, I dont use any software to do the stuff, I have my own MySQL db with all the data and work with that. If you are using Cumulus then I think it probably isnt that easy to do, unless you upload and merge all your log files into a db like me and then work with that - in that case making a PHP that would do the trick is relatively simple, but you would have to somehow aggregate the data and then make them available to php.

[WX1N]

The data is uploaded to the website and available locally in monthly/yearly tab-delineated text files, NOAA-style format (for example, http://www.eastunitynhweather.com/Reports/NOAAMO0113.txt).  I could probably also configure Cumulus to upload the day file, which would be a comma-delineated text format, with all station data.  Would either of these work?

[Jáchym]

Hi,
I am not sure, depends on how exactly you want to calculate "warmest" / "coldest" day, whether you mean the overall average temperature, or for example that particular day´s max/min. I personally never worked with TXT files, I always uploaded the data into a MySQL table. What you could try is to export cumulus data into a CSV or to excel and then create a CSV file. What you basically need is a table where each row contains date/time and the particular set of data - again it depends whether you want just daily values or all the values at all the measured intervals. Then once you have such a table, import it to MySQL and then write a script that simply goes through the DB and checks whatever you need. This is exactly how the section "weather station" on my page works, in the backend I have a MySQL DB with all my data, which is also regularly automatically updated every 5 mins by meteobridge. I dont use any other software, Meteobridge simply adds raw data as rows to that db and then the scripts do all the rest.

[WX1N]

Ok, thanks for the help.  I'm just looking to compare the current day's min/max to previous day's min/max temperatures.  I've never worked with MySQL before and I'm not sure I want this badly enough to learn how.  I think at this point I'll either try to learn enough PHP to get it to search the CSV file, or give up until someone else tries to undertake this.

[Jáchym]

Hi,
well in fact this is quite simple and I think it is much easier to import CSV to MySQL than makeing PHP work directly with CSV. I would guess you should have access to something like phpMyAdmin, from where importing a CSV and thus creating a MySQL basically requires clicking the button "Import"...

[WX1N]

I think I've seen phpMyAdmin in my hosting control panel, I'll have to take a look and see if I can figure it out.  Thanks for pointing me in the right direction!

[Jáchym]

Or if you just need to get that one single peace of information, I think it could be easier for you to use for example Excel and write a macro, which would do it. Of course you then wouldnt have it directly online, but you wouldnt have to manually look for it and it would give you the result straight away.

[Jáchym]

In fact, I already do have something like this, if you want to have a look, go to my page, change to english version, go to the "weather station" section in the menu. In the "console" that opens, find the icon "history icon" on the right (looks like a clock, labeled "past weather") - click that and you should get a table of past weather. But also underneath that you will see tabs, try clicking for example the "temperature" one and I think this is sort of very similar to what you want.

[WX1N]

Nice website!  Yes, that is very similar to what I'm looking for.  The only difference for what I want to do, is compare the current temperature instead of specific temperatures like you have.
I like the Excel idea, and I don't mind having the data locally instead of on the server, I can just incorporate it into the page before it uploads.  But if it's in an Excel file, the Excel file has to be manually opened for the formula to calculate, or for a macro to run, right?

[Jáchym]

Well I am not a programmer, so someone probably would be able to do it in a simpler way. I will do some research, but I think I should be able to figure out how to do it, if you had a csv file on the server, that would contain all the daily data - in other words for example like this:

Sep 1	15
Sep 2	20
Sep 3	55

Then what I would do is first load all the values into a PHP associative array - so date and corresponding temp, then get the current value and then loop backwards, starting from yesterday and going back, checking whether the value is smaller or larger.

Just one more thing though, if I understand it correctly, one of the values will always be sort of "yesterday" right? Because if you think about it, yesterday had to be either colder or warmer, so one of those will be like "last time it was this cold was yesterday" or "last time it was this warm was yesterday" - one of those would always be true.

[WX1N]

Exactly right - one  will always be yesterday.  That method sounds right.  If any programmers on here want to jump in and say how to do it, that would be a plus .  Otherwise, I'll look through some PHP documentation and existing websites and see if I can figure it out.  It shouldn't be too complicated.
Thanks so much for the help!

[Jáchym]

OK, here is what we could do... if you would be able to create a CSV file, in the format I mentioned - i.e. date in first column, temp second etc. for all the days you have in your db, then if I had this file, I could try to write a PHP code for you to do it. I think I know most of what I need and there are only few things I would have to check, but it shouldnt be a problem to find a solution on stackoverflow and similar websites.

Let me know if that was possible for you, if you would be able to provide such a file. This could then be uploaded to your server along with the PHP script and all you would then have to do is always upload a new version of the file and rewrite the original, so that it contains new data. Only thing this would require is you to manually enter new entry for a new day and just save and upload.

[WX1N]

If it could read my existing CSV file, which is automatically created by Cumulus, then I wouldn't have to do anything manually - I'd just have to instruct Cumulus to upload the file every day.
This file has the date in the first column, the minimum temperature for that date in the fourth column, and the maximum temperature for that date in the sixth column.
I'll try to attach that file to this post, in case you think that format will be workable.

[Jáchym]

Hi,
just looking at it quickly I think this should be ok. I am at work right now, but I will look at it later today and try to create the PHP script that will do it. Just maybe to clarify, so all you want is for the script to look for the coolest and warmest since.... if there is anything else you would like, just let me know, I could try implementing it as well. I also cannot tell you how long exactly it will take me, as there are some things I will have to check, hopefully by tomorrow the latest.

[Jáchym]

Sorry one last thing...
where is actually the "today´s" temperature?
The file you sent me only contains data until yesterday, so I am assuming the new entry is probably added at midnight every day. So you want to compare the last entry (i.e. yestrday) with previous data? or if you want today - the average of todays data up until now, those are actually not specified in the file if you see what I mean. Or I could create a form with input text, where you would enter todays data and that would be used for the calculation.

[WX1N]

Oh yeah, that would be important...  I think the best way to go is with the current day's high and low so far, so the result would be regarding "today", instead of the actual moment in time.  I know those figures are already on the server somewhere, but I'm not sure the best way to access them, so I'll just have Cumulus upload an additional file and name it "todayhighlow.csv", which will simply contain the current day's minimum so far, followed by a comma, followed by the current day's maximum so far.  Does that work?

[Jáchym]

Hi,
I am a bit confused right now, but they do not have to be in one file. In other words, I would import the CSV to get past data and then need either a new csv, or ideally just a text file, which would contain the average temperature for today so far - this would probably be updated regularly at some time interval by cumulus, so that it reflects current conditions. I would then do this:
1. import text file - get current average for today
2. import CSV - compare this number with average temperature of yesterday, day before yesterday etc. etc.
3. find first instance in the past where t(today)<t(day X) - here X would be the first day in past when it was warmer on average, and similarly t(today)>t(day Y) - which would be the day where it was colder than today
4. result would be two dates, out of which one would obviously be yesterday

[WX1N]

I was thinking to use the maximum and minimums from the current day and previous days, to compare extremes rather than averages, so:
max(today)>max(day "Y")=Today is the warmest it has been since (day "Y")
min(today)<min(day "Z")=Today is the coldest it has been since (day "Z")

[WX1N]

And yes, the current day's max and min are in the realtime.txt and realtime.xml files, however, I wasn't sure if they would be difficult to extract.  I can attach samples if you want to look at them.

[Jáchym]

Yes that would be ideal, I dont use any weather software so I have no clue what these text files look like.

With regards to the max/min, if I understand it correctly, it is basically the same thing I suggested, except you dont do it for the average, but you compare max for warmer than and min for colder than.

[WX1N]

Ok I'm attaching them to this post.

Yes, that's what I was trying to say.

[Jáchym]

The txt file seems usable, but you have to tell me which number is the max and which one is the min (there are several numbers for the different parameters and I dont know the order of them and which one is which)