Topic: Weather34 Aurora MKII csv file conversions for 2020 2019 chart years

[new2ark]

can I get the following CSV files converted?  i renamed to .txt for attachment

[Gyvate]

can I get the following CSV files converted?  i renamed to .txt for attachment

your attachments are not csv files, i.e. text files with values separated by commas
when I try to open them it's just some gibberish.
so nothing to be converted -
what do you want the csv files get converted into ?

[DGP]

I saw an available update for the Aurora template so I downloaded it. I installed the update and it went fine. But I don't see what is different. Do I now have to install the entire MKII template? And is that update for that? But what then happens to history? Is that the conversion csv file? How should I deal with that. I don't think I get it completely, I think I missed something, I didn't update an entire template before.

[Gyvate]

Normally, you just download the update zip file, make a backup of your existing version (that's the button under the update button which comes after you choose update from the setup) and then start the update. Some changes are in the detail and you may not notice them, or you don't have a constellation where they occur. Sometimes it's a minor change in the look of certain tiles (display sections). You may not see directly.

You will have to restart (with update active = get newest version in the admin section) your Meteobridge as the API, the interface by which the template receives data from Meteobridge, has changed - otherwise the template will not find data or get wrong data. Reboot has to be (at least once) after 22-Jan-2021. Then all should work. As there may be additional fields added to the monthly and  yearly .csv files, the update may convert your csv files for the current month and earlier months based on the daily csv files (meaning add the missing and now needed values). But I don't know if there is a conversion in the update and what it all does.

I could see in my case that with Aurora in the month and year csv (they have an identical structure, year = month1 + ... + month12) there were 5 new entries at the end of the record which hadn't been there before. (info about wind bearing, UV index, solar radiation and one more I don't know). They may have been added already earlier, no idea. Until now I was still using my "old" version 8.34.

Somehow the update didn't work and messed up the display of the template. So I deleted everything (except the backup  which also contains the history files - also, I have a separate file based (copy) backup of the ./weather34charts/ directory - ) and installed a complete new Aurora build 31-Jan-2021 - and then copied my csv files back.

Now (like this I probably missed the conversion if there was one) I had to manually add the daily entries in the year csv for 2021.
That was fast done - collect the daily max UV and Solar and bearing - the other new data are not shown in the year chart and January was already over, so it's now only February as current month shown => no need to update the January.csv.
I still just did it and copied the result into the 2021.csv as the have an identical structure.

As I also have the full year 2020 I also had to add these (5) fields to the 2020.csv file. Now everything is shown nicely. For 2019 the charts still work the old way (I have also 2019.csv history).

The link in my signature still points to the 8.34 pre-Aurora version. Aurora is still on my local test server only but will be migrated soon.

[DGP]

Thanks. I just installed the whole template and updated it. I assumed I wouldn't have to. Simply updating the old template is apparently not possible. I did what you said. Copied the history from the ../Weather34charts/ .. and put it in the directory of the new folder / template. I assume that the charts in the template should have some time to fill up? How long does that take?

[Gyvate]

Thanks. I just installed the whole template and updated it. I assumed I wouldn't have to. Simply updating the old template is apparently not possible. I did what you said. Copied the history from the ../Weather34charts/ .. and put it in the directory of the new folder / template. I assume that the charts in the template should have some time to fill up? How long does that take?

You shouldn't miss anything - normally - unless you had your downtime over overnight.
The monthly and yearly charts are updated only once a day - in the new setup at 00:15 (that's why you have to setup that http request for 00:15 with weather34-chart-end-of-day.php).
The daily values for this month and year are in ./weather34charts/2021/February.csv and in ./weather34charts/2021.csv.
If your end-of-day job is properly set up, the next update should happen at 00:15 tomorrow morning. If you missed one day, you can build the record manually and add it.
See below.
Can be easily opened and edited in Windows Notepad/Editor - of course also in Notepad++

The structure of the csv file is:
1: Day Month - the first entry/record is a copy of the last entry of the previous month
2: T out high
3: T out low
4: dew point high
5: dew point low
6: rain total
7: max gust m/s
8: max average wind m/s
9: average temp out
10: sea level pressure high
11: sea level pressure low
---------------------------------- since Aurora/9.34
12:uv index max
13:solar radiation max (W/m2)
14: ?? contains always "--" (without "")
15: ?? cloud base ?? - haven't seen yet where it's used in the monthly/yearly charts
16: wind bearing average
a typical day record in the month and year file would then look like:
31 Mar,10.8,-0.5,-1.1,-6.0,0.0,1.6,7.1,4.8,1025.0,1023.0,7.4,792.5,--,66.0,171.0
(of course in the March.csv fie  )

The month and year csv contain the same record - don't forget the 2021.csv 

[weather34]

Thanks. I just installed the whole template and updated it. I assumed I wouldn't have to. Simply updating the old template is apparently not possible. I did what you said. Copied the history from the ../Weather34charts/ .. and put it in the directory of the new folder / template. I assume that the charts in the template should have some time to fill up? How long does that take?

if you need help feel free to drop an email .

@gyvate

Code: [Select]

        ######################################
# CREATED FOR Aurora MKII version        
# https://weather34.com/homeweatherstation/index.html
#      Creates and adds month and year  csv data 
# Updated Release: Revised January 2021
#                                                                                                
#   https://www.weather34.com                                                                    
######################################
//0 (date-month-0)),
//1 MAX(yesterday Temp-1-$meteobridgeapi[82])
//2 MIN(yesterday Temp-2-$meteobridgeapi[84])
//3 MAX(yesterday Dewpoint-3-$meteobridgeapi[52])
//4 MIN(yesterday Dewpoint-4-$meteobridgeapi[120])
// 5 TOTAL(yesterday Rain total-5-$meteobridgeapi[100])
// 6 MAX(yesterday Windspeed-6-$meteobridgeapi[94])
// 7 AVG(yesterday WindSpeed-7-$meteobridgeapi[123])
// 8 not currently used $meteobridgeapi[--] * reserved
// 9 MAX(yesterday Barometer-9-$meteobridgeapi[135])
// 10 MIN(yesterday Barometer-10-$meteobridgeapi[137])
//11 Max( yesterday UV-11-$meteobridgeapi[114])
//12 Max( yesterday SOLAR-12-$meteobridgeapi[107])
//13 Purple Air(13-$aqiweather["aqi24h"]) *if yes
//14 AIRQ 24H(14-$meteobridgeapi[184]) *Davis Airlink
//15 yesterday Wind DIRECTION average (15-$meteobridgeapi[171])



[DGP]

You shouldn't miss anything - normally - unless you had your downtime over overnight.
The monthly and yearly charts are updated only once a day - in the new setup at 00:15 (that's why you have to setup that http request for 00:15 with weather34-chart-end-of-day.php).
The daily values for this month and year are in ./weather34charts/2021/February.csv and in ./weather34charts/2021.csv.
If your end-of-day job is properly set up, the next update should happen at 00:15 tomorrow morning. If you missed one day, you can build the record manually and add it.
........
The month and year csv contain the same record - don't forget the 2021.csv 

Thanks again. I did some puzzling and adjusted files. For some reason, no record had been created last night. That's for the 4th at 00:15 today. But that was maybe because the record for the day before (3/2) was also missing while the 2nd February had a second entry. I saw for the first time that this occurs more often in the files ... Maybe for that reason the 'end of the day' in the new template is executed at 00:15 and no longer at 11:59 pm. whether a record will be created for this day tonight. My Meteobride looks good ....

[DGP]

30 Jan,2.4,-4.0,-2.0,-5.1,0.0,4.9,1.1,-0.3,1006.1,994.1,0.6,352.0,,97.4
31 Jan,1.1,-0.3,0.2,-2.8,1.1,4.5,0.6,1.0,1002.2,991.5,0.0,63.0,,113.3
2 Feb,8.6,0.2,8.2,-0.8,0.7,4.5,0.7,8.0,1003.3,996.1,0.0,77.0,,103.8
2 Feb,9.5,7.2,8.5,6.3,24.4,4.9,0.5,0.7,1002.7,993.2,0.0,113.0,,43.8

@Gyvate
@weather34
I see there is no input on // 14 which is Davis AirLink, if I understand correctly. I did install it ... Why is that input missing?
And also this example because you see that a date was doubled and February 1 skipped. That's before the transition to MKII, where of course I don't have any examples yet. I changed the date to 1/2 with Atom. Quick and dirty but ok for now as I assume 11:59 pm was the 'problem'.

[weather34]

duplicate date entries usually indicates two instances of end of day running in meteobridge services .please post a screenshot of your services screen from meteobridge.

[Gyvate]

The double date and skipping happens - since long. No idea why exactly. And no double instances active,
Even if on Meteobridge you put "every full 10 minutes", it sometimes became 10:09, 10:49 etc. in the day file and that may have caused the double dates - the sensor values are from the proper date though.
Maybe with the delayed creating of the day record now at 00:15 in the month and year file this issue is settled - to be observed.
In my new February.csv no double dates so far. In my January.csv the 30 Jan was doubled, i.e. no 31 Jan entry (or correctly said, there was a 31 Jan entry, but it carried the date 30 Jan).

In my case they are not really duplicate records, only the subsequent day carries the same date as the record before.
Like in the example of DGP - in such cases I would have changed retroactively (manually) the first entry of 2 Feb into 1 Feb.

Remedy (for the past values) - just edit the file with Notepad - correct the double dates (overwrite them) - otherwise only one of the dates will show on the chart.

For the Davis AirLink question, Brian will have to provide an answer - at least not me.

[weather34]

DGP are we talking end of day or current rolling day. please just follow the setup guide you should only have three simple services running for this template get them correctly setup the need for editing will dissapear unless you have erroneous sensor data generated by meteobridge which can occur when sensor i.e wind is not being sent from the ISS due to failure or battery level.

https://weather34.com/homeweatherstation/weather34-standalone.html

lets resolve this then move on to  resolve airlink which basically needs to done in meteobridge mapping itself

[DGP]

@Gyvate
Thanks that's exactly what I did. But now I also get it.

@ weather34
Certainly I turned on the 3 services. Or actually 2 transferred to the new template in the meteobrigde. The last one performed a new one at 12:15 am. The old one removed. I have checked again if it is correct. The old template is offline. Tonight I will check again if the meteobridge is doing the job.
That AirLink remains. But don't be in a hurry. Will be fine.
In MB I have added the AirLink in the tab Weather Station/Station 1 more than a month ago.

 [ You are not allowed to view attachments ]

[DGP]

@Gyvate
@weather34

It's final. I've lost my mind. A graduate madman. My apologies to both of you. I checked 14 times if all the http requests were entered correctly in the MB. Among other things, the entry in the service tab of the MB must be:
https: //yourdomain/yourfolder/weather34-chart-end-of-day.php
Then it turned out that February 5th in the chart was not created after 00:15 today. I eventually discovered that in the MB it said:
https: //yourdomain/yourfolder/weather34-charts-end-of-day.php
I am ashamed.
(I wonder if the 5th will turn up with the 6th when the 'job' runs tomorrow)

[weather34]

dont worry it happens ..in future join in over on the meteobridge forum for a quicker answer most issues can be resolved unfortunately i dont frequent here much these days .
https://forum.meteohub.de/viewtopic.php?t=15243&start=510

[DGP]

That's OK thanks for the tip. But useful if I still get an idea of how I get that AirLink reported in the .csv file. The template itself states that the AirLink is visible and online.

[weather34]

good afternoon

just run the simple update for 6th February it will show month year data for Davis Airlink if Davis Airlink option is set to yes

* Adds Davis airlink air quality to month/year charts (4:15pm).
* Webcam popup improved scaling size (4:15pm).

https://weather34.com/homeweatherstation/

should resemble something like which is dummy data but should give you an idea of what the outcome should be

 [ You are not allowed to view attachments ]

[DGP]

 [ You are not allowed to view attachments ]

If I click on the AirLink in the main screen it gives datachaxrt on the times time intervals today.But in the. 2021 chart I have a time info bock instead of the Air... Do I always have to reboot the MB after an update. I assumed it no to be necessary.

[weather34]

update template only from the download on home page no need to update or reboot meteobridge. your site is not showing update for 6th February it shows 31st January release. are you familiar with the simple template update method

https://weather34.com/homeweatherstation/weather34-update-guide.html

[DGP]

Strange. I had done well. Downloaded and installed the update .zip again. Message was that update was successful. But nothing changed in the template. I've done this 3 times. Meanwhile viewed the zip files and zipped again. Discarded the file on the server and uploaded it again. It worked. But the version is still on January 31st. I don't know what was wrong. 

[weather34]

Strange. I had done well. Downloaded and installed the update .zip again. Message was that update was successful. But nothing changed in the template. I've done this 3 times. Meanwhile viewed the zip files and zipped again. Discarded the file on the server and uploaded it again. It worked. But the version is still on January 31st. I don't know what was wrong. 

its a very simple process and non technical and its showing its updated dont read too much into it there is very little technical stuff to brainstorm .
probably your browser cache the html file for update info popup..

here is screenshot of your Davis airlink data in Year charts and icon popup I just took so your good to go, have fun...Brian..


 [ You are not allowed to view attachments ]


 [ You are not allowed to view attachments ]