Topic: Meteobridge archive data

[tomascot]

Hello all. How are you?

I'm looking for options to upload archive data to a custom server using FTP and I would like to know which options do I have. Currently we are using WeatherLink Software with the archive templates for 2 (downld02) and 8 days (downld08).

Can I generate files similar to this with any of the meteobridge products? I'm particularly interested in the meteobridge standard (installed on a tplink router)


The reason we want to upload the archive data is because the station is in a place with bad internet and sometimes it goes a couple of days without an internet connection.

Thank you!!

[Gyvate]

Not sure if your question was already answered otherwise ...
however, in order to get a satisfactory reply you need to be more specific
what archive data you need to/want to upload to your custom server ?
Which format is the data supposed to have ?
do you want to upload it using Meteobridge's FTP function ?
do you you need to upload your data once or regularly ?

[tomascot]

Hello Gyvate. Happy new year.

1- The format I'm looking for is something similar to the files generated by Weatherlink Sw when using the archive data options:

Code: [Select]

                  Temp     Hi    Low   Out    Dew  Wind  Wind   Wind    Hi    Hi   Wind   Heat    THW   THSW                Rain  Solar   Solar Hi Solar    Heat    Cool    In     In    In     In     In   In Air          Soil 1  Soil 2  Soil 3   Soil    Soil    Soil    Leaf  Wind  Wind    ISS   Arc.
  Date    Time     Out   Temp   Temp   Hum    Pt. Speed   Dir    Run Speed   Dir  Chill  Index  Index  Index   Bar    Rain  Rate   Rad.  Energy    Rad.     D-D     D-D    Temp   Hum    Dew   Heat    EMC Density     ET   Moist.  Moist.  Moist.  Temp 1  Temp 2  Temp 3  Wet 1  Samp   Tx   Recept  Int.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
 1/10/21   0:10   16.2   16.2   16.1    92   14.9   5.2     S   0.87  10.4     S   16.1   16.4   16.3    ---  29.811  0.00   0.0   ---      ---    ---     0.015   0.000   23.7    58   15.0   24.1  10.56  .0727     0.00    ---     ---     ---     ---     ---     ---    ---    233    1    100.0   10
2 - I would prefer the FTP function.

3- The interval is not that important, currently with weatherlink I'm uploading the archive data once an hour.

I'm thinking about creating my own Open WRT image to be able to get the archive data from the USD datalogger.

Thank you!!!

[Gyvate]

in Meteobridge you can create a CSV export of these Data with a custom export.
Look at the existing examples and modify (create a customized copy) accordingly.
They are in \\METEOBRIDGE\data\templates extension .exp

Then you can setup FTP in MB and create a FTP event in your desired interval

[tomascot]

I'm still waiting to the get the tp link router.

What I understand about meteobridge std is that it only uploads the current data, not the archive data.

The example I shared has multiple rows, I just added the first so you could see the format. But the donwld-02 file has data for the last 2 days and there's also another file called downld-08 that has data for the last 8 days. The number of rows varies depending on the archive interval set up on the davis console. I need that because I want to avoid gaps in data when the internet connection is lost.

is it possible to upload data from the last 2 days with meteobridge std?


Thank you!!!

[Gyvate]

not sure what you mean by "std"
you can set an event in MB that creates CSV export with a time stamp (export-report-name+time stamp).
this can be run every minute etc., also every hour, also every 2 days.
The result will be a file with several rows/lines each representing the chosen period.

As an example:
I have for backup, transfer etc. reasons an export running every hour providing minute values
for the data archived every minute a row will be generated => my hourly export contains 60 rows
at the end of the day at midnight, I run the same report, now over the whole day, => 1440 rows
I could of course choose other intervals for downloading.

As far as I know MB (Nano, Pro, RaspberryPi versions) can connect directly to the VP2 and collects and archives data realtime independent of the VP archive cycle. Either via USB connection or receiving the sensors from the ISS directly via RF.

And as for uploading data from the MB database (via a beforehand export), this depends on your event definition for the file transfer. The structure and content of your export you also define.
So I don't see what MB would not be able to do to satisfy your needs.

[tomascot]

Meteobridge std is the way some people call the version that runs from a router with meteobridge's firmware (a modified version of open wrt).

In their docs it says:

Meteobridge clients will not store gathered weather data locally. The most recent data to be transmitted to a weather network is available plus some min/max values. Beside that, no data will be stored locally.

I think I cannot explain myself. Thanks for your help though.

[Gyvate]

OK - understood - you (want to) go for the modified router solution.
This doesn't keep a local archive though. So no export from a local database possible.

What I would recommend is running MB on a RaspberryPi [RPi] (an 8 GB microSD - must  be industrial SLC type, but the enforcement can be officially circumvented by a procedure published by the developer due to SLC SD card shortage in COVID-19 times - is enough to hold data for a couple of years - i.e. you can use a normal SD card, but they are likely to fail after some time).
It is fast, provides all MB features whereas the router implementation does not. The routers have too weak a processor and very little internal memory. They are from the beginning MB era ...

Then you can export your CSV files as mentioned before from the local database which will have recorded all your station data on either a minute or hour based interval. The RPi will run 24/7, consume little power.

I have a MB Pro and a RPi4B installation. The MB Pro has already capacity and performance issues (due to the amount of sensors I have and the fact that I want to have a minute based data resolution) - the RPi4 is just fast, can accommodate my data from two stations for 6 years on a 16 GB microSD (industrial SLC) - it's in my opinion and for my use case the best solution.

But you are of course free in your choice of equipment.

Bottom line: with a MB Pro or MB on Raspi your requirements can be solved. With the router solution rather not - or I don't know how

[tomascot]

Thank you so much, Gyvate.

Currently there are lot of problems with importing electronics where I live, and there are routers on the domestic market, I'm gonna check if I can get a RPi. We don't need that much precision and we only need the rain sensor.

Thank you again.

[Gyvate]

Thank you so much, Gyvate.

Currently there are lot of problems with importing electronics where I live, and there are routers on the domestic market, I'm gonna check if I can get a RPi. We don't need that much precision and we only need the rain sensor.

Thank you again.

make sure it is a RPi3 or better a RPi4B - the other earlier versions are not supported - the RPi4-2GB version is sufficient (i.e. smallest RPi4B version)
buena suerte !