Author Topic: Conversion of data to Csv or Weatherlink format for UK Met Office upload  (Read 753 times)

0 Members and 1 Guest are viewing this topic.

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Im a newbie.
I have a Youshiko YC9471 and upload weather data to Weather Underground and Weathercloud web sites.
Can anyone advise if its possible to convert this data, or the weather station data, to csv format so i can upload it to the UK Met Office Weather Observationd Website (WOW)?Alternatively, WeatherLink format?
If it is can anyone point me to instructions how to do this?
Thankyou

Offline Mattk

  • Forecaster
  • *****
  • Posts: 2381
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #1 on: January 13, 2025, 04:49:00 AM »
WOW is simply an ASCII format with details at https://wow.metoffice.gov.uk/support/dataformats#dataFileUpload

WeatherLink to WOW format I have done, just a case of following the WOW import format above

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #2 on: January 13, 2025, 05:10:11 AM »
Thankyou very much.
Much appreciated
Kind regards
Steve

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #3 on: January 13, 2025, 08:21:17 AM »
Actually just seen Weathercloud creates a monthly CSV format file so perhaps this can just be used to upload my weather station data to the UK Met office.
Would need to add 3 columns though - Site name,ID, Authentication key to it.
Ive sent it to Met Office to ask if this would suffice.

I read the link supplied above and could not find other software formatting Weatherlink with my type of weather station. I'm afraid I have very little knowledge of this type of thing.
Steve

Offline R.Sidetrack

  • Senior Contributor
  • ****
  • Posts: 211
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #4 on: January 13, 2025, 09:48:32 AM »
Spreadsheet software is often used to manipulate .csv files.  Examples of such software include Microsoft Office / Excel, and LibreOffice: https://www.libreoffice.org/download/download-libreoffice/

Import the .csv file, add columns, move them around, populate cells with text, etc. Typically, the default File Save format is NOT .csv, so when you want to output the finished file as .csv format, use something like "Save as" and choose .csv format. Terminology varies with the software, but the concepts are similar.
Graham

--- Variety of Ecowitt devices; CumulusMX on Raspberry Pi4 ---

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #5 on: January 13, 2025, 09:53:22 AM »
Thankyou very much !
That's very much appreciated
Kind regards
Steve

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #6 on: January 16, 2025, 10:00:56 AM »
Well strugging to bulk upload my Weathcloud export weather data into an acceptable UK Met office WoW csv format
Easy to do manually but the format of the csv has to be exact.Contacted Met office who advised
exact same columns need to be provided as per their template.
Cannot find any other software that will take Youshiko data and put it into exact csv format.
Might have to give up on this one.
Dont fancy fiddling around for ages moving data around.

Offline Mattk

  • Forecaster
  • *****
  • Posts: 2381
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #7 on: January 16, 2025, 04:37:09 PM »
So what's this Youshiko data format look like, is it a consistent format or variable formats depending on the station?

Same with WU and WeatherCloud formats, what do they look like and are these consistent?

I have conversion from WeatherLink.txt (which is just the .csv) to WoW relative to station specifics as it's not only specifically the sensor data format/units but things like Local Date/Time conversion to UTC+0 etc. The specific are in the detail.

Without knowing the complexities of the format to be converted it's hard to determine the difficultly in modifying existing code in a simple fashion. This conversion I use is written in F77 and part of a larger Weather module but could be re-packaged into a WoW stand-alone exe. The many variations with different formats and within formats is the issue.

But lets have a look at the format/s before thinking this is possible     

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #8 on: January 17, 2025, 04:30:41 AM »
The data from Youshiko is only extractable into tables and graphs on the Weather Underground or Weathercloud websites from which, as far as I can see, is only extractable into a set format csv file. Cannot see any way to extract into Weatherlink format.
One question: Can data in 1 excel/csv cell be transferred automatically into separate cells?
NOTE THIS POST WONT LET ME ATTACH CSV OR XLSX FILES.So I cannot attach my extracted data or Met Office files
Trouble is all the data goes into 1 csv/xlsx cell ( cell as in like on an excel sheet cell).
The UK Met office supply an example csv and data MUST be supplied in this EXACT forrmat.
Im afraid I dont have your level of knowledge regarding software transfer.I dont think I can extract in txt format. None of the software referenced on Met Office website seem to be compatible with Youshiko ie its not mentioned.
I think supplying a csv file would be best option but somehow this extract needs aligning with met office example.
If I can extract data from 1 cell into several columns on an excel sheet this might help me.

Offline box

  • Forecaster
  • *****
  • Posts: 394
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #9 on: January 17, 2025, 05:19:39 AM »
Im a newbie.
I have a Youshiko YC9471 and upload weather data to Weather Underground and Weathercloud web sites.
Can anyone advise if its possible to convert this data, or the weather station data, to csv format so i can upload it to the UK Met Office Weather Observationd Website (WOW)?Alternatively, WeatherLink format?
If it is can anyone point me to instructions how to do this?
Thankyou
an alternative is to use another data logger like cumulus mx that can upload to WOW directly

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #10 on: January 17, 2025, 05:31:44 AM »
Thanks for this.
Yes i have just see cumulus mx will take data from weathercloud.
Cumulus mx runs on win11.
Ill look into this

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #11 on: January 17, 2025, 08:02:49 AM »
Set up Cumulus MX.
I guess data would go
Weather Station >> Weather Underground >> Cumulus MX >> Met Office WoW.
Need to first work out how to use Wunderground Protocol to link WU to Cumulus MX

Offline box

  • Forecaster
  • *****
  • Posts: 394
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #12 on: January 17, 2025, 08:42:39 AM »
Set up Cumulus MX.
I guess data would go
Weather Station >> Weather Underground >> Cumulus MX >> Met Office WoW.
Need to first work out how to use Wunderground Protocol to link WU to Cumulus MX
I may have lead you astray as after a bit of research I don't think cnx can read data from youshiko devices unless it can output in easy weather format

So my brilliant idea may be a red herring

Offline Stevebarker12

  • Member
  • *
  • Posts: 16
Re: Conversion of data to Csv or Weatherlink format for UK Met Office upload
« Reply #13 on: January 17, 2025, 08:53:40 AM »
Thats ok ( i think ! )? because my weather station uploads to weather underground.I know cumulus mx can link with weatherunderground and met office wow.

 

anything