Author Topic: Parsing data from WeatherLink Live summary page?  (Read 400 times)

0 Members and 1 Guest are viewing this topic.

Offline meteoloco

  • Member
  • *
  • Posts: 2
Parsing data from WeatherLink Live summary page?
« on: November 21, 2021, 02:36:05 AM »
Hi!! I need help

Starting from the summary page of WeatherLink Live (https://www.weatherlink.com/embeddablePage/show/63d504233ca340e59b4c8cd44478f45b/summary),

How can I get data, using Php o Javascript (or both) to view them on a personal web page, without using API?

Thank you for your help!

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9297
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Parsing data from WeatherLink Live summary page?
« Reply #1 on: November 21, 2021, 11:20:35 AM »
The URL you cited does show a HTML page display -- the bad news is that the page was generated by JavaScript so scraping the page itself won't work.

Enabling the browser developer tools shows the JavaScript on the page makes two XHR queries that contain the actual data you are looking for in JSON format:

Data:  https://www.weatherlink.com/embeddablePage/summaryData/d5a47b7cb6ef4521b8dd8dfe828189a8
ExtraSensors: https://www.weatherlink.com/embeddablePage/summaryDataExtraSensors/d5a47b7cb6ef4521b8dd8dfe828189a8  (shows null for your setup)

Parsing the JSON data should be very easy as the data is already organized in machine-readable format.

What did you want to do with the data?  Save it? Show it in a different way?
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline meteoloco

  • Member
  • *
  • Posts: 2
Re: Parsing data from WeatherLink Live summary page?
« Reply #2 on: November 22, 2021, 01:15:55 AM »
The URL you cited does show a HTML page display -- the bad news is that the page was generated by JavaScript so scraping the page itself won't work.

Enabling the browser developer tools shows the JavaScript on the page makes two XHR queries that contain the actual data you are looking for in JSON format:

Data:  https://www.weatherlink.com/embeddablePage/summaryData/d5a47b7cb6ef4521b8dd8dfe828189a8
ExtraSensors: https://www.weatherlink.com/embeddablePage/summaryDataExtraSensors/d5a47b7cb6ef4521b8dd8dfe828189a8  (shows null for your setup)

Parsing the JSON data should be very easy as the data is already organized in machine-readable format.

What did you want to do with the data?  Save it? Show it in a different way?

Thank you for your answer.

What I need to do is creating e clientraw.txt  file, parsing json data, in order to show data in my own webpage.

I already use this kind of method with Weatherlink ip, starting from the xml file of weatherlink. The problem is that with Weatherlink Live there's no xml page.

Thank you for your help.


Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9297
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Parsing data from WeatherLink Live summary page?
« Reply #3 on: November 22, 2021, 10:32:02 AM »
What you want (WeatherLink JSON to Weather-Display clientraw.txt) is possible, but will be complicated.

The JSON data from WeatherLink.com appears to have the units you set to record data in your weather station.
The clientraw.txt file is always using units of C,kn/s,hPa,mm for temp,windspeed,pressure,rain so there would need to be the capability of units conversion for each data entity.  Also, there are data in clientraw.txt that do not exist in the JSON (like text description, icon number, etc.) so some provision to provide them (or use a null entry) would have to be done.

Accessing the JSON with PHP is simple.. you use the function json_decode() which can place the results in an array or in a class structure (like you use when using the XML decode function).

I'd suggest you start with what the data definitions are for each field in clientraw.txt and in a spreadsheet, annotate which ones appear in the WeatherLink JSON as available for your use. 

I did a similar thing for the WLCOM plugin to the Saratoga template set.  You might take a look at that and see what's involved.  It can create a 'realtime' file like the Cumulus realtime.txt file, so the same method could be done for a clientraw.txt file with a goodly amount of rewrite.  WLCOM plugin at https://saratoga-weather.org/wxtemplates/WLCOM-plugin.zip
The WD clientraw.txt field definitions can be seen at https://saratoga-weather.org/wdparser.php

Hope this helps...
« Last Edit: November 22, 2021, 10:34:21 AM by saratogaWX »
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

 

anything