Author Topic: weatherlink.com - import historical data files  (Read 946 times)

0 Members and 1 Guest are viewing this topic.

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
weatherlink.com - import historical data files
« on: August 14, 2019, 05:33:05 AM »
There's been no announcement from Davis, but looks to me like like the long-promised facility to import historical data from your old .wlk files has now been added (provided of course you have a Pro subscription, which is needed for all historical data). It's there as 'Import' on the Device menu.

It would be good to see these new features announced better. The 'What's new' button is still very generic and simply dating from the launch of 2.0. As it is at present, the only option is to graze over the whole feature-set on a daily/weekly basis to see what's changed. I'm just happening across these newer features almost by accident as I notice something that seems unfamiliar.
« Last Edit: August 14, 2019, 05:35:42 AM by johnd »
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 4551
    • KomokaWeather
Re: weatherlink.com - import historical data files
« Reply #1 on: August 14, 2019, 08:56:32 AM »
Thanks johnd,

I also scan the site regularly, but without a paid subscription that new feature hasn't shown up for me.

Enjoy,
Paul
Davis Vantage Pro 2 Plus 24-FARS Wireless
Envoy/USB DataLogger/Cumulus v.1.9.4   -   Vue Console/WiFiLogger/CumulusMX
Komoka, ON  Canada
www.komokaweather.ca   www.komokaweather.com   www.komokaweather.com/weather28/  
www.komokaweather.com/pws/index.php
Blitzortung Station #1076 www.lightningmaps.org/blitzortung7/america/index.php?bo_page=statistics&bo_show=network&lang=en&bo_station_id=969


Offline ssell9889

  • Member
  • *
  • Posts: 9
Re: weatherlink.com - import historical data files
« Reply #2 on: November 05, 2019, 12:40:33 PM »
Same issue
I try to uload and get an error..they tell me they fixed it and NADA....now they wont reply to emails....

Offline ssell9889

  • Member
  • *
  • Posts: 9
Re: weatherlink.com - import historical data files
« Reply #3 on: November 05, 2019, 12:41:13 PM »
Left out that import button is now there....but doesnt work and it takes days, months for them to get back to you

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #4 on: November 05, 2019, 01:33:34 PM »
Left out that import button is now there....but doesnt work and it takes days, months for them to get back to you

Same question as the other thread: Do you have a Pro subscription? If not then you obviously cannot expect to upload historical data.
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline ssell9889

  • Member
  • *
  • Posts: 9
Re: weatherlink.com - import historical data files
« Reply #5 on: November 05, 2019, 01:57:43 PM »
yes i have pro account....I have the button and I have uploaded the data then get an email saying the upload failed.

Offline GB

  • Member
  • *
  • Posts: 2
    • Swanston Weather
Re: weatherlink.com - import historical data files
« Reply #6 on: November 07, 2019, 05:01:27 PM »
I have the Pro subscription but cannot see this 'import' button anywhere. Which page is it on?

George

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #7 on: November 08, 2019, 03:48:41 AM »
Station menu (the spanner/wrench icon in the RH sidebar); then Device | Import.
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline GB

  • Member
  • *
  • Posts: 2
    • Swanston Weather
Re: weatherlink.com - import historical data files
« Reply #8 on: November 08, 2019, 06:44:09 AM »
Thanks for the reply John. Still no success however. I click on the wrench icon in top RH corner and get the 'device info' page but no sign of the elusive 'import' button.

Just to repeat I do have a Pro subscription, I've checked it through with Firefox, Chrome and Edge but nothing. Just wondering, does this require Weatherlink Live?

George

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #9 on: November 08, 2019, 08:15:49 AM »
By chance, I was looking at a WLL station when I wrote that last post. Must admit that I thought the upload was available for other logger-based stations too, but maybe I was wrong (or maybe eg it's been temporarily withdrawn for non-WLL stations).

One problem nowadays with weatherlink.com is that there are so many different combinations of station types, logger/upload devices, account types (Basic/Pro/Pro+ etc) and then the smartphone app is sometimes different from the browser app (eg repeaters can be configured for WLL on phones but not the browser) that it's becoming increasingly tricky to know if information true in one context necessarily applies to other contexts.

Let me try and check the definitive situation with Davis & apologies for any inadvertent misinformation if that proves to be the case.
« Last Edit: November 08, 2019, 08:24:37 AM by johnd »
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline dbrinza

  • North Sylmar Vantage 2 Pro PWS
  • Member
  • *
  • Posts: 1
  • David Brinza
    • Bulletin: North Sylmar
Re: weatherlink.com - import historical data files
« Reply #10 on: November 08, 2019, 08:00:33 PM »
I'm interested in loading my historic data into weatherlink.com as well.
I have a Pro account showing data since account activation on Oct 20, 2019, but have weather history back to 2005.
Looking forward to a solution to having access to all the data.
Thanks!!

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #11 on: November 09, 2019, 04:02:13 AM »
OK, some clarification from Davis: I'm afraid it looks like the ability to upload historic wlk files to weatherlink.com is indeed limited to Weatherlink Live stations**. I guess the argument is that those with USB or IP loggers can download their weatherlink.com into local Weatherlink for Windows via Web Download should they wish to do so and thereby maintain their full historical archive locally, whereas Weatherlink Live users don't have access to that same option. Apologies for any confusion. The option to upload wlk files for other (eg USB or IP) logger-based stations may be added at a later date but isn't a current development priority.

In other news: Wireless repeaters are now supported by WLL, but must be configured via the smartphone app - repeater configuration is not currently available via the Weatherlink browser UI.

** To clarify: If you now have a WLL station at weatherlink.com but previously had eg a USB or IP logger then older wlk files stored locally in Weatherlink for Windows can be uploaded to your WLL account at wl.com
« Last Edit: November 09, 2019, 06:29:03 AM by johnd »
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline Brientim

  • Senior Member
  • **
  • Posts: 78
weatherlink.com - import historical data files
« Reply #12 on: November 09, 2019, 06:02:28 PM »
Davis have managed this extremely poorly and continue to do so. They have locked themselves in to a bad implementation based on the device not user account -> station that still persists to this date with some improvements.

The problem scenario exists today with WLL and WeatherLink.com. If a WLL dies and needed replacing it will create a new device and they still havenít design a process to migrate data from one device to another. However they acknowledge the need to work on that but when is unknown.

Update I stand corrected, I notice looking at the app there is now the ability to replace a WLL
« Last Edit: November 09, 2019, 06:50:16 PM by Brientim »

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #13 on: November 10, 2019, 04:23:44 AM »
Davis have managed this extremely poorly and continue to do so. They have locked themselves in to a bad implementation based on the device not user account -> station that still persists to this date with some improvements.

I'm interested in what you think Davis could have done differently? Davis have aimed to create a single platform for all of their stations and upload devices, which I think on balance is a good idea. But data uploaded from WLL (and also, but in a different way again, EM stations) has many distinct characteristics from the traditional logger-based Davis stations and has to be handled differently at wl.com. So it's almost inevitable that different station types are likely to have different features at any given point in time.

The only mistake that Davis have arguably made was originally to allow multiple stations to share the same Pro status within a single account. This was never going to work as a business model - an account with eg 6 or 8 well-optioned EM stations is going to consume vastly more resources at wl.com compared to eg a hobbyist with a single Vue and eg IP logger. Resources on cloud platforms like AWS are relatively cheap these days but that's not the same as free and to a first approximation you get charged for how much you use. So having a per-station charging model is the only basic approach that makes sense AFAICS.

(Actually of course the charging structure is a bit more complicated than that with Davis choosing to operate a freemium model whereby current data is free for logger-based accounts, whereas historical data is the paid-for part. But all cell-based uploaders are automatically Pro with an all-in-one charge (ie data transmission + platform).)
« Last Edit: November 10, 2019, 04:26:14 AM by johnd »
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline Brientim

  • Senior Member
  • **
  • Posts: 78
weatherlink.com - import historical data files
« Reply #14 on: November 10, 2019, 01:17:09 PM »
Iím staying within the context of the thread and the technical design and implementation of WeatherLink service and not worried about their business model.

I liaised with Davis when they first rolled out the upgrade to WeatherLink.
1. The ported the original design which was based on a device ID to become an account_device.
2. They didnít allow user account management to delete a device.
3. This was a transformation and they had the chance to fully normalise the whole structure to allow the export/transform/load (ETK) to completely manage any variation.
4. A station is not based on a transmitting device but location and station
5. A device can fail, so it must be able to be replaced, this applies to all variations.

Davis acknowledges the issues with the upgrade to WeatherLink.com, especially the implementation of 4. and the problem was, a good problem to have, their devices were extremely reliable and only had limited failure so they never addressed this and went from 1 > 1 direct relationships to 1 > many but it was still tied directly to the device. This was easily avoidable by Account > station > device and therefore the device is interchangeable whilst maintaining the station history. In addition, it is only a single design of the ETL as any variation is managed by transformation
« Last Edit: November 10, 2019, 01:44:28 PM by Brientim »

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #15 on: November 10, 2019, 01:46:44 PM »
Not sure it's worth debating this in much detail because wl.com is what it is, but FWIW:

I liaised with Davis when they first rolled out the upgrade to WeatherLink.
1. The ported the original design which was based on a device ID to become an account_device.

Yes, a device is a single upload_device which is effectively a station. Not sure what else Davis could easily have done that would work across their whole range of station types including EM and WLL as well as traditional loggers. (Though no doubt other architectures are conceivable)

Quote
2. They didnít allow user account management to delete a device.

True. But admin accounts can delete devices, though these accounts aren't available to end-users.

Quote
3. This was a transformation and they had the chance to fully normalise the whole structure to allow the export/transform/load (ETK) to completely manage any variation.

It was a big project to move to wl.com v2 and some limits to the functionality available to end-users was necessary (and possibly prudent too). Again, admin accounts have pretty much full control over user accounts.

Quote
4. A station is not based on a transmitting device but location and station

See [1]. (I presume that by 'transmitting device' you mean an upload device?) Location is a valid search term for a station but it's not part of the station's identity.

Quote
5. A device can fail, so it must be able to be replaced, this applies to all variations.

As indeed it can be, but by an admin account.
« Last Edit: November 10, 2019, 01:49:01 PM by johnd »
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline Brientim

  • Senior Member
  • **
  • Posts: 78
Re: weatherlink.com - import historical data files
« Reply #16 on: November 10, 2019, 03:19:30 PM »
Totally agree it isnít worth debating but you asked why... and I manage this type of transformation within my day job on a very different scale, so I might come from a different perspective and I responded with my opinion observations.

Location in context is not the place but characteristics to describe position, albeit street, suburb city, geospatial location. So when a weather station is relocated it should not represent the same as it was before it was relocated.

They have implemented user WLL device replacement by end user. The reliance on Admin and support was reported as taking a considerable time which at present a potential loss of data.

I like Davis products and WeatherLink but there is improvement that reduce dependency of support and improves the end use over time. They can take the step now or defer and that is their choice but it would be easier now before this grows in complexity with more and more sensors and nodes being introduced.

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #17 on: November 10, 2019, 04:26:19 PM »
Don't quite understand the change of location point. Don't you just hit the blue 'Edit location' link on the Device Config page? Or do you mean something more than that?

On the speed of response of Davis support to eg delete/replace requests, we've not found that a particular problem. IME changes that are clearly requested (ie no ambiguity about the request or device in question) are usually actioned within one working day, or two at the most. Of course timezone differences wrt California do come into it too. It can be quicker to make a request via your local Davis distributor (if outside the US and you're not doing that already) - if they're on the ball, they will typically have a way to expedite the request.
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline Brientim

  • Senior Member
  • **
  • Posts: 78
Re: weatherlink.com - import historical data files
« Reply #18 on: November 10, 2019, 05:58:21 PM »
Don't quite understand the change of location point. Don't you just hit the blue 'Edit location' link on the Device Config page? Or do you mean something more than that?
Something different:
This topic is historical data. Therefore to simplify a station has certain characteristics to describe the conditions and location is one aspect.

If a user moves to a new house, and in this case, keeps the same physical equipment including a WLL, they can rename the station and change the physical location (Lon/Lat) and continue to upload to the same station. However, this effectively corrupts historical data unless the position at the point time is uploaded.

So location as a characteristic rather than station ID or Account ID, or Device ID is an extremely important data point in determining validity of historical data.


Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #19 on: November 12, 2019, 10:54:57 AM »
Ah I see, sorry. So you would presumably want to spawn a new station for the new location but retain the uploading device's ownership details, Pro status etc etc. Potentially that option will come in time - it sounds like a genuine requirement. Problem is that Davis wil I'm sure have a to-do list that's maybe 100 features and several man-years of development work long or more and it's just a question of where a given feature ranks on that list.
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline Brientim

  • Senior Member
  • **
  • Posts: 78
Re: weatherlink.com - import historical data files
« Reply #20 on: November 12, 2019, 11:55:52 AM »
Correct. I communicated some of the logic they should consider with future enhancements road map that they may consider. They acknowledged there is challenges with limited resources and one of my concerns would be in to reduce the dependency on support if a task can be managed by an end user. Given time they will only improve.

Offline johnd

  • Forecaster
  • *****
  • Posts: 3417
    • www.weatherstations.co.uk
Re: weatherlink.com - import historical data files
« Reply #21 on: November 12, 2019, 01:22:52 PM »
Correct. I communicated some of the logic they should consider with future enhancements road map that they may consider. They acknowledged there is challenges with limited resources and one of my concerns would be in to reduce the dependency on support if a task can be managed by an end user.

I suspect the concern is that giving more powers to the end-user will increase the support load at Davis (when things start to go wrong because less skilful users make the wrong move). That and the extra work involved in making the account editing options as robust as possible to user error, compared to a small and well-trained group of support staff at Davis. So progress is just going to be in a slow and steady stepwise fashion I suspect.
Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

 

anything