Author Topic: WD and error correction  (Read 10001 times)

0 Members and 1 Guest are viewing this topic.

Offline Bob65

  • Member
  • *
  • Posts: 41
WD and error correction
« on: December 05, 2007, 07:06:47 PM »
Seasons greetings all,

I am currently using WUHU for uploading my data. Once in awhile the program will discard a report because the windspeed is way out of range.

I purchased VWS and used it for two weeks and it reported those wind gusts in the 40 mph and 60 mph range plus accumulated  rain total by doubling the year to date twice for some interesting totals.

Having said all this, I question is can I expect more from WD as far as error are concerned

Regards, Bob
Ambient WS-2080 A   
Cumulus  1.9.3
CW8067
KFLBUSHN2

Offline carterlake

  • Senior Contributor
  • ****
  • Posts: 243
    • CarterLake.org
Re: WD and error correction
« Reply #1 on: December 05, 2007, 07:24:18 PM »
Bob,

What weather station are you using? I'm going to guess a La-Crosse.

If you are, then you're incorrectly laying the blame on the software, instead of the hardware.

That being said, I believe Brian has built some error checking into WD to assist with this issues.

But still.... garbage in / garbage out.  :sad:

Davis VP2 6153; Weather Display (LIVE w/ Ajax); Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; live NOAA Radio

Offline Bob65

  • Member
  • *
  • Posts: 41
Re: WD and error correction
« Reply #2 on: December 05, 2007, 07:43:26 PM »
That's correct. It's a LaCross 2310.
Should have included that in my post.

Then you're saying that the errors are caused/coming from the pws and that a Davis never does that?
Ambient WS-2080 A   
Cumulus  1.9.3
CW8067
KFLBUSHN2

Offline carterlake

  • Senior Contributor
  • ****
  • Posts: 243
    • CarterLake.org
Re: WD and error correction
« Reply #3 on: December 05, 2007, 09:18:20 PM »
That's correct. It's a LaCross 2310.
Should have included that in my post.

Then you're saying that the errors are caused/coming from the pws and that a Davis never does that?


The WS-23xx has some very well-known issues with data spikes. Actually, if you cut the "cabling" open, you'll find literally one or two strands of copper in the lines... that's just not adequate for outdoor usage. If you replace the lines with shielded twisted pair data cables, it will greatly help the problem. Others have found adding ferrite chokes helps or twisting the cables themselves.

With my old WS-23xx, I replaced all the cabling and added chokes... it helped but didn't completely eliminate the problem.

Beyond that, most hardware has some issue(s)... the WS-23xx is data spikes and lower quality humidity sensor. The Oregon Scientifics have a lower quality humidity sensor, etc. Davis is just about the best home system you can get, but obviously is a LOT LOT more expensive.

Anyway, your problem is hardware related... not software.

Davis VP2 6153; Weather Display (LIVE w/ Ajax); Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; live NOAA Radio

Offline Bob65

  • Member
  • *
  • Posts: 41
Re: WD and error correction
« Reply #4 on: December 06, 2007, 10:04:52 AM »
Thank you for your input.

If I can recap;
LaCrosse is garbage and your data is poor quality and cannot be fixed by software although WD may be able to help some.
Until you buy a Davis pws you cannot play with the Big boys nor use real software so take your toy and go home.

Interesting perspective on things.
Ambient WS-2080 A   
Cumulus  1.9.3
CW8067
KFLBUSHN2

Offline racenet

  • Forecaster
  • *****
  • Posts: 1306
    • NH Weather Data
Re: WD and error correction
« Reply #5 on: December 06, 2007, 12:36:10 PM »
Thank you for your input.

If I can recap;
LaCrosse is garbage and your data is poor quality and cannot be fixed by software although WD may be able to help some.
Until you buy a Davis pws you cannot play with the Big boys nor use real software so take your toy and go home.

Interesting perspective on things.

Your recap is an interesting perspective on things.  :roll:

The stations in question have known issues and no software can really correct that. And personally, I wouldn't want to be correcting bad data with software on a daily basis. The data coming in should be as close to correct as possible. If it isn't, it's all really a waste of time, effort and money to try correcting it.

As the saying goes, garbage in......garbage out......
No amount of software correction is going to fix that.



Bob



 
www.theamericanflagstore.com - The American Flag Store



www.nhweatherdata.com - NH Weather Data

Offline windy

  • Senior Member
  • **
  • Posts: 61
    • Grahams Beach Weather
Re: WD and error correction
« Reply #6 on: December 06, 2007, 12:54:22 PM »
if you replace the cheap data cables with the better quality shielded twisted pair cables then gets rid of the majority of electrical interference problems
WD does have data limits you can impose and it does have spike detection in the code

Offline wuhu_software

  • Forecaster
  • *****
  • Posts: 689
    • WUHU Software Yahoo Group
Re: WD and error correction
« Reply #7 on: December 06, 2007, 01:41:19 PM »
Thank you for your input.

If I can recap;
LaCrosse is garbage and your data is poor quality and cannot be fixed by software although WD may be able to help some.
Until you buy a Davis pws you cannot play with the Big boys nor use real software so take your toy and go home.

Interesting perspective on things.

Your recap is an interesting perspective on things.  :roll:

The stations in question have known issues and no software can really correct that. And personally, I wouldn't want to be correcting bad data with software on a daily basis. The data coming in should be as close to correct as possible. If it isn't, it's all really a waste of time, effort and money to try correcting it.

As the saying goes, garbage in......garbage out......
No amount of software correction is going to fix that.

Bob
 

IMHO, none of the stations on the market are 100% reliable.

Here is a station just down the road, Davis VP...

http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIWIXOM2&day=6&year=2007&month=12&graphspan=year

Yearly precip: 385.77in

I have also witnessed other Davis stations reporting bogus wind speeds, temperatures, etc, using a variety of software packages.

Although the Davis units are generally more reliable and accurate, they are not without faults.

The La Crosse units currently out there certainly have flaws, especially when it comes to wind measurement. It is a shame that they have not made much of an attempt to improve the data integrity (especially where wind measurement is concerned).

Have the software attempt to detect bogus wind speeds without tossing legitimate data is a tricky business. Winds are highly variable. No matter how sophisticated the scheme, you are likely to throw away valid data sooner or later. The best you can do is find the obviously bogus data and reject it.

One thing to keep in mind when uploading to sites like WU, you are uploading many times per hour (using rapidfire for example), if you lose one or two uploads, the impact is minimal. WU stores a record every few minutes, there is no guarentee they will store records during peak wind speeds for exampe. Only a fraction of the data is actually stored so losing a few records here and there is of little consequence.


Offline carterlake

  • Senior Contributor
  • ****
  • Posts: 243
    • CarterLake.org
Re: WD and error correction
« Reply #8 on: December 06, 2007, 03:00:34 PM »
Thank you for your input.

Interesting perspective on things.

No problem Bob. Actually, it's just the plain old truth.

Although I would call the WS-23xx wiring poorly engineered. They should add $10 to the cost and include better cabling.

I wrote up a little article on it once, let me see here... hum...

The WS-23xx is definitely a "fixer upper" weather station.

1) All the wiring absolutely MUST be replaced with better wiring. Plan on spending the first day or so after you get it cutting, crimping, and soldering shielded, twisted wire. Mine is a 2-pair phone wire specially designed to be outdoors. The cheap phone wire that comes with the WS-23xx is a joke... especially when you cut it open and see the (up to) 4 strands of copper in it... completely unacceptable for long-term outdoors use. Some have had temporary luck twisting the wire but it really needs replaced for long-term reliability.

2) Wireless rate and reliability is completely unacceptable. The unit must be wired. There's another 4-8 hours of wiring, drilling, etc.

3) The housing for the hydro/thermo sensor is a joke. Expect to replace it with a proper weather-resistant housing. Luckily, there are some very low cost options here (white plastic bowls for instance) which will work just fine. Mine is located up under an eave on the north side of the house. I use a fan to aspirate the housing to prevent a pocket of air from forming.

4) The humidity sensor is well... very low cost. It really only works between 20% and 90%. Over the short term, this affects other important things (dew point for instance). So if you live some place which is very dry or very wet, the WS-2310 could be frustrating. Some of us, ok, I had the idea, and Bob (http://www.jaxweather.net) actually installed it - are looking at replacing the humidity sensor with a 1-wire sensor.

Ok, that behind us... My WS-2310 has been remarkably accurate once I got all the wiring replaced and wired the unit. I'd say other than during times of high humidity, it's compares well with the much more expensive Davis units.

The station has some warts but can be a great station with some work.

Davis VP2 6153; Weather Display (LIVE w/ Ajax); Quickcam for Notebooks Pro; Boltek w/ Nexstorm; GRLevel3; live NOAA Radio

Offline windy

  • Senior Member
  • **
  • Posts: 61
    • Grahams Beach Weather
Re: WD and error correction
« Reply #9 on: December 06, 2007, 04:11:22 PM »
i dont know of any Davis stations myself using WD software that have problems with bogus data, I think your claim of it being a real problem wuhu might be a bit on the high side?
which software are you refering too?
the davis wmii however is prone to electrical intereference as well, but RF chokes on the cables handle that
also the Davis VP does have a known ground loop problem in heavy rain with the windspeed

Offline wuhu_software

  • Forecaster
  • *****
  • Posts: 689
    • WUHU Software Yahoo Group
Re: WD and error correction
« Reply #10 on: December 06, 2007, 06:55:33 PM »
i dont know of any Davis stations myself using WD software that have problems with bogus data, I think your claim of it being a real problem wuhu might be a bit on the high side?
which software are you refering too?
the davis wmii however is prone to electrical intereference as well, but RF chokes on the cables handle that
also the Davis VP does have a known ground loop problem in heavy rain with the windspeed

I am not saying that the Davis is not better than the rest, I am only stating that the stations are not perfect and will also cough up bad data now and then (for whatever reason).

As an example, I poked through WU history looking at only VP2 stations in my state to look for errors.

Here is a list of the more obvious errors although I did not record many that for example severely under-reported rainfall or questionable windspeeds for example.

My only point is that there is no "error free" station out there.

WeatherLink:
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIYALE1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIJAMES1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIJASPE1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIOLDMI1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIEASTP2&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIMATTA2&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIGRAND40&year=2007&month=11&day=19

VWS
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIKALAM11&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIMILLE2&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIMOUNT8&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMISAGIN4&graphspan=month&month=3&day=6&year=2007
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMITEMPE3&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMITWINL1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIWYAND4&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIWAYNE4&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMICHARL4&year=2007&month=3&day=31
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIDETRO6&day=6&year=2007&month=12&graphspan=year

WD
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIKALAM9&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMITAYLO3&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMICOMST1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIWALKE1&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIANNAR20&day=6&year=2007&month=12&graphspan=year

Unknown
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIPORTH4&graphspan=month&month=5&day=6&year=2007

WU mod
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIROCKF2&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMITROY6&day=6&year=2007&month=12&graphspan=year
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMISANDP1&day=6&year=2007&month=12&graphspan=year

Weather Tracker
http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KMIMUSKE4&day=6&year=2007&month=12&graphspan=year

Offline wuhu_software

  • Forecaster
  • *****
  • Posts: 689
    • WUHU Software Yahoo Group
Re: WD and error correction
« Reply #11 on: December 06, 2007, 06:57:00 PM »

P.S. - I am not blaming the software.

Offline windy

  • Senior Member
  • **
  • Posts: 61
    • Grahams Beach Weather
Re: WD and error correction
« Reply #12 on: December 06, 2007, 09:34:54 PM »
sometimes a spider web or similar can set the tipper in the rain gauge into a position and then the davis station keeps on recording rain...thats a known problem too

personaly I have not had any data problems apart from the spider web in the rain gauge for a long time with my davis VP
(i did have one case of a ground loop rain->wind spike, but thats fixed now with better grounding)

Offline Anole

  • Forecaster
  • *****
  • Posts: 585
    • http://pineislandweather.com
Re: WD and error correction
« Reply #13 on: December 07, 2007, 06:16:16 AM »
Here is a list of the more obvious errors although I did not record many that for example severely under-reported rainfall or questionable windspeeds for example.

My only point is that there is no "error free" station out there.

True enough, but you also have to take into consideration that some of the troubled stations in your list may not be problems with the station hardware but are instead siting and installation issues (which are of course factors for all stations regardless of manufacture).

Offline wuhu_software

  • Forecaster
  • *****
  • Posts: 689
    • WUHU Software Yahoo Group
Re: WD and error correction
« Reply #14 on: December 07, 2007, 09:51:38 AM »
Here is a list of the more obvious errors although I did not record many that for example severely under-reported rainfall or questionable windspeeds for example.

My only point is that there is no "error free" station out there.

True enough, but you also have to take into consideration that some of the troubled stations in your list may not be problems with the station hardware but are instead siting and installation issues (which are of course factors for all stations regardless of manufacture).

Siting can definitely be an issue with any station type.

I grabbed only those with either extreme temperatures, wind reports, and rainfall reports that were obviously bad at first glance.


Offline capeweather

  • Administrator
  • Forecaster
  • *****
  • Posts: 1309
    • http://www.capeweather.com
Re: WD and error correction
« Reply #15 on: December 07, 2007, 10:30:28 AM »
Here is a list of the more obvious errors although I did not record many that for example severely under-reported rainfall or questionable windspeeds for example.

My only point is that there is no "error free" station out there.

True enough, but you also have to take into consideration that some of the troubled stations in your list may not be problems with the station hardware but are instead siting and installation issues (which are of course factors for all stations regardless of manufacture).

I have to agree with that. When I first installed my station I had it elevated on my roof which was producing an upspike in temperature of about 5 degrees during the day from the reflective heat. It no longer resides there and the temperatures are normal. Siting is very important.

Chris
Cape Coral, Florida
Website: http://www.capeweather.com
Website: http://www.fortmyersweather.net

Offline Bob65

  • Member
  • *
  • Posts: 41
Re: WD and error correction
« Reply #16 on: December 07, 2007, 02:06:30 PM »
This has been a Very informative topic discussion and I thank everyone for the input.
For me, the main points are that my 2310 is a fixer upper and for now that's what I'll do. I ordered the Cat 5 stp today and have been nosing around the kitchen for a bowl  :-P . Although this may get me in trouble so maybe I should look elsewhere. I will continue to improve my siting issues and read this board for help and suggestions. Again, thanks to all.
At this point I would not buy a Davis, even if I win the lottery, on principle. I will make my pws the most accurate pws I can and if I find out that my readings are .01 deg or % off then so be it.  More than that would require some rethinking.

BTW I thinking of buying a Ford automobile. Not good huh? :-P

Best Regards, Bob
Ambient WS-2080 A   
Cumulus  1.9.3
CW8067
KFLBUSHN2

Offline Hfcomms

  • Senior Member
  • **
  • Posts: 83
    • Big Bay, Mi - Local Weather
Re: WD and error correction
« Reply #17 on: December 07, 2007, 04:46:32 PM »
I've only had the 2308 up and running for about a week now.  So far it's doing a pretty decent job for the money but you get what you pay for.  It's worth the $100 I paid but not much more.  It retails for about $275 and thats a poor joke.  So far the only problem I've had is the wind speed is registering too low.  I have my Davis anemometer about 50' away from the Lacrosse and there are times where the Davis is showing 10mph and the Lacrosse is showing 2.2mph.  And I have already been up on the roof a couple times in the last week to blow some snow out of the circular vane in the anemometer.  That is a pretty poorly designed system there.  Give me the standard cup type any day.  In the past I've had the Davis anemometer freeze up a few times but it happens rarely.  I have a feeling I'm going to be up on the roof a bit this winter with the Lacrosse.

Offline wuhu_software

  • Forecaster
  • *****
  • Posts: 689
    • WUHU Software Yahoo Group
Re: WD and error correction
« Reply #18 on: December 07, 2007, 07:46:25 PM »
I've only had the 2308 up and running for about a week now.  So far it's doing a pretty decent job for the money but you get what you pay for.  It's worth the $100 I paid but not much more.  It retails for about $275 and thats a poor joke.  So far the only problem I've had is the wind speed is registering too low.  I have my Davis anemometer about 50' away from the Lacrosse and there are times where the Davis is showing 10mph and the Lacrosse is showing 2.2mph.  And I have already been up on the roof a couple times in the last week to blow some snow out of the circular vane in the anemometer.  That is a pretty poorly designed system there.  Give me the standard cup type any day.  In the past I've had the Davis anemometer freeze up a few times but it happens rarely.  I have a feeling I'm going to be up on the roof a bit this winter with the Lacrosse.

Some Costco stores are carrying the newer La Crosse 16XX stations that use the cups rather than the turbines.

If you are unhappy with the WS-2308, you could hunt the new type down.




Offline Bruiser

  • Contributor
  • ***
  • Posts: 113
    • Miami Township (Just South of Dayton), Ohio Weather
Re: WD and error correction
« Reply #19 on: December 11, 2007, 11:34:11 AM »
Not sure about the wind data cause (I have heard of electrical shorts causing this on Davis stations connected to computers) but the rain data could be someone not clearing excessive rain data from WU after their rain tipping bucket was stuck and reporting tremendous rainfall.  I have had a stick get into the tipping bucket area and trap the bucket at a point that reports a downpour.  Tipping bucket freezing can cause this also.  Pain in the butt but I had to go into Weatherlink and do some weatherunderground entry deletions to correct the faulty data.

Uggh...

 

anything