WXforum.net

Weather Related Organizations => CWOP Forum => Topic started by: stromb0li on July 19, 2018, 10:17:49 AM

Title: Adjusting for bad data
Post by: stromb0li on July 19, 2018, 10:17:49 AM
Hey WXForum,

When reporting to CWOP, what do you do when you know your data is going to be bad?

I just noticed my rain gauge stopped reporting any rain altogether.  I went outside to investigate and it turns out the darn birds' poo had plugged the gauge.  It's been raining since very early AM (now about 4 hours) and I didn't check until now.  In this case, when the data gets reported it's going to:
1) Show a massive spike in data (I tried to leave as much rain as-is in the bucket, so once it was unblocked it'd measure what was in there)
2) for any rain that is missing, I'm sure it will be off

What is the right procedure when you know the data going to CWOP is going to be bad?  Secondly, if I have a manual rain guage, is there a right process for reporting what that has vs the weather station?

Thanks in advance!
Title: Re: Adjusting for bad data
Post by: DoctorKnow on July 19, 2018, 10:20:09 AM
I would just stop reporting until the issue can be corrected.
Title: Re: Adjusting for bad data
Post by: stromb0li on July 19, 2018, 02:16:36 PM
Thank you for the response.

That works for maintenance to just stop reporting, but what about bad data itself (i.e. rain gauge discrepancies)?
Title: Re: Adjusting for bad data
Post by: dwhitemv on August 02, 2018, 12:10:26 AM
Once data is sent to the cloud, it's sent everywhere. You would need support in the data protocol to void previous observations, and there isn't any at current. The large CWOP data aggregators (MADIS, MesoWest) have their own quality checking and may flag your erroneous observations as questionable anyway.

Weather station data is highly temporal. By the next day, nobody will notice bad data unless they are looking through historical databases.

Best you can do is be vigilant for bad data and stop sending data until its fixed. This is where the Gladstone MADIS QA emails are nice; they can alert you to abnormalities before you have a month of bad data on your hands. As with all QA systems, nothing is perfect, and you'll get used to what your "normal" reject level is.