Topic: MADIS Quality Checks

[LewisMcCallum]

Hi all,

I recently wrote a post on the weewx google group about MADIS quality checks and was pointed to here as somewhere I might get some answers. I'll just copy and paste it below:

In particular, I'm interested in the MADIS quality checks that are run. If you upload to CWOP you eventually get a site on gladstonefamily.net where you can track the quality checks of your data and then potentially improve your data quality over time. However, I was disappointed to see my station (UK) wasn't showing full analysis. I don't have an average error/standard deviation for any variable. I then did some exploring and discovered that I could find no UK station which does show this info. All US stations I checked, however, gave full analysis.

I emailed to ask and got a blunt single sentence response from MADIS saying "US and UK treated the same". I did some exploring of the actual raw MADIS QC data and discovered that for all UK stations I checked (once I'd translated the annoying bitmasks), none of them had had a "Stage 3" check carried out, while all US stations I checked did have this check (where, to my knowledge, stage 3 is the stage which compares station data to the network of other stations and produces the error in your data for each observation). I've pointed this out to MADIS support but believe I am now being ignored for pestering them so much.

Essentially I've come here to ask if anyone has a good understanding of the MADIS quality check system, and can explain to me why only North American stations are getting Stage 3 checks carried out. There must be a valid reason for this, but it's frustrating to just be told by MADIS that "there is no difference" when there clearly is. I was looking forward to improving my data quality and it looks like I just sort of can't because MADIS isn't doing the checks that they seem to be claiming to carry out.

Can anyone shed some light?

Many thanks,

Lewis

PS - Station FW8519 for the interested, but my question is about the quality checks in North America/UK more broadly.

weewx user thread - https://groups.google.com/g/weewx-user/c/kf069jzwSY8

[galfert]

I've always just accepted that this is the way that it is. But if I were to try and come up with a reason then I would say that computing power is the reason that it is limited to North American stations.

Now I will tell you that having neighbor station analysis is a mixed bag. I don't put much credibility in it. If your neighbors have good stations then all is well and you can be alerted if your station acts up. But if you have bad neighbors then you get flagged for not falling in line with others. I've also noticed that it only takes one bad neighbor to mess things up. As you can see the analysis can't be trusted. All it indicates is that someone is off but it doesn't tell you who.

What I would recommend is for you to do neighbor comparison yourself with the overlay feature of Gladstonefamily.net. This is what I do and I then can find who is the outlier station. I then contact them based on their registration email.

[LewisMcCallum]

I've always just accepted that this is the way that it is. But if I were to try and come up with a reason then I would say that computing power is the reason that it is limited to North American stations.

I also assumed this would be the case, but MADIS support told me this isn't true. Had they told me that they don't carry out the analysis outside of North America because of the computational expense I probably would have just gleefully moved on. But they thoroughly confused me when they came back and boldly claimed they do "the same checks" on US and UK stations (which is demonstrably false). I think most likely you're right, and MADIS support is just being dismissive of my questions...

And yes, I keep hearing a lot of people complaining about neighbouring stations ruining the spatial consistency checks (I'm yet to see a station which passes L3 for wind vector). My station is on the Scottish Highland boundary fault, so the MADIS map is well-populated with lower elevation stations to my South-East, but very bare to the higher elevation North-West of me. I can't imagine being right on this boundary could be very helpful for the analysis values at my station if MADIS were to start doing them.

[chief-david]

deploy it and forget it.

[CW2274]

I emailed to ask and got a blunt single sentence response from MADIS saying "US and UK treated the same". I did some exploring of the actual raw MADIS QC data and discovered that for all UK stations I checked (once I'd translated the annoying bitmasks), none of them had had a "Stage 3" check carried out, while all US stations I checked did have this check

In my area of the US, many of the stations are showing "S - Screened, passed levels 1 and 2". I don't visit this site very often, but when I do, it's hit or miss whether you'll be at an "S" or a "V". My guess is it's a MADIS timing thing on what you'll see. Whether this means you'll personally "graduate" to a "V" or not, I have no idea. All I can suggest is patience. As far as Gladstone, don't waste your time (I climbed out of that rabbit hole loooong ago), be your own QC.