WXforum.net
Weather Related Organizations => CWOP Forum => Topic started by: RickNY on October 16, 2018, 12:21:22 PM
-
For the past several weeks, I will get QC emails reporting crazy dew point errors.. My site is E3379 -- https://weather.gladstonefamily.net/qchart/E3379
From what I can tell based on all of the other stations in my area is that the analysis data itself is wayyyy off, and not my data (or my neighbor's station data)
Looking at the dew point graphs show analysis values for dew point of 3000 degrees, 8000 degrees, etc.. Which leads me to my 2nd question -- is the WXQC Mailing list gone now? When I try to use any of the WXQC Mailing List links, I end up with a custom 404 from Philip Gladstone's site..
Not sure what I should/could do in this instance..
Thanks,
Rick
[ You are not allowed to view attachments ]
[ You are not allowed to view attachments ]
-
It is not just your data - I looked through 8-10 'Neighboring Stations' to my station and they all have the same error.
-
I'm getting the same issue, and neighboring stations show the same. Looks like CWOP is messing up some calculations.
-
Hi,
Mine and all neighboring stations around me have the same issue, it's been like this for sometime now, not sure why the issue has lasted so long - sent email to Phillip asking time line for a fix.
Will let you know the answer if I get one.
Nick. dw7240.com.
-
Seems like it happens for a stretch of 2-3 days, and then goes back to being OK.. I think its been happening for 2-3 months now.
-
I'm having the same problem with dew point and humidity. Like you said it is only the analysis that is way off. I also checked all of my neighbors and I don't see any one of them being the source. For me this started a little over a week ago around October 5th.
This is very annoying. To say that it will go away seems like the wrong approach. Why hasn't gladstonefamily.net been updated in years? If the project is too much to handle or Phillip has lost interest maybe he should consider handing it over. I mean we appreciate all the work that got put into it years ago, but at some point doing the right thing for the project is just as important.
-
Same thing going on here for quite a while now. :-(
-
I did notice that on the days this occurs, there usually ends up being big gaps in the “Analysis” data as well. For example, looking at my graphs now, I see this:
[ You are not allowed to view attachments ]
-
It is probably due to the 2001 Pentium 4 CPUs. Surprised they lasted this long.
-
I did notice that on the days this occurs, there usually ends up being big gaps in the “Analysis” data as well.
I don't think the gap is in the data - it's the dewpoint that is way off the scale...
-
If you notice on my first pic, the graph does adjust scale when the analysis data is off.
-
It isn't just the graphs with thousands of degrees analysis that is off it is also the analysis data.
[ You are not allowed to view attachments ]
This message brought to you by Gladstonefamily. Red Xs for everyone !
[ You are not allowed to view attachments ]
-
It is probably due to the 2001 Pentium 4 CPUs. Surprised they lasted this long.
LOL I've still got a Dell Deminsion 2300 uploading my data flawlessly since 2007 (but I purchased it in 2002) Serial connection data logger and all.
-
Just for the record, Dew point is back to normal, thanks Phillip for your efforts - Be interested to know what the issue was, but it's fixed - that's what counts.
Nick. dw7240.com
-
Thanks Phillip? For what? I don't know that Philip has done anything recently. He certainly hasn't communicated anywhere that I know of. For all we know the problem fixed itself only because its been a few weeks without issue and maybe the servers are not glitching. Or maybe the hackers got tired of messing with old Pentium 4 hardware. There is no comfort in not knowing what happened, what was done to fix it if anything, and the uncertainty that the lack of communication would mean that any day it could happen again if the system is running without any maintenance. If you look at historical data the errors are all still there. So it hasn't been cleaned up.
I've used online tools to see what software version these servers are running and they haven't been patched nor upgraded in many years. This is no way to run servers that are exposed to the Internet that contain known security holes and unpatched exploits. The website contains numerous dead links. The website has not changed in forever. I've sent emails that get no response. And these have been emails not complaining about problems but with simple questions and with closing remarks that I would be sending in PayPal contribution money. No response, no thank you for your support. Philip is MIA.
I can't thank Philip enough for what he built but if maintenance is not possible after all this time then it is time to pass the baton like Steve of Cumulus has recently done.
-
I don't think you should pass judgement. We don't know the why's/ what's/ or where about's of Phillip. Until that information becomes public just assume that it's business as usual. Besides, it's a free site. If you don't care for the service then don't send your data.
-
I don't think you should pass judgement. We don't know the why's/ what's/ or where about's of Phillip. Until that information becomes public just assume that it's business as usual. Besides, it's a free site. If you don't care for the service then don't send your data.
+1
-
I don't think I'm passing judgment. I only asked why anyone would thank Phillip for doing anything recently when there is no evidence that anything was done. Then I listed a bunch of facts. Yes I also threw in some speculation about hackers and Pentium 4 servers, but that is not passing judgment. I'm just frustrated with how the service is run. I'm voicing my displeasure so that I can raise awareness and find a following and create buzz and offer solutions to hopefully create improvements. I'm a supporter of the service so I have no interest in abandoning what I want to see improved. If I buy a car and after owning it for a while I find out something is not to my liking that wasn't immediately obvious when I bought it I'm going to want to offer constructive criticism and feedback to cause improvement and then when I trade in I might have better choices. The service may be free to some, but some people are donors, and donors want to be recognized, have their feedback heard, and thanked for their support. That isn't happening.
If anything I said can be twisted and be called passing judgement, then a "Thanks for fixing" is a judgement that something was done, without having any information to base that response. Goes both ways. I just wanted to point out that nothing has been maintained in years with proof...facts. Therefore I feel justified to pass judgment on those facts. So depending on how you look at it I may or may not be passing judgment and the same goes for the baseless thank you.
If you want to assume it is business as usual I agree with with you. It does seem like business as usual, which is inaction, and no response to the community.
This is getting to be a bit more confrontational than I would like. I hope that we can move along and find ourselves on the same side. I love being a part of CWOP and I would like to see it improved.
There are some people that are haters and like to bash other things that they don't like nor support. They could be called trolls. Jump on those people by all means (or ignore them some people say, don't feed the trolls). But I'm not one of those. If I came off that way then I'm sorry.
-
I've always found Philip to be very responsive when I've contacted him. I worry that something may have happened to him.
- Jim
-
I do see evidence that Phillip was very involved at one point and he obviously cared.. I wish that someone that knows him could reach out and see how he is doing. I think that there are lots of people that would be willing to help.
-
It is probably due to the 2001 Pentium 4 CPUs. Surprised they lasted this long.
LOL I've still got a Dell Deminsion 2300 uploading my data flawlessly since 2007 (but I purchased it in 2002) Serial connection data logger and all.
Dimension 4700 here on XP even! A little slow but real reliable!
-
I see this issue seems to have returned.. Past couple of days, my error reports are showing dew point errors of 80-150 degrees.
-
1) MADIS is computer program that adheres to the adage "Garbage IN, Garbage OUT."
2) It only takes ONE 'bad apple' wx station to "spoil" the MADIS sample pool.
3) Most wx people do NOT check/compare what they are sending against what MADIS expects to find.
4) In Real Estate it's Location--location--location, in home wx stations it's Microclimate--microclimate--microclimate.
5) These are HOME wx stations NOT NOAA/FAA certified stations, that's why we can afford them.
And normally that would be the case -- however, when this has happened over the past few months, very widespread errors are shown in the data across the entire country. For example, I'm in NY.. If I check stations in Missouri -- I see similar dew point errors reported. Its not from a single station thats causing the sample data to be wrong, or even multiple wrong stations in a close area.
-
The NOAA/NWS Meteorological Assimilation Data Ingest System (MADIS) is NOT just for local wx, it is *THE* data collecting (and analysis) system that feeds the WHOLE USA weather mapping program. We just "see" it's local results based upon whatever stations it has selected as its "pool", which changes as the inputs change; it's NOT always the same stations being sampled and used.
Some common "input" problems that MADIS contends with are: different timestamps, units, wrong elevations, incorrect locations (lat & long), etc.
I think it would be reasonable to expect that the whole system would reject as invalid dewpoints of 20,000 degrees.. And not flag a slew of data as not meeting QC because it was 19,950 degrees low.. Just saying..
-
Old Tele man,
This is not a MADIS issue that RickNY is reporting and as others have seen. You can look at the MADIS analysis and the results are fine. The issue being reported only has to do with Gladstonefamily.net. That is a poorly run system that ingests MADIS data and then does its own separate analysis. That is where the problem is ....not MADIS.
But to your credit you are correct that there are times when MADIS does ingest bad data from bad stations. But never have I seen MADIS report thousands of degrees of error. MADIS is very good at throwing out really far off bad data. MADIS does make minor errors as you are reporting though. Unfortunately Gladstonefamily hick-ups or something with its own far off data analysis nationwide for no apparent reason sometimes.