WXforum.net
Weather Station Hardware => Ambient Weather and Ecowitt and other Fine Offset clones => Topic started by: hifi_hound on May 13, 2022, 08:56:20 PM
-
I have been trying to solve a problem with multiple data spikes showing up in my daily Weather Underground graphs. These seem to have started around Jan 31 from what I can see going back through my data. At that time the data was being supplied by my Ecowitt GW1000. The spikes look like they are happening across the board much of the time, meaning they appear in temperature, UV, solar rad, sometimes pressure, and rain. I've seen some other threads about random UV spikes, but mine are happening in these other areas mentioned as well. I've tried relocating the GW1000 to different outlets which are away from cycling refrigerators etc., but I can't seem to get rid of the problem. I recently purchased the Ecowitt Wittboy, and transfered the WU duties over to the GW2000 and I am still getting the spikes. I'm not sure what to do at this point. My Ecowitt.net data along with Weathercloud and WeatherObservations data don't seem to show the same spikes or problems. Also, the WU data temperature lows don't seem to match the graphs either, with the low temperature be stated for the day, much lower than any readings in the graphs. I just can't figure out why there are discrepancies, but I think it has to do with the data spikes/corruptions. You can look at the historical data charts and see what I mean. The Ecowitt site graphs don't show any problems.
https://www.wunderground.com/dashboard/pws/KCAEMERA6
https://www.ecowitt.net/home/share?authorize=Y6FJKF
Here is a sample from WU
[ You are not allowed to view attachments ]
I'm running 2 stations at the moment, the Wittboy, and my original station which includes an Ecowitt WS68 mounted 30ft up, and a homemade FARS with an WH32_EP temp/humidity probe. The WH32_EP and the FARS unit have been running for about a year, so it doesn't seem to be related to those. At the moment I have the GW2000 reporting to WU with WH32_EP providing the outdoor temp and the Wittboy providing the rest of the data. I'm not really sure what the issue is since the problem is across 2 different stations and 2 receivers/upload units. Also, my internet is gig fiber, and my router/wifi hasn't changed in over a year, so no recent changes there either. Any help or suggestions would be appreciated.
-
at first glance it looks like a WU issue which is not in your control.
If you used FOSHKplugin, you could make related tests right away.
for FOSHKplugin see https://www.wxforum.net/index.php?topic=39625.0
You could install FOSHKplugin if you happen to have a RaspberryPi easily - or even install and run it in the Windows 10/11 WSL (Windows Subsystem for Linux).
FOSHKplugin documents the incoming and outgoing posts in logging files. Not just with a time stamp, but the full post including the payload.
FOSHKplugin would post the data to WU just as your console would do (using custom server and WU protocol).
So you could compare the data sent with the data shown for that time stamp at WU.
Of course, you wouldn't know if for whatever reason the standard WU posting of the console behaved out of the usual and not in tune with a related custom server setup.
In order to know this you would need to set up DNS spoofing or DNS redirection by installing e.g. DNSmasq redirecting the WU post to your FOSHKplugin, forward them to WU from there and then compare the values.
-
Thanks for the reply! I briefly looked at FOSHKplugin awhile back, but haven't really looked at it lately. Just curious, what makes you think this is a WU issue?
-
Thanks for the reply! I briefly looked at FOSHKplugin awhile back, but haven't really looked at it lately. Just curious, what makes you think this is a WU issue?
your own statement ;)
My Ecowitt.net data along with Weathercloud and WeatherObservations data don't seem to show the same spikes or problems.
but to verify I proposed to use FOSHKplugin 8-)
-
Wow that's very weird!
I've just setup a Wittboy along side with my old fine-offset WH2950 unit/console. I've never seen spikes like that in the WU data. I previously had updates set at 5 minutes and now I've set them both to 1 minute intervals without problems. I set the Wittboy up on new site ID's for all the uploads (ecowitt, wu, weathercloud, wow) to see how it compares with my old unit. If as you say all looks good on the sites other than WU for your Wittboy/GW2000 then I can only think there is a problem on the WU server side. At the moment the only other thing I can think of which seems unlikely, is if you have multiple gateways/consoles sending reports to the same site WU ID.
-
Yes it is strange. I thought the same thing that somehow maybe I hadn't removed my WU credentials from my GW1000. But I have checked multiple times just to be sure. I also still have an Acurite 5 in 1 from probably 10 years ago still running, but everything is turned off on that unit as far as reporting to outside services. It only shows up on the console and on myAcurite, but nowhere else. I can't figure out anything that might have changed at the end of January either that could cause the spikes. It started while the GW1000 was in service and now it's the GW2000 with the same issue. Also 2 different sending units, other than the WH32_EP which was sending to both receivers. I might try and create a new WU setup and see if it happens there as well.
-
So I created a new WU site, fed by the GW2000 and moved my original WU station over to my GW1000. Both devices are sending the same device data from my WS90. So far after 1 day, the same spikes are showing on my old WU station, but not the new station. It definately seems to be a WU issue. I have sent them an email, but don't expect hear anything back.
Old WU site: https://www.wunderground.com/dashboard/pws/KCAEMERA6
New WU Site: https://www.wunderground.com/dashboard/pws/KCAEMERA18
-
Interesting!
It's a pity that a new pass key cannot be generated for an existing ID. Shouldn't be difficult. I find the WU stuff pretty underwhelming.
-
So, I turned off sending to my old site at https://www.wunderground.com/dashboard/pws/KCAEMERA6 at around midnite last night to see what would happen. The site was still showing as online up to around 4 pm today, but the data was not mine. It was showing a temperature of 55 degrees F, but the temperature was in the 90 degree range at that time. It seemed to be receiving upload data every 30 minutes to an hour or so from somewhere else. The data was not from any of my stations. Because I could not regenerate a new station key as Jonesy mentioned, I decided to change my Wunderground password just to see if it would affect the rogue uploads. That seemed to stop the rogue data, and the station is now showing as offline. I'm going to reactivate the station now, and see if the spikes go away. I'll let you know if this fixes it.
-
It seems I spoke too soon. I went back to my station to check on it, and it was back to showing online. Apparently it received a new data upload from somewhere, and is registering as online again. I've sent an email to WU, asking for them to regenerate the station key, but haven't received a response yet. Has anyone ever had WU successfully regenerate an upload key? If so, was it through their support email?
-
Not only is it sending data, but bad data, for 4AM, I doubt you have 700 W/M^2.
Also you have you hardware set to an Ambient WS2000, shouldn't matter but confuses the issue.
Did you ever have a WS-2000?
Did you ever send data to AmbientWeather net, and have data forwarded to WU from there?
Apparently it looks like whatever data is being sent has a wrong clock / time value
KCAEMERA18 is your new ID?
-
Apparently it looks like whatever data is being sent has a wrong clock / time value
Or the station is roughly 12 hours away around the globe?
Could it be someone made a typo entering their key and by chance hit another valid one - yours! The upload keys are quite short.
Meanwhile they are complaining to WU about their data not showing up on their station! :lol:
-
Could it be someone made a typo entering their key and by chance hit another valid one - yours! The upload keys are quite short.
Meanwhile they are complaining to WU about their data not showing up on their station! :lol:
Might be, but in that case one would expect all their data, not these huge gaps, sometimes over 2 hours. I mean maybe they have real crappy wifi / Internet.
This is strange, I've looked at the UTC values, in case they were somehow confused, and no, that doesn't line up either based on the solar values , which would i think for his time would be + 7 (UTC - 7).
-
Rover1822: Yes KCAEMERA18 is my new ID. I have never had an Ambient WS2000, but it is pretty much the the equivalent to my Ecowitt branded system, and WU doesn't list Ecowitt systems in their weather station choice's. The only alternative is to list the station as "other".
I also noticed the location difference which explains why I would get UV spikes + temperature spikes in the middle of the night. If you go back a few days or more, through the graph data for previous days, you will see what I mean. It also explains why my low for the day was often lower than the actual data that would show on my Ecowitt site. This other station is sending daytime readings at random times during my evenings and early mornings, and nightime data during my days. I don't understand the reporting intervals though. They are very random intervals.
The idea of someone typing in the wrong key by a digit seems strange though as I would think they would have to put in the station ID as well (KCAEMERA6), which would seem unlikely.
-
Unfortunately we do not know what keys WU uses for uploading other than the upload key. We , I mean I.
I would move to another ID, key. Don't hold your breath on a response from WU, blue is probably not your color. I know , you will lose history data, and yes I get it. WU is not known for end user support.
What we are pretty sure of , this is not your issue, not something you can just fix.
I would say, just to be consistent change your station type to "other", the upload protocol for WU is pretty much fixed for all station types, it will have no bearing on your data and be better than saying you have something you don't, just in case you ever do get a response from support.
Also you get wack jobs like me asking silly questions about the ws-2000
you may also want to pose your question on another forum of this board https://www.wxforum.net/index.php?board=90.0 which is for WU. The info about .. hey I'm not transmitting anything , but there is this spurious data. Might get you somewhere
At this point device doesn't matter
-
I also have spikes in the WU upload
UV, Solar and barometric pressure sometimes have spikes in the graphs
I use the GW2000 to upload my data to WU
I created a new station on WU
My existing station is fed through meteobridge and no issues
On the ecowitt.net site everything is good
It must be a bug on the WU site
-
If you can, try the experiment that hifi_hound did. Turn off the WU upload from the device that is showing spikes. Watch it for a cpl days, see if you get any data posted to WU. This is what hifi_hound discovered.
-
Also, please post links to your WU sites.
-
Rover1822: Yes KCAEMERA18 is my new ID. I have never had an Ambient WS2000, but it is pretty much the the equivalent to my Ecowitt branded system, and WU doesn't list Ecowitt systems in their weather station choice's. The only alternative is to list the station as "other".
I also noticed the location difference which explains why I would get UV spikes + temperature spikes in the middle of the night. If you go back a few days or more, through the graph data for previous days, you will see what I mean. It also explains why my low for the day was often lower than the actual data that would show on my Ecowitt site. This other station is sending daytime readings at random times during my evenings and early mornings, and nightime data during my days. I don't understand the reporting intervals though. They are very random intervals.
The idea of someone typing in the wrong key by a digit seems strange though as I would think they would have to put in the station ID as well (KCAEMERA6), which would seem unlikely.
Interesting your data for KCAEMERA6 shows 0 for solar values today, also if you compare the reporting times for the last 2 days, they are roughly (very roughly) the same -- very weird
-
Also, please post links to your WU sites.
INOARD3 is my WU id GW2000 Ecowitt
IFRIESLA82 is my WU id with meteobridge Davis
I will turn off upload to WU and see if any data comes up
-
The timing of your solar spikes, is similar but not exact to hifi_hound's data for today (he is not sending). Although , he has 0 solar values.
https://www.wunderground.com/dashboard/pws/KCAEMERA6 (in the table view, compare your solar spike times, and his entries)
almost like there is pattern
-
Just checked my WU
About 1 hour ago I quit uploading
But my data says updated 6 minutes ago and Temp is 7.9 degrees Celsius which cannot be right
Strange also barometric pressure is still updated but way lower as can be seen on the graphs
[ You are not allowed to view attachments ]
[ You are not allowed to view attachments ]
-
To completely rule out the devices/consoles , might consider completely shutting them down for a day or so, but give it a day to collect data as it is now before doing so. Just a thought.
-
OK I have been checking the WU data from both systems in this discussion, that I thought were shut down/not sending. What is on WU is wacked. You must be doing your own tests :)
-
Is it possible that you are using Weewx or cumulus, something else , and those are sending in missing records to WU? and for some reason, the timestamp is being ignored and what looks like the current time for your zones is being used?