Author Topic: Analysis of Number of Weather Stations by Brand / Type  (Read 2044 times)

0 Members and 1 Guest are viewing this topic.

Offline galfert

  • Forecaster
  • *****
  • Posts: 821
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #25 on: October 15, 2018, 10:49:25 AM »
*** New Formula for Calculating Multiple Search Matches ***

Well I finally figured it out. The problem before was that I was only matching one search result to give a tally. For example to search for all the Davis Vantage Pro 2, I was only previously looking for a match for "Vantage Pro". For Vantage Vue I was only searching for "vue". This is because some stations left out the word Davis, and others called it Vantage Pro2 or Vantage Pro II. The problem I noticed with Davis (and later with other station types too), is that there is the most common choices from the drop down selection for station type, but apparently perhaps at one point the drop down selection was not as good or even still Weather Underground may be permitting free expression of the model hardware. This lead to a few stations listed with some different names and even typos. There are stations listed as Advantage Pro. But my my search was counting those before because "vantage pro" is still a search string match for those situations. But there were also some instances like, vantagepro, davis 6153, Davis Pro 2, VP2. The typos were things like, provantage, vantage2, vantege...etc. I tried to find the most popular variations and that is why I stuck with "vantage pro" before. But now I have a formula that accounts for multiple matches. This also helped Acurite, as sometimes it was spelled Acu-rite with hyphen. Oregon Scientific also sometimes were listed as just OS WMR or WMR-xxx. Ambient was sometimes spelled Ambiant. You get the idea.

Here is the new tally using the old data with these new formulas. There is nothing terribly wrong with the previous numbers as they paint a pretty good picture of the most common name for each station type. With this new formula now we just have a better more accurate picture. Now you can almost see that if you add the VP2 + Vue stations you come pretty close to the Davis total (the balance is probably old Weather Monitors still running). The Davis total is also counting stations that are only listed as just VP2 (leaving out the word Davis) for example where as before I was only looking for the word "Davis" explicitly. This helped the Davis total go way up from the prior search method.

If anyone is curious this is the new formula (totally different than previous) for searching for multiple matches (took a while to figure this one out):
=IF(SUMPRODUCT(--(NOT(ISERR(SEARCH({"vue","view","davis 6250"},D2)))))>0,1,0)
That is an example of the Vue search. The others are formatted the same, just the search matches differ and the VP2 search is huge with so many variations.

I also noticed that the Oregon Scientific was off because it was matching positive for the State of Oregon's harware called Oregon Dept. of Environmental Quality. I had to tally those and subtract them from the Oregon total (because I was trying to pull all the Oregon models by just matching the word "Oregon".) So the Oregon Scientific total number you see reflects this correction already. You just see the Oregon Dept column because I had to tally it in order to have a number to subtract (that was easier than coming up with a custom Oregon formula with AND IF functions.)

Entire US Station List:
 [ You are not allowed to view attachments ]


Just Florida:
 [ You are not allowed to view attachments ]

This is still not perfect. When you have over 91,000 records in Excel (and multiple data sets) and you are looking and trying to account for all the variations that people mistakenly enter it gets very tedious very fast. I think these new results are good enough numbers even though probably still not perfect because of the human factor. I'm not going to sweat it trying to find every single obscure variation. I spent a lot of time on this and hopefully it was worth it to some people. Id say these number are probably 99% accurate based on the data provided by WU.

« Last Edit: October 15, 2018, 11:54:35 AM by galfert »
WS-2902A | ObserverIP | WeatherBridge (Meteobridge)
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Tele-Pole flag pole is here (not installed yet)

Offline havtrail

  • Senior Member
  • **
  • Posts: 57
    • Haverford Weather Station
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #26 on: October 15, 2018, 12:00:13 PM »
Great stuff!

So, if the brand names (as opposed to individual models) now closely represent the brand totals, then in the U.S. the leader seems to be Ambient (27,384 in Oct.) followed closely by Acurite (26,762) and then Davis (19,418). Right?

This is probably the best publicly available info on the national PWS market, which is at least 91,000+ on Weather Underground alone.

Rich K.
Onset HOBO RX3003 Cellular
http://weather.havtrail.com
WU KPAHAVER17

Offline Felix1

  • Member
  • *
  • Posts: 34
  • PhD Engineering Physics
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #27 on: October 15, 2018, 12:17:46 PM »
Sorting on your raw data, looks like only 39 WeatherHawk stations are uploading their weather info to WU. I would have assumed many more than that but I guess it's because most sales are to federal, regional & local governments, universities, industry, emergency management entities and high-end residential automation.

Offline galfert

  • Forecaster
  • *****
  • Posts: 821
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #28 on: October 15, 2018, 12:20:51 PM »
Great stuff!

So, if the brand names (as opposed to individual models) now closely represent the brand totals, then in the U.S. the leader seems to be Ambient (27,384 in Oct.) followed closely by Acurite (26,762) and then Davis (19,418). Right?

This is probably the best publicly available info on the national PWS market, which is at least 91,000+ on Weather Underground alone.

Rich K.

Yes, correct. The reason the brand doesn't exactly match the listed models mostly because there are other models not listed separately and because of obscure rare variations in station types with errors or omissions.

Too bad there isn't any data for other countries. I really would have liked to have seen analysed other countries. Also too bad that Acurite and WU don't list out model numbers separately.* Seems only Davis and Ambient do this. Well I've seen the data and other manufactures also list out model numbers separately like Rainwise. I haven't bothered to include those nor others like Vaisala and Peet Bros..etc ....because nobody has asked. By looking at the numbers you can tell that +79,000 out of +91,000 is made up of just Ambient, Acurite, Davis, and LaCrosse (to a small part). Those four make up 87% of what is on WU (in the US).

* If you have an Acurite station please update your station type in Weather Underground so that we can start to differentiate Acurite by model. This is a newly offered WU Station setting option. See this link for details.
https://www.wxforum.net/index.php?topic=35425.0

PS - I also fixed Onset HOBO to include search for both instances of "onset" and "hobo."
« Last Edit: November 01, 2018, 09:48:35 AM by galfert »
WS-2902A | ObserverIP | WeatherBridge (Meteobridge)
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Tele-Pole flag pole is here (not installed yet)

Offline galfert

  • Forecaster
  • *****
  • Posts: 821
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #29 on: October 15, 2018, 12:43:48 PM »
Sorting on your raw data, looks like only 39 WeatherHawk stations are uploading their weather info to WU. I would have assumed many more than that but I guess it's because most sales are to federal, regional & local governments, universities, industry, emergency management entities and high-end residential automation.

I counted 54 WeatherHawk in both the August and the October data sets. Yes this analysis needs to be taken for what it is. This is what reports to WU and therefore is mostly the home owner and weather hobbyists, and there are some schools sprinkled in there too. But I think that is representative enough for the people browsing this forum and looking at what to buy for themselves. I don't think federal, local and regional governments, industry ..etc care what is on WU. This data is for us the hobbyists. But sometimes it is surprising to see some state run hardware that does show up on WU. I mentioned the Oregon Dept. of Environmental Quality, but there are others like New Jersey Dept of Environmental Protection, New Hampshire Dept. of Environmental Services...etc. But when those show up and on WU there is no way of knowing what hardware they really are using as it just shows up with those department names, because that is how they are reporting.





« Last Edit: October 15, 2018, 12:50:23 PM by galfert »
WS-2902A | ObserverIP | WeatherBridge (Meteobridge)
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Tele-Pole flag pole is here (not installed yet)

Offline havtrail

  • Senior Member
  • **
  • Posts: 57
    • Haverford Weather Station
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #30 on: October 15, 2018, 04:50:10 PM »
Great stuff!

So, if the brand names (as opposed to individual models) now closely represent the brand totals, then in the U.S. the leader seems to be Ambient (27,384 in Oct.) followed closely by Acurite (26,762) and then Davis (19,418). Right?

This is probably the best publicly available info on the national PWS market, which is at least 91,000+ on Weather Underground alone.

Rich K.

Yes, correct. The reason the brand doesn't exactly match the listed models mostly because there are other models not listed separately and because of obscure rare variations in station types with errors or omissions.

Too bad there isn't any data for other countries. I really would have liked to have seen analysed other countries. Also too bad that Acurite and WU don't list out model numbers separately. Seems only Davis and Ambient do this. Well I've seen the data and other manufactures also list out model numbers separately like Rainwise. I haven't bothered to include those nor others like Vaisala and Peet Bros..etc ....because nobody has asked. By looking at the numbers you can tell that +79,000 out of +91,000 is made up of just Ambient, Acurite, Davis, and LaCrosse (to a small part). Those four make up 87% of what is on WU (in the US).

PS - I also fixed Onset HOBO to include search for both instances of "onset" and "hobo."

Back in August when the original national csv file was posted thanks to "openvista," I downloaded it and sifted through it, finding a total of 66 Onset HOBOs, reporting from 23 states. Some are just listed as "Other."

I don't think the list's choices of models is defined by the manufacturers. I think it was, as you also guessed, originally a free-form entry by the individual registrants. Later, I believe, WU instituted the drop-down choice menu with options that they created themselves, to give some level of consistency to the data. I think they included what they believed were more popular specific models in their list of choices.

I had to register our HOBO RX3003 as a HOBO U30 (an obsolete model), because that was (and still is) the only HOBO choice available. At that time I was tempted to choose "Other" instead, but went with "HOBO U30" because at least it signaled "HOBO".

Rich K.
« Last Edit: October 15, 2018, 04:54:27 PM by havtrail »
Onset HOBO RX3003 Cellular
http://weather.havtrail.com
WU KPAHAVER17

Offline galfert

  • Forecaster
  • *****
  • Posts: 821
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #31 on: October 15, 2018, 11:33:58 PM »
Back in August when the original national csv file was posted thanks to "openvista," I downloaded it and sifted through it, finding a total of 66 Onset HOBOs, reporting from 23 states. Some are just listed as "Other."

How do you know they are Onset HOBOs if they are listed as "other"?

Quote

I don't think the list's choices of models is defined by the manufacturers. I think it was, as you also guessed, originally a free-form entry by the individual registrants. Later, I believe, WU instituted the drop-down choice menu with options that they created themselves, to give some level of consistency to the data. I think they included what they believed were more popular specific models in their list of choices.

I think some manufacturers must have some part in it though Take the WS-2000 that wasn't even officially released yet till late August and it was already a choice for model type with all of 12 stations. The WS-2000 did not fully go on sale till late August after the August WU list was pulled. This model was originally suspended for sale because a bug was discovered early on that if the SD card memory reached a certain level the device would get bricked. As a new firmware was worked on the sale was suspended. I think Ambient had to of informed WU early on that this model was new and for them to add it to the list. I think WU should clean up the list of bad choices, fix the typos and consolidate.

Quote

I had to register our HOBO RX3003 as a HOBO U30 (an obsolete model), because that was (and still is) the only HOBO choice available. At that time I was tempted to choose "Other" instead, but went with "HOBO U30" because at least it signaled "HOBO".
I would reach out to WU and see if they can add the Onset HOBO RX3003 model to the list.
« Last Edit: October 16, 2018, 07:24:06 AM by galfert »
WS-2902A | ObserverIP | WeatherBridge (Meteobridge)
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Tele-Pole flag pole is here (not installed yet)

Offline galfert

  • Forecaster
  • *****
  • Posts: 821
Re: Analysis of Number of Weather Stations by Brand / Type
« Reply #32 on: November 01, 2018, 09:55:13 AM »
* If you have an Acurite station please update your station type in Weather Underground so that we can start to differentiate Acurite by model. This is a newly offered WU Station setting option. See this link for details.
https://www.wxforum.net/index.php?topic=35425.0
WS-2902A | ObserverIP | WeatherBridge (Meteobridge)
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Tele-Pole flag pole is here (not installed yet)

 

anything