I'm not an engineer, but I would think from the reading that there are a limited number of ways for a humidity sensor, of the types that electronic stations use and excluding blonde hair or horse hair gizmos of the 40s, can work.
That being said, why would a sensor be substantially different from others, if the design were nearly the same. Why would one be more sensitive to high humidity damage than others?
I know that Young and Vaisala have some very long life, extreme conditions, types of humidity sensors. Are their construction that much more expensive than the glut of low cost things that are like the SHT stuff or the little almost pennies, ones for Arduino etc?
Anyone know the science behind it?
I am no engineer and am no expert on the science, but I am familiar with the history for Davis. Johnd and Pro Data used to have this very well documented, but I am not sure they still do. I also know that the old analog film capacitor element Davis used to use was superior to the current capacitor/digital technology, despite what the specs say. Why that is is a great question.
My first VP2 was circa 2005 and had the analog temp/hum sensor. On paper, specs weren't as good as they promote with the current SHT31. The specs for the temp sensor (a platinum wire thermistor) weren't as good and at the extremes, resolution in readings degraded. But in practice, I found these sensors to be superior in the humidity realm. The specs are still available online as +/-3% in humidity and +/-4% above 90%. In real world conditions, these sensors did the 2 things most people complain about with the current Sensirion lineup: they routinely hit 100% humidity and did not exhibit a wet bias as they "dried out" better when humidity dropped.
In 2006 I bought a second VP2 and noticed almost immediately that the humidity and dew point readings weren't as accurate (it would never hit 100% humidity and at lower humidity levels it had a considerable wet bias). This new station had the Sensirion SHT11 and Davis has been using Sensirion ever since. As far as I can tell, the weaknesses of these sensors continue now nearly 14 years later. As I also understand it, the reason for the switch was the obsolescence of the humidity sensing element for the analog sensor. Initially, the SHT11 did not have any coating protecting the soldering joints of the digital sensor and they had a rash of failures. Davis then began protecting the solder joints with a coating. Davis then briefly offered the improved-spec SHT15 shortly before going to the SHT31. By all accounts, the temperature performance of this sensor is outstanding, which is not insignificant. On the humidity side however, there is no way it lives up to the +/-2% humidity spec. The sensors are not designed for high humidity environments for prolonged periods of time creating an offset in the reading. Each chip is supposed to have a temperature and linear calibration correction for humidity loaded onto the chip. Maybe these are extremely accurate in a dry, laboratory setting, but these corrections are seemingly insufficient in a weather station. Plus many of us have experienced a further wet bias offset present specifically in Davis mounted sensors compared to homemade Sensirion sensors we've begun using.
As johnd has pointed out numerous times, aside from a few of us who are frustrated by the humidity performance, most users are probably either oblivious or indifferent to the humidity performance, so if it hasn't been improved in 14 years, I can't imagine many resources are being put into improving accuracy. Plus the other issue is compared to what? The VP2 is still a great weather station for the price and even the Rainwise brand has gone to the Sensirion sensor, so it does seem that the options for accurate, low cost humidity sensors designed for meteorological monitoring is quite limited.