WXforum.net
Weather Station Hardware => Ambient Weather and Ecowitt and other Fine Offset clones => Topic started by: dolfs on March 29, 2017, 04:24:06 AM
-
Over the past 2 years I have exchanged emails with Ambient (and occasionally directly with Fine Offset) about bugs. This was not a continuous conversation, but in spurts. I figured I'd do all of us a favor of collecting all bugs and issues that I know of in one place, so here goes (hope I did not forget anything):
Bugs: Time and timing related
- Realtime updates to Wunderground are attempted, even if no station is configured. Even if stationID or password were left empty, and therefore a request to WU could never succeed, the attempt was made nonetheless. Due to the failing request, retries are attempted every 2 seconds. Request plus fail response consume about 0.5KB and doing this every two seconds means 43,200 times a day, using 21MB each day. This is all wasted bandwidth and unwanted consumption, particularly important to those of us on slow or capped connections, and not using WU. This issue has been fixed in firmware 3.1.6 where no uploads are attempted if stationID or password are empty.
- When publishing sensor data to WU, the rapid-fire API is used with a parameter of "rtfreq=5", indicating updates every 5 seconds. The reality, see above, is more like 30 seconds. The parameter should reflect the programmed interval.
- I know the reporting interval has changed over various firmware versions, so if you are running an older version, you may see a different interval.
- The very first/many publishings of sensor data to WU may have an incorrect date/time. This is due to the ObserverIP setting its own internal clock based on the response from WU. Right after a reboot/power reset it is likely to not have the correct time (for me it is usually early Jan 1 of 2000) and will include an incorrect time stamp until it successfully takes the time from a WU response.
- Needless to say this can cause the data to be discarded by WU because its timestamp is nonsensical.
- MeteoBridge/WeatherBridge are not affected by this as they are better time keepers
- The delay to get the right time is not just until one successful response is received. I waited for 5 minutes and saw many success responses from WU, yet the time was still not correct. S power-reset does force this to happen sooner, but I have not been able to find consistent behavior!
- When changing units in the station settings page, it may take 3-5 updates of the LiveData screen before the new units are reflected. This may be just an inconvenience for users that do not use MeteoBridge, but can be a real problem otherwise.
- Over time the response time of the LiveData page seems to become progressively slower. This is not 100% consistent in terms of how long it takes before it becomes noticeable.
Bugs: Sensor value related
- When using the rapid-fire API, some sensor values are reported to WU as values with two decimal places. Yet the LiveData page displays only one decimal. Products like MeteoBridge, and WeatherBrige get values from the ObserverIP by scraping the LiveData page, and they loose that precision. May be not so important, but it sure is inconsistent.
- Internal storage/calculations for temperature are limited to an accuracy of 0.1C, equivalent to 0.0555F. Yet you will not see F values with a second decimal in the live page.
- All temperatures measured in C are converted to F using a formula that computes only one decimal. It is equivalent to computing more decimals, adding 0.05, and then truncation to one decimal. 20C is in theory 68.00F, and is reported as 68.0F (good), but 20.1C is in theory 68.18F, but reported as 68.2F (not good, 0.02 error), 20.2F is in theory 68.36F, reported as 68.4F (0.04F error). Consequently this can result in errors up to 0.05F. Noteworthy, but likely irrelevant.
- Indoor and outdoor temperatures are reported to the rapid-fire API, as well as the LiveData page using only one decimal, so data between the two is consistent, but subject to the stated error.
- Error is not the same as accuracy. Due to the above, you will never see certain F values and, generally neighboring values are 0.2F apart. The stated accuracy in the spec-sheet is 0.1F.
- Wind speed and gust speed is reported to rapid-Fire with two decimals, but we have only one decimal in the LiveData page. One example is a measured wind speed of 2.7 m/s, in theory 6.03970 mph, reported wind speed is 6.04 mph (rapid-fire), but shown as 6.0mph in LiveData. As far as I can tell, the values converted from m/s are correctly rounded to two decimals (unlike the rounding for temperature). The maximum observed error I have seen is close to 0.05 mph, so the last digit of two is not all that reliable. The specified resolution is 0.1mph when, in fact it is 0.1m/s which amounts to 0.045mph, which is twice as good as stated on the spec.
- I have not researched in as much detail, but pressure is measured in hPa and converted to whatever you had configured. Specified resolution is 0.01 inHg, which amounts to .339 hPa. The spec sheet on the fosh.com site lists 0.08 inHg (or 2.71 hPa) instead (very different, same hardware). I suspect, but have no evidence, that the real value is 0.25hPa, which corresponds to 0.0074 inHg. This is reported to rapid-fire with two decimals (for inHg), and shows in LiveData with two decimals as well. When reporting in mbar (LiveData only, rapid-fire always gets inHg), it still shows two decimals, but the second one is always 0. Since I see various decimal fractions for hPa displays, I conclude that true resolution is sub 1hPa (0.03 inHg).
- Some users have reported "missing values" for the internal pressure sensor. These are visible as gaps in the pressure graph on WU. In particular these issues have been reported since WU changed to a new server farm using Amazon AWS. This has been attributed (by Ambient) to the new servers being noticably slower than the old situation and a timeout occurring in the ObserverIP causing the internal sensor unit to not be serviced on time. Subsequent firmware updates attempted to solve the problem by increasing the timeout values (3.1.6 is the first one that supposedly really fixes this, see above). This appears to have mostly alleviated the problem, but as has been explained to Ambient, this is not a good/great solution because while the OIP waits for a timeout to occur, nothing else is serviced, including web pages (see MeteoBridge). A better solution, with technical detail, has been supplied to Ambient/Fine Offset, but so far has not been implemented, nor is there promise to improve this.
- It has also been suggested that this can be due to placement of the indoor sensor unit, in particular if it is too far from the OIP. It is my opinion that if the unit is too far away it would show as trouble with all sensors in the unit simultaneously, and not just pressure, but I may be wrong. What constitutes too far depends not only on distance, but radio frequency transmission characteristics and obstructions in your particular situation. In my case 30 feet and one wall in between seems to be just fine.
Bugs: UI
- Values entered on the calibration page or not always echoed back after submitting. There are sometimes subtle changes that are confusing. See discussion below.
- While it is possible to change the WU stationID/password values in the "Weather Network" tab, it is not possible to completely remove them. Inputting an empty field for either results in an error message. Yet there are cases where this exactly what one wants to do (see introduction above). Currently the only way is to reset to factory defaults, which means all units and calibration values have to be re-entered. This is at the very least cumbersome, and good luck if you forgot to note them down first.
Bugs: Network related
Much of what I discuss here can only be observed by "packet sniffing" on the network, or sometimes indirectly if you are measuring data usage.
- It appears that the TCP network software stack in the ObserverIP produces 5-8 retransmits of the upload request to WU each time an attempt is made to upload. I have not done sufficient analysis to determine what the exact reason for the retransmit is. I have simulated the very same upload request using "curl" on my desktop Mac and observed none of the re-transmits. The only real difference in these scenarios is the TCP/IP network stack, which is why I suspect the ObserverIIP.
- The ObserverIP does not implement an NTP (Network Time Protocol) client. If it were it would not have to rely on getting the time from WU. This is important because some users, like myself, do not wish to configure a stationID and thus no calls to WU would be made. In such scenarios the unit's internal time would always be incorrect.
- I have suggested that an NTP client function be implemented. It would also be good to allow the user to configure which NTP server to use, but a suitable default might suffice. This does not have to be a full blow client (that would smoothly manipulate the system clock to avoid sudden jumps and discontinuities). A simple client, executed once every so often to set to the correct time would likely do a sufficient job. How often would be determined by the drift characteristics of the internal clock (unknown to me), but I suspect once a day might be enough, so it would not be a lot of network overhead. It would also fix the issue of incorrect timestamps on initial contact with WU.
- An alternative work-around that I have suggested is to not require that the response from WU is successful before extracting the time from it. Even an "invalid password" response carries the time header, so it could be used. This would allow a user such as myself to configure a bogus stationID and still get time updates. The downside would be that this call is now made every 30 seconds again, wasting lots of data per day, which could be hurtful to users on slow speed or capped connections. You might ask why not simply upload directly from the ObserverIP to WU. I will answer that in the "Observations" section.
Issues/Observations
- If/when station is correctly configured, uploads to WU appear to happen about once every 30 seconds, which is less than stated on the product specifications: "14 second real-time updates on the ObserverIP and the internet". It also does appear to be different from the update frequency of the LiveData page.
- The calibration page offers input fields so the user can enter corrective values for various sensors. It is very hard to use this page for several reasons:
- Each input field states acceptable range multiple times for different units. The user must remember what units were set in the station settings page so as to know what the correct value/range is to use here. It would be much, much better, if this page only display the range for the currently selected units.
- Likewise it would be much better if the LiveData page would show the units with each value.
- Due to the formulas used for conversion from the internal storage units (which I suspect to be temp in C, pressure in hPa, wind in m/s, humidity in integer percent) to the display units, and back from inputted values to internal storage, truncation and rounding errors happen. See also remarks above about sensor value issues. The back and forth conversion exacerbates the problem beyond the one for sensor values. As a result it is possible to input values that are inside the stated range, within the stated resolution, yet when submitted a slightly different value is echoed back. It is not clear to the user if the displayed value is the one actually being applied (I am pretty sure sometimes it is not). It is also sometimes possible to enter a slightly higher value than desired, only to have the re-displayed value to be the one desired. Uncertainty still exists though.
- For example an offset entered as 0.1F becomes 0C stored after truncation, and therefore 0F after display. 0.2F as input becomes 0.1C stored after truncation and therefore 0.1F for display. Yet 0.3F becomes 0.1C stored, thus 0.1F on display and 0.4F becomes 0.2C stored, and 0.3F on display. You’ll find that this formula accounts for all the displayed offsets you may encounter. It seem fairly obvious that the above is a problem for any user who chooses to work in units of F because effectively they cannot enter any offsets such as 0.2, 0.4 etc. Yet, according to the product specification sheet, the temp resolution is 0.1F
- There is also a user interface problem. When I enter a desired temp offset of 0.4 the resulting display is 0.3, as explained above. If I now go back to try and input 0.3, the resulting display is 0.3, which is inconsistent with the above, because I should have gotten 0.1. I believe this because of a misplaced optimization where nothing is done if it appears the user did not actually make a change. This is further confirmed if I enter 0.31 instead. Textually this differs from 0.3 and so the optimization does not kick in, calculations are done, and the result is as expected because 0.31 would convert to 0.1C (stored) and thus 0.1F displayed. The 0.3 should have been treated as if it was a completely fresh input and resulted in 0.1. Of course the optimization would be valid if the problem with rounding/truncating sketched above would not exist.
- Next I set my units to F and since I had originally entered a -0.4F offset, it had been displayed as -0.3F temperature offset. I then changed units to C and went back to the calibration page. The offset was now displayed as -0.2. This is consistent with the discovery above because the -0.4F would have become -0.2C stored. When the units were changed, the value of -0.2 showed (no further conversion needed). Changing the units back to F shows -0.3 again, but of course the intended -0.4F is gone forever.
From the observed behavior it also appears that calibration values are stored as strings, or fixed point numbers with only one decimal. It should be considered to store them with more precision, and still display them rounded.
- On the LiveData screen battery status for indoor and outdoor are shown as both hexadecimal values and textual labels. Even when batteries are good, my unit consistently displays "0x69/LOW STATUS" for indoor and "0x65/Normal" for outdoor. In particular the "LOW STATUS" is alarming to the user. Inspection of the indoor unit's display does not reveal the low battery indicator there so this interpretation of the hex value seems in error and unnecessarily alarming, or certainly unhelpful. In addition, the possible values and labels are not documented as far as I can tell
- The "Local Network" page has a setting for "Server Listening Port" with a default value of 5000. This is completely undocumented and certainly does not correspond to the web server port, which is the default port 80. I suspect this has something to do with the ObserverIP hardware being able to act as two virtual serial ports over Ethernet (presumably over this port), where each serial port would be an interface to the radio dealing with indoor and outdoor unit. Having access to these serial ports can be potentially very useful to products like MeteoBridge (see below), and should either be documented, or the setting should be completely removed to avoid confusion. It may also require commands to be removed from the interface available via telnet.
- Why not report directly to the WU? It has been my experience during calibration of my Ambient WS-1400-IP that the calibration method offered by the calibration page (offset only) is unsufficient. I found that I would get better results if I could apply a linear correction (offset plus slope). The ObserverIP is not capable of this, but in my case MeteoBridge is. The same might be true for other "external" solutions. The other issue is that I (and many others) not only report to WU. I, for example, also report to CWOP and MADIS. Surely one would not want to report different values to WU than to CWOP. This can be more or less guaranteed if you let all uploading be handled by MeteoBridge, but that requires you shut off the uploading directly from WU.
Such uploading, while useful for more casual users of the product, is not good for more serious users. - What about this calibration? I used a linear correction with clamping (to make sure values stay in range). So to implement this in the ObserverIP would complicate the UI quite a bit, and I am not advocating it. If you are sophisticated enough to want this kind of correction, you probably also wants something like MeteoBridge.
MeteoBrige/WeatherBridge
This is, as far as I know, identical hardware, and basically identical software, except for branding, and perhaps different versions. Caveat is that I don't have a WeatherBridge (it is several times more expensive than buying a TP-LINK TL-MR3020 and licensing MeteoBridge for it), so I am guessing here.
The manner in which both get information from the ObserverIP is to act as a web browser and request the LiveData page. Values are then "scraped" from this page. In order to get the interpretation of the values right, the station settings page is also "scraped" to determine the unit. I am not sure if the latter is scraped each time before the former is.
A possible issue exists here with respect to the delay in LiveData reflecting newly selected units. The 3-5 screen delay (around 1 minute) before new units show up create the possibility that MeteoBridge misinterprets 2-4 screens. In real life this depends on when exactly the MB calls up the LiveData page, and how frequently. I would say it is somewhat likely, but you are probably not changing the settings much, so practical impact may be low. It might be even less of an issue if MB realizes that the converted values appear abnormal and ignores them. Still, it is a problem that would not exist if the LiveData page would always use the current units.
The slower and slower responses from the LiveData page can become a problem for MeteoBridge resulting in sensor values of "--". Only a reboot of the ObserverIP seems to solve this.
The method whereby MB has to screen-scrape web pages from the ObserverIP is extremely cumbersome and potentially error prone. It also wastes bandwidth although this is likely not significant for an in-home network. It would be much, much better, if there was a JSON-API endpoint that would deliver all sensor values, and their units, in JSON format, in a single go. This should not at all be hard to create. Alternatively, documenting the serial port over Ethernet protocol and the packet format from the radios, would allow MB and others to do their own direct interpretation of the sensor values. Even better would be the inclusion of a reliable timestamp in the response, where the timestamp would correspond to when the sensors were read out. If an external application such as MB reads data too often, the timestamp would be the same in subsequent calls, and duplicate data could be avoided.
MB should be receiving the higher decimal precision that is provided to WU. This can be done either by making sure that the LiveData page uses the same precision, or (in addition) in an API implementation.
Final Notes
If anybody knows of any other bugs or issues, you can PM them to me and I can edit this post to stay up to date.
-
Thanks for the detailed write up.
-
More info:
- LiveData Page
- In version 3.1.6 the LiveData refresh interval is 16,000 ms or 16 seconds
- LiveData page refresh time (not the interval, but the actual loading of the page), is very slow at times.
- In some cases (I am using Safari), the page itself loads, but the 3 dependent files (see below) fail to load, causing the JavaScrip on the page not to execute, and thus refresh stops altogether. My research seems to indicate this happens in particular if more than two web requests are happening at the same time. This can easily happen if one requests the page in a browser while at the same time MeteoBridge is requesting a page for scraping.
- Once such a "dual" request is causing one of them to hang waiting for a long timeout, no other web requests can be handled either. So clicking around in the browser on various tabs will all cause requests to "hang"
- I suspect all of this is the direct result of the code being single threaded (as stated to me by Ambient).
- The LiveData page requires loading of 3 files (in fact all pages load these 3):
- The Ambient Logo. This file is, essentially constant, although it could change with a firmware update. Its file is not marked as cacheable, causing each page to have to fetch it. It would be easy to supply caching headers with a time to live of, let's say, 1 day.
- A css style sheet. Same considerations as the image. It is not currently being cached.
- A JavaScript file. Same considerations as the image. It is not currently being cached.
- The CSS and JS files contain comments in them, making them larger than necessary. Even without comments, the files are not "minimized", so this should be considered for optimization.
- Wind speed
- The working of the anemometer is that as the cups rotate, the shaft rotates. A magnet(s) and a coil(s) then produces a AC current which is converted to a series of pules that can be counted. The higher the wind speed, the more pulses per measurement interval. There is also a minimum wind speed required (so called "lift-off") before the cups will even start spinning. This is stated (by Ambient) as about 2.2mph or 1 m/s (for the unit associated with the WS1400-IP). The lift-off limitation is usually caused by inherent static friction in the bearings of the shaft, and therefore lower lift-off speeds are associated with higher quality bearings (and thus increased cost).
- Specification wind speed accuracy is the greater of 2.5mph or 10%, and resolution is 0.1 mph. Comments from Ambient say it is actually "about 2.4mph, and 0.1mph". The FineOffset page specifies an accuracy of 1 m/s, which corresponds to 2.24 mph. We know this is all the same hardware, so which is it?
From my measurements, I can see values such as 0, 2.5, 4.9, 7.6, 10.1, 12.5, 15, 17.4, 20.1, or 22.6 (mph). The interval between successive values seems to be either 2.4, 2.5, or 2.7. If there were just 2.4 and 2.5 I could attribute it to a rounding factor oscillating between the two. The third case, however, makes it hard to figure out what is happening. It is not the case that the interval increases as windspeed does, because after I see the 2.7 interval I see 2.5 and 2.4 again. Based on observation (which may be particular to my unit), I would say practical accuracy is 2.7mph or 1.21m/s. - The accuracy is limited by the lift-off speed and, as the speed increases, by the electronics that convert the AC current into countable pulses, which is approximately a percentage of the speed (current). The resolution, stated as 0.1mph (nothing stated for Fine Offset), corresponding to 0.04 m/s. The latter would suggest two decimal positions are warranted, and only one is provided on the LiveData page (for units set to m/s), but again, two decimals are provided to WU rapid-fire.
- It appears that the measurements are produced by counting pulses over an interval much shorter than the reporting interval of 30 seconds. Each such count then provides an "instantaneous" wind-speed value. This is repeated a number of times over the reporting interval. The maximum value seen is reported as the wind-gust speed, the average of them is presented as the wind speed. This is fairly evident from the fact that measured wind gusts are 0.0, 1.1, 2.2, 3.4, 4.5 (m/s) etc. These can be reproduced with an interval of 1.118 between them, with results rounded to one decimal. 1.118 m/s of course is the equivalent of 2.5mph. Therefore I am convinced that the read-out resolution of the pulse counter corresponds to 2.5mph.
- I observe wind speed (average) values of 0.0, 0.3, 0.4, 0.7, 0.8 m/s (not necessarily all inclusive) for a gust value of 1.1. The 0.1 increments imply that if the highest wind seen is 1.1 over the reporting interval, there must have been at least 10 measurements during the reporting interval. It also explains why the listed resolution is 0.1 m/s (or 0.22 mph). Ambient specifies 0.1mph so either that is wrong, or the actual number of samples taken is greater than the 10 mentioned above and the increments would have to be 0.05 m/s or less to support the stated 0.1 mph. That too, would speak in favor of two decimal digits in reporting.
- My unit is clearly capable of reporting gust readings of 1.1 m/s as the lowest value, which leads me to believe the lift-off is 1.1 m/s or less (2.46 mph), and possible increments are also 1.1 m/s. @kmahler reports a lift-off of 4mph. I assume that is based on the reading from his hand-held unit. If his WS1400 also reports 4mph for that occasion, his unit is different from mine. I also noticed his WS1400 kicking in once when his digital display showed 1.1/0.9 so that would suggest his conclusion about 4.0 mph is not completely reliable. It seems to me that his unit exhibits "sticking" at very low speeds (abrupt halt of the rotation), and if this stick is somehow position dependent, it could explain this. Either way, that particular unit is no good.
- If the number of samples per reporting interval is 10 or less, one decimal only is justified in m/s results, but would require 2 in mph results. If the number of samples is larger than 10, perhaps these numbers should be 2 and 3 decimals respectively.
-
I have been working with Fine Offset engineering supplying them with packet traces etc., and have been testing a newer version that appears to have solved the network problem where uploads to Wunderground are repeated many times. Still testing a little longer, but appears to improve the volume of traffic sent over the network to WU by a factor of about 6-8 (without reducing the update frequency).
-
More info:
I am having to add things as replies rather than amending the original post because of the size limits on posts. I'll be using WU for Wunderground.com, and OIP for ObserverIP.
- Networking
- I discovered that for more data is transmitted to WU than necessary, detailed in a prior message. Data was on the order of 6-8 times more voluminous than required. This has been fixed in an upcoming firmware release.
- Once the TCP session for the HTTP request/reply to WU has been completed it needs to be closed down. Current firmware versions do not close the network socket, although eventually after a timeout, WU decides to do it from their end. This causes unnecessary socket resource consumption on WU (the timeout seems to be around 2 seconds), which is, at the least, not "nice" to them. Improvements are currently being tested by me.
- This issue also affects any browser or device (i.e. MeteoBridge) that requests web pages from the OIP (such as LiveData and StationSettings). Nonetheless, it does not appear to do so in a manner that causes things not to work, but it may lead to some unwanted delays.
- Communication to WU happens using http, rather than https, and consequently your stationID and password are communicated in plaintext and are subject to "theft" by anyone using a network packet sniffer. Fine Offset has been informed about this.
- I have confirmed WU can take the update as an https request
- Introduction of https (if possible) will necessarily increase the complexity and data volume of sensor uploads to WU.
- This may be undesirable, so I have suggested that if this can be implemented at all, it should perhaps be a user configuration choice.
- The WU API is not "strong" enough to allow a better solution where a device such as OIP can use https to negotiate a session specific security token with limited time validity, (e.g. OAUTH) and then use it for http uploads. Even if stolen, the usefulness of such a token would be limited in time. WU API does not offer this.
- OIP performs DNS lookup of WU server IP address once at startup time and then never again. This means that if WU changes their server address information in DNS, the OIP will not see it, unless the unit is rebooted. This is not in conformance with DNS protocol and WU is, in fact, stating (currently) that DNS lookups should be repeated at least once a minute. It is also an explanation for all of you who experienced "unit is not reporting" messages from WU around the switch over to Amazon in early March: their DNS info had to drastically change, and running OIP would not "see" that. Reboots would have fixed it. Fine Offset has been made aware.
- Time keeping
- The OIP sets its clock from the timestamp included in successful replies from WU, detailed in a prior message. While workarounds are being researched for those of us who do not (wish to) use the uploads to WU, a problem exists that I have not identified until now.After a fresh reboot the time in the OIP will not be correct until at least one successful interaction with WU. The problem is that for each interaction a timestamp is needed that identifies the time associated with the sensor data being uploaded. Necessarily then, the timestamp in the first (several) updates to WU will be incorrect, until the time has been successfully extracted from WU.
- I have proposed to Fine Offset that they implement a (simple) NTP client to get the time. This can work because they have confirmed to me that the OIP has an on-chip RTC (Real Time Clock). As long as updates to WU would not be sent until such a time update had occurred, many issues would be resolved. Fine Offset looking into this.
- WU uploads
- After the OIP establishes a TCP session for making the HTTP request to WU, there is a several second delay before the actual request is being made. This consumes unnecessary resources on the WU side and makes the updates come in slightly delayed. I am currently testing an improved version.
- There is a small bug in the formatting of timestamps as reported to WU when the second, minutes, or hours (in UTC) are < 10. Formatted time stamps leave out the leading zero. It is currently not known if this causes an actual problem with WU. Users that use an alternate delivery host (can be set through telnet) should be aware of this though.
- Some of the issues discussed cause variable delays in overall OIP operations. This can manifest as OIP occasionally missing an update to WU so that the update interval seems to be more like 30 (32) seconds than 16. This, while perhaps undesirable, is not fatal. It can also manifest itself as delays serving web pages for the OIP, whether to the user or MeteoBridge/WeatherBridge. In fact, the requests coming from such devices may itself causes delays as well, and MeteoBridge for example, seems to poll OIP about every 5 seconds. Some of the improvements Fine Offset has been working on, in my testing seem to show better performance in this regard, but I have no absolute measurements of this.
- Sensor data
- After I wrote my observations around the way the anemometer sensor works and produces data, I discovered some excellent research by another member of this forum (http://www.wxforum.net/index.php?topic=28713.msg278935#msg278935). It includes having peeked at the actual hardware. What I found seems consistent with what he found and can be summarized as follows:
- Sensor package reports to OIP every 16 seconds (confirmed to me by Fine Offset).
- First 8 seconds of this interval nothing is measured (confirmed to me by Fine Offset).
- In the next 8 seconds the total number of revolutions is counted (1 pulse per one revolution), as well as the highest number of pulses in each of the 8 one second intervals. The latter is reported as wind-gust value, the former divided by 8 is reported as wind-speed.
- The sensor package can be configured for an 8 second interval (at manufacturing time only it seems) (confirmed to me by Fine Offset), but it is not (yet) clear why the product is configured for 16 seconds.
- One pulse corresponds to 1.118 m/s, equivalent to 2.5 mph. Thus the accuracy of the wind-gust in the last 8-seconds is 1.18 m/s, although average error will be half that. The linked article says one revolution is 1.1 m/s. This could be because that result is less accurate and essentially is the same as mine, or because it is actually different. The observed values seem to indicate 1.118 is correct (based on observed values of 0, 2.5, 4.9, 7.6, 10.1, etc.), although there is some strange rounding issue in the reported values.
- Wind-speed, being an average over 8 seconds therefore has a resolution 1/8 smaller, or 0.125 m/s or 0.28 mph and an average error of half that: 0.14 m/s.
- Since no measurements are done in the first half of each 16 second interval, it is possible to completely miss a wind-gust in that half. Basically, the one should realize the data only represents the last 8 second interval immediately preceding the report. On the other hand, since the sensor values are reported to WU along with a time stamp corresponding to the end of the 16 second interval, the data is not necessarily incorrect. Since WU does not associated the data with a time interval, but rather with a moment in time. The absolute truth is somewhere in the middle.
- MeteoStick/Airbridge
- MeteoStick (http://www.smartbedded.com/wiki/index.php/Meteostick) is an product alternative to ObserverIP. It can receive the communications from the WH24 unit (sensor package for among others the WS-1400-IP), but it cannot communicate with the indoor sensor unit. MeteoStick will not be sold to US customers, but Ambient Weather sells this unit as part of its AirBridge-Receiver WiFi (http://www.ambientweather.com/amairbridgereceiver.html) package.
- To replace functionality of the indoor unit, it seems MeteoStick has a built-in temperature and barometric pressure sensor.
- Downside for some may be price (currently around US$ 170 for MeteoStick and US$ 278 for Airbridge), and that the MeteoStick needs to be plugged into the MeteoBridge device, directly, or via USB cable, so location may not be what you want for indoor temperature (pressure should not really matter).
- Implementors of MeteoStick obviously must know the packet protocol of the frequency hopping sensor package (not just Fine Offset, but also Davis Instruments), and they obviously do. On the other hand, these formats are not published officially by the manufacturer, or the MeteoStick creators. Since MeteoStick was licensed to Ambient, it would seem that this is how the MeteoStick implementers got the necessary information from Fine Offset. There are various internet articles describing reverse engineering efforts.
- I do not own either of the two units, or have access to them, so the above is from Internet research only, and it also means that I do not know if any issues similar to what I have been reporting here apply. It would appear however that any of the reported issues that have to do with fundamental capabilities or limitations of the sensors do apply.
-
Version 3.1.8 of the firmware has been released:
- The networking code was changed so that a single upload to wunderground.com only consumes 1/8th of the amount of data, and happens faster, and releases network resources sooner. This should also help improve overall responsiveness
- If stationID and/or password are left blank in the configuration, there will no longer be any (repeated) attempts at uploading to wunderground, as this would be guaranteed to fail anyway. This is useful if you do not wish to uploaded to wunderground directly from the ObserverIP, perhaps because you are a MeteoBridge or WeatherBridge user.
- The code that sets the time based on responses from wunderground.com has been changed to change the time regardless of success or failure of the request (failure because of incorrect stationID or password). You can take advantage of this if you do not normally upload to wunderground from the ObserverIP (feft stationID and password blank). Set an incorrect stationID and password, and the requests will be made, will fail to actually upload anything, but the response will be used to set the time. Unfortunately this will still happen every 30 seconds. A suggestion has been made to Fine Offset to reduce the frequency for this scenario to as little as the accuracy of the onboard RTC clock allows.
-
Version 4.0.1 was released and in limited testing, I found:
- If you have no WU station configured (which will be the case after upgrade) and want to set the upload for ambientweather.net to "no upload" the form will give you error messages and you effectively cannot do it. To get around it, enter non-sense stationID and password for WU. Side effect will be bandwidth consumption with WU. This has been reported to Ambient.
- It seems that the outdoor sensor readout code is not working correctly. Compared to 3.8.1 I noticed that my graphs on WU tended to be fairly smooth and (generally, with few exceptions) monotonically increasing during the day, and decreasing from late afternoon. With 4.0.1 I see a lot of "jittery" behavior where, in the space of 2-3 minutes the outdoor temperature will jump up and down by 0.5 to 1F from one value to the next but over several samples I see even larger swings. In reality, the temp is not fluctuating like that. I obverse the values in the "Live Data" page and confirmed it is happening on (inside) the OIP unit. I downgraded the 3.8.1 and the behavior went back to normal smooth. I conclude that the OIP firmware is the source of the problem and exclude WU or MeteoBridge as culprits. This too has been reported. I will remain on 3.8.1 until this is resolved.
You can see the "jittery" behavior starting at 5:30PM on this graph: WU graph for 4/27 (http://"https://www.wunderground.com/personal-weather-station/dashboard?ID=KCAMOUNT75#history/s20170427/e20170427/mdaily") and continuing in the next several days. Then starting around 9PM on this day I switched back to 3.8.1: WU graph for 4/29 (http://"https://www.wunderground.com/personal-weather-station/dashboard?ID=KCAMOUNT75#history/s20170429/e20170429/mdaily"). Night time is not so great to see the effects so look at the next day and see how smooth things are again.
-
Version 4.0.8 was released.
Notable changes:
- If WU stationID and password fields are left blank no error is generated (as it was before). This allows a setup where there will be no upload attempts to WU at all. Previously this could only be achieved after a factory reset, which would clear the fields. Once the fields were set, they could no longer be cleared (due to the validation error message that would not allow empty fields).
- As previously explained, successful (no timeout) requests to WU are needed to properly set the time on the ObserverIP. Since the feature was introduced where you can upload to ambient weather.net as well, or instead, time can also be taken from there, so as long as at least one is configured and working, your time will be correct(ed) shortly after a reboot. If you do not wish to upload to WU.com and also not to ambient weather.net, leave the WU.com fields blank and set the AW.net popup to "no upload". This will put the ObserverIP in a mode where it will still make a request to WU four times a day, at midnight and then every 6 hours, on the hour. The request itself will fail to store data at WU.com due to the blank stationID and password, but the time will still be delivered in the response and will be used to correct the RTC on the ObserverIP. The 6 hour interval was chosen to keep the time correct to one second or better within the known accuracy of the on-board RTC.
- Overall performance was improved and stabilized with respect to the ObserverIP web page serving. This is not as important to a casual user, but is quite important to users of MeteoBridge or WeatherBridge. Previously I had observed occasions where the OIP performance would slowly degrade and response times to the MeteoBridge (hard wired using 100Mb connection on same network) went from a routine 3-6 ms slowing increasing until it would exceed 100 and then 200ms. This behavior has stabilized and I consistently see 3-6 ms responses.
- WU.com update frequency has been reduced to about once per minute (although it is not precise), from a previous 30 second interval and an advertised 16 second interval. This was ostensibly done to create more stable network behavior. In my opinion it effectively works around a bug in the code, at the expensive of this update interval. I also believe the problem might not exist in scenarios where either WU.com or AW.net would be the sole upload destination (i.e. not both), but I have no solid evidence of that.
- Despite the update frequency/interval for both services now being at once per minute (or longer for AW.net if configured like that), the OIP still receives data from the indoor and outdoor sensor packages more frequently, approximately every 15 seconds. This will be reflected on the LiveData page, for example. It also means that, once an upload occurs, the data is generally not older than 15 seconds.
- A bug was fixed where if both uploads to WU.com and AW.net were configured, the latter would be sending a request body of non-zero length containing partial content from the request headers to WU.com (shared buffer overwrite with incorrect length management). This appeared to not have any negative consequences, but it was sloppy and has been fixed.
Aside from testing that I perform for Ambient and Fine Offset, I operate my OIP without uploads active to either AW.net or WU.com as I use my MeteoBridge to do this instead. This way all uploads, including those to CWOP etc. will use the same sensor values, as they all go through the same calibration curves and altitude setup. MeteoBridge is able to read sensor values from the MeteoBridge ("scraping" the LiveData page) much more frequently, to take advantage (if you are so inclined) of the fact that the OIP internally still receives and updates the outdoor and indoor sensor values approximately every 15 seconds.
-
This thread did not get enough recognition for the amount of time and effort put into all of this. I want to say thank you for not only tracking all this but for working with the manufacture. So what is the status of the ObserverIP and its latest firmware? Maybe I missed a newer thread with more current updates and if so please link that as a response.
Thanks!
-
I noticed at midnight Pacific time today (June 12th) my ObserverIP started misbehaving. It has been "rebooting" itself constantly and only reported a dozen or so times to ambientweather.net and to my Meteobridge. It's bad enough that I generally can't even connect to its webpage.
On power cycling, I could connect to it for about 30 seconds before it locked up again and eventually rebooted.
I traced the issue to something with publishing to ambientweather.net. When I set that setting to "never", the ObserverIP s back to working correctly again and reporting data through my Meteobridge.
I contacted Ambient support and let them know what happened to me. If I hear back, I'll post the response here. If anyone else is seeing similar issues starting today, I suggest disabling your ObserverIP's reporting to Ambient Weather for the time being.
Edit on 6/13/18:
I heard back from Ed that this is a known issue. He suggested that I downgrade my firmware to 4.3.6.
-
We're testing an experimental new FW right now so that you'll be able to keep the other fixes and improvements made through 4.4.3 and keep publishing to AW.net. We'll let you know as soon as it is confirmed good.