Topic: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?

[drsprite]

I received an Ambient Weather WS-1200-IP for Christmas. It's my first weather station and so far I am having fun with it. I'm already setup with it sending data to Weather Underground which is great, but I'm a tinkerer, and I can't help but to want to get more out of my new you.

So I set up a Python script to screen scrape the sensor values from the ObserverIP's livedata.htm, and this is all well and good, but it's not quite what I'm after. I'm thinking that due to the 16 second polling interval, my Python polls could be mis-timed and I could have duplicate data.

So this brought up a couple of things, and am looking for feedback:

I see Ambient Weather sells a product called WeatherBridge. They claim it's compatible with the WS-1200-IP. The WeatherBridge is simply a TP-Link MR3020 (Meteobridge points this out, which is awesome because I already own one). Does anyone know exactly how this device communicates to the WS-1200-IP? The MR3020 doesn't have a 915mhz antenna built in, and there's not much room to add one. My guess is they're screen scraping the ObserverIP as well?

I'm also wondering about just intercepting the packets on the 915mhz band and decoding them directly from the device. I have my eyes on a Moteino with a 915mhz antenna and decode packets.

Anyone else do packet decoding on the WS-1200-IP?

Anyways, just looking to see if there's other tinkerers out there with this same setup!

[SpringHillWeather]

You need purchase the IPObserver module.

http://www.ambientweather.com/amobserverip.html

This captures the data from the station and then passes it to the Meteobridge.

Here is a guide which now includes the Meteobridge PRO method.
http://www.ambientweather.com/amweobsefeco.html

[drsprite]

You need purchase the IPObserver module.

http://www.ambientweather.com/amobserverip.html

This captures the data from the station and then passes it to the Meteobridge.

Here is a guide which now includes the Meteobridge PRO method.
http://www.ambientweather.com/amweobsefeco.html

Thanks

The Ambient Weather WS-1200-IP comes with the ObserverIP module already, and I'm doing screen scraping already of the livedata.htm. I just wasnt sure if there was another way to get the data off the unit. Maybe more reliable (and cheaper). To get the data off 915mhz directly, a moteino is only $25. Not $300+ like what Ambient Weather is looking for.

[redvers]

You need purchase the IPObserver module.

http://www.ambientweather.com/amobserverip.html

This captures the data from the station and then passes it to the Meteobridge.

Here is a guide which now includes the Meteobridge PRO method.
http://www.ambientweather.com/amweobsefeco.html

Thanks

The Ambient Weather WS-1200-IP comes with the ObserverIP module already, and I'm doing screen scraping already of the livedata.htm. I just wasnt sure if there was another way to get the data off the unit. Maybe more reliable (and cheaper). To get the data off 915mhz directly, a moteino is only $25. Not $300+ like what Ambient Weather is looking for.

There is a cheaper alternative, if you can get it working... There is a software package called weewx which normally would connect directly to the weather console connected to your PC. Someone has developed an ObserverIP driver, allowing it to connect to the receiver module & get the required readings that way!

While I have been able to get it to see my WS-1400-IP, I haven't yet been able to get it to read the data from it

[drsprite]

You're right. I've been using weewx for a while now, and the ObserverIP driver is just a screen scraping method to capture the values from the livedata.htm - same thing as what I'm already doing (I wrote my own driver for weewx for the ObserverIP).

I think what I'm finding out is that this screen scrape method is really the only way to get data off the ObserverIP.

I think I might get an arduino and a 915mhz transceiver and see if I can get the packets directly off the air from the WS-1200-IP itself.

[redvers]

You're right. I've been using weewx for a while now, and the ObserverIP driver is just a screen scraping method to capture the values from the livedata.htm - same thing as what I'm already doing (I wrote my own driver for weewx for the ObserverIP).

I think what I'm finding out is that this screen scrape method is really the only way to get data off the ObserverIP.

The one I have tried getting working was written by David Malick & amended by Matt Wall; I'm not sure how it works to be honest, haven't had chance to dig through the code to see why I get errors, let alone to figure out how it works

Given that the ObserverIP allows you to specify the URL/port for reporting, I wonder how difficult it would be to actually build a listener on your own server, so instead of trying to capture the data at the right moment, it gets pushed to you at the appropriate time...

[drsprite]

The one I have tried getting working was written by David Malick & amended by Matt Wall; I'm not sure how it works to be honest, haven't had chance to dig through the code to see why I get errors, let alone to figure out how it works

Given that the ObserverIP allows you to specify the URL/port for reporting, I wonder how difficult it would be to actually build a listener on your own server, so instead of trying to capture the data at the right moment, it gets pushed to you at the appropriate time...

Yeah, I've tried Matt Wall's forked driver, but it gave me errors as well, which is why I wrote my own based on theirs. Basically it just loads livedata.htm, and grabs the weather values and sends them to weewx. Then weewx repeats the process every 15 seconds. It's not ideal, and the driver likes to crash overnight (which kills weewx). So it's not a reliable solution 

Having the data pushed to me is ideal, but I don't see what you mean about the ObserverIP allowing you to specify the URL for reporting. I do see a configuration option for "Server Listening Port", and mine is set to 5000, but there's no TCP/UDP data I could find on that port - so I'm not sure what that option is for.

[redvers]

This may be a difference due to the Aercus version of the firmware; here's what I get on my UK model :



The server type allows for PHP, JSP or ASP. This to me suggests that I could have my own weather server listening out for updates from the ObserverIP, although what the communication protocol is I have no idea!

[redvers]

Yeah, I've tried Matt Wall's forked driver, but it gave me errors as well, which is why I wrote my own based on theirs. Basically it just loads livedata.htm, and grabs the weather values and sends them to weewx. Then weewx repeats the process every 15 seconds. It's not ideal, and the driver likes to crash overnight (which kills weewx). So it's not a reliable solution 

Is that a problem with weewx causing the crashing, or the way the driver is implemented? I'd be interested in trying your driver if you didn't mind

Having the data pushed to me is ideal, but I don't see what you mean about the ObserverIP allowing you to specify the URL for reporting. I do see a configuration option for "Server Listening Port", and mine is set to 5000, but there's no TCP/UDP data I could find on that port - so I'm not sure what that option is for.

As per my other post, I don't know if the option I have mentioned is specific to the Aercus firmware.

The server listening port of 5000 relates to the static DNS server, if you use that.

[drsprite]

Is that a problem with weewx causing the crashing, or the way the driver is implemented? I'd be interested in trying your driver if you didn't mind

Most surely it's my own code in the driver. I've tried to debug it, but I'm only about 5 days into owning this unit, so it's a bit trial & error right now.

The server listening port of 5000 relates to the static DNS server, if you use that.

Hmm, DNS is port 53, so I'm still confused by the port 5000 thing.

The server type allows for PHP, JSP or ASP. This to me suggests that I could have my own weather server listening out for updates from the ObserverIP, although what the communication protocol is I have no idea!

Wow, your firmware looks different than mine! That's exactly what I would need! I could put together a PHP page pretty quickly that would take the data in and see what the data is and then formulate something that way. Right now the only option is to use Weather Underground - it'd be nice to feed the data to other web sources and/or a more reliable driver to weewx / MySQL /etc.


If I telnet in though, I do see plenty of other options available, but there's no documentation (and I don't want to completely break my unit!)

[drsprite]

Well, looks like maybe there's the server option in telnet. Not sure I want to change it quite yet - still having a look around. And best I know, Weather Underground doesn't use port 5000 either.

and I wonder what the difference is between server mode and client mode on this thing?

[drsprite]

Ok I got brave and changed the hostname to my server. So far after rebooting the ObserverIP, I have no hits in the access or error logs on my server, and weather underground keeps getting updated.

This tells me the firmware doesn't even look at this option.

[redvers]

Ok I got brave and changed the hostname to my server. So far after rebooting the ObserverIP, I have no hits in the access or error logs on my server, and weather underground keeps getting updated.

This tells me the firmware doesn't even look at this option.

The Aercus firmware is available at the following link, depending on how brave you get :

http://www.aercusinstruments.com/downloads/

As far as I can tell, v2.1.9 is equivalent to the Ambient v3.0.3 as it has the ability to enter the rainfall values manually...

[drsprite]

As far as I can tell, v2.1.9 is equivalent to the Ambient v3.0.3 as it has the ability to enter the rainfall values manually...

Very tempting! I'm not brave right now, but maybe I will be this evening! Hmm.....

[drsprite]

I was just reading the user manual for the WeatherSleuth, and I don't think it'll work for me. Your system looks like it uses 433mhz, and mine uses 915mhz. So I am thinking that the firmware will only work on the 433mhz band. 

[redvers]

I was just reading the user manual for the WeatherSleuth, and I don't think it'll work for me. Your system looks like it uses 433mhz, and mine uses 915mhz. So I am thinking that the firmware will only work on the 433mhz band. 

Ah   I would have thought that the hardware was the same, and this kind of thing was controlled in firmware, but I wouldn't be willing to risk it either

What would I need to set up to listen to responses from the ObserverIP? I have two VMs available to play with; WHS2011 or CentOS 7...

[drsprite]

Ah   I would have thought that the hardware was the same, and this kind of thing was controlled in firmware, but I wouldn't be willing to risk it either

What would I need to set up to listen to responses from the ObserverIP? I have two VMs available to play with; WHS2011 or CentOS 7...

If you want to change your ObserverIP to use a custom server, use CentOS, and create a new PHP file in the root of your web directory (like /var/www/html/) and name it index.php, then try this simple code to see if you get anything:

Code: [Select]

<?php
$data = file_get_contents('php://input');
var_dump($data);
?>
As for screen scraping;

I'm using CentOS for weewx, so that's where I've been playing with my Python driver/script

Here's a quick Python script (that'll run under Linux or Mac) to just show how to get the data off the ObserverIP.

http://obrienlabs.net/code/ambient-weather-observerip-python-script/

Let me get my weewx driver updated a little and I'll upload it (It crashed twice in the last hour)

[drsprite]

I find that I have to reboot my ObserverIP about once a day or so. Wunderground will say "Station reported in 20 minutes ago" and I know it's locked up.

[redvers]

Ah   I would have thought that the hardware was the same, and this kind of thing was controlled in firmware, but I wouldn't be willing to risk it either

What would I need to set up to listen to responses from the ObserverIP? I have two VMs available to play with; WHS2011 or CentOS 7...

If you want to change your ObserverIP to use a custom server, use CentOS, and create a new PHP file in the root of your web directory (like /var/www/html/) and name it index.php, then try this simple code to see if you get anything:

Code: [Select]

<?php
$data = file_get_contents('php://input');
var_dump($data);
?>

I gave that a go, and it stopped reporting to wunderground, but how do I see the output of var_dump?

Hmm, and since changing back to wunderground & rebooting it still isn't updating :-/

[drsprite]

I gave that a go, and it stopped reporting to wunderground, but how do I see the output of var_dump?

Hmm, and since changing back to wunderground & rebooting it still isn't updating :-/

The var_dump() should output directly onto the web page. Do you see anything in your access or error logs? Maybe it never reported in?

It makes sense that when you changed it to your server that it would stop reporting to wunderground.

But it's weird that it didn't report in when you changed it back. It seems that sometimes these little buggers are overly sensitive. Sometimes even after a reboot, I have to re-save the wunderground settings and it starts working again.

[redvers]

I gave that a go, and it stopped reporting to wunderground, but how do I see the output of var_dump?

Hmm, and since changing back to wunderground & rebooting it still isn't updating :-/

The var_dump() should output directly onto the web page. Do you see anything in your access or error logs? Maybe it never reported in?

Weird that it stopped reporting to wunderground too... It seems that sometimes these little buggers are overly sensitive. Sometimes even after a reboot, I have to re-save the wunderground settings and it starts working again.

My understanding is that the ObserverIP reporting process would be what connects to my server, not the configuration page. As such there is no web page for me to view! As for access or error logs, am still taking baby steps on CentOS (installed it so that I could eventually learn) so wouldn't know where to even start looking

Had to telnet in to force the destination to be wunderground again!

[drsprite]

My understanding is that the ObserverIP reporting process would be what connects to my server, not the configuration page. As such there is no web page for me to view! As for access or error logs, am still taking baby steps on CentOS (installed it so that I could eventually learn) so wouldn't know where to even start looking

Had to telnet in to force the destination to be wunderground again!

You're right - I made a small edit to my last post but I figured you may have missed it!

It makes sense that when you changed the server, that ObserverIP would report there instead of wunderground. But I would have figured it would have gone back to wunderground when you configured it again.

Yeah, I have to do that too. Re-save the server settings for it to "wake up" wunderground again.

Do you have apache installed and is it running? SSH to your server and run "service httpd start". The access/error logs should be in /var/log/httpd/access_log and /var/log/httpd/error_log. What you would be looking for is if your ObserverIP is accessing (or causing an error) to your server. Then you at least know it's talking to your server. Then it's a matter of making a PHP page that understands whats going on.

[drsprite]

Here's my attempt at an ObserverIP weewx driver. The main differences between mine and the one already out there is I am using lxml to scrape the data, and my ObserverIP uses a static IP. So no network scanning is needed, but a static IP on your ObserverIP is needed.

The driver that's already rooted in the weewx community seems to be a bit broken at the moment (I kept getting Exception: ObserverIP network probe failed), and also it doesn't seem to handle a static IP. (The network probe failed for me with a static IP and DHCP.)

Which is why I'm giving it a go. My driver is a bit buggy right now. If the ObserverIP unit is rebooted or otherwise slow, then weewx will crash.

Anyways - anyone reading this is welcome to help out and submit a pull request!

https://github.com/poblabs/weewx-ObserverIP

[redvers]

Here's my attempt at an ObserverIP weewx driver. The main differences between mine and the one already out there is I am using lxml to scrape the data, and my ObserverIP uses a static IP. So no network scanning is needed, but a static IP on your ObserverIP is needed.

Excellent stuff! Hopefully I can get on and have a play tonight, kids permitting

As for suggestions/pull requests for improvements, I may need to learn Python first

The driver that's already rooted in the weewx community seems to be a bit broken at the moment (I kept getting Exception: ObserverIP network probe failed), and also it doesn't seem to handle a static IP. (The network probe failed for me with a static IP and DHCP.)

I had the same issue (which makes me think it is CentOS specific). I was able to fix the network probe problem with some Google-fu; seems that a sock.bind statement is missing from the next line of the driver. However I then hit a different error (Unable to load driver: string index out of range) which my limited CentOS / Python skills couldn't resolve

[redvers]

The access/error logs should be in /var/log/httpd/access_log and /var/log/httpd/error_log. What you would be looking for is if your ObserverIP is accessing (or causing an error) to your server. Then you at least know it's talking to your server. Then it's a matter of making a PHP page that understands whats going on.

A-ha!! Found it here's what is in the access log :

192.168.0.14 - - [04/Jan/2016:21:02:36 +0000] "GET /weatherstation/up
dateweatherstation.php?ID=IENGLAND1112&PASSWORD=PLAINTEXTPASSWORD&tempf=43.3&h
umidity=98&dewptf=42.8&windchillf=43.3&winddir=129&windspeedmph=0.00&
windgustmph=0.00&rainin=0.00&dailyrainin=0.04&weeklyrainin=0.04&month
lyrainin=0.91&yearlyrainin=0.91&solarradiation=0.00&UV=0&indoortempf=
76.5&indoorhumidity=49&baromin=29.05&lowbatt=0&dateutc=2016-1-4%2021:
2:35&softwaretype=Weather%20logger%20V2.1.9&action=updateraw&realtime
=1&rtfreq=5 HTTP/1.0" 404 237 "-" "-"

Interesting to see that the password for the station is sent as plain text across a http link. And the ObserverIP is passing all values in inches/Fahrenheit even though I display them as MM/Celsius. So there's no need for any listener to perform conversions or be able to handle multiple formats!