Author Topic: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?  (Read 38804 times)

0 Members and 1 Guest are viewing this topic.

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #50 on: January 16, 2016, 03:09:21 AM »
YES PLEASE!! :)

Thanks to work & kids I have been too busy to "play" with my setup to try capturing the data.

Those with the Aercus variant of the IPMonitor won't need to use a router for redirecting the traffic, as it's built in to the functionality of the firmware...

(One thing that I did see & was going to investigate more was the fact that there appears to be a frequency setting on one of setup pages, albeit greyed out. My plan had been to see if this was something I could change via telnet - may still give that a go, as if the Aercus firmware can be used on an Ambient 915MHz device, you can skip the router step too :) )

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #51 on: January 16, 2016, 09:55:22 AM »
You're right, if you have an ObserverIP that lets you select the remote server, then you can skip the dd-wrt portion; but the PHP file version may be helpful.

I'll write something up in the next day or so!

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #52 on: January 16, 2016, 01:23:52 PM »
:)

I managed to get five minutes on my laptop, so telnetted into my device to have another poke around in the settings. Couldn't find anything that explicitly allowed you to change the frequency though :(

Offline gtoal

  • Member
  • *
  • Posts: 24
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #53 on: January 16, 2016, 05:08:58 PM »
I think it's unlikely - although the same underlying module handles all three frequencies, I'm fairly sure (although not 100%) that they are enabled by supplying a crystal of the appropriate frequency.  I don't think these are dynamically tunable circuits.

G

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #54 on: January 18, 2016, 12:00:48 PM »
It's a bit long, but I tried to cover everything.

Here's my write up start to finish on how I'm redirecting the ObserverIP traffic to my server. The PHP file that accepts the data which forwards it to wunderground.com as well as to a socket server. Finally the weewx driver that connects to the socket server and listens for new weather data and submits it to the weewx archive.

Sure hope it helps someone! It's been working great for me the last few days and is much more reliable than the initial screen scraping method. The screen scraping method would lock up the ObserverIP for up to 45 minutes a day.

Redirecting Weather Station Data from ObserverIP

« Last Edit: January 18, 2016, 04:56:13 PM by drsprite »

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #55 on: January 30, 2016, 07:03:29 PM »
I finally got to give this a go tonight, but for whatever reason it just doesn't seem to work :(

I can see the GET requests coming through in my access log, so that bit is working. The server light is showing on the ObserverIP, but updates aren't getting to weather underground! Couldn't see anything on the socket I set up either, so i suspect a PHP issue!

(Have never done anything with html or PHP and have only installed Apache for this, so am likely missing something that needs to be installed!)

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #56 on: February 01, 2016, 03:16:32 PM »
I finally got to give this a go tonight, but for whatever reason it just doesn't seem to work :(

I can see the GET requests coming through in my access log, so that bit is working. The server light is showing on the ObserverIP, but updates aren't getting to weather underground! Couldn't see anything on the socket I set up either, so i suspect a PHP issue!

(Have never done anything with html or PHP and have only installed Apache for this, so am likely missing something that needs to be installed!)

hmm.. weird because the string should pass it along to wunderground without interruption... especially if you see the ObserverIP data in your access logs.

Well, you could try something like this which may clue us onto something:

Code: [Select]
$wunderground = file_get_contents("http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?" . $_SERVER['QUERY_STRING']);
syslog(LOG_INFO, "Update to wunderground: $wunderground");

To confirm: on the root of your web server, you do have a folder called "weatherstation" and a file inside it named "updateweatherstation.php" with that code above?

If so, we should see something in your syslog.

See what it says in your /var/log/messages (or wherever your syslog is). It should normally say "success", and that "success" wording is what turns the server light On. Maybe this will help clue us into other problems?


Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #57 on: February 01, 2016, 03:41:21 PM »
Well coincidently enough i was just working on this, and have swapped out my existing 5.4 PHP installation for 5.5; not sure if the old one was corrupt but either way I can now get a success message when I hit the webpage myself, and wunderground is still updating :)

Next issue, that I haven't looked at solving yet is the fact that I get permission denied on localhost:2999 from fsockopen()

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #58 on: February 01, 2016, 03:43:20 PM »
Well coincidently enough i was just working on this, and have swapped out my existing 5.4 PHP installation for 5.5; not sure if the old one was corrupt but either way I can now get a success message when I hit the webpage myself, and wunderground is still updating :)

Next issue, that I haven't looked at solving yet is the fact that I get permission denied on localhost:2999 from fsockopen()

Sounds like it might not be running. Try doing a ps aux | grep ncat and see if anything shows up (besides the grep command)? or you could also try pidof ncat. If nothing shows up, then it's not running.

Update: try also to telnet to it. telnet localhost 2999 should open it up. If nothing opens up, then it's definitely not running

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #59 on: February 01, 2016, 03:52:12 PM »

Sounds like it might not be running. Try doing a ps aux | grep ncat and see if anything shows up (besides the grep command)? or you could also try pidof ncat. If nothing shows up, then it's not running.

Update: try also to telnet to it. telnet localhost 2999 should open it up. If nothing opens up, then it's definitely not running

Good spot; although I had added the command to /etc/rc.local it hadn't been run on reboot; had to chmod rc.d/rc.local and now it's running... Next step to get weemx reading from the socket! :)

edit : gah still not working  ](*,)

Code: [Select]
[Mon Feb 01 20:55:09.556323 2016] [:error] [pid 2462] [client 192.168.0.14:26539] PHP Warning:  fsockopen(): unable to connect to localhost:2999 (Permission denied) in /var/www/html/weatherstation/updateweatherstation.php on line 35
[root@CentOSVM httpd]# ps -ef | grep ncat
root      1330     1  0 20:49 ?        00:00:00 /usr/bin/ncat -l 2999 --keep-open --broker
root      2726  2681  0 20:55 pts/0    00:00:00 grep --color=auto ncat

« Last Edit: February 01, 2016, 03:57:33 PM by redvers »

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #60 on: February 01, 2016, 04:08:28 PM »
Do you have SELinux running and/or enabled? That could be blocking it (I run mine disabled).

If so, run /usr/sbin/setsebool httpd_can_network_connect=1

More troubleshooting ideas are here: http://www.rkrishardy.com/2009/12/permission-denied-13-when-opening-socket-in-php-apache/

In the end, you should be able to telnet localhost 2999

Wait about 16 seconds +/- and you should see some weather data being dumped in there from the PHP script. If you do, then it's time for weewx!  :grin:

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #61 on: February 02, 2016, 12:39:19 AM »

Do you have SELinux running and/or enabled? That could be blocking it (I run mine disabled).

If so, run /usr/sbin/setsebool httpd_can_network_connect=1


I haven't a clue, but i ran that command and have had five minutes of access entries without corresponding error logs, so I imagine that is now working - thanks :)

Will continue this when on a laptop rather than trying to telnet from my phone!

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #62 on: February 02, 2016, 08:35:41 AM »

Do you have SELinux running and/or enabled? That could be blocking it (I run mine disabled).

If so, run /usr/sbin/setsebool httpd_can_network_connect=1


I haven't a clue, but i ran that command and have had five minutes of access entries without corresponding error logs, so I imagine that is now working - thanks :)

Will continue this when on a laptop rather than trying to telnet from my phone!

SELinux is like an annoying firewall on top of a firewall. It makes sure all the apps play nicely together, and blocks them when they don't play nicely (or what it thinks is not nice play). Such as HTTP connections to a local socket.

At least, this is *my* interpretation of it :)

I always disable it because I feel it just gets in the way.  :lol:

Hopefully when you're on your laptop the telnet and everything will be working!

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #63 on: February 10, 2016, 04:15:45 PM »
So, over a week later and I have been successfully logging to the socket but without weewx running at the other end the data was going in to a black hole (and wunderground.com :))

Have finally had the time this evening to get weewx updated and the socket logger installed and configured. Couldn't see any reports at first; the standard weewx config isn't as helpful as the documentation suggested & i had to tweak a few things to get things saved in to the correct locations!

As there is nothing to interrogate directly, weewx is unable to pull the history from the socket. I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

Have also configured weewx to send data onwards to the Met Office WOW site. What other weather sites do people suggest for logging data to?

Offline redvers

  • Member
  • *
  • Posts: 25
How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #64 on: February 10, 2016, 04:21:47 PM »
Forgot to add, following firmware discussions earlier, did you know there was a v3.0.7 for the Observer IP that isn't directly linked on the Ambient site?

http://site.ambientweatherstore.com/iptools/USA_v3.0.7.bin

It's dated August 2015 (3.0.3 is from May) and is possibly the version that includes the ability to customise the ip address used for logging to!
« Last Edit: February 11, 2016, 01:05:18 AM by redvers »

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #65 on: February 11, 2016, 07:12:35 PM »
I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

Have also configured weewx to send data onwards to the Met Office WOW site. What other weather sites do people suggest for logging data to?

There's this script, it's a bit backwards where it syncs missing data TO wunderground. However, maybe it can be tweaked to download missing data FROM wunderground? Worth checking out at least: http://www.weewx.com/wunderfixer/

I upload my data to CWOP, PWSWeather.com, Wunderground and WeatherBug. I think WeatherBug is States only, but I think you could do CWOP and PWSWeather from where you are. I see a lot of CWOP sites in UK.

Forgot to add, following firmware discussions earlier, did you know there was a v3.0.7 for the Observer IP that isn't directly linked on the Ambient site?

http://site.ambientweatherstore.com/iptools/USA_v3.0.7.bin

It's dated August 2015 (3.0.3 is from May) and is possibly the version that includes the ability to customise the ip address used for logging to!

I did notice this actually, but I was hesitant to try it in fear of it not being fully baked and breaking my Observer IP :)

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #66 on: February 11, 2016, 07:36:31 PM »
I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

Looks like this might help you too?

https://groups.google.com/d/msg/weewx-user/TkuP0B48EpY/F_lxP6x1AAAJ

You could search that user group to find other examples. Hope it helps!

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #67 on: February 12, 2016, 01:01:49 AM »

I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

Looks like this might help you too?

https://groups.google.com/d/msg/weewx-user/TkuP0B48EpY/F_lxP6x1AAAJ

You could search that user group to find other examples. Hope it helps!

Good find! Will be giving that a go at some point :)

Am not too fussed about backfilling to WOW or CWOP but would be good to have Weewx keeping a copy of all my data!

Offline mwall

  • Contributor
  • ***
  • Posts: 135
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #68 on: February 12, 2016, 07:50:25 AM »
I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

gary is working on a utility to do this:

https://github.com/weewx/weewx/issues/97

perhaps you could help him with the implementation?  at the very least i'm sure he could use some help testing it.

m

Offline redvers

  • Member
  • *
  • Posts: 25
Re: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?
« Reply #69 on: February 12, 2016, 09:39:37 AM »

I suspect that there is no way of getting back the history from wunderground and importing that back in to weewx... Is there?

gary is working on a utility to do this:

https://github.com/weewx/weewx/issues/97

perhaps you could help him with the implementation?  at the very least i'm sure he could use some help testing it.

m

Looks good, will certainly look to help out where I can :)

Offline CRichard

  • Member
  • *
  • Posts: 3
Could this device be used to grab real time data from the sd card slot on the display console and if so how do I use it?

http://www.saelig.com/product/MIO001.htm

Offline creeble

  • Member
  • *
  • Posts: 14
Ok I got brave and changed the hostname to my server. So far after rebooting the ObserverIP, I have no hits in the access or error logs on my server, and weather underground keeps getting updated.

This tells me the firmware doesn't even look at this option.

This is getting to be a pretty old thread, but I just got my AmbientWeather 1400 and played with the setting.

You need to do a "saveconfig" after changing the hostname, then a "reboot".

Then it starts calling your hostname (port 80 only, the other port numbers in the firmware don't affect this).  So as long as you can live with the 16s update frequency...

I'd like to decode the "seteep" params and see if there's a way to change the update frequency, but seems rather tedious.  For now I'm just going to redirect it to another host, log it, then call Wunderground.

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
It is pretty old, but I still watch it since I started it :P

I gave up on modifying the ObserverIP. I instead implemented my own solution to archive the data and still update weather services like live data on my own website, Wunderground, PWS Weather, CWOP, etc.

Details here: http://obrienlabs.net/redirecting-weather-station-data-from-observerip/

Offline creeble

  • Member
  • *
  • Posts: 14
Details here: http://obrienlabs.net/redirecting-weather-station-data-from-observerip/

Ah, I read that post!  I'll let you know if I find anything out about the "seteep" params.

Offline drsprite

  • Senior Member
  • **
  • Posts: 86
I'm curious now; I'm pretty sure I did the saveconfig and reboot and it didn't work for me. Do you have the Ambient Weather version of the ObserverIP?

 

anything