Topic: How to directly log data from the Ambient Weather WS-1200-IP / ObserverIP?

[drsprite]

Ok my couple of hours turned into a couple of couple hours

Changed the weewx configuration with station_type = SocketLogger and put in the socketlogger section at the end. I've rebooted the Pi. Something still wrong.

If you still want to nerd on the other option, there's gotta be something simple slowing things down.

The traffic comes into that updateweatherstation.php file BEFORE ncat, so once we can verify there's activity there, then we can focus on ncat.

But if the weewx-interceptor driver is working for you, then it could be wasted effort

I ended up going back to the Pi image I had with weewx-interceptor configured, and once I had the router with the firewall rule configured, it works!

Ah cool, than maybe you're all set.

For some reason I also cannot get weewx to upload to WU

You might have a bad password or something? What does the log file say?

Restart weewx and then check the log file. This command will do both in 1 copy/paste.

sudo service weewx restart; sudo tail -f /var/log/syslog

If it's not obvious what's wrong, set debug = 3 in weewx.conf and restart weewx and check the log file again.

Alternately, if someone knows of a way I can configure the router so I don't lose the ambientweather.net connection, but still maintain the connection to a local Apache server, that would be ideal.

Hmm. This might be able to be accomplished if we knew the format of Ambient's API. We'd have to sniff the traffic to find out, which may not be worth the effort.

I'd have to imagine it's a bit of a clone of WU with an updated API backend.

[dkrug]

Quote from: dkrug
Alternately, if someone knows of a way I can configure the router so I don't lose the ambientweather.net connection, but still maintain the connection to a local Apache server, that would be ideal.

Hmm. This might be able to be accomplished if we knew the format of Ambient's API. We'd have to sniff the traffic to find out, which may not be worth the effort.

I'd have to imagine it's a bit of a clone of WU with an updated API backend.

Here's reference docs to their full REST API - https://ambientweather.docs.apiary.io/#

[drsprite]

Here's reference docs to their full REST API - https://ambientweather.docs.apiary.io/#

hmm, this looks like the API to retrieve data, not submit data.

[dkrug]

I can't offer more than that API link. Not worth the effort. I'll just need a slick weather page like yours to go along with this setup 

You might have a bad password or something? What does the log file say?

Restart weewx and then check the log file. This command will do both in 1 copy/paste.

sudo service weewx restart; sudo tail -f /var/log/syslog

If it's not obvious what's wrong, set debug = 3 in weewx.conf and restart weewx and check the log file again.

I had not noticed before that the station and password fields were commented out. I don't recall removing the hash from the PWSWeather fields, but I guess I did.

Working now! Thanks for your help. See you at the Hubitat forum! Message me when you get there.

[drsprite]

I'll just need a slick weather page like yours to go along with this setup 

I had not noticed before that the station and password fields were commented out. I don't recall removing the hash from the PWSWeather fields, but I guess I did.

Working now! Thanks for your help. See you at the Hubitat forum! Message me when you get there.

My website is heavily customized to my specific setup - but I'm getting a lot of requests for those who want it for their own stations. It's not a weewx skin, it's a full site. That being said, I'm working on making the website available for others to install. Might take some time to make it un-customized and able to configure easily.

weewx sanitized the passwords in the log to XXX for security purposes. You would see other error messages if anything was wrong or not being published from weewx thought.

Edit: a weewx skin of my website is now available!

[gtoal]

Wow, this really is the thread that never dies

Just dropping in to mention that the Observer IP firmware has been updated to support both wunderground and ambient, so the hack of intercepting port 80 has to be updated to support two destinations now.  Either catching them at the iptables layer or just by supporting two URLs in the replacement web server, i.e. http://rtupdate.wunderground.com/weatherstation/updateweatherstation.php?... and http://api.ambientweather.net/endpoint?...

My wife recently added a PM2.5/PM10 sensor and we want to inject the values into those feeds.

by the way they both have published APIs now (which wasn't the case when we started this thread):
https://github.com/ambient-weather/api-docs/wiki/Device-Data-Specs for ambient, and
https://feedback.weather.com/customer/en/portal/articles/2924682-pws-upload-protocol?b_id=17298 for wunderground.

Setting up the PM2.5 sensor was trivial and cheap.  https://www.ebay.com/itm/FOR-NOVA-SDS011-Laser-Dust-Sensor-PM2-5-PM10-JIA/292914367168 and https://hackernoon.com/how-to-measure-particulate-matter-with-a-raspberry-pi-75faa470ec35

regards,

Graham & Anne Toal

[gtoal]

I'm working on the MITM software to insert the PM data into the observer ip's payload to send to Wunderground.

Using the stationdata interface I can see that the pm10 data has been accepted but not the pm2.5 data.

http://stationdata.wunderground.com/cgi-bin/stationlookup?station=KTXEDINB9

<aqPM10 val="25"/>

This is the URL I'm tweaking:

        GET /weatherstation/updateweatherstation.php?ID=KTXEDINB9&PASSWORD=idontthinkso&tempf=98.1&humidity=59
              &dewptf=81.3&windchillf=98.1&winddir=165&windspeedmph=1.79&windgustmph=2.24
              &rainin=0.00&dailyrainin=0.00&weeklyrainin=0.00&monthlyrainin=0.07&yearlyrainin=6.85
              &solarradiation=814.27&UV=7&indoortempf=78.6&indoorhumidity=43&baromin=29.88
              &AqPM2%2E5=24&AqPM10=28
              &lowbatt=0&dateutc=now&softwaretype=WH2602%20V4.5.7
              &action=updateraw&realtime=1&rtfreq=5 HTTP/1.0

And I have tried several alternatives for the field name... AqPM2%2E5   AqPM2.5   AqPM25  aqpm25  AqPM_25

- none have been accepted.

I've only found 2 other implementations of software that reference these fields but I've tried the field names that both use.
( https://gist.github.com/brandonregard/32709a55e56a61bd9e9b6399db09f56b   and https://s.campbellsci.com/documents/us/miscellaneous/blog-resources/wunderground-new.dld )

Could someone with the ambient PM sensor please sniff their connection and see what is being sent?

thanks

G
PS by the way the cheap sensor is returning values that are bang on target according to other sites with data for this region.
Once we sort out how to upload the data, I can't see any justification for using $200 equipment when these are so easy to use and cost about $20!

[astrosteve]

Graham (gtoal),

I've got my ObserverIP setup with the router to forward that to my local WAMP server, and used (and modified) the updateweatherstation.php script, to save the data to sql, and then forward it to weather underground. Do you have this setup for ambient forwarding too? I am not sure how to set that up, although you are saying it's the "endpoint" page I'd need to setup. I tried setting up a script, but am not sure it would run that file as php. I created the file "endpoint" and tried unsuccessfully with the following:

Code: [Select]

<?php

$ambient = file_get_contents("http://api.ambientweather.net/endpoint?" . $_SERVER['QUERY_STRING']);
echo $ambient;
exit();
?>
Any help in getting this setup to forward the info to Ambientweather.net is appreciated. I'm also liking your homemade and cheaper air quality sensor. I'll be checking that out in more detail!

[gtoal]

Actually the 'endpoint' script in /var/www/html is quite simple - almost cheating ...

Code: [Select]

#!/bin/bash
wget -U "WH2600" --save-headers --output-document -  "http://api.ambientweather.net${REQUEST_URI}&pm25=`cat /var/www/html/data/PM25`" -o /dev/null -O /dev/stdout | tail +2
exit 0
It just adds the pm25= parameter to the existing parameters and passes it on to Ambient.  I have  separate process which writes the most recent pm10 and pm2.5 data to files in /var/www/html/data/ - your script will have to extract those values some other way.

The trickiest part was actually enabling scripts in the root directory.

This is added to the end of /etc/apache2/sites-available/000-default.conf

Code: [Select]

<Directory />
<FilesMatch "^endpoint$">
Options +ExecCGI
SetHandler cgi-script
</FilesMatch>
</Directory>

[gtoal]

by the way, this has all been running smoothly for the last couple of years, but now I have a few RTL-SDR dongles and I've started transitioning to reading the sensors directly by radio.  I think the ObserverIP may be obsolete now but even if not, it's always nicer to run your own server.

You need to sniff both 915MHz and 433MHz.  If anyone else is looking at this thread again and considering alternative ways to get data, here's the decoded data that rtl_433 returns, to help you decide... (and the uploads to ambient and wunderground).

Mar 14 15:32:09 linux Weather: Sun Mar 14 15:32:09 CDT 2021 station: {"time" : "2021-03-14 15:32:09.404436", "protocol" : 78,
 "model" : "Fineoffset-WH24",
 "id" : 84,
 "battery_ok" : 1,
 "temperature_C" : 24.800, "humidity" : 55, "wind_dir_deg" : 14, "wind_avg_m_s" : 3.500, "wind_max_m_s" : 4.480, "rain_mm" : 2028.000, "uv" : 1604, "uvi" : 4, "light_lux" : 57877.000,
 "mic" : "CRC", "mod" : "FSK", "freq1" : 914.860, "freq2" : 914.937, "rssi" : -1.623, "snr" : 22.528, "noise" : -24.151}

# neighbour outdoor?  Note humidity 127 signifies that the humidity sensor is absent in this device
{"time" : "2021-03-14 16:10:29.147742", "protocol" : 40,
 "model" : "Acurite-Tower",
 "id" : 10204,
 "channel" : "A", "battery_ok" : 1,
 "temperature_C" : 24.400, "humidity" : 127,
 "mic" : "CHECKSUM", "mod" : "ASK", "freq" : 432.956, "rssi" : -8.766, "snr" : 5.677, "noise" : -14.443}

# freezer
{"time" : "2021-03-14 16:05:51.433011", "protocol" : 41,
 "model" : "Acurite-986",
 "id" : 7689,
 "channel" : "2F", "battery_ok" : 1,
 "temperature_C" : -19.444,
 "status" : 0, "mic" : "CRC", "mod" : "ASK", "freq" : 432.948, "rssi" : -0.123, "snr" : 15.548, "noise" : -15.670}

# fridge
{"time" : "2021-03-07 14:56:20.326326", "protocol" : 41,
 "model" : "Acurite-986",
 "id" : 49269,
 "channel" : "1R", "battery_ok" : 1,
 "temperature_C" : 2.778,
 "status" : 0, "mic" : "CRC", "mod" : "ASK", "freq" : 433.938, "rssi" : -0.937, "snr" : 9.229, "noise" : -10.166}

# living room
Mar 14 15:29:43 linux Weather: Sun Mar 14 15:29:43 CDT 2021 indoor1: {"time" : "2021-03-14 15:29:43.164734", "protocol" : 78,
 "model" : "Fineoffset-WH32B",
 "id" : 110,
 "battery_ok" : 1,
 "temperature_C" : 25.200, "humidity" : 52, "pressure_hPa" : 1005.600,
 "mic" : "CRC", "mod" : "FSK", "freq1" : 914.862, "freq2" : 914.911, "rssi" : -0.138, "snr" : 24.931, "noise" : -25.068}

# back room
Mar 14 15:30:50 linux Weather: Sun Mar 14 15:30:50 CDT 2021 indoor2: {"time" : "2021-03-14 15:30:50.826468", "protocol" : 113,
 "model" : "AmbientWeather-WH31E",
 "id" : 224,
 "channel" : 2, "battery_ok" : 1,
 "temperature_C" : 24.300, "humidity" : 57,
 "data" : "b900000000", "mic" : "CRC", "mod" : "FSK", "freq1" : 914.883, "freq2" : 914.956, "rssi" : -0.131, "snr" : 25.024, "noise" : -25.154}

# bedroom
Mar 14 15:24:38 linux Weather: Sun Mar 14 15:24:38 CDT 2021 indoor3: {"time" : "2021-03-14 15:24:38.699753", "protocol" : 113,
 "model" : "AmbientWeather-WH31E",
 "id" : 196,
 "channel" : 3, "battery_ok" : 1,
 "temperature_C" : 22.600, "humidity" : 57,
 "data" : "5500000000", "mic" : "CRC", "mod" : "FSK", "freq1" : 914.879, "freq2" : 914.929, "rssi" : -0.109, "snr" : 24.401, "noise" : -24.510}

# kitchen
Mar 14 15:29:08 linux Weather: Sun Mar 14 15:29:08 CDT 2021 indoor4: {"time" : "2021-03-14 15:29:08.161605", "protocol" : 113,
 "model" : "AmbientWeather-WH31E",
 "id" : 183,
 "channel" : 4, "battery_ok" : 1,
 "temperature_C" : 27.500, "humidity" : 48,
 "data" : "a800000000", "mic" : "CRC", "mod" : "FSK", "freq1" : 914.875, "freq2" : 914.882, "rssi" : -6.915, "snr" : 17.448, "noise" : -24.363}


192.168.2.240 - - [14/Mar/2021:15:09:50 -0500] "GET /endpoint?
&PASSKEY=...
&stationtype=WS-1501-IP
&dateutc=20210314+20:09:49

&winddir=350
&windspeedmph=6.93
&windgustmph=10.29
&maxdailygust=27.29

&tempf=74.8
&battout=1
&humidity=62

&hourlyrainin=0.00
&eventrainin=0.00
&dailyrainin=0.00
&weeklyrainin=0.07
&monthlyrainin=0.07
&yearlyrainin=1.56
&totalrainin=1.56

&tempinf=77.2
&battin=1
&humidityin=53
&baromrelin=29.43
&baromabsin=29.70

&uv=4
&solarradiation=429.40

&temp2f=75.74
&humidity2=58
&batt2=1

&temp3f=72.68
&humidity3=56
&batt3=1

&temp4f=81.50
&humidity4=49
&batt4=1
 HTTP/1.1" 200 836 "-" "WH2600"


192.168.2.240 - - [14/Mar/2021:15:09:56 -0500] "GET /weatherstation/updateweatherstation.php
?ID=...
&PASSWORD=...
&tempf=74.8
&humidity=62
&dewptf=61.0
&windchillf=74.8
&winddir=350
&windspeedmph=6.93
&windgustmph=10.29
&rainin=0.00
&dailyrainin=0.00
&weeklyrainin=0.07
&monthlyrainin=0.07
&yearlyrainin=1.56
&solarradiation=429.40
&UV=4
&indoortempf=77.2
&indoorhumidity=53
&baromin=29.43
&lowbatt=0
&dateutc=now
&softwaretype=WH2602%20V4.5.6
&action=updateraw
&realtime=1
&rtfreq=5
 HTTP/1.0" 200 174 "-" "-"