Topic: Implementing a Si1000 based wireless receiver for Davis ISS data

[franzz]

People playing with this stuff should keep in mind that these frequencies might vary a tiny bit from one console to another (I think).  I believe the console keeps track of the frequency errors it measures between itself and the ISS and applies that as an offset as it does its thing, so different combinations of consoles and ISS's will give slightly different frequencies.  Because of this, it is very likely that you would get slightly different numbers if you re-sniffed the register settings.

That's an interesting point. I will sniff the register setting process again if the temperature difference between console and transmitter crytal oscillator is a little bit higher. In this case it could be that there is a temperature drift between the crystal oscillators.
What I can also do is to check if the AFC register will be read out by console. According to the datasheet this register value gives the information how to change the frequency registers to compensate drifts.
One point which doesn't match with the "vary of tiny bit theory" is, that the frequency register settings are taken from the same pool if you are using more than one transmitter. In my case I have 2 different transmitters (ISS and one add. temp/hum. sensor). Both of them are hopping in the same frequeny pool. It could be that the crystal osc. have the same frequency drift or there is no frequency correction.

BR
franzz

[wylee]

I want to give a big thanks to Tridge and all the others that have worked on deciphering the wireless data. I have an Ebay clone 3DR ground radio attached to Raspberry Pi reading wireless data from a Vantage Vue. It is only showing the temperature in the reports but I can't see the wind, rain, and humidity data.

When the loops log to screen using  ./bin/weewxd weewx.conf everything appears to be there. The data is obviously there I just need to figure out why weewx is not processing it.

edit: Wow, I must have been tired when I posted that nearly incoherent mess....fixed to make it more readable. I also realized the humidity  (sorted below) was the only data weewx was not processing; the wind and rain were not reporting anything.

[wylee]

Got the humidity working. 

There was a typo in the Si1000 driver listing humidity instead of outHumidity. Took a bit but I figured it out after I got phpLiteAdmin running to get a good look at the db.

[wylee]

I got a PM inquiring about installing the firmware. Replying via PM wouldn't be very helpful to anyone else so I'll post here.

My process went something like this using Ubuntu:

Verify my laptop was seeing my Si1000 ground module:

Code: [Select]

lsusb
Show the port that the Si1000 module was attached to:

Code: [Select]

ls /dev/*USB*
Load the Davis firmware on the ground module: (my module was attached to port ttyUSB0)

Code: [Select]

sudo ./uploader.py --port /dev/ttyUSB0 --resetparams --baudrate 57600 "radio~hm_trp.ihx"

[brycemercer]

Darn it wylee, I was going to post the humidity fix!  You beat me to it, nice work.

I too would like to thank everybody for their work on this.  It's been a fun project to replicate!

Since the Vantage Vue doesn't have a barometer I'm looking at adding one separately.  If anybody has any advice on this I'd love to be pointed in the right direction.  Otherwise I'll check back in once I've received my BMP085 off ebay and played around with it a bit.

Thanks again!

[wylee]

My BMP085 should be here any day now. It looks to me like the biggest challenge there will be getting the data into the database.

If anyone figures out how to get the Si1000 driver to properly handle the dewpoint and windchill calculations please post it.

[brycemercer]

I've inserted the following three lines in the genLoopPackets definition in the Si1000.py driver right above the yield _packet line using the same indentation:

Code: [Select]

            _packet['windchill'] = weewx.wxformulas.windchillC(_packet['outTemp'], _packet['windSpeed'])
            _packet['dewpoint'] = weewx.wxformulas.dewpointC(_packet['outTemp'], _packet['outHumidity'])
            _packet['heatindex'] = weewx.wxformulas.heatindexC(_packet['outTemp'], _packet['outHumidity'])

Weewx now calculates and reports the three measurements.  Since there is currently no heat index or windchill (due to there being no heat or wind in Boise) I am unable to compare those values to my Davis console.  However, the dewpoint reported by Weewx is 15.1 while my console shows 13.  I'm going to let it run overnight and see what happens.

Also, as a disclaimer, I am a coding rookie, so the above code might be in a completely wrong place (or might be a completely wrong idea altogether).  Please feel free to correct my errors, I'm trying to learn!

Bryce

[wylee]

Thanks! That works. I just changed the formula suffix from C to F since I use Fahrenheit.

I tried that same code but I had put it right under the 'usUnits' : weewx.US } line and I was getting errors.

I think the rounding might be off but it is very close. My current temp of 30.2°F and Humidity of 90% has weewx giving a dewpoint of 27.7°F. I calculated the dewpoint to 27.62°F

Thanks again. I was pulling my hair out thinking it just had to be something simple.

[wylee]

I think I've found another driver typo.

Each spoon should be .01 instead of .1. I thought I had my station pretty close to level and couldn't understand how with a very light rain overnight, weewx recorded 4" of rain. I don't have a console to compare to but I'm pretty sure this is the problem. The forecast was for .25 to .5 of rain.

Code: [Select]

        # each spoon is 0.1"
        return ret * 0.1
should be:

Code: [Select]

        # each spoon is 0.01"
        return ret * 0.01

[wylee]

I got my BMP085 a few days ago but I forgot to order jumper wires. I finally got my jumper wires and have the inTemp and barometer readings working. 

[kobuki]

People playing with this stuff should keep in mind that these frequencies might vary a tiny bit from one console to another (I think).  I believe the console keeps track of the frequency errors it measures between itself and the ISS and applies that as an offset as it does its thing, so different combinations of consoles and ISS's will give slightly different frequencies.  Because of this, it is very likely that you would get slightly different numbers if you re-sniffed the register settings.

That's an interesting point. I will sniff the register setting process again if the temperature difference between console and transmitter crytal oscillator is a little bit higher. In this case it could be that there is a temperature drift between the crystal oscillators.
What I can also do is to check if the AFC register will be read out by console. According to the datasheet this register value gives the information how to change the frequency registers to compensate drifts.
One point which doesn't match with the "vary of tiny bit theory" is, that the frequency register settings are taken from the same pool if you are using more than one transmitter. In my case I have 2 different transmitters (ISS and one add. temp/hum. sensor). Both of them are hopping in the same frequeny pool. It could be that the crystal osc. have the same frequency drift or there is no frequency correction.

BR
franzz

Did you manage to advance on this? I'm trying to repeat what DeKay did (see here), but with an EU ISS on 868 MHz. I'm using a Moteino too, and I have updated the FRF table to reflect the correct 868 frequencies, but it doesn't seem to pick up anything but interemittent noise. I've even written some code to scan the whole 868 MHz range using small steps, to no avail. I'm starting to doubt this is working with an RFM69W (or my ISS is faulty, but it has been tested before, according to the seller). Maybe some parameters like preamble or sync word are different? Any hints?

[kobuki]

Well, I solved my problems by using a battery to power the receiver, now I see the ISS packets. However, I'm blocked again since I only know the temperature formula for fahrenheit, but not for degrees celsius (I have an EU metric unit). F is simply the raw value divided by 160. The C value is the raw value divided by something around 500, guessing by the room temp displayed on a desktop T/RH meter (which is not terribly accurate). It could be exactly 500, 3*160 (based on the F formula) or 512. Anyone with suggestions?

[jrojko]

I think, raw is always in F*160 so use Fahrenheit to Celsius formula C = 5*(F-32)/9

room temperature 70F => raw = 70*160 = 11200

C = 5*(raw/160-32)/9 = 21,1  (~raw/530 for this temperature)

[kobuki]

Hm, I've actually thought of that at least once If I interpret the values as F, they seem to be consistent with the "reference" (alcohol thermometer). I can only hope that the wind speed is km/h and not mph... One thing I noticed tho is that the LSB nibble is always 0xD. I wonder if it should be treated as zero or as is, a 0xD. It steered me off track since I started to think that only the first 3 nibbles are important.

Thanks for the hint!

[DeKay]

One thing I noticed tho is that the LSB nibble is always 0xD.

It is?  See this explanation which uses an actual example from my own ISS.

Message 8:

Byte 3 and 4 are temperature. The first byte is MSB and the second LSB. The
value is signed with 0x0000 representing 0F. This reading in the old version
of the ISS was taked from an analog sensor and measured by an A/D. The newer
ISS uses a digital sensor but still represents the data in the same way. 160
counts (0xa0) represents 1 degree F. A message of

80 04 70 0f 99 00 91 11

represents temperature as 0x0f99, or 3993 decimal. Divide 3993 by 160 to get
the console reading of 25.0F

Also, I'd bet that all the values coming back from your ISS are in Imperial units.

[kobuki]

Yeah, it looks like the ISS is always sending data in imperial units, however, the rain meter is calibrated for 0.2 mm, it even says so on a sticker on the bucket. So one pulse should mean 0.2 mms of rain on a metric unit for every flip. I wonder about other differences. I've already read your great protocol writeups, too. Yes, the last temp nibble is always 0xD, as strange as it sounds. The sensor could be broken...

Some samples from the last few hours:

Code: [Select]

4 - Data: 80 0 0 1A 6D 0 2D F4   RSSI: -65dBm   CRC: 2DF4 2DF4 CRC OK 2561
1 - Data: 80 0 0 1A 5D 0 28 61   RSSI: -66dBm   CRC: 2861 2861 CRC OK 7686
1 - Data: 80 0 0 1A 3D 0 23 4B   RSSI: -65dBm   CRC: 234B 234B CRC OK 5122
...
4 - Data: 80 0 0 27 1D 0 A2 59   RSSI: -88dBm   CRC: A259 A259 CRC OK 5123
1 - Data: 80 0 0 27 1D 0 A2 59   RSSI: -87dBm   CRC: A259 A259 CRC OK 7684
2 - Data: 80 0 0 27 2D 0 A7 CC   RSSI: -36dBm   CRC: A7CC A7CC CRC OK 10246

The last number is the time difference between subsequent reads in ms. I don't have the anemometer connected (thus the many zeros).

[jrojko]

My guess: 160 is quite strange number. What if temperature is only in first 3 nibbles? Then raw data is temperature multiplied by 10. 10 is nice number that make sense. See this table:

Code: [Select]

raw hex  raw dec   temp [°F]  raw hex  raw dec   temp[°F]
0f99     3993      24.95625   0f9      249       24.9
1A6D     6765      42.28125   1A6      422       42.2
1A5D     6749      42.18125   1A5      421       42.1
1A3D     6717      41.98125   1A3      419       41.9
271D     10013     62.58125   271      625       62.5
272D     10029     62.68125   272      626       62.6

Also note that temperature resolution of the weather station is 0.1°F. This corresponds with the last column of the table.

[kobuki]

Yes, that's what I thought at a point too. But DeKay's raw temperature readings show a changing LSB so it's not entirely justified yet. Maybe someone can look at the raw data on the console of a metric (or a US) unit? It has a serial command that dumps the raw data received from the ISS, IIRC it's "STRMON".

[jrojko]

This is from description of LOOP data format: The value is sent as 10th of a degree in F. For example, 795 is returned for 79.5°F.

[kobuki]

Yeah. But regarding the exact data format and the significance of the nibbles it's just as vague as it gets. I'll go with the scheme you proposed, regardless. This is the most feasible so far.

[jrojko]

From what I read so far, I am pretty sure format of the raw data is the same for US/EU (imperial/metric) stations. Units are transformed by the console and/or software only.

[DeKay]

Yes, that's what I thought at a point too. But DeKay's raw temperature readings show a changing LSB so it's not entirely justified yet. Maybe someone can look at the raw data on the console of a metric (or a US) unit? It has a serial command that dumps the raw data received from the ISS, IIRC it's "STRMON".

I'm starting to think that you and jrojko might be on to something here.  The difference in the lower nibble might be some status bits or other information we haven't figured out yet instead of real data.  Even if it is real, it is beyond the resolution of the console and the accuracy of the device, so it doesn't matter anyway.  I think I'll just bit shift this lower nibble out of existence in the packet processing I do: that directly gives me degrees F in tenths of a degree just as pumped out by the console without any further math.

[rdsman]

The LSB of the raw data coming from the Vue ISS (US) changes as in:

Code: [Select]


Packet: 80  08  46  1C  08  0F  D1  4C
Packet: 80  09  4A  1C  1A  0A  01  9B
Packet: 80  09  47  1C  28  02  DA  E2
Packet: 80  0E  4C  1C  39  08  32  21
Packet: 80  0A  50  1C  38  05  0D  2E


The current outside temp displays 45 (F).

[kobuki]

Thanks. Your lowest nibble seem to be changing for every packet too. Though after this discussion I think that it can be simply discarded.

[wylee]

Anyone have any suggestions for implementing rain rate?

I'm thinking the best way would require a firmware modification for the Si1000 module but maybe a weewx driver modification will be close enough.