Author Topic: Implementing a Si1000 based wireless receiver for Davis ISS data  (Read 38932 times)

0 Members and 1 Guest are viewing this topic.

Offline tridge

  • Member
  • *
  • Posts: 20
Following on the great work by DeKay and others, I've started implementing a Si1000 based wireless receiver for the Davis ISS 900MHz protocol.
The motivation for this work is that some vandals smashed the Davis console on a weather station I had setup for my local RC model flying club (see http://weather.cmac.org.au). I thought that instead of buying a new console I would build my own. I'm the author of the firmware for an open source telemetry radio commonly used in UAVs, so I just took one of our Si1000 based radios and adapted it for receiving the Davis messages. That involved looking at the CC1101 register settings from the code that DeKay and others had done, and creating equivalent register settings for the Si1000 transceiver.
In case it helps anyone else, the key wireless information is:

Symbol rate: 19.191 kbaud
Modulation: GFSK, no manchester
Deviation: 9.521 kHz
Band: 900MHz
Preamble: 32 bits
Preamble detection: 16 bits
Sync words: 0xCB 0x89 (MSB)
Packet format: fixed packet length, 10 data bytes
Checksum: CCITT over 8 bytes, results in constant. Constant varies by protocol version
Trailer: fixed 2 bytes of 0xC2 0x16 (bytes 8 and 9)
Frequency hopping pattern: 911380218, 902349090, 911882476, 922921051, 914892852, 906363189, 925931427, 918405487, 908872100, 920412139, 913387664, 903854278, 916899505, 924426239, 910377288, 904858001, 915896575, 921415863, 907366912, 926935150, 912886199, 903352813, 917401763, 923422515, 909373565, 926432891, 905861724, 914391387, 919408416, 924927703, 902850555, 910878753, 921917327, 915394317, 906864654, 917903228, 927436614, 920913604, 908369842, 912383941, 918906951, 904355743, 923923980, 916398040, 909875030, 919910675, 905359466, 922418792, 907868377, 913889129, 925429962

I have put a current version of my code in the davis-wip branch of my SiK tree, at http://github.com/tridge/SiK
The code works on the 3DR 900MHz radio (see http://store.diydrones.com) and should also work on the various clones out there.

Now the remaining puzzle is decoding the leading 8 bytes of the protocol. For my ISS the protocol is clearly different to the protocol decoded by DeKay and others, although it is closely related.

The first 4 bits of the leading byte is the packet type, but the packet types don't match what previous people have found. For example, DeKay found that type 8 was temperature, whereas in mine it became clear that temperature is type 7.

My current theory is that the console uses the result of the checksum calculation to determine the protocol version, or perhaps use the two trailer bytes. The fact that the freq hopping and modulation is the same would mean that can have one console that supports many different protocols.

The bottom 3 bits of byte 1 are the transmitter ID. Changing the ID of the ISS changes these bits, and doesn't affect anything else in the protocol (so same hopping sequence).

The second byte is the wind speed, but it isn't in the simple linear format that others have found. It seems to be byte1 xor 0xE. I tested this by driving around with my ISS hanging out the window of a car, and logging the driving speed with a GPS. This formula gave a good match.

The 3rd byte is wind direction, but it is permuted in a strange way. The two nibbles are permuted in the following way (python code):

Code: [Select]
def wind_direction(b):
    n1 = [0xD, 0xE, 0xF, 0x0, 0x9, 0xA, 0xB, 0xC, 0x5, 0x6, 0x7, 0x8, 0x1, 0x2, 0x3, 0x4]
    n2 = [0xD, 0xC, 0xF, 0xE, 0x9, 0x8, 0xB, 0xA, 0x5, 0x4, 0x7, 0x6, 0x1, 0x0, 0x3, 0x2]
    v = (n1[b>>4]<<4) | n2[b&0xF]
    return v * 360 / 255
That gives the direction in degrees.

The other readings all seem to be based on a 12 bit number derived from the 4th and 5th bytes. The basic formula for the 12 bits seems to be something like this:

Code: [Select]
def decode(b1, b2):
    n1 = [ 0xF, 0xE, 0xD, 0xC, 0xB, 0xA, 0x9, 0x8, 0x7, 0x6, 0x5, 0x4, 0x3, 0x2, 0x1, 0x0 ]
    n2 = [ 0x6, 0x7, 0x4, 0x5, 0x2, 0x3, 0x0, 0x1, 0xE, 0xF, 0xC, 0xD, 0xA, 0xB, 0x8, 0x9 ]
    n3 = [ 0xC, 0xD, 0xE, 0xF, 0x8, 0x9, 0xA, 0xB, 0x4, 0x5, 0x6, 0x7, 0x0, 0x1, 0x2, 0x3 ]
    v = (n1[b1>>4]<<8) | (n2[b1&0xF]<<4) | n3[b2>>4]
    return v

I derived these permutations by putting the ISS in a freezer to make the 12 bits cover a good range, with slow movement of the temperature. The n1 permutation is not really certain, and contains quite a lot of guesswork (as temperature only covered two of these values), but n2 and n3 are more certain. This results in the following temperature formula for message type 7:

Code: [Select]
def temperature(b1, b2):
    v = decode(b1, b2)
    farenheit = v / 7.0 - 30.0
    celcius = (farenheit - 32) * 5 / 9
    return celcius

That seems to match the temperature recorded by an independent sensor (my EnviR home monitor) quite well.

I haven't yet worked out the message types and scaling of barometric pressure, humidity and rain. I'll put the ISS under a slow tap tomorrow to get the rain readings.

I'm planning on making the firmware react much like a real ISS console, so it can be used as a simple USB dongle that behaves just like a real davis console, so the various commands work. I'd like it to work "out of the box" with the weewx weather program, which is what we use at the flying field.

Cheers, Tridge

PS: Thanks to the people at Davis for making the protocol interesting enough to be a challenge, but easy enough to work out with a bit of effort!

Offline DeKay

  • Forecaster
  • *****
  • Posts: 399
    • Mad Scientist Labs
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #1 on: April 09, 2013, 10:57:35 PM »
Following on the great work by DeKay and others...

No.  Thanks to you, tridge   =D&gt;

[dk@laptop ~]$ pacman -Q samba rsync
samba 3.6.13-1
rsync 3.0.9-5

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #2 on: April 09, 2013, 10:59:42 PM »
I've now moved the code to its own git repository, here:
 
https://github.com/tridge/DavisSi1000

One of the things I'd like to do is make this work with the other protocol variants that Davis ISS versions use. Once I have the emulation sufficient for weewx to run I'll see if I can work in support for the ones that RayDees and DeKay have described.

Cheers, Tridge

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #3 on: April 10, 2013, 01:11:39 AM »
[dk@laptop ~]$ pacman -Q samba rsync
I guess I am not as anonymous as I expected to be in a weather forum :-)

I loved your MAME/MESS posting btw - very nice hack! It tempted me to try to load the current davis fw in MAME, although I suspect I can work out all the protocol bits we need just by putting the ISS in unusual conditions. I wondered if I can work out the humidity field by having the ISS in the bathroom when I take a shower?

I really wish I'd started on this protocol when I still had a working Davis console. It would have been so much easier just to generate packets and watch what the console displayed.

Cheers, Tridge

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #4 on: April 10, 2013, 08:00:46 AM »
Tridge...

I might be able to help.

I have a Vantage Pro2 wireless (f/w 3.0) only purchased a few weeks ago from USA. Have the usa to aus a/c adapter and have the unit up and running. Would love to  upgrade to f/w 3.12 as I can then connect to my Raspberry Pi..This is not possible with f/w 3.0.

Have u a Davis rs-232 cable that would allow u to upgrade my unit? If u have I am happy to loan you my unit for a few weeks for testing if you can.

Ideas?

BTW I'm in Sydney and will be in Canberra on the weekend of the 27th April

Offline torkelmj

  • Contributor
  • ***
  • Posts: 145
    • My weather station, etc.
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #5 on: April 10, 2013, 11:38:54 AM »
Upgrading FW 3.00 to 3.12 on a new console (2012-...) still won't give you access to the serial line. Yet.

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #6 on: April 10, 2013, 05:30:39 PM »
I not really familiar with the firmware differences. I'd just bought a ISS and console on ebay, then soldered a TTL serial adaptor onto the connector on the console to get serial onto a RaspberryPi for weewx.
I still have the box the ISS and console came in, and it says:
 CON FW: 2.14
 SIM FW: 2.5.56
I presume those are the console firmware versions and ISS versions. So presumably the protocol decoding I'm doing is for the protocol used by the 2.5.56 firmware.
Is it possible to upgrade/downgrade the firmware on the ISS? I don't see any exposed ports to attach to for fw upload.

I did the shower test on my ISS last night. I put it under a slow drop in the shower, and collected the water in a bucket. I collected around 2.63L of water in 4 hours. Looking at the logs, it seems that the main rain message is type 1, which wrapped through a 11 bit range 6 times over the 4 hours, giving a total delta of around 14000. The manual says that each tipping spoon is 0.01", and the area of collection is 18.1 square inches, so that means each spool is 2.95 millilitres. That means 2.63L is 891 spoons, yet the counter went up by around 14000.

Hmm, looking at the data more closely, the counter always goes up by a multiple of 16, so it seems very likely that message type 1 is number_of_spoons*16, with an 11 bit range.

So that means I now have wind speed, wind direction, temperature and rain decoded. Just humidity and barometric pressure to go. I need to think of some creative ways to decode those :-)

Cheers, Tridge

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #7 on: April 10, 2013, 05:39:02 PM »
I might be able to help.
That's very kind of you!

I'm not really familiar with the implications of the different fw versions. If you wanted to drop past my place we could get a capture of the raw wireless data from your ISS and see what it looks like. If you had time, we could also do the "drive with the ISS out the window" test to get the wind scaling, and the "in the freezer" test for temperature. We could put my ISS in the same freezer to compare. We could also do the "run a tap through the ISS" test for rain.

Quote
Have u a Davis rs-232 cable that would allow u to upgrade my unit? If u have I am happy to loan you my unit for a few weeks for testing if you can.
sorry, I don't have any special cables, but I do have a soldering iron  :-)
Quote
BTW I'm in Sydney and will be in Canberra on the weekend of the 27th April
Great! I should be around on that weekend. I'm in the white pages under Andrew Tridgell if you want to get in touch. It would be fun to compare the differences in the wireless protocol.
Cheers, Tridge


Offline BCJKiwi

  • Forecaster
  • *****
  • Posts: 302
    • Silver Acorn Weather - N.Z.
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #8 on: April 10, 2013, 07:17:57 PM »
Don't see where you have mentioned which Davis model you purchased.

Con FW: 2.14 suggests a Vantage Vue which may account for some of the differences you are seeing relative to DeKay's work which I think is on Vantage Pro 2.

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #9 on: April 10, 2013, 07:48:37 PM »
Upgrading FW 3.00 to 3.12 on a new console (2012-...) still won't give you access to the serial line. Yet.

Torkelmj

In your write up about WOSPi (2 April 2013) software you did mention that f/w 3.12 now allows comms. with RS-232 on certain models, one of which is the 6312. Looking through the docs there doesn't seem to be a huge difference between the 6312 and the 6152 (maybe under the hood there is) - is there something specific to the 6152 or is it a build date issue. Thanks in advance.

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #10 on: April 10, 2013, 07:50:32 PM »
I not really familiar with the firmware differences. I'd just bought a ISS and console on ebay, then soldered a TTL serial adaptor onto the connector on the console to get serial onto a RaspberryPi for weewx.
I still have the box the ISS and console came in, and it says:
 CON FW: 2.14
 SIM FW: 2.5.56
I presume those are the console firmware versions and ISS versions. So presumably the protocol decoding I'm doing is for the protocol used by the 2.5.56 firmware.
Is it possible to upgrade/downgrade the firmware on the ISS? I don't see any exposed ports to attach to for fw upload.

I did the shower test on my ISS last night. I put it under a slow drop in the shower, and collected the water in a bucket. I collected around 2.63L of water in 4 hours. Looking at the logs, it seems that the main rain message is type 1, which wrapped through a 11 bit range 6 times over the 4 hours, giving a total delta of around 14000. The manual says that each tipping spoon is 0.01", and the area of collection is 18.1 square inches, so that means each spool is 2.95 millilitres. That means 2.63L is 891 spoons, yet the counter went up by around 14000.

Hmm, looking at the data more closely, the counter always goes up by a multiple of 16, so it seems very likely that message type 1 is number_of_spoons*16, with an 11 bit range.

So that means I now have wind speed, wind direction, temperature and rain decoded. Just humidity and barometric pressure to go. I need to think of some creative ways to decode those :-)

Cheers, Tridge


Tridge,

I am using the Vantage Pro2 Wireless M/n 6152. What system are you using? Your f/w numbers would indicate a different Davis Console.

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #11 on: April 10, 2013, 08:14:57 PM »
Con FW: 2.14 suggests a Vantage Vue which may account for some of the differences you are seeing relative to DeKay's work which I think is on Vantage Pro 2.
yep, it is a Vantage Vue.

Does the console for a Vantage Pro2 work with a Vue and vice-versa? My guess is that the console will detect the protocol version based either on the checksum value or the last 2 bytes.

In other news I've worked out another protocol type. Message type 8 seems to be light level. In a fairly dark room I get a value of around 200. In bright (winter) sunlight I get over 3000. I'm not sure what the units are. Message 8 is sent every 54 seconds.

I suspect message type 12 is pressure. I'll need to either take my ISS up a hill, or wait for a low pressure system to come over canberra to confirm that.

Cheers, Tridge

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #12 on: April 10, 2013, 08:20:28 PM »
I am using the Vantage Pro2 Wireless M/n 6152. What system are you using? Your f/w numbers would indicate a different Davis Console.
The sticker on the back of what is left of the console says "Product #6351" and "Model No 6250"
It was part of a packaged set of a Vantage Vue sensor suite and console.
The vandals must have had some serious anger to vent. They did a pretty thorough job of smashing it.
Cheers, Tridge

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #13 on: April 10, 2013, 08:27:45 PM »
btw, one of the things that could come out of this is receiving data from a Davis weather station at much longer ranges. The firmware I'm writing also works on this radio:
http://rfdesign.com.au/index.php/rfd900
That has a 20dB LNA, and a nice diversity antenna. I suspect it would pick up the signals from a ISS at many times the range that a 3DR radio would. It would still need line of sight (as 900Mhz is lousy through hills), but I'd expect many kilometers of range - perhaps 10-20km or so would be feasible with a cheap antenna. With a good antenna it could be more.
Cheers, Tridge

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #14 on: April 10, 2013, 09:54:35 PM »
I am using the Vantage Pro2 Wireless M/n 6152. What system are you using? Your f/w numbers would indicate a different Davis Console.
The sticker on the back of what is left of the console says "Product #6351" and "Model No 6250"
It was part of a packaged set of a Vantage Vue sensor suite and console.
The vandals must have had some serious anger to vent. They did a pretty thorough job of smashing it.
Cheers, Tridge


Tridge,

Page 1 of the Vantage Pro2 Console Manual states that it can receive data from the Vanatge Vue ISS. Progress.
Before your console was trashed how did you connect it to the PC (RS-232 or USB ?)

Offline torkelmj

  • Contributor
  • ***
  • Posts: 145
    • My weather station, etc.
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #15 on: April 10, 2013, 11:28:14 PM »
In your write up about WOSPi (2 April 2013) software you did mention that f/w 3.12 now allows comms. with RS-232 on certain models, one of which is the 6312. Looking through the docs there doesn't seem to be a huge difference between the 6312 and the 6152 (maybe under the hood there is) - is there something specific to the 6152 or is it a build date issue. Thanks in advance.

Sorry I wasn't more specific.

FW v. 3.12 on "old" consoles poses no problems with the serial line communications.
FW v. 3.12 on "new" consoles (originally delivered with FW v. 3.00) still requires the original data logger. My guess is that the FW simply reads the model number/board ID and applies the restrictions accordingly.

...but we're working on that.

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #16 on: April 10, 2013, 11:39:16 PM »
Page 1 of the Vantage Pro2 Console Manual states that it can receive data from the Vanatge Vue ISS. Progress.
ok, nice to know
Quote
Before your console was trashed how did you connect it to the PC (RS-232 or USB ?)
I wired it straight into recv, send, gnd on the connector on the back of the console, using the wiring from this posting by DeKay:
http://madscientistlabs.blogspot.com.au/2011/10/build-your-own-davis-console-datalogger.html
I connected it straight to the TTL serial pins on the RaspberryPi, and then ran weewx on the RPi to log the data. The resulting archives were then copied over a 3G link from the airfield to my home server, using rsync. Another copy of weewx runs on my server to display the results, with that 2nd copy pointed at the rsync archive directory.
Cheers, Tridge

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #17 on: April 10, 2013, 11:42:32 PM »
FW v. 3.12 on "old" consoles poses no problems with the serial line communications.
FW v. 3.12 on "new" consoles (originally delivered with FW v. 3.00) still requires the original data logger. My guess is that the FW simply reads the model number/board ID and applies the restrictions accordingly.

...but we're working on that.
It may be simpler just to buy a 3DR radio and grab the data directly. You can get a 3DR radio pretty cheaply ($35 for a original version with a USB connector, cheaper for a clone).
Cheers, Tridge

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #18 on: April 10, 2013, 11:45:08 PM »
In your write up about WOSPi (2 April 2013) software you did mention that f/w 3.12 now allows comms. with RS-232 on certain models, one of which is the 6312. Looking through the docs there doesn't seem to be a huge difference between the 6312 and the 6152 (maybe under the hood there is) - is there something specific to the 6152 or is it a build date issue. Thanks in advance.

Sorry I wasn't more specific.

FW v. 3.12 on "old" consoles poses no problems with the serial line communications.
FW v. 3.12 on "new" consoles (originally delivered with FW v. 3.00) still requires the original data logger. My guess is that the FW simply reads the model number/board ID and applies the restrictions accordingly.

...but we're working on that.

Sweet.. got it now. Thanks for the clarification and still buggered - but maybe our time will come.

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #19 on: April 10, 2013, 11:50:38 PM »
FW v. 3.12 on "old" consoles poses no problems with the serial line communications.
FW v. 3.12 on "new" consoles (originally delivered with FW v. 3.00) still requires the original data logger. My guess is that the FW simply reads the model number/board ID and applies the restrictions accordingly.

...but we're working on that.
It may be simpler just to buy a 3DR radio and grab the data directly. You can get a 3DR radio pretty cheaply ($35 for a original version with a USB connector, cheaper for a clone).
Cheers, Tridge


Tridge,

Sounds very interesting..I will need to look into 3dr radio...more of a Radio National man myself.

Offline dmurphydrtc

  • Member
  • *
  • Posts: 19
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #20 on: April 10, 2013, 11:53:29 PM »
Page 1 of the Vantage Pro2 Console Manual states that it can receive data from the Vanatge Vue ISS. Progress.
ok, nice to know
Quote
Before your console was trashed how did you connect it to the PC (RS-232 or USB ?)
I wired it straight into recv, send, gnd on the connector on the back of the console, using the wiring from this posting by DeKay:
http://madscientistlabs.blogspot.com.au/2011/10/build-your-own-davis-console-datalogger.html
I connected it straight to the TTL serial pins on the RaspberryPi, and then ran weewx on the RPi to log the data. The resulting archives were then copied over a 3G link from the airfield to my home server, using rsync. Another copy of weewx runs on my server to display the results, with that 2nd copy pointed at the rsync archive directory.
Cheers, Tridge


Unfortunately I have one of the newer builds from Davis and even with an upgrade can't use the rs-232 comms. port. Hopefully that will change in the coming months. The 3dr radio is looking like a work around. I will send u an email and discuss loaning the Vantage Pro2 regardless.

Offline DeKay

  • Forecaster
  • *****
  • Posts: 399
    • Mad Scientist Labs
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #21 on: April 11, 2013, 04:43:18 PM »
The first 4 bits of the leading byte is the packet type, but the packet types don't match what previous people have found. For example, DeKay found that type 8 was temperature, whereas in mine it became clear that temperature is type 7.

I haven't studied your code at all but have noticed that you are doing a lot of bit shifting and flipping that I don't seem to have to do in the protocol I'm working with.  This wouldn't have anything to do with missing the fact that the ISS sends the least significant bits of each byte first, does it???

My current theory is that the console uses the result of the checksum calculation to determine the protocol version, or perhaps use the two trailer bytes. The fact that the freq hopping and modulation is the same would mean that can have one console that supports many different protocols.

Old Vantage VP2 consoles with old firmware can receive the transmissions from the newest Vues, if I understand correctly.  I'm surprised that Davis would have planned out various protocol versions so far ahead of time and baked in support for that in the original code.

The bottom 3 bits of byte 1 are the transmitter ID. Changing the ID of the ISS changes these bits, and doesn't affect anything else in the protocol (so same hopping sequence).

This is really surprising to me.  I always expected that changing the ID would change the sequence.  What is known for sure is that the different IDs have a different spacing between transmissions (62.5 ms for each increment in the ID value IIRC).  So I guess that is all Davis does to mitigate interference between stations???  Lazy buggers  :grin:

I'm planning on making the firmware react much like a real ISS console, so it can be used as a simple USB dongle that behaves just like a real davis console, so the various commands work. I'd like it to work "out of the box" with the weewx weather program, which is what we use at the flying field.

I suspect message type 12 is pressure. I'll need to either take my ISS up a hill, or wait for a low pressure system to come over canberra to confirm that.

The Pro2 consoles have the pressure sensor in the console and I suspect the Vue does as well.  The sensor used by Davis is also an expensive beast.  I think I saw it for around $30 on DigiKey.

If you are going to try to do a proper emulation of the Davis console for WeeWx, I'd recommend picking up a cheap BMP085 on a breakout board on EBay.  This is the one I bought and I just got it working on my Stellaris Launchpad this morning.  While you're at it, you'll also want to pick up a cheap temperature / humidity sensor so you can properly emulate the indoor temperature and humidity readings.  The latest consoles use a DHT11.  The DHT22 I linked is higher res but still comes in under five bucks   \:D/

I loved your MAME/MESS posting btw - very nice hack! It tempted me to try to load the current davis fw in MAME, although I suspect I can work out all the protocol bits we need just by putting the ISS in unusual conditions. I wondered if I can work out the humidity field by having the ISS in the bathroom when I take a shower?

Thanks!  It started off well but got horribly messy very quickly.  The problem is that all these little Atmel chips have different registers scattered in different memory locations with different bit definitions within those registers.  The current AVR emulator in MAME isn't very flexible in this respect, so modifying the AVR core to reasonably faithfully emulate the ATMEGA128L in my console would be a ton of work.  The instruction set itself is easy compared to a decent emulation of all the peripherals like SPI, I2C, and timers.

In other news I've worked out another protocol type. Message type 8 seems to be light level. In a fairly dark room I get a value of around 200. In bright (winter) sunlight I get over 3000. I'm not sure what the units are. Message 8 is sent every 54 seconds.

The Vantage Pro2 Plus has a proper solar sensor.  The regular Pro2 and the Vue do not.  I suspect you've tripped over the next best thing: a reading of the solar panel voltage in units of mV.  Try hooking a DMM to the thing and see if the values agree.  There might be some small diode drop offset between the two though.

btw, one of the things that could come out of this is receiving data from a Davis weather station at much longer ranges. The firmware I'm writing also works on this radio:
http://rfdesign.com.au/index.php/rfd900
That has a 20dB LNA, and a nice diversity antenna. I suspect it would pick up the signals from a ISS at many times the range that a 3DR radio would. It would still need line of sight (as 900Mhz is lousy through hills), but I'd expect many kilometers of range - perhaps 10-20km or so would be feasible with a cheap antenna. With a good antenna it could be more.

Cool!  I bookmarked the DIY Drones thread on this radio ever since it was mentioned on Hack-A-Day months ago.  That is a great looking radio.  This would be a really fun hobby I'd like to get in to, but I don't have the time for all the other hobbies I have now.  Any update for where RCLink is at for it that would add long range RC control?

By the way tridge, you might be interested in something that comes out of the different firmware versions discussed here.  For a while, Davis introduced some code to prevent unauthorized DIY dataloggers from working with the console.  They did this by writing a one-time programmable key to the serial EEPROM based on a 64-byte non-alterable key written to the flash by Atmel.  Have a look at this thread if you have some time on your hands and comment in there if anything jumps out at you.  If this encryption could be broken, we could go back to DIY dataloggers / interfaces for those who bought a console with 3.0 firmware in it.

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #22 on: April 12, 2013, 01:49:42 AM »

I haven't studied your code at all but have noticed that you are doing a lot of bit shifting and flipping that I don't seem to have to do in the protocol I'm working with.  This wouldn't have anything to do with missing the fact that the ISS sends the least significant bits of each byte first, does it???

I'm doing a MSB -> LSB conversion of the data bytes in the Si1000 fw, so the data my davis_show.py script operates on is LSB. I had noticed the LSB/MSB stuff in your code, so I did initially wonder if it was just a bit-endianness issue, but after looking at the data it became clear that it is a real permutation of the values, not just bit order.

The simplest case where this is clear is the wind speed byte. It shows 0xE with zero wind. After plotting a few combinations it became clear it is XORd with 0xE.

The wind direction has a complete permutation of the bytes. I rotated the vane slowly to get the byte ordering, and then worked out the ordering from the transitions as it rotates. It ended up like this:

Code: [Select]
def wind_direction(b):
    '''return wind direction in degrees. Zero degrees is the arrow pointing at the front of the unit, away
    from the solar panel'''
    n1 = [0xD, 0xE, 0xF, 0x0, 0x9, 0xA, 0xB, 0xC, 0x5, 0x6, 0x7, 0x8, 0x1, 0x2, 0x3, 0x4]
    n2 = [0xD, 0xC, 0xF, 0xE, 0x9, 0x8, 0xB, 0xA, 0x5, 0x4, 0x7, 0x6, 0x1, 0x0, 0x3, 0x2]
    v = (n1[b>>4]<<4) | n2[b&0xF]
    return v * 360 / 255

I don't see how a MSB/LSB change would help with this, but maybe I'm just suffering from brain fade :-)

The nice thing is that all the other message types seem to use the same decoding, using the 12 bits starting at byte 4, with this decoder:

Code: [Select]
def decode(b1, b2):
    '''decode the 4th and 5th bytes of the packet, using 12 bits to give a value for some field'''
    n1 = [ 0xF, 0xE, 0xD, 0xC, 0xB, 0xA, 0x9, 0x8, 0x7, 0x6, 0x5, 0x4, 0x3, 0x2, 0x1, 0x0 ]
    n2 = [ 0x6, 0x7, 0x4, 0x5, 0x2, 0x3, 0x0, 0x1, 0xE, 0xF, 0xC, 0xD, 0xA, 0xB, 0x8, 0x9 ]
    n3 = [ 0xC, 0xD, 0xE, 0xF, 0x8, 0x9, 0xA, 0xB, 0x4, 0x5, 0x6, 0x7, 0x0, 0x1, 0x2, 0x3 ]
    v = (n1[b1>>4]<<8) | (n2[b1&0xF]<<4) | n3[b2>>4]
    return v

The n1 pattern is just subtraction, but the other two seem to be permutations. Maybe it is some simple bitop that I've missed but it doesn't really matter, as the table works fine.

Quote
Old Vantage VP2 consoles with old firmware can receive the transmissions from the newest Vues, if I understand correctly.  I'm surprised that Davis would have planned out various protocol versions so far ahead of time and baked in support for that in the original code.

Perhaps the bit permutations are computed from some key in the packets?

Quote
The bottom 3 bits of byte 1 are the transmitter ID. Changing the ID of the ISS changes these bits, and doesn't affect anything else in the protocol (so same hopping sequence).

This is really surprising to me.  I always expected that changing the ID would change the sequence.  What is known for sure is that the different IDs have a different spacing between transmissions (62.5 ms for each increment in the ID value IIRC).  So I guess that is all Davis does to mitigate interference between stations???  Lazy buggers  :grin:

I think it is not too surprising that the ID doesn't change the sequence. The ID allows for one console to receive data from multiple sensor sources. If they changed sequence of even spacing then you couldn't do that.

Interference isn't really a problem I think. The packets are only 10 bytes long, plus 2 sync bytes and 4 preamble bytes, so that is 16 bytes at 19.2kbaud. That's less than 7ms to send a packet. Packets are sent every 2.7s, so the chances of a collision with 51 channels is around 1 in 20655. So you'd need around 140 ISS devices in the same area before you get a 50% chance of two of them colliding. The timing is predictable enough that if they don't collide in the first run through 51 channels then they won't collide on the next either. So if it works today it will work tomorrow too. You'd need a lot of weather enthusiasts in an area before you had a problem!

Quote
The Pro2 consoles have the pressure sensor in the console and I suspect the Vue does as well.  The sensor used by Davis is also an expensive beast.  I think I saw it for around $30 on DigiKey.

oh, that explains why I can't find a pressure value!!

Quote
Cool!  I bookmarked the DIY Drones thread on this radio ever since it was mentioned on Hack-A-Day months ago.  That is a great looking radio.  This would be a really fun hobby I'd like to get in to, but I don't have the time for all the other hobbies I have now.  Any update for where RCLink is at for it that would add long range RC control?
Seppo is working on that I think - the main issue is the right compromise between very fast channel switching and small packets for RC control, and slow switch with large packets for efficient telemetry. I don't know if he's solved that yet.
Quote
By the way tridge, you might be interested in something that comes out of the different firmware versions discussed here.  For a while, Davis introduced some code to prevent unauthorized DIY dataloggers from working with the console.  They did this by writing a one-time programmable key to the serial EEPROM based on a 64-byte non-alterable key written to the flash by Atmel.  Have a look at this thread if you have some time on your hands and comment in there if anything jumps out at you.  If this encryption could be broken, we could go back to DIY dataloggers / interfaces for those who bought a console with 3.0 firmware in it.
I think I'll resist jumping in on that one - I'm happy playing with radios for now :-)

Cheers, Tridge

Offline aweatherguy

  • Senior Contributor
  • ****
  • Posts: 288
    • Weather Station Data Logger
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #23 on: April 12, 2013, 04:41:27 AM »
Hi, just a passive observer here with a comment. It might lead nowhere except down a rat-hole but I wanted to mention it in the hopes it might help.

Looking at all the bit wiggling you are going through makes me wonder if Davis has enabled the FEC option in the CC11xx transmitter. If you have not already found it, TI has an app note, "swra313.pdf". It describes how to un-mangle the FEC performed in the transmitter...perhaps this could un-scramble your bits...(or perhaps not).

Anyway...like I said...I'm just a passive observer and have not even read this whole thread. Please accept my apologies if I've brought up territory you've already covered.

Cheers, and best of luck in your endeavour...sounds like something I'd enjoy muddling with, except I've already got too many other irons in other fires.


« Last Edit: April 12, 2013, 04:45:54 AM by aweatherguy »

Offline tridge

  • Member
  • *
  • Posts: 20
Re: Implementing a Si1000 based wireless receiver for Davis ISS data
« Reply #24 on: April 12, 2013, 06:38:19 AM »
Looking at all the bit wiggling you are going through makes me wonder if Davis has enabled the FEC option in the CC11xx transmitter. If you have not already found it, TI has an app note, "swra313.pdf". It describes how to un-mangle the FEC performed in the transmitter...perhaps this could un-scramble your bits...(or perhaps not).
Nice idea, but I don't think it's likely. The SPI sniff in the earlier threads shows the register settings on the CC1101 as having FEC off, plus if it was a FEC scheme (eg. a golay code or similar done in software) then I'd expect the other bits to vary along with the data bits. In the data I've been examining the other data bytes are either constant, or change over only a few values.
Cheers, Tridge

 

anything