WXforum.net

Weather Station Hardware => Davis Instruments Weather Stations => Topic started by: DeKay on February 21, 2011, 02:40:21 PM

Title: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 21, 2011, 02:40:21 PM
As some people might be aware, I've been investigating the idea of using a Pretty Pink Pager (http://madscientistlabs.blogspot.com/2011/01/alternative-davis-vp2-console.html) as an alternative console to my wireless VP2.  It seems that there are at least a couple other people out there interested in the wireless side of things, so I thought I'd start a thread about this to help share the knowledge and pass ideas back and forth.  I'd be posting the hardcore details I work out in posts to my blog (http://madscientistlabs.blogspot.com) and post the odd summary here.

The first of these posts can be read at this link (http://madscientistlabs.blogspot.com/2011/02/davis-weather-station-wireless-sniffing.html).  There, I show how the console sets its receiver chip up for receiving ISS data every 2.5 seconds.  It shows which of the chip registers are configured and what they are set to.  Between that and the datasheet, you can get a very clear idea of how it does what it does.  You can also read about an extemely tenuous link I uncovered between the Davis console firmware and an amatuer satellite system.

There is still a lot of work to do of course, such as

Anyone who wants to dig in and lend a hand, please do.  Or just follow along and enjoy the show   :-)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on February 22, 2011, 02:17:28 PM
I noticed the sat guy was in the 400 mHz range.  Any issues translating that up to Davis' 902-928 DSS?
Also, I assume you have perused the docs over  at the FCC for Davis's ISS.  teh test lab reports are illuminating.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 22, 2011, 11:13:09 PM
The key for my purposes is to sniff the writes that Davis is doing to the radio chip so I can be compatible.  The satellite application is very different but might have a few useful tidbits in it.  And I linked to the FCC docs for the Vue transmitter in my blog post.  Do you know of a link to the older VP2 ISS report?  I looked around a bit but couldn't find it.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 27, 2011, 01:59:09 PM
Thought a few folks might be interested in my new development platform (http://madscientistlabs.blogspot.com/2011/02/one-small-step-for-man.html).   I believe the ISS transmissions are that bump on the left.  8-)

(https://lh4.googleusercontent.com/-oo1DciHuMwA/TWqLQL0BURI/AAAAAAAAAMI/FYNTD5V3500/s400/IMME.jpeg)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on March 20, 2011, 11:53:05 PM
Finally got my logic analyzer and have done more sniffing on how the VP2 console processor configures the radio chip.  The work this weekend was figuring out the frequency hopping sequence.  I think I've got that sorted out now, at least for Transmitter ID 1.  More at this link (http://madscientistlabs.blogspot.com/2011/03/davis-frequency-hopping-sequence.html) if you are interested.  Still lots to do, but I'm having fun doing it.  Obligatory interesting picture:
(https://lh5.googleusercontent.com/-sX_IHffUCWs/TYa6a-En6KI/AAAAAAAAANM/Wy_rWH69lhk/s400/sniffing.jpg)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: xykotik on March 21, 2011, 10:24:47 AM
It is obvious what you are up to with that prototype, DeKay.  Just like "big tobacco" uses ad agencies to create cool cartoon camels to get young boys to start smoking, "big weather" is using you to turn our innocent pre-teen txt-princesses into gadget-geeks.  4000 members here can tell you what an expensively obsessive habit that can become.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: SoMDWx on March 21, 2011, 11:23:15 AM
Someone has way too much time on their hands....
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: xykotik on March 22, 2011, 12:15:09 AM
Someone has way too much time on their hands....

It's fun, and a great service to mankind!  Far from a waste of time, unless you consider these...

...Charles Darwin sketched a lot of finches...
...Ben Franklin liked to fly kites in storms...
...Isaac Newton did a lot of daydreaming under apple trees...
...Eratosthenes liked measuring shadows...
...Archimedes spent a lot of time in the bathtub...

If you like the fact that personal computers are cheap and abundant, you will appreciate the term "reverse engineering."  If you ever destroyed an alarm clock or radio as a kid just to figure out how it worked, then you understand "hacking."  For those who still don't get it I recommend this book (http://www.amazon.com/Pleasure-Finding-Things-Out-Richard/dp/0738203491).

I for one feel I have become a better person by witnessing both the making of sausages and laws.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: SoMDWx on March 22, 2011, 07:52:21 AM
It was meant to be taken as a joke... :roll:
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: xykotik on March 22, 2011, 09:15:51 AM
Of course.  Your post sounded like my wife.  Self-defense is a natural instinct.  My response was to that voice in my head, not you in particular.

It's hard to type in this canvas jacket.  They put it on backward and the sleeves are waaaay too long.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on April 03, 2011, 03:10:22 PM
The time I've had for weather station hacking has been somewhat limited lately.  I had to:

With that now out of the way and My Lovely Wife now happily playing Angry Birds on the latter, I've finally been able to sniff how the Davis console configures the key register settings in its radio chip as it powers up.  This is key if I have any hope of to building a compatible ISS receiver.

I now know that the number of known unknowns are small.  But, of course, we all know that it is the number of unknown unknowns that kill ya.  More here (http://madscientistlabs.blogspot.com/2011/04/peeling-back-layers-of-onion.html) if you are interested.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on April 11, 2011, 11:37:31 PM
So I spent a little time this weekend figuring out some real world numbers out of the register definitions the Vantage Console does as it comes out of reset.  Some things make perfect sense.  For example, the console receives data from the ISS at the same 19.2 kHz rate that the console's expansion port runs at.  Some things make less sense, like a frequency spacing that isn't as regular as once thought.  Regardless, there is that much more information now to support the build of a compatible receiver.  More here (http://madscientistlabs.blogspot.com/2011/04/crunching-numbers.html) if you are interested.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on March 11, 2012, 01:35:01 PM
I thought a few folks might be interested in my latest work in developing an alternative receiver to the Davis console.  I've been able to sniff the raw data stream that comes from the ISS and make sense out of it.  This screenshot shows that there is a lot more to it than just the seven data bytes you see from the STRMON output.  I'm just happy that I didn't fry my console again in the process.  More details in this blog post (http://madscientistlabs.blogspot.com/2012/03/first-you-get-sugar.html). 

(http://3.bp.blogspot.com/-9zh3RPr2Z68/T1yLRA3epaI/AAAAAAAAAbY/rSySAoqZjGA/s1600/Closer+First+Look.png)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: dalecoy on March 11, 2012, 03:19:22 PM
That's very nice work.  Looking forward to the next chapter.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on March 11, 2012, 11:18:21 PM
This screenshot shows that there is a lot more to it than just the seven data bytes you see from the STRMON output.

Take another look at the CC1021  datasheet, each transmission has a preamble of 0x55, 0x55, 0x55, 0xD3, 0x91. Each packet also has a CRC suffix.


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on March 12, 2012, 10:33:52 PM
This screenshot shows that there is a lot more to it than just the seven data bytes you see from the STRMON output.

Take another look at the CC1021  datasheet, each transmission has a preamble of 0x55, 0x55, 0x55, 0xD3, 0x91. Each packet also has a CRC suffix.

The datasheet is nowhere near that precise.  My blog post quotes the datasheet directly and demonstrates how vague it is.  The datasheet simply recommends a sync word that can be two to four bytes long but nothing in hardware enforces that.  It also states what can be achieved with a three byte preamble but nothing says it can't be longer or shorter (and my blog clearly shows a preamble four bytes long from the ISS).
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ajayabb on March 13, 2012, 10:26:22 AM
I often wonder why Davis didn't release the Echo receiver for the VP2 like they had for the VP1?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on March 14, 2012, 10:22:23 PM
The datasheet is nowhere near that precise.  My blog post quotes the datasheet directly and demonstrates how vague it is.  The datasheet simply recommends a sync word that can be two to four bytes long but nothing in hardware enforces that.  It also states what can be achieved with a three byte preamble but nothing says it can't be longer or shorter (and my blog clearly shows a preamble four bytes long from the ISS).

I thought that was in the datasheet... In any case, when a console retransmits, the preamble is 3 * 0x55 and then the sync word, followed by the data. Which reminds me of a bug in the retransmit code. A low battery in the transmitter being retransmitted, will be reported as a low battery in the transmitter retransmitting.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on April 08, 2012, 01:07:56 PM
I can now receive data from the ISS.  More here (http://madscientistlabs.blogspot.ca/2012/04/achievement-unlocked-im-me-weather.html) in case you are interested.  Anybody want to help build an open source receiver?   \:D/

(http://3.bp.blogspot.com/-FToxp1ye0cs/T4EiDF4xVkI/AAAAAAAAAds/uBikjekWUOA/s400/IMME+Display.JPG")
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Devonian on April 24, 2012, 05:26:02 PM
Interesting project.
I've landed here as I've been Googlin' around, finding info on wireless weather stations and the interception and re-use of the data.
There are several snippets on various forums, using various different radio modules, mostly on 433MHz.

I've recently been playing with the HopeRF modules and a couple projects based around them using Arduino to get them to talk and listen, most recently, a Spectrum Analyser, based on RSSI within a frequency band.  All open source of course.
I'm no code writer, but can glue some bits to a PCB to get a project running.

I like the idea of a PC-less, online station using small, low power, embedded device.
http://wiki.meteohub.de/Introduction

Watching with interest...

Nigel.



Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on April 24, 2012, 11:38:33 PM
Interesting project.

Thanks.

I've landed here as I've been Googlin' around, finding info on wireless weather stations and the interception and re-use of the data.
There are several snippets on various forums, using various different radio modules, mostly on 433MHz.

I've recently been playing with the HopeRF modules and a couple projects based around them using Arduino to get them to talk and listen, most recently, a Spectrum Analyser, based on RSSI within a frequency band.  All open source of course.
I'm no code writer, but can glue some bits to a PCB to get a project running.

If you're playing with HopeRF, I'd recommend a JeeNode (http://jeelabs.com/products/jeenode) from JeeLabs (http://jeelabs.org/).  That gets you HopeRF built right onto an Arduino compatible board with lots of libraries of goodies.  And cheap too.  I've got a couple down in the basement but haven't played much with it yet.

I want to move from the IM-Me to an XRF module (http://shop.ciseco.co.uk/xrf-wireless-rf-radio-uart-rs232-serial-data-module-xbee-shape-arduino-pic-etc/) at some point because of the compatibility with the radio chip in the Davis ISS.  I have one of these XRF modules in the basement too.

I like the idea of a PC-less, online station using small, low power, embedded device.
http://wiki.meteohub.de/Introduction

Watching with interest...

Nigel.

And I like the idea of driving all this stuff with a Raspberry Pi (http://www.raspberrypi.org/).

An interesting project would be to try receiving ISS transmissions with a HopeRF module.  My skim of the datasheet shows it might be possible.  All you need to know is either posted on this forum somewhere or on my blog.  Give it a shot!

Unfortunately, my time to play around with this stuff really tails off in the summer months.  It won't be until the snow flies once again until I expect I can put significant amounts of time in to this again.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Devonian on April 25, 2012, 03:14:30 AM
I'm pretty much the same, time wise as during the summer, I fly radio control model aircraft.
A number of us are also using the HopeRF modules for R/C...
http://flytron.com/16-openlrs
and some are now making their own boards for the same thing + other ideas
http://arduino.cc/forum/index.php/topic,93777.0.html

I have my name down for a Raspberry-Pi, but it's a waiting game.

I've seen others beginning to consider using the HopeRF modules to intercept the signals from various WX transmitters and thought it might be an interesting project.
Once you know the WX Tx protocol, you're in with a chance.

I'll keep my eye on this and we'll see what develops...

Nigel.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on September 30, 2012, 11:01:33 AM
DeKay:

You have certainly put a lot of effort into this!  I had some spare time this weekend and I thought I would start helping out.  I have the Vantage Vue installed and fully operational.  My first thought was to peek inside the console just to see what is in there.  No luck - The keyboard overlay is glued so well to the main board, that I was afraid to pry it loose.  I'll just have to try another way.

Maybe I could just ask the console?  Lets try a few commands:

TEST<CR>
TEST

TST 1<CR>
OK

CHAN 0<CR>
OK

GETREG
OK
==============================================
IOCFG2(00) 2E           IOCFG1(01) 2E
IOCFG0(02) 2F           FIFOTHR(03) 42
SYNC1(04) CB            SYNC0(05) 89
PKTLEN(06) 0A           PKTCTRL1(07) 04
PKTCTRL0(08) 00        ADDR(09) 00
CHANNR(0A) 00          FSCTRL1(0B) 06
FSCTRL0(0C) 00          FREQ2(0D) 22
FREQ1(0E) B4             FREQ0(0F) B4
MDMCFG4(10) C9         MDMCFG3(11) 83
MDMCFG2(12) 12         MDMCFG1(13) 21
MDMCFG0(14) F9         DEVIATN(15) 24
MCSM2(16) 07             MCSM1(17) 00
MCSM0(18) 18             FOCCFG(19) 16
BSCFG(1A) 6C              AGCCTRL2(1B) 43
AGCCTRL1(1C) 40        AGCCTRL0(1D) 91
WOREVT1(1E) 87         WOREVT0(1F) 6B
WORCTRL(20) F8         FREND1(21) 56
FREND0(22) 10            FSCAL3(23) EF
FSCAL2(24) 2B            FSCAL1(25) 28
FSCAL0(26) 1F            RCCTRL1(27) 00
RCCTRL0(28) 00          FSTEST(29) 59
PTEST(2A) 7F              AGCTEST(2B) 3F
TEST2(2C) 88              TEST1(2D) 31
TEST0(2E) 0B              PARTNUM(30) 00
VERSION(31) 04          FREQEST(32) 00
LQI(33) FF                  RSSI(34) 80
MARCSTATE(35) 01        WORTIME1(36) 00
WORTIME0(37) 00         PKTSTATUS(38) 90
VCO_VC_DAC(39) F4       TXBYTES(3A) 00
RXBYTES(3B) 00          RCCTRL1_STATUS(3C) 00
RCCTRL0_STATUS(3D) 00   PATABLE(3E) C6

==============================================

I'm out of time for now, I'll continue this later!




Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: chief-david on September 30, 2012, 11:04:57 AM
Sometimes there are posts that just make me feel dumb   ](*,) ](*,)

-I know that is not your intent.

I have no clue what any of that means.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on September 30, 2012, 01:26:23 PM
It's a dump of the radio registers. Already aware of it and they are playing with some risky commands to be using when one doesn't know what they do.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 08, 2012, 07:34:12 PM
I've sworn off "risky commands".  I think that I will just "sniff" the SPI bus directly and see what I get for the Vantage Vue Hop Sequence.  Here it is:

Index: 00, FREQ2: 23, FREQ1: 0D, FREQ0: 97
Index: 01, FREQ2: 22, FREQ1: B4, FREQ0: AB
Index: 02, FREQ2: 23, FREQ1: 12, FREQ0: 89
Index: 03, FREQ2: 23, FREQ1: 7F, FREQ0: 39
Index: 04, FREQ2: 23, FREQ1: 30, FREQ0: 2D
Index: 05, FREQ2: 22, FREQ1: DC, FREQ0: 31
Index: 06, FREQ2: 23, FREQ1: 9C, FREQ0: DD
Index: 07, FREQ2: 23, FREQ1: 52, FREQ0: C3
Index: 08, FREQ2: 22, FREQ1: F4, FREQ0: E5
Index: 09, FREQ2: 23, FREQ1: 66, FREQ0: 85
Index: 10, FREQ2: 23, FREQ1: 21, FREQ0: 5B
Index: 11, FREQ2: 22, FREQ1: C3, FREQ0: 7D
Index: 12, FREQ2: 23, FREQ1: 43, FREQ0: EF
Index: 13, FREQ2: 23, FREQ1: 8E, FREQ0: 0B
Index: 14, FREQ2: 23, FREQ1: 03, FREQ0: B7
Index: 15, FREQ2: 22, FREQ1: CD, FREQ0: 5F
Index: 16, FREQ2: 23, FREQ1: 3A, FREQ0: 0F
Index: 17, FREQ2: 23, FREQ1: 70, FREQ0: 67
Index: 18, FREQ2: 22, FREQ1: E6, FREQ0: 13
Index: 19, FREQ2: 23, FREQ1: A6, FREQ0: BF
Index: 20, FREQ2: 23, FREQ1: 1C, FREQ0: 6B
Index: 21, FREQ2: 22, FREQ1: BE, FREQ0: 8D
Index: 22, FREQ2: 23, FREQ1: 48, FREQ0: E1
Index: 23, FREQ2: 23, FREQ1: 84, FREQ0: 29
Index: 24, FREQ2: 22, FREQ1: F9, FREQ0: D5
Index: 25, FREQ2: 23, FREQ1: A1, FREQ0: CD
Index: 26, FREQ2: 22, FREQ1: D7, FREQ0: 41
Index: 27, FREQ2: 23, FREQ1: 2B, FREQ0: 3D
Index: 28, FREQ2: 23, FREQ1: 5C, FREQ0: A3
Index: 29, FREQ2: 23, FREQ1: 92, FREQ0: FB
Index: 30, FREQ2: 22, FREQ1: B9, FREQ0: 9B
Index: 31, FREQ2: 23, FREQ1: 08, FREQ0: A7
Index: 32, FREQ2: 23, FREQ1: 75, FREQ0: 57
Index: 33, FREQ2: 23, FREQ1: 35, FREQ0: 1D
Index: 34, FREQ2: 22, FREQ1: E1, FREQ0: 21
Index: 35, FREQ2: 23, FREQ1: 4D, FREQ0: D1
Index: 36, FREQ2: 23, FREQ1: AB, FREQ0: AF
Index: 37, FREQ2: 23, FREQ1: 6B, FREQ0: 75
Index: 38, FREQ2: 22, FREQ1: EF, FREQ0: F3
Index: 39, FREQ2: 23, FREQ1: 17, FREQ0: 79
Index: 40, FREQ2: 23, FREQ1: 57, FREQ0: B3
Index: 41, FREQ2: 22, FREQ1: C8, FREQ0: 6D
Index: 42, FREQ2: 23, FREQ1: 89, FREQ0: 19
Index: 43, FREQ2: 23, FREQ1: 3E, FREQ0: FF
Index: 44, FREQ2: 22, FREQ1: FE, FREQ0: C5
Index: 45, FREQ2: 23, FREQ1: 61, FREQ0: 95
Index: 46, FREQ2: 22, FREQ1: D2, FREQ0: 4F
Index: 47, FREQ2: 23, FREQ1: 7A, FREQ0: 47
Index: 48, FREQ2: 22, FREQ1: EB, FREQ0: 03
Index: 49, FREQ2: 23, FREQ1: 26, FREQ0: 4B
Index: 50, FREQ2: 23, FREQ1: 97, FREQ0: ED

This is the actual data being clocked into the radio chip.  Index "00" corresponds with the Vue console of display "0", it could be off by one.

Now, back to my real job.........

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 09, 2012, 02:38:00 PM
I've sworn off "risky commands".  I think that I will just "sniff" the SPI bus directly and see what I get for the Vantage Vue Hop Sequence.  Here it is:

Index: 00, FREQ2: 23, FREQ1: 0D, FREQ0: 97
Index: 01, FREQ2: 22, FREQ1: B4, FREQ0: AB
Index: 02, FREQ2: 23, FREQ1: 12, FREQ0: 89

<snip>

Been there, did that. (http://madscientistlabs.blogspot.ca/2011/03/davis-frequency-hopping-sequence.html)

And if you read my earlier posts in this thread, you'll see the rest of the details on how the radio is configured to receive transmissions from the ISS.  Click this link (http://madscientistlabs.blogspot.ca/search/label/Davis%20VP2) for more than you'll likely ever want to know about this stuff.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 09, 2012, 04:20:57 PM
DeKay:

I've followed your progress!  Nice work, by the way.  My next goal is to get "Hopping".

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 09, 2012, 11:32:38 PM
DeKay:

I've followed your progress!  Nice work, by the way.  My next goal is to get "Hopping".


Sorry.  I don't quite catch your meaning.  You know what the hop sequence is.  Do you mean to build your own bit of hardware of some kind???
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 10, 2012, 04:58:33 PM
Quote
Sorry.  I don't quite catch your meaning.

DeKay:

I don't have the CC1101 hopping yet, just monitoring a single channel.  The hopping sequence was captured by a microprocessor running in SPI Slave mode.  The data was captured from the expansion connector and filtered to provide only frequency data.   I have not analyzed a long sequence to see if it is random in nature or simply repeats every time.

My plan (goal) is to have a second CC1101 running in parallel with the Vue console.  I will initialize it, but feed it the frequency information from the Vue.  This way I can start and stop it, read registers or do anything else needed to perfect the hop sequence.  I can simultaneously print the STR information also.

Now all of this may seem necessary, but I have real reason for doing this.  If I program the CC1101 and park it on a channel, it only takes about 5 seconds for my screen to be completely full of data packets, depending on the channel.  I don't live in a metropolis, but talk about RF congestion!

This is what I meant by get "Hopping".  I will not be back home until October 21, so for now I can only dream about it.......

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on October 10, 2012, 10:01:09 PM
It's not random. The sequence is hard coded in the firmware, but DeKay has documented it, so you don't even have to figure out where it is.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 10, 2012, 11:43:54 PM
I didn't know that the Vue used the CC1101 vs. the CC1021 in the Pro2.  Developing a compatible receiver would have been much easier had I access to the CC1101.  For one thing,  I wouldn't have had to open my console and probe the pins of the radio chip with a sewing pin attached to the lead of my logic analyzer to sniff the actual data from the ISS.  Kids today have it easy, I walked to school uphill both ways, yadda yadda.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 11, 2012, 07:13:06 AM
C5250:

Thanks!

DeKay:

It's either using the CC1101 or a real close relative based upon the register values it writes.  (Barefooted in the snow!)

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 14, 2012, 11:55:31 AM
I don't have the CC1101 hopping yet, just monitoring a single channel.  The hopping sequence was captured by a microprocessor running in SPI Slave mode.  The data was captured from the expansion connector and filtered to provide only frequency data.   I have not analyzed a long sequence to see if it is random in nature or simply repeats every time.

What is the specific setup you are you using to capture this?  Would you share this code assuming it is on a platform commonly available?

My plan (goal) is to have a second CC1101 running in parallel with the Vue console.  I will initialize it, but feed it the frequency information from the Vue.  This way I can start and stop it, read registers or do anything else needed to perfect the hop sequence.  I can simultaneously print the STR information also.

It would be nice to see the sequence of register writes to the CC1101 as it prepares to read each packet.  It will be repetitive except for the specific frequency bytes.  I tried by best to translate the sequence for the CC1021 in my Pro2 to the CC1110 in my IM-ME, and it works OK, but it might not be optimal.  There is also a bunch of stuff written only when the console first boots up that would be good to know as well.

Another thing that would be very interesting is to set the console to re-transmit and analyze the command sequence.  This would give a clear picture on how to build a compatible transmitter.  Knowing this, it is quite likely that we could build our own temperature / humidity stations for peanuts.  I probably have most of the info I need to do this now, but it would go a lot quicker with data you might be able to supply.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 16, 2012, 10:52:50 AM
I'm using an Arduino running on 5.0 volts at 16 mHz with 3.3 to 5 volt logic level conversion to capture the data.  The program was compiled with Arduino 1.0.  (I have a data analyzer, but I've never been able to capture more than 50 seconds of data with it.)  

The data is taken from the expansion connector using SCK, MOSI and GND.  The program simply captures the data and prints it via the Serial Port.  It is shown below:


Code: [Select]
/*
  
  Arduino program to decode the Davis Vantage Vue
  Hop Sequence.  Index displays "99" until the
  program catches up with the console.
  
  by rdsman
    
  07 OCT 2012
  
  YOU MUST USE AN ARDUINO RUNNING ON 3.3 VOLTS
  OR PROVIDE LOGIC LEVEL CONVERSION!
  
  YOU HAVE BEEN WARNED!
*/
#include "Arduino.h"
//
//  Arduino Digital Pin Numbers.
//
#define SS   10  //  Forced LOW.      
#define MOSI 11
#define MISO 12  //  Not used in this program.
#define SCK  13
//
//  Global Variables.
//
volatile byte comp, freq, sync;
volatile byte freq2, freq1, freq0;
volatile byte index = 99;
//
//  Setup Loop.
//
void setup()
{
  delay(100);
  Serial.begin(115200);
  delay(100);
  Serial.println("Ready to begin processing....");
  Serial.println();
  delay(1000);
  initialize();
}  
//
//  Main Loop.
//
void loop() // run over and over
{
  if (comp)
    {
      comp = 0;
      Serial.print("Index: ");
      if (index < 10)
        Serial.print("0");
      Serial.print(index);
      Serial.print(", ");
      Serial.print("FREQ2: ");
      printHex(freq2);
      Serial.print(", ");
      Serial.print("FREQ1: ");
      printHex(freq1);
      Serial.print(", ");
      Serial.print("FREQ0: ");
      printHex(freq0);
      Serial.println();
      delay(2000);  // Yes, we want it to stop!
    }  
}
//
//  Hex print utility - Prints leading "0".
//
void printHex(byte value)
{
  Serial.print(value >> 4 & 0x0F, HEX);
  Serial.print(value >> 0 & 0x0F, HEX);
}
//
//  Setup the microprocessor.
//
void initialize()
{
  cli();                       //  Disable global interrupts while we make our changes.
    
  pinMode(SS, INPUT);          //  Set up the SPI pins.
  digitalWrite(SS, LOW);            //  SS is tied LOW.
  pinMode(MOSI, INPUT);
  pinMode(MISO, OUTPUT);
  pinMode(SCK, INPUT);
                               //  Turn off everything that might interfere!
  ADCSRA = 0x00;               //  Disable the analog comparator.
  TCCR1B = 0x00;               //  Disable Timer 1.
  TCCR2B = 0x00;               //  Disable Timer 2.
  
  SPCR = (1 << SPIE | 1 << SPE);  //  Initialize the SPI.
  
  sei();                       //  Enable global interrupts.
}
//
//  SPI Interrupt Service Routine.
//
ISR(SPI_STC_vect)
{
  byte temp = SPDR;
  
  if (!sync)
    {
      switch (temp)
        {
          case 0x0D:
            freq = 1;
            break;
          case 0x0E:
            freq = 2;
            break;
          case 0x0F:
            freq = 3;
            break;
          default:
            freq = 0;
            sync = 0;
            comp = 0;
            return;
        }
      
      sync |= 0x01;
      return;
    }    
  
  if (sync)
    {
      switch (freq)
        {
          case 1:
            freq2 = temp;
            break;
          case 2:
            freq1 = temp;
            break;
          case 3:
            freq0 = temp;
            comp |= 1;
            if (index < 51)
              index++;
            break;
          default:
            break;
        }
    }  
  
  sync = 0;
  if (freq2 == 0x23 && freq1 == 0x0D)  //  The index gets reset here, could be off by one!
    index = 0;                         //  Matches the Vue console - for what its worth.
}
//
//  The End.
//


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 16, 2012, 02:17:11 PM
I'm using an Arduino running on 5.0 volts at 16 mHz with 3.3 to 5 volt logic level conversion to capture the data.  The program was compiled with Arduino 1.0.  (I have a data analyzer, but I've never been able to capture more than 50 seconds of data with it.) 

The data is taken from the expansion connector using SCK, MOSI and GND.  The program simply captures the data and prints it via the Serial Port.  It is shown below:

Excellent!  As my dad used to say, this is as handy as a back pocket on your underwear.  My Jeenode (http://shop.moderndevice.com/products/jeenode-kit) is 3.3V, so I'm good to go without needing to do any level conversion.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 18, 2012, 09:55:19 AM
Previously I posted that one of my goals was to synchronize an external CC1101 to the frequency data coming from the Vantage Vue.  I've modified the capture code to implement a "Poor Mans" STX/ETX protocol.  The incoming data is captured via the SPI port and sent out the UART at 115200 bps.  The protocol looks like this:

STX FREQ2 FREQ1 FREQ0

The spaces were added to make it easier to read.  We don't really need the ETX, so it's left off.  The question is will we be fast enough.  The screenshot below shows the incoming data from the Vue on the top trace.  The outgoing serial data is on the bottom trace.  The outgoing is completed before the Vue has finished writing to the internal radio registers.


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 18, 2012, 06:17:51 PM
If you read my last post, you must read this one.  I have Bad news and Good news.  First the Bad, it was a Bad, Bad, Bad idea to think that using the serial port would work.  It looked good on paper and on the Logic Analyzer.  But in reality, by the time I was ready to receive data on the new frequency, the Vue had already received it and was moving on.

Now the Good news, I have it working!  I'm still getting the data via SPI, but sending it out via I2C.  Here is a snipet of the result:

E0  00  4B  01  00  0E  2D  45
50  00  4B  FF  70  0B  76  46
A0  00  4B  AE  2A  09  C0  28
E0  00  4B  01  00  07  BC  6C
50  00  4A  FF  70  03  81  FA
20  01  07  D4  C2  40  0B  C0
E0  01  2A  01  02  0A  8E  94
50  01  2A  FF  75  08  A5  70
70  01  2A  99  02  85  F1  CF
E0  01  2A  01  01  05  2A  28
50  01  2A  FF  71  05  21  8E
70  10  44  65  98  22  A3  2D
30  01  32  FF  C0  84  4C  67
E0  01  32  01  03  05  D2  2E
50  01  32  FF  70  07  AC  99

I am completely locked to the frequency pattern of the Vue console.  I get all of the 20, 30, 50, 70, 90, E0 and A0 messages.  What about the 80 series?  It doesn't appear to come from the ISS.  I've monitored it for hours without ever receiving a single 80 series message.  I have some other revelations to report, but I will save them for now.

The capture code is posted below.  The receiving end is so ugly, that my dog ran away.  I will clean it up and post it in the near future.  There is a lot of mystery left before we move on to frequency hopping on our own!



Code: [Select]
/*
 
  Arduino program to extract the Davis Vantage Vue
  frequency information and send it via I2C to the
  RF Bee or some other device.
 
  by rdsman
   
  18 OCT 2012
 
  YOU MUST USE AN ARDUINO RUNNING ON 3.3 VOLTS
  OR PROVIDE LOGIC LEVEL CONVERSION!
 
  YOU HAVE BEEN WARNED!
*/
#include "Arduino.h"
#include "Wire.h"
//
//  Arduino Digital Pin Numbers.
//
#define SS   10  //  Forced LOW.
#define MOSI 11
#define MISO 12  //  Not used in this program.
#define SCK  13
//
//  Global Variables.
//
volatile byte comp, freq, sync;
volatile byte freq2, freq1, freq0;
//
//  Setup Loop.
//
void setup()
{
  delay(100);
  initialize();
  delay(100);
  Wire.begin();

//
//  Main Loop.
//
void loop() // run over and over
{
  if (comp)
    {
      comp = 0;
      Wire.beginTransmission(0x22);  //  The I2C address was picked "Out Of The Air".  It means nothing....
      Wire.write(freq2);             //  It wasn't even converted to 7 bit format, so on the Logic Analyzer
      Wire.write(freq1);             //  it will show as 0x44!
      Wire.write(freq0);
      Wire.endTransmission();
    } 
}
//
//  Setup the microprocessor.
//
void initialize()
{
  cli();                       //  Disable global interrupts while we make our changes.
     
  pinMode(SS, INPUT);          //  Set up the SPI pins.
  digitalWrite(SS, LOW);       //  SS is tied LOW.
  pinMode(MOSI, INPUT);
  pinMode(MISO, OUTPUT);
  pinMode(SCK, INPUT);
                               //  Turn off everything that might interfere!
  ADCSRA = 0x00;               //  Disable the analog comparator.
  TCCR1B = 0x00;               //  Disable Timer 1.
  TCCR2B = 0x00;               //  Disable Timer 2.
 
  SPCR = (1 << SPIE | 1 << SPE);  //  Initialize the SPI.
 
  sei();                       //  Enable global interrupts.
}
//
//  SPI Interrupt Service Routine.
//
ISR(SPI_STC_vect)
{
  byte temp = SPDR;
 
  if (!sync)
    {
      switch (temp)
        {
          case 0x0D:
            freq = 1;
            break;
          case 0x0E:
            freq = 2;
            break;
          case 0x0F:
            freq = 3;
            break;
          default:
            freq = 0;
            sync = 0;
            comp = 0;
            return;
        }
     
      sync |= 0x01;
      return;
    }   
 
  if (sync)
    {
      switch (freq)
        {
          case 1:
            freq2 = temp;
            break;
          case 2:
            freq1 = temp;
            break;
          case 3:
            freq0 = temp;
            comp |= 1;
            break;
          default:
            break;
        }
     
     sync = 0;   
       
    }   
}
//
//  The End.
//


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 18, 2012, 06:35:21 PM
I am completely locked to the frequency pattern of the Vue console.  I get all of the 20, 30, 50, 70, 90, E0 and A0 messages.  What about the 80 series?  It doesn't appear to come from the ISS.  I've monitored it for hours without ever receiving a single 80 series message.  I have some other revelations to report, but I will save them for now.

Well, 80 is temperature, so either you are doing something wrong or the temperature has reached absolute zero at your house.

You might be synced in frequency but perhaps you are losing something in the data.  I've documented the STRMON output here (https://github.com/dekay/im-me/blob/master/pocketwx/src/protocol.txt), so check the values you are getting to see if they make any sense.  You should check the CRC as well.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on October 18, 2012, 10:10:31 PM
Absolute zero?  So THAT is where my ex-wife went.  ;)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on October 18, 2012, 10:23:27 PM
I've documented the STRMON output here (https://github.com/dekay/im-me/blob/master/pocketwx/src/protocol.txt), ...

Uhm... this:
Code: [Select]
The lowest three bits in the low order nibble is the transmitter ID.
  0 = ISS
  1 = Anemometer transmitter
  4 = Temperature / humidity station
  5 = Envoy retransmitting ISS
  7 = Temperature / humidity station
is not correct.

The first three LSB bits of the first STRMON byte are the absolute TX ID, that is as configured with the dip switch -1. In no way do they signify what type of station is transmitting. Keep in mind that the transmitter doesn't know what it is.  :-)

Further, certain sensor packets are also used for other purposes.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 19, 2012, 09:51:50 AM
I've barely scratched the surface on using the CC1101 correctly, so I am probably doing many things wrong!  However, these are my initial observations:

I have not received one single 80 message.
I get all other messages every time that STRMON prints them!
I don't get a 40 or 60 message - Neither does my STRMON output.
I get a 20, 30 and 70 message - Same as my STRMON output.
The 30, 50, 70, 90 and E0 messages are 8 bytes long that match exactly to the STRMON output - every time.
The 20 and A0 messages appear to be 8 bytes long, but the CRC bytes never match the STRMON output.  If I calculate the CRC using my first 6 received bytes, it matches the STRMON output.


 
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 19, 2012, 04:28:27 PM
Gentlemen:

By using my finely honed engineering skills, I have discovered (stumbled upon) the problem with my code.  Here is a snipet of the results:

80  03  34  2F  39  0C  08  9C
30  03  2E  FF  C2  89  EB  BE
E0  03  35  01  02  09  35  3D
50  04  38  FF  72  01  E1  C1
80  04  40  2F  3B  05  10  87
90  04  3F  0D  92  34  F5  AA
E0  05  39  01  00  06  20  07
50  07  53  FF  72  01  48  DE
80  07  58  2F  38  0A  C4  8D
A0  05  5E  CA  08  0A  DC  CB
E0  04  5E  01  00  05  B2  CA
50  03  5E  FF  71  07  CD  CB
80  02  5E  2F  3A  0A  A6  21
20  02  53  D4  C2  88  46  8E
E0  03  43  01  02  0F  30  17

My latest version matches byte for byte with STRMON.  There is still plenty left to do!


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 27, 2012, 09:08:56 PM
I haven't worked on this since my last post, so there is no progress to report.  However, for those of you who want to follow along or help, I've posted my current work on my website.  The link below will take you to a site where you can download Davis.zip.  

http://www.raydees.com/Downloads.html


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on January 06, 2013, 12:44:04 PM
I've updated my work on emulating the Davis protocol with the RFBee.  The library has been cleaned up and now implements a super simple hopping scheme.  It can currently maintain a PER of less than .2%   There are two example sketches included.  The first one simply prints out the received packets as:

Index: 04  Packet: E0  03  87  45  00  08  9E  AA  FF  FF  RSSI: -75  LQI: 30  Count: 15766  Miss: 4

The second one uses Dekay’s formulas and outputs something useful, such as:

SPD: 3  DIR: 155  TMP: 49  HUM: 73  RSSI: -77

Hopefully, this will be interest some of you!

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on January 06, 2013, 07:46:07 PM
I've updated my work on emulating the Davis protocol with the RFBee.  The library has been cleaned up and now implements a super simple hopping scheme.  It can currently maintain a PER of less than .2%   There are two example sketches included.  The first one simply prints out the received packets as:

Index: 04  Packet: E0  03  87  45  00  08  9E  AA  FF  FF  RSSI: -75  LQI: 30  Count: 15766  Miss: 4

The second one uses Dekay’s formulas and outputs something useful, such as:

SPD: 3  DIR: 155  TMP: 49  HUM: 73  RSSI: -77

Hopefully, this will be interest some of you!


Very interesting indeed!  I'm impressed by the low packet error rate!  I was thinking of doing something similar and you've beaten me to it.    \:D/

Next step would be to pick up a BMP085 for humidity and a DHT22 for temperature / pressure.  These are five or six bucks each with free shipping on EBay and there are already existing Arduino libraries to drive them.  From there the Davis LOOP and a few other commands could be cloned and whammo... a DIY console minus the display suitable for connecting to Cumulus and the like.

Again, great job   =D&gt;
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on January 06, 2013, 08:17:56 PM
Forget the console - build low cost temp/hum sensors and the world is your oyster!
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: belfryboy on January 07, 2013, 03:09:29 AM
Forget the console - build low cost temp/hum sensors and the world is your oyster!
I already do, pin compatible with davis, and all for around £35
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on January 07, 2013, 10:02:54 AM
Forget the console - build low cost temp/hum sensors and the world is your oyster!
I already do, pin compatible with davis, and all for around £35

More info PLEASE!!!  Pics, links etc.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on January 07, 2013, 12:39:15 PM

Next step would be to pick up a BMP085 for humidity and a DHT22 for temperature / pressure.  These are five or six bucks each with free shipping on EBay and there are already existing Arduino libraries to drive them.  From there the Davis LOOP and a few other commands could be cloned and whammo... a DIY console minus the display suitable for connecting to Cumulus and the like.


The RFBee doesn't have enough horsepower to do the full emulation.  One of the ideas that I have is to use the RFBee as just another sensor attached to the I2C bus.  It's contents could be read just like a real time clock, BMP085 or the DHT22.  Then use a MEGA based unit to tie everything together.  Any thoughts on this approach?

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on January 07, 2013, 07:47:55 PM

Next step would be to pick up a BMP085 for humidity and a DHT22 for temperature / pressure.  These are five or six bucks each with free shipping on EBay and there are already existing Arduino libraries to drive them.  From there the Davis LOOP and a few other commands could be cloned and whammo... a DIY console minus the display suitable for connecting to Cumulus and the like.


The RFBee doesn't have enough horsepower to do the full emulation.  One of the ideas that I have is to use the RFBee as just another sensor attached to the I2C bus.  It's contents could be read just like a real time clock, BMP085 or the DHT22.  Then use a MEGA based unit to tie everything together.  Any thoughts on this approach?


Daisy-chaining arduinos just seems so wrong.  It is a shame these things aren't a little more capable.

What about scrapping the Arduino altogether and going with a straight CC1101 module (http://dx.com/p/cc1101-wireless-module-w-external-antenna-blue-160898) running off a Raspberry Pi?  Because at the end of the day, you'll want to hook those two Arduino's up to something with real horsepower anyway.  Check out what is going on at JeeLabs (http://jeelabs.org/2013/01/07/technology-decisions/) these days.  Hang a bunch of raw sensors off the Pi which can then serve a web page so your phone takes the place of the console LCD.  The Pi could also push data out to someplace like wunderground, archive the data on a monster flash card or USB stick, collect data from other devices, etc.  Might be able to write all the code in Python as well (see here) (http://hackaday.com/2013/01/06/hardware-spi-with-python-on-a-raspberry-pi/) to make things more accessible.

Thoughts on this approach?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on January 08, 2013, 09:45:12 AM
DeKay:

The RFBee is fully capable of being a stand-alone receiver.  It could be used to monitor an ISS and provide an alarm if the temp drops below freezing or windspeed exceeds 25 mph.  It could be used as a stand alone range extender (One of my goals).  You could hook a low cost serial or I2C display and
monitor your neighbor's ISS and really save some money!  I'm just not sure how well (if at all) it would perform trying to run a rtc, temp, humidity, etc
and process the Loop command.  The RFBee only has 16k of flash and 1k of sram.  I will attempt it, however.

I do like your idea with the Raspberry Pi, but I also like this:

http://www.seeedstudio.com/depot/dragrove-generic-gateway-for-internet-of-things-p-1118.html?cPath=139_141

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on January 08, 2013, 03:26:15 PM
DeKay:

The RFBee is fully capable of being a stand-alone receiver.  It could be used to monitor an ISS and provide an alarm if the temp drops below freezing or windspeed exceeds 25 mph.  It could be used as a stand alone range extender (One of my goals). 

Or a standalone transmitter for a DIY temp / humidity station...

You could hook a low cost serial or I2C display and monitor your neighbor's ISS and really save some money! 

 8-)

I'm just not sure how well (if at all) it would perform trying to run a rtc, temp, humidity, etc
and process the Loop command.  The RFBee only has 16k of flash and 1k of sram.  I will attempt it, however.

You could always pile in the drivers for the DHT22 and BMP085 (both available at the AdaFruit website) and see how much room you have left to give you an idea of how much room you'd have for your own code.  Also, temp, humidity, and pressure don't need to be read very often.  I think the console does it every 50s or so.  That helps lightens the load.

Remember too that the processor in the console only runs at 1.8432 MHz.  You have clock speed on your side.

I do like your idea with the Raspberry Pi, but I also like this:

http://www.seeedstudio.com/depot/dragrove-generic-gateway-for-internet-of-things-p-1118.html?cPath=139_141


Hadn't seen that one before.  I'm assuming it just has an Arduino stuffed inside it somewhere, and you could still end up hitting the same limitations on code and data space?

Another thing I've bounced around in my head is extending their ISS protocol, essentially defining new station types or stuffing new things into the existing messages.  A Davis console would ignore what it didn't understand.  A DIY console would interpret these things as light sensor outputs or contact closures for home monitoring.

Sounds like you are keen to dig into the coding side so I'll see if I can dig more into the reverse engineering side of what is there already.  Unfortunately, my time is going to be pretty limited for the next six weeks.
Title: Re: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: meteobreda on March 29, 2013, 02:20:46 PM
Hello,

I'm searching information about Arduino and ISS connection.
My idea is use a ISS with no Console.
Title: Re: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on March 29, 2013, 03:44:30 PM
I'm searching information about Arduino and ISS connection.


This is what I have:

http://www.raydees.com/uploads/Davis.zip

It uses the RFBee from seeedstudio, which is Arduino based.   What we really need is the CC1101 on it's own breakout board without a processor attached.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: meteobreda on March 29, 2013, 04:33:44 PM
Hi,

Not is necessary Arduino Uno or similar? or the module is plugged to Arduino card.
Please could you send a list of necessary materials.

Thank You.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on March 30, 2013, 02:09:39 PM
I used these two devices:

http://www.seeedstudio.com/depot/rfbee-v11-wireless-arduino-compatible-node-p-614.html?cPath=139_140

http://www.seeedstudio.com/depot/uartsbee-v4-p-688.html?cPath=132_135

They are all you need to get started decoding ISS transmissions.  I'm sure there are other products out there that would do just as well.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: meteobreda on March 30, 2013, 03:18:18 PM
Hello,
This is for EU Davis?
868MHz is European or US Davis?  #-o
My Davis is US I think the frequency is 466MHz
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on March 30, 2013, 08:24:58 PM
This pretty much explains it:

Code: [Select]
//
//  Table for FREQ2 settings.  (Vantage VUE - US 900 mHz)
//
static const uint8_t __attribute__ ((progmem)) FREQ_2[51] =
{
  0x23, 0x22, 0x23, 0x23, 0x23, 0x22, 0x23, 0x23, 0x22, 0x23,
  0x23, 0x22, 0x23, 0x23, 0x23, 0x22, 0x23, 0x23, 0x22, 0x23,
  0x23, 0x22, 0x23, 0x23, 0x22, 0x23, 0x22, 0x23, 0x23, 0x23,
  0x22, 0x23, 0x23, 0x23, 0x22, 0x23, 0x23, 0x23, 0x22, 0x23,
  0x23, 0x22, 0x23, 0x23, 0x22, 0x23, 0x22, 0x23, 0x22, 0x23,
  0x23
}; 

I don't know anything about the 466 mHz version.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: johnd on March 31, 2013, 05:42:43 AM
There is no 466MHz version, just generic OV (eg EU) @ 868MHz, US in the 900MHz band (and, for completeness, AU and possibly NZ (not sure about NZ) in what I think is a shortened US band).
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: meteobreda on March 31, 2013, 06:47:40 AM
I'm sorry  :oops:

Now this could capture ISS America and Europe because works with frequency 868MHz to 915MHz
http://www.seeedstudio.com/depot/rfbee-v11-wireless-arduino-compatible-node-p-614.html?cPath=139_140

Thankyou.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on June 23, 2013, 12:01:49 PM
DeKay:

I found these and ordered two each:

http://www.elechouse.com/elechouse/index.php?main_page=product_info&cPath=90_92&products_id=2148

I'll let you know how they work out......

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on June 28, 2013, 02:29:22 PM
DeKay:

I found these and ordered two each:

http://www.elechouse.com/elechouse/index.php?main_page=product_info&cPath=90_92&products_id=2148

I'll let you know how they work out......



Please do.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on July 04, 2013, 02:57:18 PM
DeKay:

I found these and ordered two each:

http://www.elechouse.com/elechouse/index.php?main_page=product_info&cPath=90_92&products_id=2148

I'll let you know how they work out......



Complete waste of money and time!  Using the exact firmware as the RF Bee, it receives nothing from the ISS.  The RF Bee rssi runs around -70, which is 10 less than the console.  If I just let it look for any packets, it does receive constantly.  Tried both of them, gave up!

 
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: intra_au on July 30, 2013, 04:47:29 AM
I've used rdsman arduino library to write sketch displaying the actual data on every packet:
Below is an example of output:

Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 319 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 319 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165

I can share the sketch if anyone wants it...

I've also decoded packet 5 - it has RAIN rate data, timer that can run up to 15 minutes and indicates umbrella on the console.
MSB is seconds
(LSB >> 4) is a multiplier

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on July 30, 2013, 09:06:57 AM
Quote
I've used rdsman arduino library to write sketch displaying the actual data on every packet:

Good job!  Are you using the RFBee or some other version of the CC1101?  I've modified the original library extensively.  It now corrects the frequency error on a per channel basis.  I'll be posting the latest version soon.

I did a project that combines the RFBee, SHT25, MS5611 and the DS3231 to emulate the Vue console.  The RFBee runs out of sram before you can make it too fancy.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on July 30, 2013, 11:07:11 AM
Quote
I've used rdsman arduino library to write sketch displaying the actual data on every packet:

Good job!  Are you using the RFBee or some other version of the CC1101?  I've modified the original library extensively.  It now corrects the frequency error on a per channel basis.  I'll be posting the latest version soon.

I did a project that combines the RFBee, SHT25, MS5611 and the DS3231 to emulate the Vue console.  The RFBee runs out of sram before you can make it too fancy.

It sucks that any decently performing CC1101 is tied to a processor so low on resources.  I have been keeping my eye out on a just released module from HopeRF called the RFM69W.  An older module, the RFM12B, was not flexible enough to be made compatible with the CC1101 because you could program the sync word.  I believe after a very quick look that the new RFM69 would work.  It can be bought on a board with an Atmega328 (double the SRAM) for a very reasonable price.  See http://lowpowerlab.com/shop/moteino-R3.

The module can also be bought standalone for an even more reasonable price for connection to the processor of your choice.  This is the route that has me interested.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: linuxfreak on July 30, 2013, 10:49:58 PM
And with a pair, ISS & console replacement?  ;)

Just thinking out loud.  :cool: :-k

George
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: intra_au on July 31, 2013, 05:18:38 AM
Quote
I've used rdsman arduino library to write sketch displaying the actual data on every packet:
Good job!  Are you using the RFBee or some other version of the CC1101?  I've modified the original library extensively.  It now corrects the frequency error on a per channel basis.  I'll be posting the latest version soon.
Yep, I'm using just RFbee. My idea is to feed the data to my home automation network via zigbee (XBee) so I can access it on any device and link some actions to it (i.e. close rolling shutters during heavy wind :)
I'll be happy  to get your updated library, the one I'm using loosing the channel sometimes and comes back in a minute or so...

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Naby on December 11, 2013, 01:30:58 PM
New forum member here, I'm resurrecting an old thread. I have a wireless Davis Vantage Pro 2 and I'd like to get the data from it and transmit it via ham radio APRS. I don't have it yet but I will be buying a Weatherlink so that I can get the data on my computer and post it to the web. There is an APRS version of the Weatherlink but I think that it only connects to a TNC and not both a TNC and a computer. My radio is also located away from where I want my console and computer, even if the APRS weatherlink would do both. So I to get data on my computer and send it via APRS I'd need either a second console or Weather Envoy and another Weatherlink module. That is simply more than I want to spend.

So here is what I want to accomplish. I want to wirelessly capture the data from my weather station and then send it to an APRS module and from there to a TNC/radio. I don't have any need to view that data as on a console or save it as my primary console will do that. I figure a Raspberry Pi or Arduino is a good platform for this. There seem to be enough APRS sheilds and stand alone modules around to handle the APRS side of things. I think I would just need to be able to capture the wireless data. I believe that is what this thread is about, has anybody been able to capture the data?

Thanks,
Chris
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: dalecoy on December 11, 2013, 01:45:38 PM
WeatherLink software will directly upload data to APRS/CWOP.  And your weather station probably isn't going to change location.

So, if you just want to do this (Davis wireless to receiver to Ardino to whatever), then this is the right thread.  But if you just want to get your weather data to APRS, this isn't necessary.

http://www.findu.com/cgi-bin/wxpage.cgi?last=24&call=W5VBQ
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Naby on December 11, 2013, 01:58:43 PM
WeatherLink software will directly upload data to APRS/CWOP.  And your weather station probably isn't going to change location.

So, if you just want to do this (Davis wireless to receiver to Ardino to whatever), then this is the right thread.  But if you just want to get your weather data to APRS, this isn't necessary.

http://www.findu.com/cgi-bin/wxpage.cgi?last=24&call=W5VBQ

Thanks for the info. I have a Mac and I was planning on using WeatherCat software. I'm not sure is that supports APRS or not but it doesn't look like it. Anyhow, I want to actually transmit the APRS data, not just upload it to the APRS server (just because I want to). You are right that my station will not be moving unless the weather is bad enough. It also sounds like a fun project.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Scott216 on July 13, 2014, 07:35:35 PM
I've used rdsman arduino library to write sketch displaying the actual data on every packet:
Below is an example of output:

Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 319 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 319 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 4 Dir 283 UV 2.2 SolRad 328 Rain 165
Temp 18.4 Hum 67 Wind 3 Dir 283 UV 2.2 SolRad 328 Rain 165

I can share the sketch if anyone wants it...

I've also decoded packet 5 - it has RAIN rate data, timer that can run up to 15 minutes and indicates umbrella on the console.
MSB is seconds
(LSB >> 4) is a multiplier

I'd like to see the sketch you're using for this.  I'm using DeKay's example sketch (https://github.com/dekay/DavisRFM69/blob/master/Examples/VP2/VP2.ino), but he doesn't have the rain, solar radiation or UV index formulas.  Rain is the one I really need.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on July 14, 2014, 11:03:56 PM
I'd like to see the sketch you're using for this.  I'm using DeKay's example sketch (https://github.com/dekay/DavisRFM69/blob/master/Examples/VP2/VP2.ino), but he doesn't have the rain, solar radiation or UV index formulas.  Rain is the one I really need.

I do rain, but not rain rate.  intra_au was a bit vague on how exactly to figure out rain rate.  I would like to see his code as well.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Scott216 on July 29, 2014, 10:01:08 PM
Using Dekay's awesome library (https://github.com/dekay/DavisRFM69) that decodes the wireless Davis data from the ISS, I've got my project working where a Moteino (https://lowpowerlab.com/shop/index.php?_route_=Moteino/moteino-r4) receives the wireless weather data then sends it up to Weather Underground's PWS (http://www.wunderground.com/weatherstation/setup.asp).  No Davis console or PC needed!  I put everything on Github (http://github.com/Scott216/Weather_Station_Data).  I've wanted to do this for a long time and I really appreciate the tons of hard work (https://github.com/Scott216/Weather_Station_Data/wiki/DeKay-Blog-Posts) DeKay put into reverse engineering the Davis communication.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on July 29, 2014, 10:56:14 PM
Impressive!  Thanks for sharing.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on July 30, 2014, 12:00:18 AM
From Scott's wiki page...

Quote
Here's a list of the many blog posts DeKay posted as he was figuring out the Davis protocols.

(Many of the blog titles don't tell you too much)

I LOL'ed.

Glad you were able to make use of the Moteino stuff.  My work on using its flash chip to archive the data will resume once the weather turns crappy again in the fall.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on August 10, 2014, 01:31:18 PM
DeKay:

I've owned a Vantage Pro 2 since early 2005.  Several years ago, I bought and tried the WeatherLink Data Logger.  I wasn't impressed -- over priced, over promised, under delivered and clunky.  I returned it and told them why, but got no response other than my money back.

I've gotten that itch to have my own real-time weather database again, so thought Davis must have improved on their data logger and software by now.  Not !  Unfortunately, Davis is just milking the same old product with only minor adjustments over the intervening years since I last tried it.

For the rest of the world, technology has marched forward at warp speed.  I'm a bit of a geek and a hardcore DIYer.  I learned how to hack my DirecTivo and my cell phone, so thought maybe I'd see if there was info on the web to do something similar with my weather station.  I've been searching to see if some enterprising soul had finally figured out how to build and set up a link from a Vantage Pro 2 console to a PC without using a WeatherLink Data Logger.  This led me to this forum and your blog.

I've been reading your posts on this topic and have learned a lot from them.  Thank you for sharing your technical knowledge and being such a clear and entertaining writer on a subject that would normally put people to sleep !  As I'm working my way through your posts from the beginning, I've noticed that some of the document links you provide, especially those to davisnet.com, now return "Page Not Found" errors.  I couldn't find where anyone else had noted this, but thought you might want to know.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Scott216 on August 10, 2014, 09:22:52 PM
You may want to read DeKay's last blog post (http://madscientistlabs.blogspot.ca/2014/02/build-your-own-davis-weather-station_17.html).  He pretty much wraps things up in it.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on August 10, 2014, 10:22:55 PM
I've been reading your posts on this topic and have learned a lot from them.  Thank you for sharing your technical knowledge and being such a clear and entertaining writer on a subject that would normally put people to sleep !  As I'm working my way through your posts from the beginning, I've noticed that some of the document links you provide, especially those to davisnet.com, now return "Page Not Found" errors.  I couldn't find where anyone else had noted this, but thought you might want to know.

Thanks for the kind words.  Always good to hear from people what have gotten something out of this.

I'm probably the cause of all those broken links.  Davis took a fair bit of material out of circulation when I started writing this stuff up.  Maybe Davis will restore those links someday,  Yeah... maybe I'm a Chinese jet pilot (https://www.youtube.com/watch?v=cbuQHwsOqW4).
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on August 14, 2014, 09:10:12 AM
Quote
I'm probably the cause of all those broken links.  Davis took a fair bit of material out of circulation when I started writing this stuff up.  Maybe Davis will restore those links someday,  Yeah... maybe I'm a Chinese jet pilot.

It's a sad commentary on business in America today that they work so hard to keep their most ardent customers from realizing the true potential of the products they make and sell.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 08, 2014, 10:09:51 AM
This link was posted in a comment on my blog and I had to share it.  This guy has done some phenomenal work in decoding some until-now unknown wireless transmissions.  Want to know the voltage on your supercap?  Wind gust info?  Need more info on rain rates?  This and more can be found here:

http://www.carluccio.de/davis-vue-hacking-part-2/

My hat is off to this guy   =D&gt;
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Scott216 on October 08, 2014, 10:19:17 AM
This link was posted in a comment on my blog and I had to share it.  This guy has done some phenomenal work in decoding some until-now unknown wireless transmissions.  Want to know the voltage on your supercap?  Wind gust info?  Need more info on rain rates?  This and more can be found here:

http://www.carluccio.de/davis-vue-hacking-part-2/

I looked at the data coming from my Vantage Pro2 ISS and I didn't see any data for the supercap voltage.  Perhaps it's only available on the Vue. 
I wish he posted this a week earlier, it would have saved me hours of staring at my spreadsheet trying to figure out the seconds between bucket tips.

Next time I'm at my weather station I want to do another test with an Arduino simulating bucket tips to fine tune that formula.  His calculation is a little different then mine.  He did a great job with everything.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Scott216 on October 08, 2014, 10:36:32 AM
I had a rain storm come through my station this morning and I compared the ISS data I'm sending to weather underground against the normal VP2 data (see attached screenshots).  The way my setup works is I'm uploading data to two weather underground stations.  The first (http://www.wunderground.com/personal-weather-station/dashboard?ID=KVTWESTD3) gets it's data from the VP2 console using a dongle that's connected to a Hautewind (http://www.wunderground.com/wximage/hautewind/1) internet appliance (no longer made).  The second (http://www.wunderground.com/personal-weather-station/dashboard?ID=KVTDOVER3) uses a Moteino to intercept the wireless ISS data and I have an Arduino program that deciphers the data and uploads it to the second WU station. My Arduino program is currently using the seconds between bucket tips that's being sent from the ISS to calculate the rain rate.  The two charts match up pretty well, not perfectly, but I expect some minor differences.  In a week or two I want to run a bucket tip simulation and compare again between the two WU stations.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 08, 2014, 12:24:17 PM
Quote

I wish he posted this a week earlier, it would have saved me hours of staring at my spreadsheet trying to figure out the seconds between bucket tips.


I own a Vue and have invested a considerable amount of time in decoding the messages.  Once you get past wind speed, we have very little in common.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 10, 2014, 05:30:38 PM
Now that I'm back home, I thought that I would paste some code (rough draft version) for the Rain Rate calculation (in inches) that I did a while back.  It could probably be simplified - but it matches the Vue exactly.....

Code: [Select]

case 0x50:  //  Rain Rate Message.
            data[1] = Radio.read();  //  Incoming Byte 3.
            data[2] = Radio.read();  //  Incoming Byte 4.
            rate = data[2] & 0xF0;   //  rate is a byte.
                 
            switch (rate)
              {
                case 0x00:
                case 0x10:
                case 0x20:
                case 0x30: 
                            temp = rate << 4 | data[1];  //  temp is a word.
                            rainRate = 576.0 / temp;      //  rainRate is a float.
                            break;
                     
                case 0x40:
                case 0x50:
                case 0x60:
                case 0x70:
                            temp = ((rate & 0x30) << 8) | data[1] << 4;
                            if (temp >= 0x3FF0)
                              rainRate = 0.0;
                            else 
                              rainRate = 576.0 / temp;
                            break;   
                                         
            default: 
                            break; 
              }   
            break;


Hope this helps out.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on October 11, 2014, 12:21:01 PM
Code: [Select]

case 0x50:  //  Rain Rate Message.
            data[1] = Radio.read();  //  Incoming Byte 3.
            data[2] = Radio.read();  //  Incoming Byte 4.
            rate = data[2] & 0xF0;   //  rate is a byte.
                 
            switch (rate)
              {
                case 0x00:
                case 0x10:
                case 0x20:
                case 0x30: 
                            temp = rate << 4 | data[1];  //  temp is a word.
                            rainRate = 576.0 / temp;      //  rainRate is a float.
                            break;
                     
                case 0x40:
                case 0x50:
                case 0x60:
                case 0x70:
                            temp = ((rate & 0x30) << 8) | data[1] << 4;
                            if (temp >= 0x3FF0)
                              rainRate = 0.0;
                            else 
                              rainRate = 576.0 / temp;
                            break;   
                                         
            default: 
                            break; 
              }   
            break;


I haven't been following this too closely.  So there are two separate messages for rain rate?  Would the same type of station send the two messages, or is one for Vue and the other for VP2?  If the former, does it send either 30 or 70 without interspersing them?

Nice work, BTW.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 11, 2014, 02:07:47 PM
I think that the VP2 and the Vue are very similar when it comes to rain.  The Type E0 message is the actual bucket tip count, which the console has to keep up with - daily, weekly, etc.  The Type 50 message is the actual elapsed time between bucket tips.  The actual formula is derived from:

57600 * (.01 inches / time)

I think for the mm users it would be:

57600 * (.2 / time) or simply 11520.0 / time.

I haven't found any other message that appears to be determined by rain.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 14, 2014, 05:30:03 PM
Not sure if this will help the VP2 users, but it should enlighten the Vue users.  Try this code somewhere in your scheme:

Code: [Select]

word rawDirection = (radio.DATA[2] << 2) | (radio.DATA[4] & 0x02)  // Using DeKay's numbering scheme
float windDirection = rawDirection * .3515625;

if (windDirection > 360.0 || windDirection == 0.0)
    windDirection = 360.0;


Let me know if it matches your Vue!
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: EA1EF on October 16, 2014, 01:20:21 PM
On RF world one of interest way for easy and cheaper testing are the USB dongles RTL2832U R820T or E4000 thah can buy online fron 7$ and open a waste horizonts for esperiment.

Only need the USB dongle, drivers (for al sistems Android, Windows Linus Raspberry etc etc) and antenna.  Its cheap and easy (step by step guided with drivers).

Many people are receiving with USB dongles and http://www.coaa.co.uk/ software " all move it"...

I thing its a good and popular tool for your poupose.

example where buy
http://www.ebay.com/itm/RTL2832U-R820T-DVB-T-SDR-DAB-FM-USB-2-0-DIGITAL-TV-Tuner-Receiver-Digital-/231071399525?pt=US_Video_Capture_TV_Tuner_Cards&hash=item35ccedbe65

where DIY installation drivers and info
http://www.m9t.co.uk/
http://www.rtl-sdr.com/
etc

where RX software
http://sdrsharp.com/
http://www.rtl-sdr.com/big-list-rtl-sdr-supported-software/

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on October 16, 2014, 01:33:39 PM
EA1EF:

I already have the dongle and I'm very familiar with SDR#.  I believe this would be an excellent project.  Maybe after I retire...............

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on October 25, 2014, 04:40:29 PM
Well, now that I had some time to permanently hook up my weather station to my home server (and some popular sites) I started looking around and read some of the posts around here. First, Scott216, great work! You've finished a project similar to the one I started last winter. Naturally, I used the heapload of info gathered by the great folks in these forums. Now I have an Envoy instead of a homebreed receiver but I'll eventually finish my old project someday too. I've also created a standalone anemometer transmitter, for which I published the code here (https://github.com/kobuki/DavisRFM69/blob/master/Examples/AnemometerTX/AnemometerTX.ino). Reading your project wiki I discovered that the ISS/SAT is able to transmit wind gust figures as well, which I ignore now.

Now my question is: how is the ISS calculating the wind gust? @Scott216: how did you calculate it? At one point you had youd own calculator. What kind of information or algorithm is it based on? I want to add this extra piece to my SAT code, making it almost complete. I also recommend you having a peek at my code for the wind vane speed compensation calculations based on material published by Davis itself. Your project doesn't take these error correction figures into account AFAIK.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on October 25, 2014, 05:13:18 PM
In the meantime I realised that my question is a little misdirected at Scott216 since in the SAT I work with different variables. I have direct access to the pulses from the anemometer and can do a lot of different calculations. But my question still hods of course, I'd like to hear some ideas. Naturally, I have my own theory too but that isn't necessarily the one I should follow.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on November 14, 2014, 08:27:30 PM
This thread and the blog that spawned it has led me to attempt to create a stand alone wireless VP2 data logger that collects the data directly from the ISS and stores it until the unit is plugged into a USB from a computer running Cumulus. 

I purchased a Moteino USB r4 with the integrated RFM69W and flash from LowPowerLab.  I also ordered and am awaiting the arrival of a DS3231 clock chip, BMP180 Barometric pressure/temperature/altitude sensor (BMP085 has been discontinued) and a DHT22 temperature/humidity sensor.  I downloaded DeKay's sketches and libraries as noted in his summary blog posting.  After many false starts, I finally have the Moteino receiving and displaying the ISS transmissions using DeKay's ISSRx sketch and a serial monitor.  Using the wire antenna supplied with the Moteino, I've noticed that about 7 out of every 51 transmissions isn't received by the Moteino.  Is this level of packet loss normal ?  If not, should I be concerned ?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on November 15, 2014, 05:24:11 AM
This thread and the blog that spawned it has led me to attempt to create a stand alone wireless VP2 data logger that collects the data directly from the ISS and stores it until the unit is plugged into a USB from a computer running Cumulus. 
Awesome!

I purchased a Moteino USB r4 with the integrated RFM69W and flash from LowPowerLab.  I also ordered and am awaiting the arrival of a DS3231 clock chip, BMP180 Barometric pressure/temperature/altitude sensor (BMP085 has been discontinued) and a DHT22 temperature/humidity sensor.  I downloaded DeKay's sketches and libraries as noted in his summary blog posting.  After many false starts, I finally have the Moteino receiving and displaying the ISS transmissions using DeKay's ISSRx sketch and a serial monitor.  Using the wire antenna supplied with the Moteino, I've noticed that about 7 out of every 51 transmissions isn't received by the Moteino.  Is this level of packet loss normal ?
No

If not, should I be concerned ?
Yes

You'd have to give some background on your setup.  Is your ISS quite far away from the Moteino or are there substantial obstructions in the signal path vs. where your console is sighted?  Is it always the same channels or any other pattern to the dropouts?  What is the reported RSSI value?  I think what others have seen is that the reception stats of the Moteino generally hold up extremely well.

By the way, I haven't looked at this stuff in quite a while.  This is a winter(s) project for me.  I do hope to get back to it soon though.  Last I left off was datalogging in my VP2 sketch.  That wasn't working yet, mostly because the flash chip in the real logger is a strange beast.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 15, 2014, 05:58:28 AM
@AnOldFella: How is your console signal? Is it in the upper 90%? I'd say 7 out of 51 is not really a big deal (around 14% loss). Keep in mind that he small wire monopole antenna is not as stable and sensitive as the accurately tuned dipole of the console. I've been running such a receiver for more than 7-8 months. It can be disturbed easily by any metallic object in the path of the signal. TBH, I'm receiving 2 stations, an ISS and a SAT, but while the received packet ratio is around 99% most of the time, I get dips for seemingly no reason for hours sometimes, while the console pays no mind and stays at around 95% or more. I'm going to attach a factory-made dipole soon. A hint: put the moteino receiver on a plain metallic surface such as the top of a PC or something. This dramatically helped me gain signal strength (10+ dB).
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on November 15, 2014, 04:23:17 PM
Quote
Is your ISS quite far away from the Moteino or are there substantial obstructions in the signal path vs. where your console is sighted?

My ISS is mounted on a metal pole about 300' from my VP2 console with only the outer wall of the house blocking the ISS signal.  However, I moved the console to right next to the Moteino while I was testing.  The Moteino will be located about 25' further from the ISS along the same line of sight as the console, but with two more walls intervening.

Quote
Is it always the same channels or any other pattern to the dropouts?

I'm assuming that the 0-50 line #s reported are the 51 channels in the frequency hop sequence.  I ran a test of 16 hop cycles giving 816 packets with 101 lost equaling a little over 12% loss ratio.  The RSSIs were mostly in the -78 to the mid -80s with just a few in the low -100s.  I'm not very  radio or hardware savvy, so I don't know exactly what this represents.  However, I am a numbers guy and the frequencies of the lost packets definitely show some patterns.  More than half the lost packets (53) were concentrated in four frequencies - 28, 29, 36 & 45.  Six more frequencies (10, 12, 20, 26, 40 & 50) accounted for the next quarter (26) of all losses.  The remaining frequencies only had 0-2 lost packets.

Quote
How is your console signal?

I placed my VP2 console next to the Moteino for the testing presented above and put it in diagnostic mode.  Signal strength runs from 27-38 (don't really know what this represents), background noise is solid 9, good packets are almost 100% with 32 bad out of 15300 total.

Quote
Keep in mind that he small wire monopole antenna is not as stable and as sensitive as the accurately tuned dipole of the console.

I've been reading the LowPowerLab Forum thread "Antenna Tutorial or Antennas in a Mote" to learn more about antennas and see what I can do to increase my Moteino reception strength.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 15, 2014, 04:44:38 PM
RSSI around -80 dB is OK. It's enough for a good reception with near 100% ratio. However a better antenna is always a good idea.

About the console RSSI values. If we go by the logic in this thread (http://www.wxforum.net/index.php?topic=22700.10), then the "real" RSSI values of -100..-40 dB are mapped to 20..60 on the console. That would map 27..38 to -89,5..-67,6. I'd say it's close to the Moteino, albeit goes a lot higher. I'd try increasing the antenna gain. For me a metallic surface helped, though I'll ttach a real antenna sometime soon.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on November 16, 2014, 11:53:36 AM
RSSI around -80 dB is OK. It's enough for a good reception with near 100% ratio. However a better antenna is always a good idea.
Thanks for the info, it means that I'm at least in the ballpark.  I have just enough electrical/radio knowledge to be dangerous, but I'm still capable of learning and mastering new skills. ;)

... For me a metallic surface helped ...
Per your comment, I cut a can lid down to a 78mm radius and placed it under the Moteino.  Should this ground plane be wired to the Gnd ?  If so, what type/length of wire ?

With the can lid centered under the antenna and on a sticky note for insulation the signal is generally running 74-82 with occasional spikes to 85 and very infrequent jumps to 100-109.  Now after 16 hop cycles my error rate is running under 2%.

... a real antenna ...
What do you mean by "...a real antenna..." ?  After market or homemade ?  Since both ends of my system are in fixed positions, it seems that the antenna on the Moteino would work best if it was highly directional over a small arc.  Do you know what type of antenna that would be ?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 16, 2014, 12:13:40 PM
Well, the interesting thing is that my metallic surface is just the top of the server-pc I've put the breadboard on, so no intended direct connection to the PC case. It's not meant to be a ground plane, but to some extent, it might pose as one, since the USB ground (and thus tme Moteino's ground) is probably galvanically connected to the PC case for grounding. It increased my RSSI, however, purely by experience. I'm by no means an antenna expert, but I think it would worth a try to connect your disc to the ground of the moteino to create a ground plane.

OTOH, I call "real antenna" one that has been tuned and tested either by factory (I have several 868 MHz rubber ducky dipoles laying around) or made with a little more precision than cutting a small wire to size :) Of course, a home made dipole with proper 50 Ohms connection should be equally good. Another attempt at a dipole could be soldering a wire to GND on the opposite side of the Moteino where the antenna is sticking out on top. It should be the same length as the monopole wire and the 2 together would be forming a dipole.

But anyway, error rate of 2% is pretty good already!
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: EA1EF on November 16, 2014, 12:18:04 PM
a good antenna for all type of test and mods are PCB 3 element from http://www.wa5vjb.com/  have 868Mhz for europe models and 900mhz for USA ISM Band

http://www.wa5vjb.com/pcb-pdfs/Yagi900.pdf

(also sold in europe by http://g4ddk.com/ )

I test this antenna radomized hide in rain sensor with good results)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: AnOldFella on November 19, 2014, 12:00:57 PM
I'm adding a DHT22 to my project, but I'm confused about how to wire it correctly.  On DeKay's summary blog, it shows
"DHT22 Pin 1 -> Moteino 3.3V Out
DHT22 Pin 2 -> Moteino D4 with 10K Pullup resistor to Moteino 3.3V Out
DHT22 Pin 3 -> Not connected
DHT22 Pin 4 -> Moteino GND".

My confusion comes from the differences between DeKay's set up and the Data Sheets on Adafruit and SparkFun.  The circuit diagram on the Data Sheet at SparkFun shows a 1K resistor between Power and DHT22 Data, but the written portion of the Data Sheets from both Adafruit and SparkFun state "One capacitor valued 100nF can be added between VDD and GND for wave filtering."  The DHT22 came with a 10K resistor, but no instructions.  Any help understanding all this would be greatly appreciated.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 19, 2014, 12:17:21 PM
The exact value of pullup doesn't matter. 1k or 10k should be good. It only prevents the input from "floating" by pulling it near Vcc. Or you could leave out the pull-up entirely.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on November 19, 2014, 07:50:51 PM
The exact value of pullup doesn't matter. 1k or 10k should be good. It only prevents the input from "floating" by pulling it near Vcc. Or you could leave out the pull-up entirely.

I'd hesitate to leave the pullup out altogether.  To say it "only" prevents the input floating is actually quite important.  The DHT22 is known for being a bit of a fickle beast.

AnOldFella: The 10K value has given me dependable operation so you should be fine with that.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 19, 2014, 08:01:03 PM
It might depend on the actual sample, but 2 different pieces worked without problems for me. I'm aware of the importance of pull-ups, and using one doesn't hurt. But we're getting a little off-topic here.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: BCJKiwi on November 19, 2014, 08:18:09 PM
The I2C specification requires a Pullup resistor.
The ideal value depends on;
Supply voltage, Clock frequency and Bus capacitance (type and length of cable).

You may well get away without adding one to the Bus if the Master (typically the microcontroller or I2C I/O device) has pull-ups built into the port but you would need to know this is the case and that it is a suitable value else the circuit may be unstable.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 19, 2014, 08:24:39 PM
Yeah, though the DHT22 is not I2C. It uses some proprietary serial protocol. And we're talking about very short wires on a breadboard. But yes, always use a pull-up according to specs.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ct on November 20, 2014, 03:09:29 AM
I have built a DIY weather station using an Arduino and some Davis sensors, such as the rain bucket and anemometer, but it has no console. 

With the work that Dekay has done, it is possible to send weather data to a Davis console?

I like Davis hardware, but prefer using my own microcontroller for increased flexibility.  Davis communiction accessories are also very expensive.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 20, 2014, 05:28:35 AM
So you have a console but not an ISS/SIM? Yes, it's possible to send data from the individual sensors. I've made an anemometer transmitter (using a Moteino). It could be used as a basis for other sensors, source is available, if that's what you have in mind. I want to make it more power-friendly because the current implementation expects the CPU to be alive all the time. I'd suggest opening a new thread for this kind of stuff.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ct on November 20, 2014, 08:46:26 PM
So you have a console but not an ISS/SIM?

A already have a VP1, but have built a DIY station for another location.  I was wanting to use some kind of console, preferably a Davis console which I would buy if it is possible to send data to it.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on November 20, 2014, 10:48:07 PM
So you have a console but not an ISS/SIM?

A already have a VP1, but have built a DIY station for another location.  I was wanting to use some kind of console, preferably a Davis console which I would buy if it is possible to send data to it.

There is probably enough data on my blog (http://madscientistlabs.blogspot.ca/search/label/Davis%20VP2) to figure out how to do it, but nobody to my knowledge has actually done it yet.  I gave it a shot once and didn't get it to work, but I didn't spend much time playing with it either.  One reason might be that Davis does some funky stuff after the data in the packet has been transmitted that I haven't dug into yet.

Lots of things to do.  Not a lot of time.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 21, 2014, 04:42:47 AM
A already have a VP1, but have built a DIY station for another location.  I was wanting to use some kind of console, preferably a Davis console which I would buy if it is possible to send data to it.
If you're willing to buy a VP2 or a Vue console, yes, it's possible as I've already mentioned in my previous post. I have working SAT (Standalone Anemometer Transmitter) code you could use as a base if you're interested. It would be wise to test it first though on a test console you can borrow or something, athough it should work without problems. The transmitter code is a modified version of DeKay's receiver, btw. Of course, I'm willig to help and interested in your progress, should you ever start on this project.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 21, 2014, 05:05:43 AM
There is probably enough data on my blog (http://madscientistlabs.blogspot.ca/search/label/Davis%20VP2) to figure out how to do it, but nobody to my knowledge has actually done it yet.
Well, I did (for the VP2, not the VP1, of course). I exchanged some PMs with you on the subject this spring and have mentioned it in several forum threads too. Never mind.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ct on November 21, 2014, 05:08:35 AM
. I have working SAT (Standalone Anemometer Transmitter) code you could use as a base if you're interested.

Yes, I'm very interested in what you have done.  Is the code available somewhere?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 21, 2014, 05:16:06 AM
Yes, I'm very interested in what you have done.  Is the code available somewhere?
Sure, it's in a branch here (https://github.com/kobuki/DavisRFM69/tree/master/Examples/AnemometerTX). It's a branch of DeKay's stuff, but I'll prolly make it a standalone repo sometime soon, since now I have a different approach in mind than a console emulator. The current transmitter code needs to be a little more power-friendly. I have a plan for that in mind but no time to code.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on November 21, 2014, 05:59:40 PM
There is probably enough data on my blog (http://madscientistlabs.blogspot.ca/search/label/Davis%20VP2) to figure out how to do it, but nobody to my knowledge has actually done it yet.
Well, I did (for the VP2, not the VP1, of course). I exchanged some PMs with you on the subject this spring and have mentioned it in several forum threads too. Never mind.

Proof once again that my memory is absolutely horrible.  :-)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Bushman on November 21, 2014, 06:24:21 PM
 "A Waste Is A Terrible Thing To Mind" :)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on February 22, 2015, 07:54:56 PM
I would first like to thank everyone who has helped discover and document the wireless protocol used by Davis.  One of the main reasons I chose Davis products for my project was that I could inexpensively access the data computationally.

I needed to receive data from multiple weather and soil moisture/temperature stations, so I initially modified Dekay's DavisRFM69 library to keep track of up to 8 transmitters.  The range on the Moteino with the included antenna was inadequate for my project, so I rewrote the receiver code to work with a CC1101-CC1190 evaluation module from TI (http://www.ti.com/tool/cc1101cc1190emk915).

I figured out how the Davis Leaf and Soil Moisture/Temperature Stations encode their data.  I'll add that information to the wiki.

My project's code is posted at https://github.com/cmatteri/CC1101-Weather-Receiver/.  Instead of processing the packets on the microcontroller, I send them to a linux host and process them in weewx drivers.  The code is still alpha.

For anyone interested in using the CC1101-CC1190 EM, they are $50 a piece, but only come in pairs.  The connection to the module is a 1.27 mm header, so I used a PCB to connect it to an Arduino.

I didn't know about rdsman's CC1101 code until several days ago.  I based all of the radio settings off of Dekay's work.  Interestingly, several of the radio parameters differ between Dekay's and rdsman's work.  The DEVIATN register setting on the Vue console is 0x24.  Assuming the crystal frequency is 26 MHz, the deviation is 26e6 / 2^17 * (8 + 4) * 2^2 = 9.5 kHz.  In the IM-ME and DavisRFM69 libraries, the deviations are 4.5 kHz and 4.8 kHz, respectively.  Dekay, could you have miscalculated the deviation on the CC1021?  If not maybe Davis got it wrong on one of them, or I'm missing something. 

The MDMCFG4 register on the Vue is set to 0xC9, giving a channel filter bandwidth (ChBW) of 26e6 / (8 * (4 + 0) * 2^3) = 101.6 kHz.  The setting on the IM-ME is the same (though the crystal frequency on the IM-ME is 27 Mhz so they're not exactly the same, but they're as close).  The ChBW in the DavisRFM69 lib is 25 kHz (it is the one sided bandwidth.  I believe ChBW for the TI chips is double sided).  Section 12.2 of the CC1021 datasheet describes how to estimate the minimum ChBW.  The minimum ChBW with a 10 ppm crystal would be 19.2 kHz + 2 * 9.5 kHz + 4 * 10 ppm * 915 MHz = 74.8 kHz.  I believe the ChBW for the RFM69 should be increased.

The frequencies sniffed by Dekay from the VP2 console and by rdsman from the Vue console are off by about 30 kHz.  Observing the value of FREQEST on my receiver with both Dekay's and rdsman's frequencies shows an error of about 15 kHz for both frequencies, but in opposite directions.  Frequency compensation must be performed by the microcontroller for the CC1012 (section 12.13 of the datasheet), so it is possible that the sniffed frequencies included an offset calculated to compensate for error/drift in the crystals of the consoles and weather stations used by Dekay and rdsman (though there is a dedicated offset register on the CC1101, so that wouldn't be necessary, but perhaps they are reusing code, or perhaps rdsman read the nominal frequencies and the error I'm showing is due to my crystals being off).  Probably the only way we're going to find the nominal frequencies is to have more people report the error they're observing.  With the large channel filter bandwidth and automatic frequency offset compensation, either set of frequencies seems to work.  I plan to implement permanent frequency compensation based on FREQEST readings, which would negate offset errors in the frequency table, but it would still be nice to know the nominal frequencies.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 22, 2015, 08:19:22 PM
Cmatteri, great work. I've also spent a lot of time coding/testing Davis competible receivers, using Moteinos. My code is based on Dekay's original work, and I've also written code for configuring and receiving up to the max. 8 stations. You can see an older version of my code on Github (https://github.com/kobuki/DavisRFM69/blob/master/Examples/VP2/VP2.ino). I'll commit newer, updated code soon.

I'm very surprised you found the reception abilities of the Moteino's RFM69 inadequate. There is no antenna included in the package, just a piece of wire you can solder on or you can attach a factory-made antenna in some way. I've done both, with somewhat better and more stable RSSI using rubber duck dipoles. My Moteino receiver is far better in reception quality and dropped packets ratio than the original Davis Envoy I have. A product called Meteostick is also using this receiver module.

You observed a few interesting things about radio paramteres. I've been strugling with a Moteino-based Davis compatible transmitter recently, and I'll try to employ some of your findings - I wonder if I can finally create a stable Moteino transmitter.

Keep up the good work, and please update us if you make any progress.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on February 22, 2015, 11:17:07 PM
I only used the Moteino with wire monopole it came with.  It wasn't working with a station that was about 700 ft away, with the receiver inside a window, and we're adding new stations that will be even further.  I could try using different antennas, but those would also improve the module I'm using right now, and the CC1190 gives an additional range boost.

I should have posted earlier.  It's amazing how much of other peoples' work I missed while searching for what had been done with Davis receivers.  I'll try to stay more in the loop going forward.  Hopefully the wiki will make it easier for new people to find what's been done.  It will be fun to compare our implementations for tracking multiple stations.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on February 23, 2015, 02:49:11 PM
I believe I have made an additional discovery about message_9.  The upper nibble of byte 5 (byte5[7:4]) appears to indicate the interval between message_9 transmissions to which the max wind speed in byte3 corresponds.  E.g. if byte5[7:4] is 0, then the max wind speed occurred in the time since the previous message_9, if it is 1, it occurred between the previous message_9 and the message 9 before that.

When byte3 increases, byte5[7:4] always goes to 0.  Sometimes byte5[7:4] goes to 0, but byte3 remains unchanged.  This would indicate that the wind rose to the speed in byte3 again in the previous interval.  byte3 only ever decreases after byte5[7:4] is 9.  This makes sense if byte3 is the max wind speed in the last 10 message_9 intervals.  When byte3 does decrease, byte5[7:4] is often set to a nonzero value less than 10.  This makes sense too because the new max wind speed could have occurred in any of the last 10 intervals.

It doesn't look like the ISS gives us the direction of the max wind speed.  If we keep track of the direction of the wind between message_9 intervals, we can at least know the wind direction during the interval (the length of which depends on the transmitter ID) that the max wind was from.  If the max wind was ever in byte1 of any packet, then we know exactly what the direction was.

Here are a bunch of message_9s for anyone who wants to confirm this:
https://drive.google.com/file/d/0B-dARPEa08c9bnZQejdVZjB6Z0U/view?usp=sharing
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 23, 2015, 04:36:22 PM
Without analysing your data thoroughly, but assuming that the upper nibble of byte 5 is always between 0..9, are you suggesting that it is a backreference to the current wind gust speed out of the last 10 ones? I guess it would make sense. I have a simple graph of the 3 wind data values superimposed, and it might explain why the wind gust values come in in an odd manner. I'm attaching the graph for the last 6 hours so you can see what I'm talking about.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on February 24, 2015, 01:42:45 AM
I'm not sure what you mean by backreference to the current wind gust speed, so I'm not sure if we're on the same page.  I'll elaborate on my thoughts.

Peak wind speed is an important reading.  Everyone wants to know the maximum wind that was detected on their station.  If the protocol relied on the wind speed transmission in each packet, the peak wind could occur between packets.  So a separate packet is used to transmit the max wind speed (message_9).  But if message_9 only transmitted the max wind speed since the last message_9, then a single packet being missed could mean a potentially very interesting max wind reading is lost.  By transmitting the max wind speed over the last 10 message_9 intervals, you get 10 chances to get an interesting reading to the receiver.  That gives you a fairly good system, but it it's hard to tell exactly when the gust occurred.  That's what the index in byte5[7:4] tells us--which of the last 10 message_9 intervals the max wind reading occurred in.

I'm not sure why the gust speed is zero when the wind speed is not in your plot.  Shouldn't the gust speed be at least as high as the average wind speed?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 24, 2015, 05:30:41 AM
OK, you described exactly the same observation with slightly different words. First, AFAIK, packet type 9 contains wind gust values. We can assume I have a good understanding of what you want to say. When you refer to a previous value in one packet in the previously received stream of packets of a certain type, it can be called a backreference - a reference to a value that has already been measured and stored or buffered for later retrieval; it is not a current value, but a value from the past. It can still be interpreted as a maximum value for a certain period of time. That is basically the definition of wind gust.

I'd like to understand it fully so I can potentially add this bit to my own code. So if you think I still haven't grasped the gist of what you want to say, please elaborate more, or try to draw an explanation if your time allows. It might be beneficial for others following this thread, too.

OTOH, wind gust values in packet type 9 packets are transmitted for every 20th packet, so their interval is (depending on station ID) somewhere between 51 secs and 1 minute. Yes, they can be lost. My assumption up till now was the same for this type of packet as for all others: values can be lost by missing a reception and the last known valid value (potentially including your proposed algoritm) can be repeated or dropped. In the case of a wind gust value, I think it should be just dropped, since it's among the 3 rarest packet types.

About my plot: yes, that's a good question :) In the interpretations I've seen so far the wind gust value is assumed to be equal to the last sensed wind speed value (contained in every packet) if it's correctly received and is zero (I can't recall if the gust calculation interval is considered for this case, though). Your proposed algorithm with that certain nibble might shed some light on it, but I'm not sure.

My post has become rather long, but finally let me ask a question: are you sure that barring factory interpretation of the values without original Davis equipment will be sufficient for your needs? You mentioned simple temperature measurements but we've been posting about wind gust interpretations for some posts now... There exist of the shelf, cost-efficient third party solutions for receiving up to 8 stations simultaneously.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 24, 2015, 09:35:07 PM
I figured out how the Davis Leaf and Soil Moisture/Temperature Stations encode their data.  I'll add that information to the wiki.

<snip>

My project's code is posted at https://github.com/cmatteri/CC1101-Weather-Receiver/.  Instead of processing the packets on the microcontroller, I send them to a linux host and process them in weewx drivers.  The code is still alpha.

Thanks for figuring this stuff out and for sharing!  It would be great if you'd update the wiki with what you've learned.  I get pretty psyched when I see other people dig into this stuff and contribute improvements.

I based all of the radio settings off of Dekay's work.  Interestingly, several of the radio parameters differ between Dekay's and rdsman's work.  The DEVIATN register setting on the Vue console is 0x24.  Assuming the crystal frequency is 26 MHz, the deviation is 26e6 / 2^17 * (8 + 4) * 2^2 = 9.5 kHz.  In the IM-ME and DavisRFM69 libraries, the deviations are 4.5 kHz and 4.8 kHz, respectively.  Dekay, could you have miscalculated the deviation on the CC1021?  If not maybe Davis got it wrong on one of them, or I'm missing something. 

Feel free to check my math (http://madscientistlabs.blogspot.ca/2011/04/crunching-numbers.html).  From the spreadsheet in the linked post, you'll see how all the other numbers check out when a crystal frequency of 14.7456 MHz is used in the VP2 console.  It is very possible that I missed out on the one sided vs. two sided bandwidth bit you mention below.

As it sounds like you are aware, there are also two versions of IM-ME out there, BTW.  I think one of my blog posts stated which mine was but it was different from most of the rest.  I think it might have to do with the fact that I ordered mine out of the UK vs. North America.

The MDMCFG4 register on the Vue is set to 0xC9, giving a channel filter bandwidth (ChBW) of 26e6 / (8 * (4 + 0) * 2^3) = 101.6 kHz.  The setting on the IM-ME is the same (though the crystal frequency on the IM-ME is 27 Mhz so they're not exactly the same, but they're as close).  The ChBW in the DavisRFM69 lib is 25 kHz (it is the one sided bandwidth.  I believe ChBW for the TI chips is double sided).  Section 12.2 of the CC1021 datasheet describes how to estimate the minimum ChBW.  The minimum ChBW with a 10 ppm crystal would be 19.2 kHz + 2 * 9.5 kHz + 4 * 10 ppm * 915 MHz = 74.8 kHz.  I believe the ChBW for the RFM69 should be increased.

If my bandwidth is too narrow, this might explain why some people report problems with the odd channel not working right.  If one frequency is out a bit from the rest, maybe it gets chopped off vs the others.

The frequencies sniffed by Dekay from the VP2 console and by rdsman from the Vue console are off by about 30 kHz.  Observing the value of FREQEST on my receiver with both Dekay's and rdsman's frequencies shows an error of about 15 kHz for both frequencies, but in opposite directions.  Frequency compensation must be performed by the microcontroller for the CC1012 (section 12.13 of the datasheet), so it is possible that the sniffed frequencies included an offset calculated to compensate for error/drift in the crystals of the consoles and weather stations used by Dekay and rdsman (though there is a dedicated offset register on the CC1101, so that wouldn't be necessary, but perhaps they are reusing code, or perhaps rdsman read the nominal frequencies and the error I'm showing is due to my crystals being off). 

I'm about 99% certain that this is the case.  One of my blog posts tries to figure out a pattern in the frequency hops, but I never got it as close as I'd have expected.  I'm pretty sure (http://madscientistlabs.blogspot.ca/2011/04/peeling-back-layers-of-onion.html) the console just writes raw frequency registers to the CC1021 without using any kind of offset correction programmed in.

Probably the only way we're going to find the nominal frequencies is to have more people report the error they're observing.  With the large channel filter bandwidth and automatic frequency offset compensation, either set of frequencies seems to work.

There are a few other ways I can think of
-  set the console to factory defaults and sniff the SPI bus as the console started hopping after that.  The console does store the error for each frequency.  After a reset, the offsets it uses should all be zero and the commanded frequencies would be nominal. 
- figure out how in the code the console calculates the frequency in the first place. C5250 had some of that code disassembled but it got a little hairy.
- sniff the radio chip in the ISS !

I plan to implement permanent frequency compensation based on FREQEST readings, which would negate offset errors in the frequency table, but it would still be nice to know the nominal frequencies.

I wanted to do this on the Moteino but had trouble getting it working.  Just another item on my long list of things to do that I never seem to get to, unfortunately.

And great work, BTW   =D&gt;
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on February 25, 2015, 10:05:53 PM
Kobuki, I didn't know about the Envoy8X until fairly recently.  If I had, I would have used one and tried to discover the serial protocol it uses.  That said, my goal with the weather system I'm setting up is as much to learn and have fun as it is to monitor weather.

Here's a text figure that demonstrates how message 9 works.  Each vertical bar represents a message 9 packet.  The number between the vertical bars is the maximum wind speed that occurred between the two packets represented by the two adjacent vertical bars (this is not real data).
packet      0 1 2 3 4 5 6 7 8 9 ...
max wind    |1|1|2|3|2|3|1|2|1|1|1|1|1|1|1|1|1|1|1|1|1|1|
byte3         1 1 2 3 3 3 3 3 3 3 3 3 3 3 3 2 2 1 1 1 1 1
byte5[7:4]    0 0 0 0 1 0 1 2 3 4 5 6 7 8 9 8 9 0 0 0 0 0
              X X X X   M M X                   X X X X X


That should help explain how it works, but the question remains whether byte5[7:4] is actually useful for anything.  I think it could be used to only record gust readings that are new.  Gust readings would be recorded if byte5[7:4] is 0, or if packets were missed, if it was 0 for one of the missed packets.  We would record the gust readings for all the packets marked with X in the above figure (where M represents a missed packet).  There are two advantages to doing this:
1. It increases the likelihood that the gust reading in an archive record actually represents a max wind from that archive interval, rather than from the previous one.
2. The wind direction for the gust needs to come from byte 2 of some packet, and by using byte5[7:4] we can get a wind direction from within 30 seconds of the gust, whereas ignoring byte5[7:4], the gust direction could be as much as 10 minutes out of date.

Dekay, we were indeed missing something.  I found this note on page 72 of the CC1021 datasheet:
"Note: The RX frequency deviation should be close to half the TX frequency deviation for GFSK at 100 kBaud data rate and below. The RX frequency deviation should be close to the TX frequency deviation for FSK and for GFSK at 100 kBaud data rate and above."

OTOH, the CC1101 datasheet specifies:
"the DEVIATN register specifies the expected frequency deviation of incoming signals in RX and should be the same as the TX deviation for demodulation to be performed reliably and robustly."

I believe you miscalculated the TX deviation for the CC1021.  The formula for the 915 MHz band is fdev = FREF * TXDEV_M * 2^(TXDEV_X - 15).  Plugging in FREF = 7.3728 MHz, TXDEV_M = 11 and TXDEV_X = 2 from your table we get
deviation = 7.3728 MHz * 11 * 2^(2 - 15) = 9.9 kHz.  You must have used the formula for the 402 - 470 MHz band.  Your calculation for the RX deviation looks right.

Thus for the CC1101 I believe 9.5 kHz is the correct deviation.

As for the channel filter bandwidth, DEC_DIV[2:0] is set to 5, so ChBW = 307.2 / (5 + 1) kHz = 51.2 kHz.  That would require a roughly 4 ppm or lower crystal, which is possible.

I can't find the tolerance of the crystal used on the RFM69W, but given the possibility that its tolerance is higher than 4 ppm, you may want to increase the channel bandwidth in the DavisRFM69 code.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 25, 2015, 11:19:37 PM
Dekay, we were indeed missing something.  I found this note on page 72 of the CC1021 datasheet:
"Note: The RX frequency deviation should be close to half the TX frequency deviation for GFSK at 100 kBaud data rate and below. The RX frequency deviation should be close to the TX frequency deviation for FSK and for GFSK at 100 kBaud data rate and above."

OTOH, the CC1101 datasheet specifies:
"the DEVIATN register specifies the expected frequency deviation of incoming signals in RX and should be the same as the TX deviation for demodulation to be performed reliably and robustly."

I believe you miscalculated the TX deviation for the CC1021.  The formula for the 915 MHz band is fdev = FREF * TXDEV_M * 2^(TXDEV_X - 15).  Plugging in FREF = 7.3728 MHz, TXDEV_M = 11 and TXDEV_X = 2 from your table we get
deviation = 7.3728 MHz * 11 * 2^(2 - 15) = 9.9 kHz.  You must have used the formula for the 402 - 470 MHz band.  Your calculation for the RX deviation looks right.

I was indeed using the calculation for 915 MHz, but I totally botched the formula in the spreadsheet.  It was a fluke that the number came out looking reasonable.  Sigh.  Your number of 9.9 kHz is correct and my spreadsheet is now updated.

BUT, I think this explains why earlier attempts by rdsman, kobuki, and me to get our own transmitters going had failed - our transmit deviation was too low!  However, it does look like that got working in this thread (http://www.wxforum.net/index.php?topic=24981.50).  I did a quick search here but couldn't find the critical incantation that got this working where others had failed.  Can somebody point me to this?  Was it the transmit deviation or something else???

As for the channel filter bandwidth, DEC_DIV[2:0] is set to 5, so ChBW = 307.2 / (5 + 1) kHz = 51.2 kHz.  That would require a roughly 4 ppm or lower crystal, which is possible.

I can't find the tolerance of the crystal used on the RFM69W, but given the possibility that its tolerance is higher than 4 ppm, you may want to increase the channel bandwidth in the DavisRFM69 code.

What would be preferable is to acquire the carrier in each channel and determine the offset in each using a wide bandwidth, then narrow the bandwidth and periodically recalculate the offset.  This would give a better signal to noise ratio and prove more reliable over longer distances.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 26, 2015, 04:19:28 AM
@cmatteri: thanks, it confirmed what I've understood about that index. I think the most importand bit is to be able to more accurately determine the direction of a gust. Losing packets happens all the time, but it can help reconstructing some missing ones as well as you noted. OTOH, for "direct" reception it has little importance unless archiving and aggregations are calculated for the archive perionds, like the original console does, for example.

Also you discovered the Envoy 8x, which is AFAIK not very popular since its software support is rather poor. I'm trying not to look as an advocate for them but it's often left unnoticed, so have a look at the Meteostick and the Weatherbridge. These are 3rd party solutions. They offer a lot more than the factory devices, including arbitrary station reception configurations.

@DeKay, cmatteri: DeKay spent so much time and so many blog posts were devoted to the subject that I never thought that such an important parameter as the deviation frequency might be wrong. But it seems to be off for transmissions only. I'll definitely have a look very soon and try to find out if it helps with my RFM69 transmitter.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 27, 2015, 12:29:50 PM
Gentlemen, I have good news to report. I've set the deviation to 9.9 kHz on my anemometer transmitter sketch and now the Envoy is receiving it quite stably. A big thanks goes to cmattery for unknowingly solving this mistery with my transmitter. My code is still emulating the Davis packet format using data bytes only but I'll probably be able to switch it back to standard automatic packet assembly mode. I'll leave it on for a few hours to test the transmission quality. I've set my Envoy to handle it as an extra temp sensor.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 27, 2015, 02:09:07 PM
Gentlemen, I have good news to report. I've set the deviation to 9.9 kHz on my anemometer transmitter sketch and now the Envoy is receiving it quite stably. A big thangs goes to cmattery for unknowingly solving this mistery with my transmitter. My code is still emulating the Davis packet format using data bytes only but I'll probably be able to switch it back to standard automatic packet assembly mode. I'll leave it on for a few hours to test the transmission quality. I've set my Envoy to handle it as an extra temp sensor.

 \:D/

This is great!  Another mystery solved.  Homebrew Temperature / Humidity sensors have just become pretty trivial.  Next we'll need to work on the low power aspects to see if we can get a remote sensor working off a solar cell and some kind of supercap / battery backup.  This should be reasonable to expect given that T/H are transmitted less often than wind, etc from the ISS, so the sensor can spend more time sleeping.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 27, 2015, 02:30:11 PM
Yeah, I'm already thinking of making a Davis-compatible T/H transmitter with one of my Moteinos, for the fun of it. Sadly the extra temp sensors are only represented as integer digits, no fractional part. The show-stopper however can be the requirement for precise transmission timing. To be able to make the 328p sleep as much as possible, the WD timer can be used for periodic wake-ups and subsequent sample/transmit procedures, but it's very imprecise. An external clock source would probably be needed, such as an RTC clock chip or maybe a low-power 32k crystal. The problem could indeed be alleviated by using solar power and a chargeable battery or similar.

Regarding the radio parameters, I think I've missed something, since cmattery said: "Thus for the CC1101 I believe 9.5 kHz is the correct deviation." - though the calculated value is 9.9k. Is it a CC1101 limitation?

Also, the mentioned bandwidth values and exact hopping frequency values will still need to be tuned, although I'm quite satisfied with the capabilities of the RFM69 receiver as it is now. But I guess reception can still be improved as it has also been mentioned. The RFM69 being weaker than the TI at reception could also be caused by these slightly off parameters and the wrong Fdev value.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on February 28, 2015, 01:50:44 PM
Since a few people asked me in PMs for code and I wanted to upload it anyway, I thought it's better to put it all on GitHub for easier access. I removed my fork of DeKay's old lib, which is my radio code is based on, since we're going in different directions as far as the rest of the code goes. I still want to contribute to existing stuff, of course. My lib is more like an attempt at a general, bare-bones solution to jump start other projects.

Have fun: https://github.com/kobuki/VPTools
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on February 28, 2015, 02:05:06 PM
Since a few people asked me in PMs for code and I wanted to upload it anyway, I thought it's better to put it all on GitHub for easier access. I removed my fork of DeKay's old lib, which is my radio code is based on, since we're going in different directions as far as the rest of the code goes. I still want to contribute to existing stuff, of course. My lib is more like an attempt at a general, bare-bones solution to jump start other projects.

Have fun: https://github.com/kobuki/VPTools

Makes sense.  I'm totally cool with this  :-)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on March 03, 2015, 10:50:49 PM
Is it a CC1101 limitation?

Yes.  The next highest possible deviation on the CC1101 (with a 26 MHz crystal, which is what the Vue receiver uses, based on the FREQ register values sniffed by rdsman) after 9.521 kHz is 10.315 kHz.  9.521 kHz is what the Vue receiver uses.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: docbee on March 14, 2015, 05:39:29 PM
Sniffing the air for Vantage packets, I tried to make use of re-transmitted packets a Vantage console does send out. When I get it correctly these packets have "FF 00" following the two byte CRC. What stumbles me is, that I can see these packets only rather sporadic. I would have thought the retransmit follows the same frequency hopping as the other transmitting sensors.

Was anyone successful in reading retransmitted data periodically?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on March 14, 2015, 06:02:58 PM
Yeah. See this thread (http://www.wxforum.net/index.php?topic=24981.0).

The console has a simple repeater function that's not the equivalent of the stand alone repeater functionality. It simply retransmits the received packets of a single channel on another channel, using the appropriate timing and schedule. You cannot just repeat the packets with the same timing on the same channel. This will confuse the receiver. In essence retransmitted packets must follow the standard hopping pattern, sequence and timing wise too. Naturally the CRC needs to be recalculated when changing the channel ID.

You can ignore those 0xff00 bytes for this case.

EDIT: I realised you only talk about the reception. But yes, it follows the same pattern. You should be able to receive it per usual, on the retransmitted channel.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: docbee on March 14, 2015, 06:21:36 PM
Dekay has written near the beginning of this thread that re-transmitted packets have 3x 0x55 instead of 2x 0x55 as preamble which might explain why my RFM69 code is not receiving retransmitted data accordingly. Do you really succeed reading retransmitted data on a reliable basis?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on March 14, 2015, 06:37:34 PM
Dekay has written near the beginning of this thread that re-transmitted packets have 3x 0x55 instead of 2x 0x55 as preamble which might explain why my RFM69 code is not receiving retransmitted data accordingly. Do you really succeed reading retransmitted data on a reliable basis?

Actually, it was C5250 that said it was 3x 0x55.  I disagreed (http://www.wxforum.net/index.php?topic=10739.msg144940#msg144940) and was able to back it up (http://madscientistlabs.blogspot.ca/2012/03/first-you-get-sugar.html).   :-)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on March 14, 2015, 06:37:58 PM
I didn't remember that. Also, there are many small corrections he made or I discovered since then. It's several years old info. In essence, a retransmitted packet is indistinguishable from a non-retransmitted packet. There is no special setting needed on the receiver. Something else must be off. It has recently been discovered that the Fdev is wrong in his original code. It must be doubled for correctness on the RFM69. I'm not sure you've introduced that fix in your receiver, but it might help with the Vue.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on March 14, 2015, 06:39:43 PM
Actually, it was C5250 that said it was 3x 0x55.  I disagreed (http://www.wxforum.net/index.php?topic=10739.msg144940#msg144940) and was able to back it up (http://madscientistlabs.blogspot.ca/2012/03/first-you-get-sugar.html).   :-)

Right. Though it shouldn't make a difference in reception. I'm thinking the Fdev correction might help.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: rdsman on March 15, 2015, 10:59:57 AM
Quote
Was anyone successful in reading retransmitted data periodically?

docbee:

The retransmitted data from the Vue follows the same hop sequence, but is behind the ISS hop index numerically.  (At least it starts out that way.)  This is shown in the attached picture.  On the left is the ISS data, on the right is the repeated data.

The difference in the hop index doesn't stay the same amount because they roll over past 51 differently.





 
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on March 15, 2015, 11:06:18 AM
Well, rather, they are interleaved and after some time they interleave differently, in such a way that one that is behind the other now, will be ahead of the other later. And if you think it a little further, in the long run there isn't something you can call first or second or whatever. The point is, use the known jumping algo, always.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on June 29, 2015, 02:30:50 AM
Hi,

My Davis receivers have always needed to jump to a channel significantly earlier than I would have expected based on the timings used by the Davis protocol in order to not miss packets and lose track of the transmitter.  Several months ago I figured out why (I thought I posted it here but apparently I didn't).  The problem is that many Arduinos, including the Moteino, use ceramic resonators to clock the microcontroller.  These have tolerances of up to 0.5 %, plus variance from temperature.  For the 2.5 - 3 s period of Davis transmitters, this translates into up to 12 - 15 ms error.  A typical crystal could have a tolerance of 25 ppm, which would give an error of 75 us for the 3 s period.  The Davis devices probably use crystals, and probably assume crystal accuracy in their designs.  Thus for those who are building transmitters or repeaters it may help to use a microcontroller with a crystal.

edit: You might be able to ask Felix Rusu to build a Moteino with a crystal.  The connections are the same and I believe you can find a crystal with the same dimensions as the resonator he uses.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on June 29, 2015, 05:27:00 AM
I never had this problem. As soon as I receive a packet on the current channel, I skip to the next right away and listen on it until a packet arrives or times out with a configurable interval. IIRC the accuracy of the resonators used in the Moteionos are better than that. Not by paper, but by experience.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: cmatteri on June 29, 2015, 02:01:27 PM
Resonators are fine for receivers if you use them like you are.  There's a small chance of intercepting someone else's transmitter with that scheme though.  Resonators may even be fine for transmitters or repeaters, I haven't been paying too close attention to projects that use those, but if a resonator board isn't working as a transmitter that's a possible reason why.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on June 29, 2015, 02:19:15 PM
Well, yes, it's an inavitable drawback and can happen with a more accurate crystal and guarding timeframe, albeit with a lot less probability. For the purposes and intents I've made my receiver it's perfectly fine. It's been running it 24x7 for more than a year without much problems. Sadly I don't have much time to play around with these projects since winter :(

I think there are a few folks with working DIY transmitters/repeaters here, maybe they'll say hi.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ct on July 02, 2015, 12:22:13 AM
I have been unable to get the VP2 console to reliably sync to a RFM69 transmitter, but my microcontroller has a crystal.

I have posted more information in this thread http://www.wxforum.net/index.php?topic=24567 (http://www.wxforum.net/index.php?topic=24567)

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on August 18, 2015, 09:12:13 PM
I have my RFM69HW working picking up my ISS unit data.   I have a couple of problems.   The barometric readings are not working.   The inside temperature is also not working.

What are the defines for both barometer and inside temperature parameters?

// Davis packet types, also defined by kobuki
#define VP2P_UV             0x4 // UV index
#define VP2P_RAINSECS       0x5 // seconds between rain bucket tips
#define VP2P_SOLAR          0x6 // solar irradiation
#define VP2P_TEMP           0x8 // outside temperature
#define VP2P_WINDGUST       0x9 // 10-minute wind gust
#define VP2P_HUMIDITY       0xA // outside humidity
#define VP2P_RAIN           0xE // rain bucket tips counter
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: C5250 on August 18, 2015, 10:19:38 PM
Barometric and indoor temp are not transmitted by the ISS, they are local parameters to a Console/Envoy.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on August 18, 2015, 10:54:10 PM
Barometric and indoor temp are not transmitted by the ISS, they are local parameters to a Console/Envoy.

ahhhh...  Yeah, I guess that's why everyone is using a BMP085.   Feeling stupid right now.  Thanks for point that out...


Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: ct on August 19, 2015, 06:08:06 AM
I guess that's why everyone is using a BMP085

If building a new receiver, it may be worth checking out the BME280.  This sensor does pressure, temperature and humidity.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on August 19, 2015, 10:10:59 AM
I guess that's why everyone is using a BMP085

If building a new receiver, it may be worth checking out the BME280.  This sensor does pressure, temperature and humidity.

I have a bmp180 sensor already from spark fun. I hooked it up and now have it feeding weather underground. I'm using Dekay git software and it's working great.   The Weather Underground wunder station is a nice console replacement.  I just have to work on the rain data now.

Using an IPad beside my nest thermostat on the wall.  Running the power cable through the wall to the electrical box

My original Davis console is dedicated to my Kenwood D710 and APRS. So I needed another console to feed the Weather Underground.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on August 19, 2015, 10:35:13 AM
The Weather Underground wunder station is a nice console replacement.

Too bad they don't have it for Android. It could have been a nice and cheap console replacement, using a cheapo 5-7" tablet...
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on August 20, 2015, 09:40:00 AM
Yeah....  It would be nice if they had it for Android.  All those surplus computer shops with those tablets for dirt cheap. Plus you could hookup the receiver module by usb to the tablet.  Then send it to Weather Underground by tablet.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: DeKay on August 22, 2015, 12:43:34 AM
I guess that's why everyone is using a BMP085

If building a new receiver, it may be worth checking out the BME280.  This sensor does pressure, temperature and humidity.

Interesting.  Hadn't heard of the BME280 before.  You can get it on a breakout board from AliExpress for $8.54.  Specs on its temperature accuracy aren't very good though, so you end up with a two part solution again if you want decent performance.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on August 24, 2015, 11:19:52 PM
I like the sht25.....  I don't see anyone throwing one of those into their Davis station though.  I notice that there will be an update to the SHT3x line in Q4 2015.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: CW2274 on August 25, 2015, 12:45:59 AM
I like the sht25.....  I don't see anyone throwing one of those into their Davis station though.  I notice that there will be an update to the SHT3x line in Q4 2015.
What's that have to do with the price of tea in China?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on September 04, 2015, 07:46:07 PM
Using Dekay's awesome library (https://github.com/dekay/DavisRFM69) that decodes the wireless Davis data from the ISS, I've got my project working where a Moteino (https://lowpowerlab.com/shop/index.php?_route_=Moteino/moteino-r4) receives the wireless weather data then sends it up to Weather Underground's PWS (http://www.wunderground.com/weatherstation/setup.asp).  No Davis console or PC needed!  I put everything on Github (http://github.com/Scott216/Weather_Station_Data).  I've wanted to do this for a long time and I really appreciate the tons of hard work (https://github.com/Scott216/Weather_Station_Data/wiki/DeKay-Blog-Posts) DeKay put into reverse engineering the Davis communication.

Scott, I'm using some of your code for the weather underground data transfer.   Using the SIM900 GSM module, Moteino USB/RFM69HW; running off battery/solar.  So far I have the GSM module with the Moteino working off battery and solar just fine.  Working on getting yours and Dekay code mixed together transferring data from the sensors to weather underground.

I still have to figure what's wrong with my UV and solar sensor readings.  I used the example(s) but it does not match up with the real Davis console.

Thanks to everyone on this thread.   I have my project going.    I'll be happy when I finally get a enclosure with the solar panel.    Why is it so hard to find a nice outside enclosure with a solar panel of 5volts @ 2Amps.   I like the design of the Daivs Solar enclosures.

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on September 04, 2015, 07:46:36 PM
I like the sht25.....  I don't see anyone throwing one of those into their Davis station though.  I notice that there will be an update to the SHT3x line in Q4 2015.
What's that have to do with the price of tea in China?

You have a point...  :grin:
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 04, 2015, 07:52:24 PM
blacklistedcard: sorry, but I just LOL'd seeing your nick :)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 06, 2015, 01:55:22 PM
A heads-up for those who have used my Arduino library (https://github.com/kobuki/VPTools). I finally had some time to tend to this project. I've modified the code a little. I'm accounting for the repeater bytes in the receive buffer and added 2 more "carrier start bytes" at the beginning of the packet for transmissions. I've just set my Envoy to receive a secondary temp. sensor at station 3 and I can see it picking up packets steadily, both in my WeeWX raw data collector database and when I check by using the STRMON command directly on the serial console. I'll leave it running a few hours and report back. Unfortunately I can't use the RFM69 chip to transmit using its automatic packet assembly mode, I still need to send all packet components as data bytes (preamble, sync word, etc), but it seems to be stable so far.

Edit 1: I almost forgot that I fixed the CRC calculation for TX mode. This bug effectively disabled sending...
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on September 18, 2015, 01:18:39 PM
Almost there.   I'm getting different data for the UV sensor than the console.  The UV on the console this second is 1.3, I'm getting 0.1.

case VP2P_UV:
    if ( radio.DATA[3] != SENSOR_OFFLINE )
    {
      tt = ((radio.DATA[3] << 8 | radio.DATA[4]) >> 6) - 1;
      loopData.uV = (float)(tt / 50.0);
    }
    else
    { loopData.uV = 0.0; }

#if 1
    Serial.print(F("UV: "));
    Serial.println(loopData.uV);
#endif

I got a solar battery to power the Moteino and sim900 shield (http://www.voltaicsystems.com/9-watt-kit).   It's been running non-stop for 4 days.  No http get commands to really juice the battery, so that will be the next test.

Hopefully in the next couple of weeks I'll get into an outside enclosure and have it running off the solar battery and sim900 to weather underground.  Completely self-contained and piping the data to weather underground by cell.

Thanks everyone...
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 18, 2015, 01:34:51 PM
      tt = ((radio.DATA[3] << 8 | radio.DATA[4]) >> 6) - 1;
      loopData.uV = (float)(tt / 50.0);

I think I've made a mistake here. Could you please try without that -1 at the end? I'll revise this formula and fix it ASAP in the repo.

Other than this, everything else is spot on? Some values are averaged in the console over some minutes, IIRC, so not everything will be a 1:1 match.

EDIT: no, that -1 won't cause an error of this order of magnitude. That is a correction I found somewhere I can't find the reference for at the moment. That 1 divided by 50.0 at the end accounts only for a 0.02 of difference.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: blacklistedcard on September 18, 2015, 07:58:02 PM
I'm thinking there is a problem with the sensor(brand new).   It's night here and the UV is at 3.  lol.

http://www.wunderground.com/personal-weather-station/dashboard?ID=IONTARIO1050#history

Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 18, 2015, 08:17:43 PM
Perfect. I suggest you put on sunglasses and tanning lotion 8-)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: docbee on September 20, 2015, 12:34:37 PM
A heads-up for those who have used my Arduino library (https://github.com/kobuki/VPTools). I finally had some time to tend to this project. I've modified the code a little. I'm accounting for the repeater bytes in the receive buffer and added 2 more "carrier start bytes" at the beginning of the packet for transmissions. I've just set my Envoy to receive a secondary temp. sensor at station 3 and I can see it picking up packets steadily, both in my WeeWX raw data collector database and when I check by using the STRMON command directly on the serial console. I'll leave it running a few hours and report back. Unfortunately I can't use the RFM69 chip to transmit using its automatic packet assembly mode, I still need to send all packet components as data bytes (preamble, sync word, etc), but it seems to be stable so far.

Edit 1: I almost forgot that I fixed the CRC calculation for TX mode. This bug effectively disabled sending...
Applying the more relaxed bandwidth settings does improve reception of retransmitted packets for me as well. It also helps to dim down the original sensor data by reduction of RF threshold level.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 20, 2015, 12:39:32 PM
@docbee: to amend your post, the new change in the lib is very recent, it was not included in the change set when I wrote the previous notice.

I don't exactly get what you mean by "It also helps to dim down the original sensor data by reduction of RF threshold level" - would you elaborate on it a bit more, please? I haven't experienced worse reception, in fact, it made reception a bit more stable (packet reception ratio has increased) even for existing standard transmitters.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: docbee on September 20, 2015, 12:49:11 PM
I did 2 things to improve console retransmit reception.
1) changing REG_RXBW and REG_AFCBW to RF_RXBW_EXP_2
2) cutting off general RF noise by setting REG_RSSITHRESH to 100 (equal -50db)

Both changes have side effects but without these the reception of retransmit packets is not really good (lots of blank phases).

BTW: I have tested on a European Vantage version.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on September 20, 2015, 12:53:43 PM
Oh, I see. The fact that raising the RSSI threshold can diminish a lot of noise is well known, even DeKay noted it more than once. Widening reception range is something new I tried after writing my new OOK library for the RFM69, where it also plays an important role.

My recent changes in vptools allow simple reading of FEI, it would be interesting to know how much the frequencies are off in case of the retransmitting console.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on November 30, 2016, 06:08:03 AM
I've not seen anything in my searches, but has anyone decoded packet type 0xC?

From my ISS ("old" firmware) is see the well documented packet sequence...
Code: [Select]
8 E 5 4
8 E 5 9
8 E 5 A
8 E 5 A
8 E 5 6

But from my anemometer transmitter ("newer" firmware, but still years old)  I am seeing the sequence...
Code: [Select]
8 E 5 4
8 E 5 9
8 E 5 C
8 E 5 A
8 E 5 6

Example packets...
Code: [Select]
C0-01-A4-00-03-00-61-B5-FF-FF
C0-01-A7-00-01-00-9C-0B-FF-FF
C0-03-A7-00-03-00-BE-EA-FF-FF
C0-00-A7-00-03-00-50-38-FF-FF
C0-00-A5-00-03-00-BD-50-FF-FF
C0-00-A9-00-01-00-94-00-FF-FF
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 30, 2016, 06:28:02 AM
Nothing is known avout packet type 0xC at the moment. It can be transmitted from T/H and standalone anemometer transmitters, and might has some relation to their nature, ie. only one specific sensor is attached. I can also see them in my packed dumps for my anemometer transmitter, though I attached the solar/UV sensors there as well. Maybe newer firmware sends it as some additional info. I think it's safe to use the old sequence for your relay.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on November 30, 2016, 09:33:49 AM
Thanks, I'm trying to get the reception better before I move on to transmission. Picking both stations can take a long time, and though it works pretty reliably once it is locked on to both transmitters, the packet drop rate is still much higher than my consoles (success = 80's% v. 90's%).
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on November 30, 2016, 09:51:54 AM
20% packet loss is a little excessive. Try to align the wire antenna perpendicular to the board, or attach a decent rubber ducky dipole. With 3 transmitters I used to receive above 95% and with the 2 constantly running transmitters it's around 97% all the time. EU frequencies but after lock-on it shouldn't make a difference.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on January 14, 2017, 11:27:14 AM
OK, I finally have my relay code working reliably. I thought I had it working last week, but I found a difference in reception requirements between the console firmware versions I am running.

The code worked fine on my 'backup' console which is running ancient firmware - Nov 13 2004 - but when I tried to connect my 'production' console (v3.12) it failed to lock on the transmissions (well it did one time I tried then I couldn't repeat it). It turns out that the later firmware requires an additional 0xff byte to be transmitted after the data. Once I implemented this both consoles will now lock on to the first packet they see - result! - and I get all but 100% reception stats, the odd dropped packets are probably 3G phone interference as I'm sharing a band with them (sshhh!  ;)).

So to sum up I'm receiving data from a Davis ISS transmitter (2) and anemometer (1) and rebroadcasting as a single ISS (3). This allows me to move my solar and UV from the 'real' ISS to the anemometer.

My little Moteino project is based on a fork of Kobuki's excellent VPtools (https://github.com/kobuki/VPTools)  =D&gt; to which I have made a few tweaks.


My fork of VPtools is here (https://github.com/mcrossley/VPTools) - I'll make a pull request back to Kobuki if he wishes to incorporate the changes.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on January 14, 2017, 06:55:22 PM
Nice work! I'd definitely like to add changes aiming robustness or tunings necessary for certain board samples (eg. to adjust Moteino frequency skew). But I think I'll first have to look deeper into the changes we haven't discussed yet. It would be nice to have a single library supporting all basic cases of operation, including a relay such as yours.

  • Made the first detection of a transmitter much more robust, the Moteino now locks on after seeing the first packet

What does this change exactly do to make it more robust? In my tests it almost always locked on the first packet on the current probe channel.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on January 15, 2017, 09:17:12 AM
AS I mentioned, I effectively just broke the combined && clauses in the if statement in your code into separate if's. I also added an extra check, but that is only a belt'n'braces as I am transmitting as well as receiving. I cannot see anything wrong with your logic or syntax.

I find with your original code, and with my extra clause it would often see the first packet on the channel it was listening to for new transmitters, but then fail to pick up the subsequent one. Often it would only 'lock on' to the first transmitter when it saw the first packet from the second transmitter - or it timed out and went into 'broadcast listening' mode again.

With the separated if clauses it picks up the a new transmitter and always get the channel 1, 2, 3 packets. I cannot explain why as the code logic is the same!

I have just found a problem with my relay code - one a error on my part, I was not resetting the bad battery status after it cleared, now fixed.

I had a weird packet from my anemometer last night...
Code: [Select]
C8-C0-00-00-A8-40-5C-D9-E5-C9Which the console interpreted as a wind speed of 194 mph direction 0. The VPtools decode said it was an unknown packet without valid wind data. This is what also triggered me to find the battery status bug.

Whats concerning is that the check sum apparently passed, and then I "lost" that transmitter for 80 seconds, then picked it up again without having to re-sync on it. So did the transmitter have a funny turn?

EDIT: Budder! My rain gauge reed switch has packed up today - anyone know what the replacement component is off the tops of their head?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on January 15, 2017, 11:07:26 AM
I see, thanks for the explanation. Sorry for yet another nitpicking, but you write: "It turns out that the later firmware requires an additional 0xff byte to be transmitted after the data." - it already sends 2 0xff bytes (repeater info placeholders), and I couldn't find anything in your code that adds more of them, did I miss something?

The reed tube is this one here (https://www.scaledinstruments.com/product/davis-7120-031-reed-switch-for-tipping-bucket/). I used to see it on Ebay too but not any more. I'd probably measure the dimensions of the old failed one and try to buy something similar.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on January 15, 2017, 12:03:10 PM
Hi, line 434 in DavisRFM69.cpp, I send DATA[9] a second time...
Code: [Select]
  // transmit dummy repeater info (always 0xff, 0xff for a normal packet without repeaters)
  DATA[8] = DATA[9] = 0xff;
  SPI.transfer(DATA[8]);
  SPI.transfer(DATA[9]);

  // extra 0xff byte required after message for console to sync reliably
  SPI.transfer(DATA[9]);

I had an idea, my old Fine offset was still in the back of my shed somewhere - dug it out and it has 9x NO reed switches in it (8 in the direction, one in the speed). Salvaged one of those and soldered into the Davis - and its working again.  \:D/ The leads have been clipped too short on the FO reeds, and they look really cheap switches but it will do for now as I'm away with work again tomorrow.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: Nyowe_Nkanye25 on January 24, 2019, 10:02:19 AM
Kindly share with me.
Thanks
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: safuser on January 24, 2019, 12:31:12 PM
mcrossley,  Is your current code available on github?  Would it be possible to get a copy if not.  I am currently having mixed results with the current github code on several moteinos and a VP2 console.

Thanks.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on January 24, 2019, 12:38:31 PM
I'm not sure it's current (https://github.com/mcrossley/VPTools) - but other people have successfully used it.
It is also way out of date with kobuki's master copy - I'd recommend you try his first.

I have about 4 copies knocking around my laptop all with uncommited changes, and to be honest I've forgotten which one is running on my Moteino!  ](*,)
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on January 25, 2019, 02:55:38 PM
If you don't need mcrossley's relay functionality, you can use my driver, but only on EU frequencies. The US reception setting has a weird bug that makes the receiver skip every other packet on average, but since I don't have access to US transmitters, I can't fix it. Maybe if someone wants to sell a used SIM for cheap I can work on it again. On EU it works perfectly. The timing is the same so I have no idea why it's broken. Mcrossley does the timing a bit differently but the code is almost the same. The WxReceiver sketch and the weewx-meteoRX driver are a good woking combo in my repo (https://github.com/kobuki/VPTools/) that is used at a lot of places.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: mcrossley on October 06, 2020, 04:31:03 AM
Thread resurrection!

But I had not seen this documented anywhere before - maybe I'm wrong?

The rain packet 0x0E data in byte 3 contains a flag in bit 8 for a counter reset.

When first powered on the ISS sends x80 in byte 3.
This then increments for each rain tip until it reaches xFF.
It then wraps to x00 and increments up for each tip to x7F where it wraps to 0x00 again.

So only the lower 7 bits are the counter, with bit 8 being set after a power reset of the ISS for the whole of the first counter cycle.

I'm surmising this is to tell the VP2 console to reset its own internal counter to match to the ISS counter the first time it receives a packet with bit 8 set and thus avoid a large spike in rainfall if the ISS is power cycled.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on October 06, 2020, 05:04:34 AM
Thanks for the heads-up. I'll also implement it in our WeeWx driver.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: snapper on August 06, 2021, 03:14:34 PM
Hi,

Has anyone managed to get the kobuki WxReceiver sketch working with 3 transmitters?
I have 3 separate transmitters for my Davis station (ch1=Wind/UV/Solar, ch2=TempHum, ch3=Rain), the sketch compiles and runs fine on my Moteino R4 with rubber antenna, however after a short while it starts dropping packets and eventually stops.Then I’m guessing there is a timeout and it starts working fine again, and degrades…
I know packets are being transmitted as I can see the wind changing on the VP2 console, but the motino doesn’t.
It’s the same even if I use the t command to only listen to less transmitters (i.e. t1, t3 & t7 all show the same behaviour)

I know the hardware is good as the mcrossley RelayRxTx sketch will run for hours in the same location, with only minimal packet loss.

Any ideas?
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: kobuki on August 06, 2021, 03:18:38 PM
This question belongs to the VPTools GitHub issue page. Please ask there and I'll try to answer your questions.

One quick note though: the US band is still problematic as I haven't got around fixing the timing skew on the US band (on the EU band, no problems with 3-4 or more transmitters). Matt's sketches require manual freq. calibration first to alleviate the issue.
Title: Re: A Walk on the Wireless Side: Deciphering ISS to Console Transmissions
Post by: snapper on August 06, 2021, 03:27:51 PM
Will do, thanks for the quick reply.

This is EU band as I’m UK.