WXforum.net

Weather Station Hardware => LaCrosse Technologies/Hyundai => Topic started by: iplay1515 on January 12, 2012, 10:43:58 AM

Title: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: iplay1515 on January 12, 2012, 10:43:58 AM
I'm interested in a discussion about the tcp/ip and http messages/packets that the LaCrosse Gateway uses in conjunction with it's communication with the host website: box.weatherdirect.com and decoding the protocol used to send the temperature and humidity data to the website.

My objective is to determine a method to capture the data from the LaCrosse sensor units such as the TX60U-IT at the local network level rather than having to resort to page scraping and similar methods to monitor the data.

Since I am new on this forum, I would first like to verify that this area is the appropriate place for the topic, and if not, I would ask a moderator to advise and to relocate this post.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: thobas on March 01, 2012, 01:47:44 AM
I would be interested in what you find out. In fact, I signed up here because your post came up in my looking for these same answers. I am still doing my research and am currently on the fence as to whether to actually buy this device, because of the circumstances you describe. I do not want to have to rely solely on an external site, to get the measurements of local devices.

I have read some reviews that describe the reporting method used as a 'push' type data stream, instead of being able to poll the device directly, with say a HTTP GET request. Although, requests to port 80 of the gateway device IP do seem to return configuration information about the gateway device, I am not yet clear that any sensor information can be gathered this way.* Capturing the network traffic may be the only option, perhaps with a tool like tcpdump or WireShark.

I'm almost tempted to just bite the bullet and buy it, just to see what rube-goldberg type system I end up hacking together...

* I have only read this as reported by others, and not personally verified it is actually true.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: thobas on March 13, 2012, 07:31:34 PM
This is just to followup with this discussion in case anyone else comes across the need to do this;

I finally broke down and purchased the gateway+temp/hum sensors, and have some bad news to report for those wishing to keep the data local. After installation, the gateway took some cajoling to finally connect to the correct server(since the IP changed since it was made it seems). I was able to get it to register successfully, and then started to capture TCP/IP packets coming from the device. Here is what happens;

1)device connects out with a TCP handshake
2) once complete, the device sends out an HTTP PUT request to the weather directservers;
http://box.weatherdirect.com/request.breql
3)Inside this request, is an HTTP_IDENTIFY header, which I assume is to uniquely identify your router/device
4)The put request is an unspecificed type application/octet stream
---The part that is making me return this device and look elsewhere for this
5)The octet stream is not in any way human readable, so I tried other possible encodings.
It is not UTF-8
It is not ASCII
It is not JSON compatible
It is not MIME

A request to the technical support email for the device and asking for clarification or documentation on this octet stream, was returned many days later. It was short and to the point;

"you are requesting proprietary information that we will not disclose.
The information will only go to the weather direct servers."

Which is unfortunate, because the connection to the servers from the device is marginal, at best. I have a SLA for my internet connection, and packets are rarely dropped, so I know it is not my end with the problem. I do not understand how they expect to maintain a business with a randomly available webserver to view this data on, and no way for the user to directly see their own measurements locally.

The temperature inside my house is not their proprietary information. This is BY FAR the worst device of this type I have ever encountered from a users perspective. I'm halfway torn between sending this back for a return, or spending a few more hours to crack this ridiculous hiding of my own data and post it for all others to use. It's your(and my) data after all, not someone else's property to be hidden at all costs from the person making the measurement.
Title: Re: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on October 05, 2013, 02:40:21 PM
I've got one of these as part of the C84612 weather station. I'm still playing with it. the system works fine stand alone but won't connect to the server. My browser can see it fine, just not getting out.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on October 07, 2013, 01:48:44 AM
Since I posted the previous note I've been using Wireshark to sniff my network and can in fact see traffic between my gateway and box.weatherdirect.com, the server at 192.151.160.14.  See topic=16683.0 for more info.  The server is still not registering my gateway though. The saga continues...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 14, 2013, 04:32:54 PM
---The part that is making me return this device and look elsewhere for this
5)The octet stream is not in any way human readable, so I tried other possible encodings.
It is not UTF-8
It is not ASCII
It is not JSON compatible
It is not MIME

Ah!  A puzzle!  I like puzzles and am a senior software engineer.  I am sniffing packets now to see if I can figure out the data format.

Did you use a dumb hub or did you sniff your switch/router in promiscuous mode?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 23, 2013, 05:08:06 PM
Just saw the reply from skydvrz.  I finally got around to trying my ERF-100 registration on a DSL connection and it worked just fine.  Apparently my problem was coming from my wimax internet provider in Missouri.  Now I think I'll go buy another C84612 (Costco has them in stock again) and start playing with it again.  @skydvrz, I used a dumb hub when I was sniffing packets.  I haven't looked at the packets on this network but probably will once I get a new 84612 to play with.  Right now I just have the gateway.  I left the rest of the stuff installed back home in Missouri.  Have you had any luck?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 23, 2013, 10:04:32 PM

>Had any luck?

A little  8-)

I have analyzed several hours of data from the IG module.  I used a dumb hub to tee the connection between the IG and my router. 

The IG sends the server 3 different commands, denoted by the two bytes before and after the 64-bit hex number in the HTTP header. I believe the 64 bit number is cryptographically related to your "Activation Code", so that they can ignore your IG for "revenue control" purposes.  Someone earlier in the thread found the IG MAC address in the hex string before the commands - I believe they are correct.

Commands in no particular order:

Command 01:/:00 is some sort of ack/status command - the packet sent with it contains 5 bytes that are mostly the same in each packet, but change occasionally.  The trailing two bytes are probably a 16-bit checksum.  The entire packet could also be the IG probing the server to check if it is valid, but that is a long shot.

Command 00:/:70 is some sort of ping / RTS command.  The server usually responds with 16 null bytes, but sometimes not.  Sometimes the server fails to respond at all.  I believe the trailing two bytes of the server's response is a 16-bit checksum, but I have not verified this.  This command may be a throttling command to lower bandwidth on the server, or to verify that the server is up, working and ready to receive data.  I will have to look at the timing data to see if this is a throttling mechanism for the server or something else.

Command 01:/:01 is the sensor data packet.  It *looks* like unencrypted data fields.  But seriously!  197 bytes to send this teensie amount of data?!  I could send full-precision data for all the sensors in one 8-9 byte packet.  I suspect the data fields are sent in some sort of record or structure, probably full of double precision numbers in binary format.  They may or may not be packed, but I doubt they are encrypted.  Unpacked data would have "padding bytes" stuffed in between fields with odd-numbered field-width byte counts.  Most of the payload records have only minor changes when compared to previous records;  They should look completely random if the fields are encrypted.  I believe the trailing two bytes of the sensor data payload is a 16-bit checksum - probably CRC16.

The only thing to figure out is the data type and field locations within the record for each sensor reading, and how to simulate a server - you would need to "keep the IG talking".  It may clam up if it cannot contact the mother ship.   I doubt a server simulator would take much rocket science.  It is possible to set the DNS server IP in the IG to an address of your choice, so it would be trivial to hijack it and have it chat with your PC instead of its normal server.

I noticed occasional data packets of 0x25 length.  These appear to be alarm packets produced if you set a min/max alarm value on your LCD controller.  They do not appear to be encrypted.  More research needed here.

Note: encryption may be in use, but if so it is probably simple XOR encryption with a fixed key.  Clueless newbie stuff.

I have no idea why they used checksums on all the data - the TCP/IP hardware layer insures there are no data errors, so it was a waste of time and bandwidth.

I am designing a program to do further analysis on the sensor data payload packets.  More to follow...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 26, 2013, 02:41:40 PM
When the server responds with 38 bytes of data in response to an 01/00 command from the gateway, the contents sent by the server is the time of day, DMY info.  Used to keep the LCD controller clock accurate I would assume.  I am not sure what the leading 22 bytes do, and some of the trailing bytes remain a mystery.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 27, 2013, 12:47:31 PM
great! I sprung for another weather station and have it up and running with lacrossealerts.com. Their server seems a bit flakey imho. I'll have to get my sniffer back up and running to see if I can replicate what you found and add to it.
Good work so far.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 27, 2013, 03:06:21 PM
I shot close-up video of my LCD display all night long and sniffed packets at the same time.  I got 7 gigabytes (18 hours!) of some of the most boring video ever shot.   :lol: 

I created a spread sheet of (some of) the values on-screen when the big data packet went out. I noted packet numbers in the spread sheet, local time, Etc.  I will use it to correlate measured values with hex values in the packets. 

The LCD display blinks the "INTERNET" indicator when the big data packet goes out.  This indicator shuts off if the server does not ACK the packet. It turns back on after the next successful transmission.  The interval between large data packets is anywhere from 1-5 minutes or so, but averages about 4 minutes.

I also blasted the LCD display (location of the inside temp/humidity sensors) with a blow dryer to make a large change in readings during the recording period.  I will probably have to do something similar with the outside sensors.

Analyzing packets now, but it looks like they are shifting XOR keys or whatever encryption they use periodically. There is some suspicious hex in the big data packet:

AA AA AA, BB BB BB, EE EE EE found at offset 0x95 and 0xA4.  They are duplicated in the two offset locations.  The 3-byte hex values seem to perturb the other hex values as they change.

The repeated hex above may be the XOR key they used, but it would be silly to send it each time.  It might also be that they accidentally encrypted some fixed 0x00 or 0xFF bytes in the data record with the XOR key they are using.  If this is the case, then they are using a short key (1 byte) and it should be trivial to decrypt the fields.  Or not...

I am hoping that the GW is light on CPU power and memory space, and that the designers did not go to heroic lengths to encrypt the data.

Here is a good Wireshark _capture filter_ (not a display filter) for sniffing just the important data traffic to/from the GW:

host 192.168.xxx.xxx and tcp port http

where 192.168.xxx.xxx is the actual address of the GW. 

This keeps you from "drinking out of the firehose" during long sniffs.

The GW also chats with the DNS server to get the current server IP, and does the usual TCP/IP muttering, but this filter blocks that.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 27, 2013, 05:49:20 PM
Wow, I'm impressed.
You might just try taking the external sensors out of range. or just take the batteries out of the hum/temp sensor. Its the gateway for the other two.
I guess I'd be surprised if they encrypt weather data.
Have you peeked inside the gateway to see if you can tell what uP they're using?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 27, 2013, 08:47:06 PM
I suspect that the GW would probably just repeat the last valid values if I somehow disabled a sensor.  I have seen the LCD display show ___.__ for missing values, so maybe there would be some value to remove batteries, Etc.  The weather outside is really crappy, so I'd rather not climb out on my roof right now  :-)

As for the uP used - it really doesn't matter, since the task at hand is interpreting the TCP/IP data and then writing a server simulator - plus I don't want to break my GW module.  The case looks pretty cheesie.

If you haven't registered your unit yet, I'd be interested in seeing the entire process in Wireshark PCAP format.  It took me 2-3 tries to get my unit registered, so I don't want to go through that again!  ](*,)

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 27, 2013, 10:48:38 PM
I registered my old GW. Thats why I bought another system for use in TX. I've got the new GW that came with the new system and will register that before I go back to MO. It wouldnt register but it does communicate with the server so perhaps if I register it here it'll work with the system back in MO. Ive got part of the registration captured with wireshark from back in MO if you think thatll help.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 28, 2013, 01:03:46 AM
I've got the new GW that came with the new system and will register that before I go back to MO. It wouldnt register but it does communicate with the server so perhaps if I register it here it'll work with the system back in MO. Ive got part of the registration captured with wireshark from back in MO if you think thatll help.

We may need a complete registration "conversation" to see if there is a way to simulate it with a replacement server.   For folks that have an existing registration with the actual server, I don't think there will be a problem.  Hijack the DNS (for the GW only), point box.weatherserver.com your own server as a replacement and it should continue to work as long as the GW thinks everything is normal.

There is much work to do  #-o

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 28, 2013, 07:06:43 PM
Now THAT'S an understatement.
Just got wireshark installed down here and ready to start sniffing again. I may try registering the new GW later. I'm reluctant to use my new activation code since I think there is a year limit on it. Maybe not. I also think I saw that it just reverts to display of current data with no history if you dont extend the subscription.
Later.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 28, 2013, 07:50:28 PM
I made a lot of headway today, figuring out many of the data fields in what I call the SDP (Sensor Data Packet).  There are a lot of unknown/guessed-but-unconfirmed fields, but I have verified a few by comparing the hex data capture against the video I shot of my LCD panel over about 18 hours time.  The packet contains a lot of min/max values, timestamped with date/time.  I verified the following fields:

Code: [Select]
0x46 for Inside Humidity
0x53 for Outside Humidity
0xA7 Barometric Pressure MSD
0xA8 Barometric Pressure Center Digits
0xA9 Barometric Pressure LSD (only the upper nibble is used)
0xAA Barometric Pressure Delta since last reading MSD
0xAB Barometric Pressure Delta since last reading LSD

Typically, but not always, the byte value in hex is the actual digit displayed on the LCD panel.  Example LCD says 30.04 for barometric pressure - the hex would be 0x03 0x00 0x41;  If humidity is 92%, then the hex is 0x92.  I have to admit that using display values in hex to transmit telemetry to the server threw me for a loop.  You typically compress data like this to a smaller numeric format.

I suspect the wind direction vane output is some sort of 3-sensor angle detector.  There are some really weird values in the SDP and I suspect it is this sensor.  My wind readings are all over the place, with the light and variable winds we have going on right now.  Maybe someone that does not have to climb out on a roof can tie theirs down and capture all the outputs at the various points of the compass rose.

I am still looking for temperatures - they may be in centigrade and then converted at the LCD - I don't know.  They may also be offset by a fixed value.  More research required.  I did notice that the LaCrosse web site shows 32F if you disconnect your outside sensors.  Maybe this means that temps are passed as Degrees C, zero (disconnected) being freezing (32F).

I made up a spread sheet of many other suspected data columns in the SDP.  If you are interested in obtaining a copy or contributing, PM me here.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: asquaredancer on December 28, 2013, 09:01:00 PM
sounds like data may be BCD.
I'm still playing with wire shark. my wind sensor is still on the porch so it would be easy to get angle data when I get to that point.
I'll PM you.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 29, 2013, 12:42:27 PM
I think I just figured out the wind direction bytes (0x95->0x97).  \:D/

Each hex byte is divided into two 4-bit numbers.  There are 16 points to the compass rose, so you can represent all points in 4 bits.  There are 6 samples, right shifted (I think) across the three bytes. Each time a new sample comes in the bytes are shifted right by 4 bits and new sample goes into the upper 4 bits of offset 0x95.

So:

Code: [Select]
0 = N
1 = NNE
2 = NE
3 = ENE
4 = E
5 = ESE
6 = SE
7 = SSE
8 = S
9 = SSW
A = SW
B = WSW
C = W
D = WNW
E = NW
F = NNW

So if the bytes were

BC CC CC

Then the wind is currently blowing WSW.  A while ago (and for 5 sample periods) it was steadily blowing W.

Now... I have very little data to correlate, so I could be 180 degrees out of phase.  I believe the "big blip" on the LCD wind direction compass is the number found in the upper part of the 0x95 byte.  The "minor blips" are found in the other bytes.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 29, 2013, 01:43:07 PM
So, to roll up all the findings so far...

The Sensor Data Packet (SDP) is not encrypted, and contains "display values" in most of the fields.  This means they are not the usual integers, long words, double precision numbers one usually finds in telemetry packets.  The values are (usually) the actual digits in hexadecimal format that appear on your LCD screen.  So if you screen says "98" then the corresponding value in the SDP data will be 0x98.  Decimal points are not found in the SDP, since all screen values use a fixed DP.  Dates and times are stored as "display values" too:  13 12 29 02 06 (in hex) is 2013 December 29th at 02:06 in the morning.  Note the Y3K problem with this format  ;)

The SDP contains a lot of historical min/max values, paired with the date/time of the event.  That is the major reason the packet is so large (192 bytes).

What do you need to do to read/store your own weather data?  Here are two different methodologies:

For legal reasons, I strongly advocate the second approach.  :grin:

Simulate the Lacrosse server:

Create a DNS server to fake out the GW into using your own local server instead of theirs.  The GW can be reprogrammed to use a static IP address on your LAN, and look for the DNS server at an address of your choice (your own PC).  Redirect the GW to use your own HTML server instead of theirs.

Create an HTML server to handshake with the GW, so that it thinks it is talking to the real one.  You may need to capture the magic numbers that are exchanged during the registration process or simply mimic the normal chit-chat the GW does with the real server.  This new server should parse the SDP packets and store the data to a database.

Next, you need some sort of viewer program to watch the DB in real-time.  This is optional and probably only useful if you only want to watch your historical data at your house.  I suppose you could expose your DB server on a public IP and watch your data with a client/server app remotely.  A web server/browser would be simpler and platform agnostic...

So, to put you info on the web, you need some way to put sensor readings, graphs, Etc to something a web browser can read - and access over the Internet.  There are a many ways to do that - create an ISAPI DLL that hooks into an Apache or IIS web server.  The DLL reads the DB and creates HTML and JavaScript to render the data in a web browser.  Another way would be to create a small, stand alone web server that only does weather data display.  If you are a .NET weenie, you could do it that way and use IIS.

Use the Lacrosse server (and pay):

A second way to read/manage your own data, but only if you pay Lacrosse to show it on their server would be to snoop the SDP packets going to their server, parse them and store them locally in a database.  If you are a good programmer, there are libraries available to create your own WinPCap-compatible app to snoop the SDP packets I mentioned.  Parsing the SDP contents is relatively simple.  Next you would have to write some sort of viewer program to watch your historical data (see above).  Using Wireshark works for snooping, but it is a royal PITA to get usable data out of it.  I strongly recommend using an integrated snooper/parser/database updater solution.  Oh yeah - you need to acquire an old-school "hub", not to be confused with a "switch" or "router".  You need one of the now-obsolete CAT-5 hubs to snoop on the GW traffic.  There are lots of them on eBay (where I just bought mine).  They run $20-$30 USD.  Warning: many do not come with a power supply. Make sure the hub ships with one if you are not technically savvy enough to cobble one together.  Mine arrived sans power cube.  Luckily, I had one in my junk box that worked.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on December 30, 2013, 09:17:01 PM
I wrote my own Internet Gateway packet sniffer today!  \:D/  It listens in on the Internet Gateway chitchat with the Lacrosse server and displays sensor readings on my computer screen in real time (well, as fast as the Lacrosse server gets updated, which is about every 4 minutes or so).

The app is complete with a partially working SDP parser and data viewer. It runs on Windows - probably XP or higher.  Tested on Win7-64 Pro.  Written in Embarcadero Delphi XE, using the Magenta PCAP libs (free) and ICS (free).

It is a stand-alone app that does not need Wireshark or any other sniffer utility.  It does use WinPCAP, which is a free utility that is typically installed along with WireShark.  You fire up my app, select the Ethernet port you want to use to sniff (I have two ports on my machine) and set the IP address of your GW.  Click the start button and off it goes.  It can run without WinPCAP, but it might not work on certain network cards.  I plan to support both modes.

You do need a special (read obsolete) Hub, but the hardware is cheap. 

Lots of work to do (it was a really quick & dirty hack), but it does correctly display Barometric pressure and wind direction in real time.  I hope to get the rest of the sensors reading correctly tomorrow, if I have time.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 03, 2014, 08:45:28 PM
Inside Temp seem to be encoded thusly:

Fahrenheit = (0.18033 * N) - 40.0

Where N is the BCD value found at offsets 0x13  and 0x14

Example:

0x06 0x27 (hex)  = 627 (base 10)

F = (627 * 0.18033) - 40.0

F = 73.1

0x03 0x06 = 306

F = (306 * 0.18033)

F = 14.8

I have confirmed the formula, but it may need minor tweaking to make it track the display value.  It seems pretty close though. 

I hit the LCD controller with a blow dryer, producing an 0x08 0x45 at around 112F

The 14.8F reading was arrived at by putting my LCD controller in a chest freezer for about 30 minutes.  The funny part was that it continued to chat with my outside sensors and the Internet while chillin' with the pork chops.  I guess my freezer makes a lousy Faraday Cage at 900 MHz :-)

I imagine Outside Temp is encoded similarly.

Conversion to degrees C will require a different formula - sorry metric heads :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 06, 2014, 12:47:35 AM
Here is the alpha release of my La Crosse Gateway app.  It listens in on the Gateway's communications with the La Crosse server and displays the current sensor data on your Windows PC screen.  It also saves sensor readings to a CSV file.  Run the EXE and it should decompress into the app itself plus much of the source code used to create it.

Enjoy!

https://www.dropbox.com/s/zzlbbwrrvu0iwql/lcazSourceEXEInstall.exe (https://www.dropbox.com/s/zzlbbwrrvu0iwql/lcazSourceEXEInstall.exe)

Open the included ReadMe.HTML file with your web browser to see required hardware, settings, screen shots, Etc.

Note:  The above link is obsolete.  Please look later in this thread for updates to this program.

Please comment here.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 07, 2014, 01:38:43 PM
Nice job for an alpha, thanks.  I have the program working and I am getting data.  The only problem I have is that it has quit gathering data twice.  Once after ~2 hrs and the second after ~5 hrs.  Nothing unusual that I can see occurred.  No error messages and the computer did not go to sleep.  Getting ready to start it again. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2014, 02:28:51 PM
Yes, unfortunately there is a problem with the packet capture library I used.  It gives the rest of the app heartburn when it updates the screen. If some rare timing conditions occur, the program will freeze while trying to update the sensor readings fields.

I think the important part of this release is the conversion algorithms and SDP field mapping appears to work and the sniffing technique is sound. 

I'd like some feedback on Wind Speed.  Does the LCAZ wind speed reading track the speed showing on the LCD display?  In my neck of the woods, we don't see much higher than 1-2 MPH this time of year, and most of the time it is dead calm.  It is a bit difficult to test this sensor reading when it is zero most of the time :-)  My anemometer is mounted on my roof at about 40' up, so I am not too inclined to climb up there to play with it.

I am aware of the 0.1-0.2F error on both temperatures.

I am already working on the next generation of this app.  It will probably use a database to store raw data, HTML headers and sensor readings.  This will be harder to deploy, because in addition to the other hardware, you'd need to install a (free) database server.  I will be using MySQL, as I already use it for a number of different software development projects.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on January 07, 2014, 06:53:45 PM
I am already working on the next generation of this app.  It will probably use a database to store raw data, HTML headers and sensor readings.  

Can I suggest that it have the ability to choose where the output goes?  My preference would be to simply append the data in CSV format to a user-specified file.  This would allow the user to post-process as desired.

I have a Lacrosse C84612 I got for Christmas.  I just ran across this topic and am interested in what you are doing since I want to upload the data from my weather station to the Weather Underground.  

Not having as much of a hardware/software background as you, I wrote a Perl script that goes to the www.lacrossealerts.com website and scrapes my data.  I then upload it to Weather Underground.  I trigger the script every 15 minutes on my Windows 7 machine via the Task Scheduler.  The script works ok, the problem is that the lacrossealerts website does not show updates from every 4 minutes.  In fact I regularly see delays of an hour or more between readings available on their website.  I don't know if the packets are being sent up from my device every 4 minutes and being dropped, or if the lag has some other cause.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2014, 07:00:04 PM
The GW attempts to send data to the server every 2-4 minutes.  It doesn't always get there, but you can sniff each outbound SDP  transmission.  The server sometimes fails to ACK, or physically tells the GW to come back later.  After La Crosse gets the data, who knows? 

That said, do you really need to have that granularity of sensor information?  The SDP actually contains averages, as well as the current readings.  I am not sure which ones get displayed on the La Crosse server.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on January 07, 2014, 07:11:54 PM
That said, do you really need to have that granularity of sensor information?  The SDP actually contains averages, as well as the current readings.  I am not sure which ones get displayed on the La Crosse server.

No, I don't need every 4 minutes.  But I would like to be able to post a reading every 15 minutes, which I think is the Weather Underground suggested interval.  I keep a log of the update timestamp LaCrosse reports. In about 7 days of running (I just wrote the script recently) I see 47 delays of 30 minutes or more between updates posted on their server, 13 of more than 45 minutes, and 6 of more than an hour.  So their server isn't the best source of data for reposting, which is what makes the snooping that you are doing sound intriguing.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2014, 07:59:57 PM
Feel free to give my app a try.  Download, run the "installer" and open the ReadMe.HTML file found within.  It lists the hardware and software you need (reasonably cheap - way less than the weather station) and diagrams for hooking everything up. 

There is source code for the packet data structure and algorithms to convert the raw binary into sensor readings.  Maybe you could write your own custom sniffer/submission tool that takes the 2-4 minute GW samples, averages them and ships the results off to Wunderground?

You can run your spider and my app side by side to see if you get comparable results.  My app makes a CSV text file, so you can open it in a spread sheet.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 07, 2014, 11:31:00 PM
From quick checks it seems like wind speed seems about right on.  I'll spend more time checking it closely in the next couple of days (tomorrows my wife's birthday and I may not have much time). I was going to play with the log file you have generated but I think I'll wait until I see what you have in the database first.  Like I said I like what you got so far.  It's been running since the last message I left and still collecting data.  Please let me know what else I can do for you.  We will be getting some wet windy weather beginning tomorrow night and Thursday so that may be a good time to match the WS data and the packets.

Thanks again. :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 08, 2014, 12:26:33 AM
Lurkers:  I am very interested in any sort of LCAZ inaccuracy.  If you can tell me the amount of error, I can correct it, but you too need to be precise.  I am especially interested in when it is 70F+ (toasty) inside your house and less than zero outside.  The wider apart the temperature between inside and outside, the greater the error will be - and the easier it will be to fix. I live in Western Washington state, and it rarely gets into the teens in winter. I need more gnarly weather! ;)

I put my LCD display in my freezer for a while, but it didn't get below about 14F.  Not a really good data point.

Please send the temperatures the LCD display says, versus what LCAZ says.  It is easy to adjust the calibration, but I need to recompile the entire app to change what LCAZ says on it's screens right now.

I am working on a version that can save to a real SQL database, Hex logs and CSV files - all at the same time. The full-blown database is necessary to gather enough raw data to make the next generation of the app. 

And... I absolutely have to come up with a better product name  :oops:

In case you were wondering - all versions will be free for the downloading.  This is way too much fun to charge money for  8-)

Just a question at this point, but how are you guys submitting weather data to Wunderground and other sites?  How do you identify your data as coming from "you" and how is it submitted? Do you use a user name and password and transmit data somehow?  I don't need user names and passwords - just a URL or FTP address + instructions for sending weather data.  I might be able to do this automatically - actually I am pretty sure it will be easy compared with figuring out the GW data format.   :lol:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 08, 2014, 12:33:38 AM
Can I suggest that it have the ability to choose where the output goes?  My preference would be to simply append the data in CSV format to a user-specified file.  This would allow the user to post-process as desired.

I am working on it.  I need full logs, but I have no problem with saving CSV files full of converted sensor readings for the folks that only need minimal logging.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 08, 2014, 01:45:23 AM
In my interests to uploading to Wunderground I found this.

http://wiki.wunderground.com/index.php/PWS_-_Upload_Protocol

In there it looks like it's straight data that gets uploaded.  Sample--

http://weatherstation.wunderground.com/weatherstation/updateweatherstation.php?ID=KCASANFR5&PASSWORD=XXXXXX&dateutc=2000-01-01+10%3A32%3A35&winddir=230&windspeedmph=12&windgustmph=12&tempf=70&rainin=0&baromin=29.1&dewptf=68.2&humidity=90&weather=&clouds=&softwaretype=vws%20versionxx&action=updateraw


Hope this helps

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 08, 2014, 10:20:53 AM
Perfect.  Yes, I can do that - as soon as I figure out where in the SDP the wind gust speed is stored. 

Apparently dew point is calculated like so:

Td = T - ((100 - RH)/5.)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Brandan on January 11, 2014, 03:42:21 PM
Has anyone figured out a way to get it to think It has a subscription, can someone send me the packets? I can't get mind to send anything.

I've been capturing them though my laptops LAN port.

What LA Crosse is doing is kinda scaumful. Atleast it violates the ACM Code of ethics.

They should really make an API.

On port 80 it says there's a data server also that's on a different IP address than the box.blah.com addresses.

Also it has options disabled for the users about uploading to an FTP server. Maybe it's possible to break into that and have it upload data to my FTP server.

I've written similar programs that emulate servers in the past but. I need to know the packages on which the GW communicates with the server and itself.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rippleeffect on January 11, 2014, 07:36:27 PM
I would love to be able to have this gateway send data to Weather Underground. I highly doubt my wife would let me drop any more money on the setup I already have, so I'm stuck with finding an option.

I'll be following this thread for hopes of an "easy" way of doing this in the future.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 11, 2014, 08:46:16 PM
I don't see the gateway itself ever uploading to Wunderground, but it looks like skydvrz will provide us with the data that can be uploaded to WU.  If you download the package he uploaded a few messages ago, you'll see that some of the data is already available. It will have to be averaged out and formatted as listed above.  I personally am looking forward to the next version skydvrz releases.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rippleeffect on January 11, 2014, 08:53:06 PM
I've got a little Atom processor PC in the kitchen that doesn't use much power. If I was able to snatch the data and use the PC to send data to wunderground, that work probably work.

The PC only uses 40w.
Title: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Brandan on January 11, 2014, 08:53:13 PM
The gateway won't but the software that intercepts it or simulates the server can.

I use a raspberry pi for those kinda of projects since it uses USB power
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 11, 2014, 09:39:56 PM
I just completed a snooper program that is much more reliable than LCAZ - the program I posted here on the forum a few days ago.  It requires a full-up database to record the data it captures from the GW.  The good news is that the database server software (MySQL) is free and fairly easy to install.  LCAZ was a proof-of-concept program and a really fast-hack.  Unfortunately it hangs after a few hours, but it does successfully capture data sent from the GW to the LCX cloud server.

The hardware necessary to make LCAZ work is under $30 USD.  The same hardware will be necessary for my new second generation snooper: SkySpy Monitor.  Download the LCAZ package and look at ReadMe.HTML found within the archive to see how you need to hook things up.

I am now working on a database viewer program SkySpy Viewer that can look at weather station sensor readings in real-time.  Again, this requires the database mentioned above.

Once that is done, I may invite a few people to submit data to my database server so that I can look at a larger sample of GW data packets.  Using the Viewer, they will be able to see their own sensor readings in realtime.  I will not be leaving my database server up indefintely - this is a couple month experiment.  I don't have the disk space for everyone's data.  The software will be free.

I am not sure of the legality of completely replacing the LCX cloud server.  It can be done.... easily.  I invite advanced programmers that wish to take that step to look at our captured data and create a replacement solution.  I will do my own, but I probably won't publish that.

Sending data to Wunderground is very simple.  I will probably add that to SkySpy Monitor.  You do need a valid Wunderground account though.  The GW can only send data to the LCX Cloud Server - nowhere else.

All of my programs will be Windows-only, but I will post much of the source code.  The monitor program requires that your computer remain on 24/7/365 for obvious reasons.  I don't think it is possible to run any of this on a mobile device, even if you were able to rebuild the code for that device.  

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Brandan on January 12, 2014, 04:49:53 AM
It's very possible and not to difficult to get it to run on a mobile platform. Does this require a subscription still? I'm not going to pay more than $3 to access something I can build myself.

I managed to capture the advanced setup packets, it sends it in plain text.

I should be pretty easy to change the servers address from there.

(http://brandanlasley.com/fakeout.PNG)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 12, 2014, 11:23:57 AM
Brandan,

The packet you are looking at is a broadcast packet used by the Gateway Advanced Setup (GAS) utility to determine the GW's IP address on your LAN.  This is how GAS fills in all of its visible fields with IP addresses and hostnames.

The GW uses TCP/IP HTML headers with a binary payload to send/receive information from the La Crosse (LCX) Cloud Server.  There are 6 or 7 different HTML packet types that are exchanged to/from the server.

If you have not sniffed any HTML packets going to/from your GW, then you have not installed a "dumb hub" to allow interception of this traffic.  Without one, you will only be able to sniff broadcast packets that are sent to every device on your LAN.  Broadcast packets contain no sensor information.

If you are using a "switch" or a "router" on your LAN, then you can only sniff packets that are addressed to your computer's IP.  Since the GW is sending packets to an external server on the WAN, you will never see them.  There is a diagram in my LCAZ installer that shows how to connect a hub to remove this restriction.

Note:  You need a dumb hub to do any sort of serious sniffing on a switched LAN.  No exceptions.

The Sensor Data Packet (SDP) is 192 bytes of binary data attached to an HTML packet. None of the SDP is human-readable.  You need to capture this packet, strip off the HTML text, and then convert the binary information back into its sensor readings if you want to tap into the data being sent to the LCX Cloud server.  The SDP is sent every 2 to 4 minutes.

See my LCAZ code (posted earlier in this thread) for the methods I use to parse the SDP.  Most of the source code is included.

Faking out the GW into thinking it is talking to LCX when in fact it is talking to a server under your control is not simple.  Possible, but not simple.  You would not need a hub to do this, but you would need a computer running 24/7/365 to replace the LCX server.

I don't know if GW continues to send the full SDP to the LCX Cloud Server after your 3-month free Extended Service trial period expires.  If it does, then you would not need to pay for the extended service.  I also noticed that freebie Basic Service on the LCX server is limited to 3 years.  I don't know what happens after that.

I think the Extended Service costs around a buck a month.  Not too bad, price wise.  I think their server software sucks, and that is why I am interested in replacing it in some way.

As for changing the GW's server address - not so easy.  You would have to use GAS to set the GW's DNS IP to a fake DNS server on your LAN.  When the GW calls your fake DNS and asks for the IP of box.weatherdirect.com, it would reply that the new IP is an address on your LAN - the address of your fake cloud server.  A fake DNS server is pretty simple to write. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Brandan on January 12, 2014, 11:45:36 AM
Most of that work was cut out when I plunged the GW into my laptop, bridged the connections between LAN and WLAN and then sniffed the packets that way. All gateway communications goes though my laptops wireless card, but its on its own IP so its easy to get the packets from them.

What you saw there was a UDP return from the GW after the GAS paged it with a binary information.

I've also seen how the GAS changes the settings on the GW by sending more binary. All you really need to do is make a UDP program in something like perl and rebroadcast what was just sent.

But of course you'll have to see how the GW talk to the server before changing the address. No DNS involved.

The problem is i'm not paying for a subscription and I would rather engineer a way to capture from the weather station directly. But thats a much harder task to complete.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 13, 2014, 03:23:42 PM

This is a great thread, just got a costco station with this gateway.   I've disassembled the gateway and am trying to get  a serial console on it.  I will let you guys know what I find out.

Mike
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 03:27:04 PM
Serial console?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 03:53:23 PM
Can I suggest that it have the ability to choose where the output goes?  My preference would be to simply append the data in CSV format to a user-specified file.  This would allow the user to post-process as desired.

I was just thinking about that and did some quick calculations.  Excel would bomb after around 6 months of data collection if you imported a CSV that large into it.  I think that the file name will have to be rotated periodically so that the CSV files do not grow too large.

The Viewer program I am working on will have Excel, PDF export out of all the data grids.  Data grids also allow user-designed printed reports.

I added all the date/time fields I found in the SDP - they are being logged to my SQL database, so I should be able to figure out which sensors they are attached to.  I believe that the LCD display keeps track of min/max values and dates.

I attached some screen shots of the SDP viewer.  The Current Readings grid updates in real-time, as new SDP records are added to the database by the GW.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 04:07:45 PM
Another screen shot, after an GW update.  The app allows you to pick from 45 different "skins" or themes.  This is the Xmas skin.  The previous skin was London Liquid Sky (my personal favorite) :-)

Note the popup on the current readings grid - you can export/print from any grid in the app.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 13, 2014, 04:09:38 PM
Serial console?


Yes so far no joy.

(https://dl.dropboxusercontent.com/u/12351460/weather_gateway.jpg)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 04:14:58 PM
Serial console?


Yes so far no joy.

I would be surprised if you could get anything out of it via serial comm.  All the GW smarts are devoted to TCP/IP.  You probably found a firmware programming port or maybe a production-test I/O port.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 04:19:48 PM
The ssMonitor app, showing the debug screen.  You can see the HTTP packet header and binary data payload - this is the raw, unparsed data sent to or received from the LCX Cloud server. 

I will add a full-up hex viewer soon.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 13, 2014, 05:12:13 PM
Serial console?


Yes so far no joy.

I would be surprised if you could get anything out of it via serial comm.  All the GW smarts are devoted to TCP/IP.  You probably found a firmware programming port or maybe a production-test I/O port.

Yea just in case it is linux or some other know quantity.   In lots of embedded devices the serial port is very handy.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 13, 2014, 08:55:59 PM
Mycal,

That is some very strategic potting compound over the GW processor and RF circuitry.   :grin: I wonder whats underneath? 

Do you think the connector you hooked up to is actually a JTAG header?  If so, you need some different hardware to snoop it.  Also, you could brick the GW in a heartbeat. Be careful.

I saw a hack for a different model home-weather station where the guy tapped into the data stream coming out of the receiver for all the wireless sensors.  The wireless data signal was simple ~915 MHz, PCM modulation.

Since all of the sensor data is routed through the GW, you might be able to do a complete end-run around the whole TCP/IP web server thing.  You would need some sort of hardware UART or do some clever software to turn the PCM bits into bytes and then parse all of that back into sensor readings.  I don't know what would be harder - hacking the hardware or snooping the TCP/IP  #-o
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 14, 2014, 01:09:48 AM

Yes it is very securely potted, even the jelly bean parts.   I don't think it is jtag, I soldered the header on and it has +3.3v, gnd and 2 wires that go into the big potted chip.  This screams serial, but so far I haven't seen anything out of it,  My DSO is packed away so I need to break that out for a peek.

I'm also guessing the 3 pin header that I did not stuff is the Manchester encoded data from the radio to the CPU.

I'm not worried about bricking it,  I design stuff like this for my job so I'm familiar.

Yea it might just be better to build a new gateway that can talk to the wireless parts.   But hijacking the pipe and emulating  weather direct server as skydvrz is doing is likely a pretty good way to go, especially if your running DDWRT or OpenWrt as it becomes trivial.

I may actually go down that path as I'm working on modules for OpenWrt at the moment for other stuff.  I can post a pic of the other side of the board if anyone is interested.

-M





Mycal,

That is some very strategic potting compound over the GW processor and RF circuitry.   :grin: I wonder whats underneath? 

Do you think the connector you hooked up to is actually a JTAG header?  If so, you need some different hardware to snoop it.  Also, you could brick the GW in a heartbeat. Be careful.

I saw a hack for a different model home-weather station where the guy tapped into the data stream coming out of the receiver for all the wireless sensors.  The wireless data signal was simple ~915 MHz, PCM modulation.

Since all of the sensor data is routed through the GW, you might be able to do a complete end-run around the whole TCP/IP web server thing.  You would need some sort of hardware UART or do some clever software to turn the PCM bits into bytes and then parse all of that back into sensor readings.  I don't know what would be harder - hacking the hardware or snooping the TCP/IP  #-o
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 14, 2014, 04:16:45 AM

So I have figured out you can talk to the gateway via UDP and set the parameters, reboot and lots of other stuff.  There is also a UDP broadcast search mechanism so you can find the gateways on the network..

So you can just program the gateway to talk to your own server, no prob.

I'm pretty tired, but I will write this up and post tomorrow.  Maybe we can write a simple configure program, then just write our own linux module and be done with it.

-M

PS the port is 61751 on UDP,  example 00 05 [MACADDRESS] 00 0A reboots the gateway.

IE 00 05 01 02 03 04 05 06 00 0A

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 14, 2014, 04:50:20 PM
Ok here is the UDP spec I've created and a test PHP script (command line) to exercise it.

I think there is more, but I haven't setup my weather stuff yet, so I've just been playing with the gateway only.

But you can definably point the server to your own box and emulate.   Likely there is a UDP query you can get the stats too, but I haven't found that yet.

-M

https://dl.dropboxusercontent.com/u/12351460/Lacross_UDP_Protocol.txt
https://dl.dropboxusercontent.com/u/12351460/lax_send.php

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 14, 2014, 05:59:54 PM
I noticed that my TCP/IP capture/analysis program was crashing every 120 minutes.  It turns out that there is a short form version of the SDP that has a payload size of 30 bytes that gets sent every 2 hours - on the dot.  The short packet was causing my conversion routines to throw an exception on the missing data.  I assumed the "01:01" packet always had the same length binary payload.  I was wrong...  :???:

The data in the short-SDP has some invariant fields, but most of it changes in subsequent transmissions from the GW to the LCX server.  Interesting... Could this be the only Sensor Data packet that gets sent if you are on the Basic Service plan?

This packet needs to be analyzed by someone and have the fields identified for later parsing.  Later, I will add parsing rules to my app for this packet.

The HTML header for this short packet has the same packet type identifier as the SDP:

HTTP_IDENTIFY: 8009A417:01:1ACD276DFAC43232:01\r\n 

But instead of a Content-Length of 197, it has a length of 30.

I also found out that the 64-bit hex number found above changes during the registration process, so it is some sort of server-issued key Probably used to verify that you have completed the registration process - or to tie your sensor data to your user account. 

The GW does some crazy things with DHCP during the registration process too. It requests new IP addresses from your DHCP server, hopping from from IP to IP 3 times (for no apparent reason).  It should be happy with the first one.  Maybe a firmware bug?

There were several new HTTP_IDENTIFY packet ID codes that are only used during registration.  I'll smoosh the the leading/trailing hex together, since everyone's 64-bit ID number (stuffed between the two numbers) will be different:

0010 - some sort of registration preamble
0020 - actual registration request
0030 - registration complete/finalize?
7F10 - 13 bytes sent - mysterious stuff
0114 - 14 bytes sent - more mysterious stuff
0001 - 210 bytes sent - even more mysteries


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 16, 2014, 04:22:02 AM
Just an update

I tired all packet types in the UDP protocol from 00 00 to ff ff and they only ones that return anything are the ones I've previously documented.

There may be other formats, but for now this is all I know.  It might change once I register my gateway.

-M
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 16, 2014, 10:17:27 AM
If you haven't registered already, how about capturing the GW TCP/IP conversation during registration. Wireshark is a good tool, if you are not already using it.  I have only one sample right now, and it would be interesting to compare more. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 16, 2014, 05:10:50 PM
If you haven't registered already, how about capturing the GW TCP/IP conversation during registration. Wireshark is a good tool, if you are not already using it.  I have only one sample right now, and it would be interesting to compare more. 

I sent you a PM.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 16, 2014, 05:33:26 PM
I sent you a PM.

I see no PM  :-(
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on January 16, 2014, 05:35:23 PM
I resent it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 17, 2014, 05:11:17 AM

I registered my device.

This is what it looks like to me, taking skydvrz HTTP_IDENTIFY as an example:

HTTP_IDENTIFY: 8009A417:01:1ACD276DFAC43232:01\r\n 
                         ^^     ^^ ^^        ^^                    ^^
                          A       B    C          D                       E

Each packet types seems to consist of

A=80, always 80 so far.
B= MAC address less vendor ID
C= Packet Code 1
D=RegistrationCode or Device Serial Number or other Identifier
E= Packet Code 2

AB never change
D changes on registration, maybe this gets read from wetherstation, or is generated in reg process, probably easy to check with a fake reg.
C and E form the packet type identifier,  This packet identifier is Acknowledged by the server's reply in the HTTP reply header in the HTTP_FLAGS Field.  Example

Packet request:
HTTP_IDENTIFY: 8009A417:00:1ACD276DFAC43232:20

Would get replied:
HTTP_FLAGS: 20:00    <-----------I'm changing my mind on this, not always the case.

This is outside of any other Post or Reply data.


Packets that I have Identified (just showing C:D) :

00:10 --> power up packet for unregistered device.
00:20 --> button Push Packet, note that this packet may or may not have data in the reply based on if your in registration on the website or not.
00:30 --> Ping?
00:70 --> Ping with Data?  Or Poll Data,  After Registration this is the power up packet.
7F:10 -->
01:14 --> Pushes data
01:01 --> Likely Data Packet Push as explained by Skydivr  (seen HTTP_FLAGS reply 00:00 here)
01:00 --> Push 5 bytes with data reply


That's it so far, I have not analyzed the data in the packets yet.


-M







Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 17, 2014, 10:39:10 AM
My packet sniffer app is working well (running 48 hours or so without a glitch), and my monitor program is just about ready for release - probably this 3-day weekend. Note that it uses MySQL database server to save packet and sensor readings.  I can temporarily provide an open MySQL server for a few beta testers. 

MySQL is free/open source.  I can provide a file needed to create the schema/table structures I used if you want to set up your own private database server (recommended).  Don't let "server" scare you.  MySQL is just a program that runs on your computer.  You can put it on the same PC as your other programs or run it on a spare computer somewhere on your home network.

Screen shots:

Current sensor reading in realtime.  I figured out most of the date/time fields in the SDP packet:
https://www.dropbox.com/s/j6twypw2pnzzp8x/Current%20Readings.png (https://www.dropbox.com/s/j6twypw2pnzzp8x/Current%20Readings.png)

Historical readings mode:
https://www.dropbox.com/s/1kzxud7jf5ujwt3/historical%20readings.png (https://www.dropbox.com/s/1kzxud7jf5ujwt3/historical%20readings.png)

Packet debug/analysis mode:
https://www.dropbox.com/s/58uos81e3gmjyfj/Packet%20Debug.png (https://www.dropbox.com/s/58uos81e3gmjyfj/Packet%20Debug.png)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 17, 2014, 01:06:36 PM
Sky,

So I was thinking with your software that maybe we can make it simple without having to sniff or use a ethernet hub.

2 possible ways:

1) One is build the interface into your software to be a proxy, this way we can tell the gateway that your software is the proxy, you add the socks proxy bits, but you get all the packets from the gateway, and forward them on.

2) Might be simple, just act as a router and point the weather gateway config to your software.   So instead of your router, the packets get sent to your software, then you just forward them on to your gateway.  (you'd have to handle reply packets too.)

Your software should be able to automatically configure the gateway for this using the UDP config protocol.

This way there is no special hardware needed, just run your software and it works.

-M






Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 17, 2014, 02:20:40 PM
1) One is build the interface into your software to be a proxy, this way we can tell the gateway that your software is the proxy, you add the socks proxy bits, but you get all the packets from the gateway, and forward them on.

Possibly.

Quote
2) Might be simple, just act as a router and point the weather gateway config to your software.   So instead of your router, the packets get sent to your software, then you just forward them on to your gateway.  (you'd have to handle reply packets too.)

Won't work, because the GW wants to carry on a specific conversational "dance" with the LCX Cloud server.  I'd have to simulate the server in order to fool the GW into submitting data.  My guess is the GW/Server designers already thought of that and made it a bit complicated to protect revenue.  Remember that they do charge you money for accessing that server....  Not that it could not be done. 

There is way too much mysterious data sent back and forth.  Much of that would have to be decoded and simulated to fool the GW. 

We may be screwed if there is some sort of challenge-reply code: The GW sends a random string to the server, the serve encrypts the data in a very specific way and sends it back to the GW.  The GW decrypts it and if it does not equal what it just sent then it quits.  This is something I would do in LCX's place to prevent guys like me doing what I am doing now.  :grin:

An actual server simulator would not be a simple proxy, but an entire Web Server simulator having the same business rules as the actual one.  Since the LCX server hostname is hard coded into the GW, then we'd also need a fake DNS server (easy) to tell the GW to send its packets to the server simulator instead of the actual LCX Cloud server.



The way it works now is I simply listen in on the conversation between the GW and LCX server.  Since it is non-invasive, the GW is not interrupted, has a nice warm server to talk to and thinks everything is Hunky Dorey.   Much simpler than simulating an entire server.

Quote
Your software should be able to automatically configure the gateway for this using the UDP config protocol.

Possibly, but why reinvent the wheel?  It is easier to use LCX's own GAS utility.  You can use it to change the GW DNS address to a fake DNS server located on your home network. When GW "dials up" the fake DNS and asks for the IP of box.weatherdirect.com, it always replies with an IP on your LAN - the location of the server simulator.  That would only work with a full-blown server simulator though.

Maybe if you could simulate the entire registration process.  Then you would not need UDP at all.  I think I saw the LCX server configuring the host name for the GW to use.  That was over TCP/IP.

To my knowledge, UDP is for use by the GAS utility only.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 17, 2014, 03:54:40 PM

Quote
2) Might be simple, just act as a router and point the weather gateway config to your software.   So instead of your router, the packets get sent to your software, then you just forward them on to your gateway.  (you'd have to handle reply packets too.)

Won't work, because the GW wants to carry on a specific conversational "dance" with the LCX Cloud server.  I'd have to simulate the server in order to fool the GW into submitting data.  My guess is the GW/Server designers already thought of that and made it a bit complicated to protect revenue.  Remember that they do charge you money for accessing that server....  Not that it could not be done. 


Myabe I'm not being clear, I relooked at option 2 and I think it is only a few more lines of code than what you have now. In option 2 your basically doing what your doing now except you don't need a hub.

All you have to do is set the "router" gateway to your software host IP address, All packets sent by the weather gateway will go to your host.  All you have to do is change the destination MAC of the packet you receive from the gateway to your "internet router" address and send the packet.   Thats it.  You don't change any of the data.  The replies will bypass you and go directly to the weather gateway from your router.

Like I said this is just an idea to toss around.















Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 17, 2014, 05:04:59 PM
Ok, I guess I misunderstood.  I am working on a couple major software development projects right now (not related to SkySpy) and mental multitasking isn't working for me this morning :-)

Can you tweak the outbound MAC address without messing with the NIC settings? 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on January 19, 2014, 01:57:00 PM
Ok, I guess I misunderstood.  I am working on a couple major software development projects right now (not related to SkySpy) and mental multitasking isn't working for me this morning :-)

Can you tweak the outbound MAC address without messing with the NIC settings? 

Well I guess I am assuming a lot.   I am assuming your using something like pcap/winpcap to sniff the packets, if you are then it is trivial.   You can send any MAC address you want.  You don't even have to recalculate any checksums since it is link level and doesn't even touch the IP layer.

I don't know what language your using either, but it shouldn't matter.

-M
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: slim83301 on January 29, 2014, 01:12:32 PM
I just wanted to say thanks for all that you guys have done so far!!!

I work in a school district as their tech for computers and we have a teacher that received one of these PWS from her parents for Christmas. She would like to get it out to a site so that students can do some predictions and experiments which are weather related.

I have access to most of the hardware you are using but the software is my challenge.  :? This has been a fun read! I guess I need to go back to school! :lol:

Don't stop now!

Again thanks!

Will
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 29, 2014, 01:23:30 PM
Will,

I have a system up and running that can capture all of the sensor readings, display them and export to Excel.  I have it running at two sites right now.

There are a couple issues to iron out, but I should be able to publish something soon.

As for displaying the data on a web site - that is a while off.  Right now you need a PC and a program that connects to my database of weather samples.

Here is a screen shot of the latest version of the SkySpy monitor program, displaying weather readings in real time:

(https://www.dropbox.com/s/qc0j7n8ub7e6lkt/ssMonitorNewHist.png)

https://www.dropbox.com/s/qc0j7n8ub7e6lkt/ssMonitorNewHist.png (https://www.dropbox.com/s/qc0j7n8ub7e6lkt/ssMonitorNewHist.png)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on January 29, 2014, 09:22:48 PM
How is the CPU usage when your sniffer is running?  It sounds like all the network traffic going out the router needs to be inspected by the sniffer program in order to find the packets we care about (going to the LaCrosse server).  There are other devices on my home network such as a Tivo and Apple TV and I'm  and I'm wondering if checking all the traffic would load down the CPU.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 29, 2014, 09:26:29 PM
Data packets and other network chitchat from the Gateway happens every couple minutes.  This is not a big burden for most PCs  :lol:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on February 02, 2014, 06:12:02 PM
Did you ever force the gateway?

Sorry I haven't been around, I deployed the weather station, but I will be getting another next week for this location, then I can help test/debug more.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 02, 2014, 07:13:14 PM
Mycal,

I am mostly working on getting the SkySpy Capture and SkySpy Monitor working 100%.  I have it running at my home and another beta site, with over 45000 packet samples collected.  A server simulator may come later, but it will be a ways off.

Today I am adding graphs to the monitor program.

You are welcome to join us when you get your hardware situated.  At this point in development, you either need an "old-school" hub or a dual-Ethernet port PC.  You might be able to get it to work with a WiFi laptop that also has an Ethernet port - where both can be run at the same time. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on February 03, 2014, 01:12:17 AM

Yes I have a 10/100 hub and a access point sniffer setup in my lab.  I like the old 10/100 hubs with 2 collision domains rather than a 10mbps hub as they are not usable for general use.

I'm well versed in wireshark, routing and pcap so I can help you do the gateway trick, so you don't have to listen to the whole network or use a hub too.

-M

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 03, 2014, 10:19:49 AM
Quote
Yes I have a 10/100 hub

Great!  ssCapture uses WinPCap to do its magic, so you are all set up.  You can run ssMonitor on any PC with Internet access.

I will send you a SkySpy download link and login credentials via PM tonight.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 05, 2014, 09:51:15 PM
Here is the latest SkySpy release:

https://www.dropbox.com/s/vkhkxmrgnbolarf/SkySpyInstall.exe (https://www.dropbox.com/s/vkhkxmrgnbolarf/SkySpyInstall.exe)

This one includes graphs of your data.

Contact me via PM or Email to request database server login credentials.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 09, 2014, 11:21:58 AM
I got the Gateway to work through a Proxy/Sniffer (without a hub).  Unfortunately, I can't easily integrate the proxy software I used into ssCapture.  The proxy components are too old for my development environment and it would take weeks to rewrite them.

If anyone wants to follow in my footsteps using an HTTP proxy/sniffer then make sure you click the Reboot button on Gateway Advanced Setup utility.  Proxy settings do not "take" until you reboot - that drove me nuts for about an hour...

The good news is that the full HTTP and binary stream can be viewed in real-time using the output of the proxy server, and the Gateway maintains its connection to the LCX Cloud Server.  This would make a lower cost alternative to using an old-school hub, but you still need to parse the binary data somehow.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 09, 2014, 09:15:32 PM
Interesting... The so called "proxy" feature in the GW simply redirects its normal HTTP traffic to the designated "proxy server" IP on whatever port you select.  It does not use SOCKS or do anything special.  The gateway continues to use HTTP PUT, not CONNECT.

I did get a proxy working that acts as an intermediary with the LCX Cloud Server, but I now think I will just abandon that in favor of writing a cloud server clone. 

It may be possible to use the clone as a personal web server, as well as a data bridge for the Gateway.  I think it could display real-time weather sensor readings on a mini-web site, store your weather data to a database, possibly send warning emails, and post updates to Wunderground.

This is getting interesting...

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on February 10, 2014, 12:58:40 PM
That would be awesome if you wrote a cloud server!  Right now I have a perl program that goes to the LaCrosse website and scrapes my data.  Which would be fine if the data upload was reliable.  However I see gaps on a daily basis of several hours between updates on their site. A cloud server hosted on my PC would be much more reliable, or so I hope.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 10, 2014, 01:34:11 PM
Yes, I believe it is quite possible to combine the gateway "data getter" and a personal web site server.  You would be able to store/view sensor samples with a granularity of ~4 minutes.  Using a dynamic DNS service, you should be able to access your personal web server running on your home PC from remote locations or your smartphone.

I started designing the server yesterday - I have all the necessary libraries to complete it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on February 10, 2014, 01:50:38 PM
With the replacement cloud server will we still need the hub?  If I've understood correctly the GW will be told to use a proxy (not sure how that is done) and the "proxy" will be the replacement cloud server.  So no need to sniff packets. Also no communication with the LaCrosse webserver but given how spotty that communication is turning out to be that seems to be no loss.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 10, 2014, 02:04:23 PM
With the replacement cloud server will we still need the hub? 

Nope.  Just use the GAS utility to set the "proxy" IP address to a PC on your LAN that is running the replacement "cloud server" (I hate that name BTW :-) )

Quote
If I've understood correctly the GW will be told to use a proxy (not sure how that is done) and the "proxy" will be the replacement cloud server. 

Exactly.  The replacement server will mimic the LCX server, so there is no need to sniff packets or have a valid LCX connection. 

You can download the Gateway Advanced Setup utility from the WeatherDirect web site.  You can configure the Gateway's so-called proxy setting by typing in a valid IP into one of the fields.  You click Set and then click the Reboot button. 

Earlier in this thread I though we might have to play DNS games to trick the gateway module, but LCX made it way too easy.  The proxy setting also allows you to use an alternate TCP/IP port, so you can avoid conflicts with existing web servers on the same PC.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on February 10, 2014, 03:00:49 PM
Here's a link to the software.

http://www.lacrossealerts.com/support/gatewayhelp-staticip

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on February 10, 2014, 03:32:08 PM
With the replacement cloud server will we still need the hub? 

Nope.  Just use the GAS utility to set the "proxy" IP address to a PC on your LAN that is running the replacement "cloud server" (I hate that name BTW :-) )

Quote

Have you considered using the GAS utility and just set the Gatway to your software device?  In this case you will get all the packets bound for the service but on the link level they would be sent to you.   You could then just sniff the packets, replace the destination MAC address to your real gateway (your router) and send it on.   This way you see everything, and you don;t have to emulate a proxy.  The replies would bypass your device and go straight to the gateway.  All you need is the raw send function of winpcap to send the packets.

I will be back in my lab tomorrow, so I can look at this.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mikemac99 on February 10, 2014, 03:41:29 PM
It may be possible to use the clone as a personal web server, as well as a data bridge for the Gateway.  I think it could display real-time weather sensor readings on a mini-web site, store your weather data to a database, possibly send warning emails, and post updates to Wunderground.
This is an cool set of features!  I hope that you make it selectable which ones a user enables.  For example I don't want to set up a mini-web site since I'd be happy getting the data from the Wunderground.  And I try to minimize the number of ports I open on my PC.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 10, 2014, 03:53:19 PM
In my experiments, I found that pretty much any old proxy software can work as a middleman between the gateway and the LCX Cloud Server.  There are some headaches associated with that. The proxy server components I tried typically strip off the HTML header and output only the "document" itself.  I need the header, because it contains a lot of information needed to classify and parse the binary "document" - the sensor data. 

I think it is time to move beyond packet sniffing...

I think the consensus is that most people want an easy to install replacement server.  I also think the technical expertise required to set up a wired LAN and old school hub are a bit beyond the skill-set of the average backyard weatherman  :grin:

So, given those two design requirements, the best way forward is to create a complete web server that talks to the gateway as well as local and remote browsers.  Luckily, this is not that hard. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 10, 2014, 04:07:08 PM
This is an cool set of features!  I hope that you make it selectable which ones a user enables.  For example I don't want to set up a mini-web site since I'd be happy getting the data from the Wunderground.  And I try to minimize the number of ports I open on my PC.

Your PC will have to have at least one open port, so that the gateway can talk to it.  If you don't open up a port on your router, then you can only look at your personal web site from inside your LAN.  The bad guys can't attack your PC.

You typically use the "port forwarding" feature of your firewall/router to run a public server on your LAN. Running a personal server like the one I have in mind should be pretty secure.  The server will be a "one trick pony" that can only display one web page and maybe some graphs.  About the worst an outsider could do is a DOS attack on your server, and that would be pointless and boring for the attacker.

The server will do Wunderground auto-updates at some point.

For the lurkers:

If you do open a port on your router, then you can look at your private web site from outside your home network.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on February 11, 2014, 10:57:50 AM
Just want to add my word of thank for all the time you guys have put into this.

I have the C84612 and have been very disappointed in the LCX Cloud server and it lag in updating. You guys are giving me hope for something better. This has been a very interesting thread. I also have some of the older Weather Direct TX-60u-it that use the same gateway and am wondering if anyone has worked with them.

Thanks again keep up the good work
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 11, 2014, 11:18:55 AM
Here are some screen captures of the SkySpy graphs - a feature I added recently to ssMonitor.

https://www.dropbox.com/s/3hwd1cus348mjcb/temp.png (https://www.dropbox.com/s/3hwd1cus348mjcb/temp.png)

https://www.dropbox.com/s/ggi9pg3i9e49yzt/humidity.png (https://www.dropbox.com/s/ggi9pg3i9e49yzt/humidity.png)

https://www.dropbox.com/s/f5o7urwku9204b1/bp.png (https://www.dropbox.com/s/f5o7urwku9204b1/bp.png)

https://www.dropbox.com/s/c3nnknyp5erph8t/wind.png (https://www.dropbox.com/s/c3nnknyp5erph8t/wind.png)

https://www.dropbox.com/s/fgsr0kmazwdz8g2/rain.png (https://www.dropbox.com/s/fgsr0kmazwdz8g2/rain.png)

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 12, 2014, 12:03:32 AM
I made a little bit of headway tonight.  My server app can successfully respond to a Gateway ping packet.  That makes the Gateway happy enough to offer either a Sensor Data Packet (SDP) or a clock time reset request.  I hope to be able to parse the SDP with the new server software and save it to the database - probably tomorrow.

Hey guys - I could really use some more data feeds.  Those of you that can hook up the necessary hub and start sending me data using SkySpy can be a really big help.  I need more samples to make this thing work for other people besides me.  I am worried that some of the mysterious data fields the Gateway sends out are critical for operation.  I feel confident that I can get something working for my particular Gateway, but I am not so sure about *yours*.

Example:  There is some sort of Epoch field that increments every few minutes in steps of 12, until it reaches a magic value and then all sorts of data changes.  This is at offset 0x16 and 0x17 in the 01:xxxxxx:00 packet the LCX Cloud server sends back to the Gateway.  I am afraid that if my server responds in a way the Gateway does not expect, it may stop working. It may be nothing, but more data from other people would be a big help!

I have sent free log in credentials to a number of people.  Please consider running SkySpy at this critical time!

 



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: maraford on February 13, 2014, 02:28:39 PM
I joined the forum so I could reply to skydvrz.
First, thank you for all your work in helping to make the c84612 more useful. A few days ago, I found a method for stripping the data from the Lacrosse site and uploading it to Weather Underground using a php script. The link is http://www.ilasoft.net/professional-weather-station-100/ (http://www.ilasoft.net/professional-weather-station-100/). The script is here: http://www.ilasoft.net/wunderground-pws-la-crosse/ (http://www.ilasoft.net/wunderground-pws-la-crosse/). The author is Ilya Pupko who helped me get it to work  using a WAMPServer on a Windows 7 machine (I'm a novice in anything more involved then simple html).
Your approach appears far more sophisticated and complete. I post the above in case Ilya's method has any value (especially in the data conversion challenge).
Look forward to your posts. -- Ford
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 13, 2014, 03:22:15 PM
Maraford,

Scraping or spidering a web site is cheating  :grin:

Some other problems with using that technique are:


There really isn't much comparison between the scraping data from the LCX web page and what must be done to glean sensor data from the raw data packets sent to the server.  That said, we have figured out almost all of the raw sensor data and designed ways to convert it into a readable format. 

The current SkySpy software requires a valid LCX server connection but it does not depend on anything the LCX server displays on the web page.  It listens in on the server conversation and gathers sensor information before it ever gets to the server.

The part that remains is making a portable system that people can use to host their own "mini server" without the need for an LCX server account/connection. I am also getting rid of the requirement for any special hardware.  That is the state of the project at this point. 

I am making good progress, but need more data samples from folks like you.  Pitch in and help  :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 14, 2014, 04:31:48 PM
There will be a brief outage of the SkySpy data server today at around 16:30 Pacific.  SkySpy Capture and Monitor should keep trying to reconnect and start logging again after the outage is over.  If you are unlucky, you may have to log back in.  You don't have to shut down SkySpy - Just keep an eye on it.

I need to update the software on my database server - It should take 20-30 minutes max.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 15, 2014, 08:39:36 PM
Life got in the way of my database server software update yesterday.  A component library I updated last night broke my compiler so I spent most of yesterday and part of today fixing it.  I'll do the MySQL server update later tonight.

Some good news...  My SkySpy Server is coming along nicely.  It logged an hour or two of valid SDP packets today, without using a hub or help from the LCX cloud server.  \:D/  Woo hoo!!!!

The only problem so far is the LCD display loses its "INTERNET" connection indicator.  Something my server is doing gives the Gateway indigestion.  Fortunately, the Gateway continues to send all the usual chitchat packets to my server.  I will let my SkySpy server run over night and see if the Gateway gives up after too long of a "no internet" failure.  I am hopeful.

If anyone can figure out what the exact handshake is to turn the Internet indicator back on, let me know.  It may have something to do with the 5-byte packet the GW sends to the LCX server.  So far, I have seen 14 possible payload values to that packet.  Each Gateway sends out a different numbers, but always from the same group of 14.  My guess is that the server embeds a reply based on the last one the Gateway sent in one of its responses back to the Gateway.  If this response is correct, then the Gateway tells the LCD that everything is hunky dory; If not, then the LCD shuts off the Internet indicator.  What that response is and where it is hidden, I have no idea.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 16, 2014, 07:17:25 PM
My server clone can now display the last 8 hours of weather sensor readings in a web browser while simultaneously accepting sensor readings from the Gateway.  Here is a screen shot of the quick and dirty web page it generates:

https://www.dropbox.com/s/wx1hyu9g0d6zsgv/Captured001.png (https://www.dropbox.com/s/wx1hyu9g0d6zsgv/Captured001.png)

You can still look at graphs and grids using ssMonitor.

My server clone seems to work OK as a complete replacement for the LCX Cloud server.  However... It may only work for me, since I mimic the packets the LCX Cloud server sends to my gateway (not yours).

I have had my server clone up for about 14 hours and it has not skipped a beat, so I think I am on the right track.

I will run some more tests to see how loose the LCX Cloud server <-> Gateway conversation can get, before the Gateway stops sending data.  By loose, I mean conforming to the expected secret handshakes and magic packets they trade.  Everybody's data packets are somewhat different, based on your registration number.  I am hoping to come up with a one-size-fits-all server.  Time will tell.  You may have to train the server clone first, before switching from the LCX Cloud server to this one.  I am not sure yet.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: trellend on February 25, 2014, 03:28:44 PM
http://www.ti.com/tool/cc1111emk868-915

USB Dongle, packet captures 915 Mghz.   

Analyze on computer, use this page for protocols: www.f6fbb.org/domo/

Little bit of coding and you wouldn't need the base station or gateway.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 25, 2014, 05:00:30 PM
Trellend,

The C84612 weather station ships with an included Internet Gateway containing a built-in RF interface.  There is no need to listen in on the sensor RF signals.

My latest software works with the hardware that ships with the weather station - you don't need to buy anything else.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: trellend on February 25, 2014, 05:34:41 PM
I have some 915 mghz junk here.  I could rig up it up on the scope and see if it picks up a signal.  Might be a nice alternate route as I don't have a dumb hub, my router won't fake DNS, and port 80 is occupied on my machine. 

I have a PIC32MX795 here with USB and an antenna.  Firmware for a HID device (for the chip) and Delphi code for HID USB devices.  I've done that before obviously.  I don't have to buy anything in reality.  I might try it as a fun way to gather the data.

I already have 3 of these: https://www.sparkfun.com/products/retired/8971   (similar) so just a few lines for receiver to power/data/ground and i'm set.  If I commit to doing this, I'll start a new thread.

I figured I'd just buffer all the incoming noise from each device and let the computer sort it out on a request basis.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on February 25, 2014, 05:46:48 PM
Trellend,

The C84612 weather station ships with an included Internet Gateway containing a built-in RF interface.  There is no need to listen in on the sensor RF signals.

My latest software works with the hardware that ships with the weather station - you don't need to buy anything else.

I am currently testing it and it runs great.  No need to buy anything and you don't need to use port 80.
 =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 25, 2014, 06:00:12 PM
Might be a nice alternate route as I don't have a dumb hub,

You won't need that in the next release.  I found a way to redirect the Gateway into talking to your PC without additional hardware or WinPCAP.

Quote
my router won't fake DNS,

You don't need to do that either.  You change the settings on the Gateway with the Lacrosse GAS utility (free).  It takes about 15 seconds.

Quote
and port 80 is occupied on my machine. 

Mine is too.  My ssServer software currently runs on port 8000.  A future version will run on any port you like.

You can use your router to port forward as you like, if you want to make your mini-server visible on the Internet.  You'd probably need a DynDNS account unless you have a static public IP address.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: trellend on February 26, 2014, 04:21:54 PM
Perhaps when you make the kind that skips LaCross all together, you could do it on a USB NIC direct to gateway.   Something like that.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 26, 2014, 04:31:13 PM
Perhaps when you make the kind that skips LaCross all together, you could do it on a USB NIC direct to gateway.   Something like that.

I already did that.  ssServer uses whatever NIC or WiFi connection you use on your server PC to connect to the Internet and the other devices on your LAN.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: trellend on February 26, 2014, 04:35:11 PM
Alright, I'll pick up another NIC next trip to civilization, I'll be ready to test in a 2-3 weeks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 26, 2014, 04:37:29 PM
Alright, I'll pick up another NIC next trip to civilization, I'll be ready to test in a 2-3 weeks.

???!!! You don't have an existing NIC or WiFi connection on your PC?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 28, 2014, 12:01:41 AM
I got the SkySpy Server talking to Wunderground tonight!   \:D/  I can't fully test it, since I don't have an account with them yet, but the Wunderground server does complain that I have an invalid station ID or password - I have both  :lol:

The next release will allow you to enter your station ID and password.  SkySpy Server will then update Wunderground at the interval you specify - around 2 minutes minimum, even if you have a rapid fire account.  That is the fastest this weather station can go :-)

Those of you on the SkySpy Server beta program should update when I make the new release - probably tomorrow night, west coast time.

https://www.dropbox.com/s/35f3wbf57pam7ri/wundergroundsetup.png (https://www.dropbox.com/s/35f3wbf57pam7ri/wundergroundsetup.png)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Cayuse on February 28, 2014, 12:57:06 AM
I got the SkySpy Server talking to Wunderground tonight!   \:D/  I can't fully test it, since I don't have an account with them yet, but the Wunderground server does complain that I have an invalid station ID or password - I have both  :lol:

The next release will allow you to enter your station ID and password.  SkySpy Server will then update Wunderground at the interval you specify - around 2 minutes minimum, even if you have a rapid fire account.  That is the fastest this weather station can go :-)

Those of you on the SkySpy Server beta program should update when I make the new release - probably tomorrow night, west coast time.

https://www.dropbox.com/s/35f3wbf57pam7ri/wundergroundsetup.png (https://www.dropbox.com/s/35f3wbf57pam7ri/wundergroundsetup.png)

Very cool!  Looking forward to the update.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 28, 2014, 02:32:40 PM
I opened a new Wunderground account this morning.  I then configured SkySpy Server to send data to the Wunderground PWS server.

Success!

Wunderground accepted my actual sensor readings!

https://www.dropbox.com/s/ol5gvsssr20hay6/WundergroundSuccess.png (https://www.dropbox.com/s/ol5gvsssr20hay6/WundergroundSuccess.png)

I have one minor tweak to do to the formatting of the sensor readings, but things are looking very good for a new release tonight.  =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on February 28, 2014, 02:34:11 PM
I opened a new Wunderground account this morning.  I then configured SkySpy Server to send data to the Wunderground PWS server.

Success!

Wunderground accepted my actual sensor readings!

https://www.dropbox.com/s/ol5gvsssr20hay6/WundergroundSuccess.png (https://www.dropbox.com/s/ol5gvsssr20hay6/WundergroundSuccess.png)

I have one minor tweak to do to the formatting of the sensor readings, but things are looking very good for a new release tonight.  =D&gt;

You the man.

Cannot wait!

Tee
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 28, 2014, 02:44:16 PM
Wow!  That was quick.  My station shows up on the Wunderground map already with my current readings.  The site says it takes 24 hours.

Oops.  I think I have an incorrect dew point calculation.  I'll fix that tonight too.

Edit: 2 Mar 2014:

Dew point is now fixed.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on February 28, 2014, 09:38:39 PM
Those of you running the beta SkySpy Server version 1.0.0.10 can update to the latest release by restarting your ssServer software.  After log in, a dialog box should appear that lists the new features (dew point and Wunderground updates).  Click OK to upgrade to the latest version.  ssServer will disappear for a few seconds and then pop up the log in form.  You should see a few more fields on the main form.

(http://idk.serveftp.net/photos/ssServerV1_1_0_7.jpg)

See the readme file in the installation directory for instructions on how to configure ssServer to update Wunderground.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Cayuse on February 28, 2014, 09:58:27 PM
Woohoo!  Simple, effective and I'm up and running already, first update showed in wunderground moments after the install finished.  So much better than the lacrosse solution.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 01, 2014, 12:19:51 AM
Woohoo!  Simple, effective and I'm up and running already, first update showed in wunderground moments after the install finished.  So much better than the lacrosse solution.

Thanks for the kind words.  I've been working on this for quite a while - since just before Christmas in almost every free moment until now.  I hope it makes people happy.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: trellend on March 01, 2014, 11:38:35 AM
Ever think about allowing read only access to your remote database?  Then we can pull some reports too.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on March 01, 2014, 03:02:46 PM
Woohoo!  Simple, effective and I'm up and running already, first update showed in wunderground moments after the install finished.  So much better than the lacrosse solution.

Thanks for the kind words.  I've been working on this for quite a while - since just before Christmas in almost every free moment until now.  I hope it makes people happy.
All your hard work is greatly appreciated and yes you do make us happy. I have been running ssCapture and ssMonitor for over two weeks and find it very useful. =D&gt; I am looking to the release of your ssServer software and can't wait to try it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 01, 2014, 04:08:04 PM
Ever think about allowing read only access to your remote database?  Then we can pull some reports too.

You already have that feature in ssMonitor.  Open the Historical tab, filter the readings by whatever criteria you like, reorder the columns to taste and then right click on the grid.  A popup menu will appear.

Select Excel export and take it from there.  The spread sheet will look just like your grid and contents.

It can also export in PDF format and print to your printer.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 01, 2014, 04:24:30 PM
All your hard work is greatly appreciated and yes you do make us happy. I have been running ssCapture and ssMonitor for over two weeks and find it very useful. =D&gt; I am looking to the release of your ssServer software and can't wait to try it.

I'm glad you like it!  I am holding off on doing a wide release of the server until I can settle on a feature set.

I need to get a consensus:

1) I am considering removing the database completely and have ssServer submit it's samples to Weather Underground (WU) exclusively.  I might remove the mini-website feature too.  Your thoughts?

You would have to download a CSV file from WU if you wanted to run special reports.

2) I want to turn ssServer into a Windows Service that runs invisibly in the background on your computer.  I will probably add a companion program used to configure ssServer and maybe display the last sensor readings. The companion program can be run continuously or only as needed.  The service would run just fine without it.

A service would survive being rebooted when Microsoft automatically updates Windows in the wee hours of the morning - you would not lose more than a couple minutes of sensor data on WU, even if you got logged out of your windows session. It would also continue running if you log out, or someone else logs in to your computer.

I think the transition from desktop app to Windows Service is important, so that is tops on my list.  I really want to wrap this thing up quickly, so that is my motivation for paring down the feature list to the minimum everyone wants and can actually use.

Feedback is appreciated!

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: redsbeach on March 01, 2014, 05:30:28 PM
First and foremost, thanks for all the effort you've put into this. (http://forum.daviddarts.com/mods/smileys/images/smiley15.gif)
I've been looking for something exactly like this to replace the unreliable Alerts website.

As far as preferences are concerned, I run my own servers and would really like the ability to keep the database local.
This also increases the flexibility for future use.
Including the mini website and the ability to also upload to WU is fantastic.

So I'd vote no change to these existing features that you've currently included.
Running the server as a service with an associated configuration app is a great idea!

Again, thanks for all the time you've put into this.


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 01, 2014, 05:55:12 PM
As far as preferences are concerned, I run my own servers and would really like the ability to keep the database local.
This also increases the flexibility for future use.
Including the mini website and the ability to also upload to WU is fantastic.

So I'd vote no change to these existing features that you've currently included.
Running the server as a service with an associated configuration app is a great idea!

Ok.  Lets get some more votes :-)

I don't want to host the DB forever - too much responsibility  :-|

The current product is MySQL-based.  For speed and efficiency, SkySpy uses a native MySQL TCP/IP interface (no DLLs or ODBC).

I am not too motivated to change DB engines, since that is the database I program with at my day job and other consulting work. I know it inside out and backwards and am not too keen on learning a new one. The good news is that the community version of MySQL is open-source/free.

The trouble with a local/remote DB is you have to log in to use it.  I suppose I could save login credentials in the registry, but that is not too secure.  You would need some sort of automatic login, in a service environment.

You can use your own DB right now - just change the host name / port # using the Advanced button on the Login form.  Note that you would have to create and populate a bunch of DB tables before use.  I have not published the DB schemas and tables, but I could if there is any interest in private / local MySQL servers.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 01, 2014, 06:04:29 PM
I also want to thank you for all the hard work that you have put into this.  You've far exceeded my expectations.  I know you have others duties that keep you busy, so I am REALLY appreciative for everything you've done thus far. 


I'm glad you like it!  I am holding off on doing a wide release of the server until I can settle on a feature set.

I need to get a consensus:

1) I am considering removing the database completely and have ssServer submit it's samples to Weather Underground (WU) exclusively.  I might remove the mini-website feature too.  Your thoughts?

You would have to download a CSV file from WU if you wanted to run special reports.


I would like the database local I think it would be a big benefit.  Setting up MySQL is not that difficult and it would allow us to be able to design custom reports or web pages.

I'm guessing, but I think moving the database to our computers would not be that difficult and would take the burden off your system.

I would not mind using my web server in lieu of your mini-server.


2) I want to turn ssServer into a Windows Service that runs invisibly in the background on your computer.  I will probably add a companion program used to configure ssServer and maybe display the last sensor readings. The companion program can be run continuously or only as needed.  The service would run just fine without it.

A service would survive being rebooted when Microsoft automatically updates Windows in the wee hours of the morning - you would not lose more than a couple minutes of sensor data on WU, even if you got logged out of your windows session. It would also continue running if you log out, or someone else logs in to your computer.

I think the transition from desktop app to Windows Service is important, so that is tops on my list.  I really want to wrap this thing up quickly, so that is my motivation for paring down the feature list to the minimum everyone wants and can actually use.

I do like the idea of running this as a windows service.  It would make it a lot easier.

Feedback is appreciated!

SKYDVR, thanks again for all that you have done for all of us so far.  Your code thus far, far exceeds the garbage that Lacrosse has on it's pay-to-view.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 01, 2014, 06:18:45 PM
I hope it makes people happy.

 :grin:  =D&gt;    :grin: =D&gt;    :grin: =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Cayuse on March 01, 2014, 06:30:33 PM
I think that having it run as a service would be of the most benefit. 

Whether it stores to a local DB or only uploads to WU doesn't really matter to me at this point although I can see some advantages to having the granularity available at the local level but for my use I'm not sure I want/need that.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: redsbeach on March 01, 2014, 07:02:31 PM
As far as preferences are concerned, I run my own servers and would really like the ability to keep the database local.
This also increases the flexibility for future use.
Including the mini website and the ability to also upload to WU is fantastic.

So I'd vote no change to these existing features that you've currently included.
Running the server as a service with an associated configuration app is a great idea!

Ok.  Lets get some more votes :-)

I don't want to host the DB forever - too much responsibility  :-|

The current product is MySQL-based.  For speed and efficiency, SkySpy uses a native MySQL TCP/IP interface (no DLLs or ODBC).

I am not too motivated to change DB engines, since that is the database I program with at my day job and other consulting work. I know it inside out and backwards and am not too keen on learning a new one. The good news is that the community version of MySQL is open-source/free.

The trouble with a local/remote DB is you have to log in to use it.  I suppose I could save login credentials in the registry, but that is not too secure.  You would need some sort of automatic login, in a service environment.

You can use your own DB right now - just change the host name / port # using the Advanced button on the Login form.  Note that you would have to create and populate a bunch of DB tables before use.  I have not published the DB schemas and tables, but I could if there is any interest in private / local MySQL servers.

Ooops.  Didn't mean that I was expecting you to host the MySQL DB.
I'm with 10ACTony in preferring to host the DB locally on my own server, as well as hosting a web page to display the output dynamically.
Sorry for the misunderstanding. 

Retaining the ability to upload to WU would accommodate those that don't want to maintain a local instance would be the best of both worlds.

Can the skyServer mySQL credentials be kept in an .ini or .conf to be passed at startup?  It's no more secure than the registry, but it might be less
work. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on March 01, 2014, 08:11:53 PM
My Vote is to have the database local as I would like the software to run independent of a internet connection. And also like having the mini-website feature. Having it run as service would be nice.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 01, 2014, 08:30:18 PM



Retaining the ability to upload to WU would accommodate those that don't want to maintain a local instance would be the best of both worlds.


Definitely keep the ability to upload to WU.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 02, 2014, 11:31:14 AM
So I'd vote no change to these existing features that you've currently included.
Running the server as a service with an associated configuration app is a great idea!

Noted.

Quote
I'm with 10ACTony in preferring to host the DB locally on my own server, as well as hosting a web page to display the output dynamically.

SkySpy needs some sort of DB, to save rainfall data.  WU needs 1-hour and 24-hour Inches of Rain readings.  I save the total accumulated rain inches samples to the DB every couple minutes and the figure out how much it changed in the last hour/day using a SQL query.

What I might do is add a setting that selects a simple 1-user database (just raw tables actually) to store sensor readings for the folks that don't want to install MySQL.  This arrangement would NOT allow reading the tables or data within them with PHP or some other method.

An advanced DB setting would use full-blown MySQL, which as many people have pointed out is relatively easy to  install and is free for the downloading.

Quote
Retaining the ability to upload to WU would accommodate those that don't want to maintain a local instance would be the best of both worlds.

WU updates are my primary goal.  But as I note above, there has to be some sort of sensor data storage - for at least 24 hours.

Quote
Can the skyServer mySQL credentials be kept in an .ini or .conf to be passed at startup?  It's no more secure than the registry, but it might be less
work.  In this mode, you could do what you like with the data.

Its six of one, a half dozen of another (Registry vs INI).  Reading/writing to the registry is pretty simple.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 03, 2014, 09:53:58 PM
I started working on the SkySpy Server-Service (that is a mouthful!) tonight.

The plan (although things may change):

1) The service runs in the background and does all the non-visual stuff that ssServer does now - GW comm, WU Update, Mini-web site, DB update.

2) A Small app will configure the service/show the latest sensor readings in real-time if you leave it running on your desktop.  This is optional - except for initial setup.  Might be good to let you know if your sensors are working OK and if communications are happening in a timely manner.

3) The database will work in one of two modes:

3a) The service will be able to use stand-alone database tables (no MySQL database install required).  This mode does not allow external programs to read the database tables:  Data is for the exclusive use of the service.

3b) The service will be able to use a local or remote MySQL database engine (but not mine).  Login credentials will be stored in the registry.  You can do what you like with the data in the tables.

4) All configuration information will be stored in the registry.  Update it with the small setup app.

5) Absolutely No Promises, but I am looking at a Cumulus interface.  If it turns out to be a pain, this will be the first thing to get tossed overboard.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 03, 2014, 10:17:59 PM
I started working on the SkySpy Server-Service (that is a mouthful!) tonight.

The plan (although things may change):

1) The service runs in the background and does all the non-visual stuff that ssServer does now - GW comm, WU Update, Mini-web site, DB update.

2) A Small app will configure the service/show the latest sensor readings in real-time if you leave it running on your desktop.  This is optional - except for initial setup.  Might be good to let you know if your sensors are working OK and if communications are happening in a timely manner.

3) The database will work in one of two modes:

3a) The service will be able to use stand-alone database tables (no MySQL database install required).  This mode does not allow external programs to read the database tables:  Data is for the exclusive use of the service.

3b) The service will be able to use a local or remote MySQL database engine (but not mine).  Login credentials will be stored in the registry.  You can do what you like with the data in the tables.

4) All configuration information will be stored in the registry.  Update it with the small setup app.

5) Absolutely No Promises, but I am looking at a Cumulus interface.  If it turns out to be a pain, this will be the first thing to get tossed overboard.

Sounds good.

If someone uses the local DB, will the monitor program still work if it's directed to the local DB?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on March 04, 2014, 02:28:49 AM
I started working on the SkySpy Server-Service (that is a mouthful!) tonight.

The plan (although things may change):

1) The service runs in the background and does all the non-visual stuff that ssServer does now - GW comm, WU Update, Mini-web site, DB update.


I just discovered this thread. Your program looks great, and I appreciate all the hard work that you've put into it. I have a new LCX station and was getting ready to connect it to their server, but really would like to keep the data local so I can access it whenever I want.

I would use your program, but I only use Linux. I would like to write a server like you've done, but using python in linux and have it write data to a local MySQL server. Then write some PHP scripts to view data and graphs with a web browser. I've done this sort of thing before in python and in C. Is it possible to get the source for your program, or perhaps the part that communicates with the gateway device and keeps it happily sending data.

I'm happy to help this project however I can, and of course make available any code I write.

Thanks a lot for your help.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: redsbeach on March 04, 2014, 02:06:11 PM
I started working on the SkySpy Server-Service (that is a mouthful!) tonight.

The plan (although things may change):

1) The service runs in the background and does all the non-visual stuff that ssServer does now - GW comm, WU Update, Mini-web site, DB update.

2) A Small app will configure the service/show the latest sensor readings in real-time if you leave it running on your desktop.  This is optional - except for initial setup.  Might be good to let you know if your sensors are working OK and if communications are happening in a timely manner.

3) The database will work in one of two modes:

3a) The service will be able to use stand-alone database tables (no MySQL database install required).  This mode does not allow external programs to read the database tables:  Data is for the exclusive use of the service.

3b) The service will be able to use a local or remote MySQL database engine (but not mine).  Login credentials will be stored in the registry.  You can do what you like with the data in the tables.

4) All configuration information will be stored in the registry.  Update it with the small setup app.

5) Absolutely No Promises, but I am looking at a Cumulus interface.  If it turns out to be a pain, this will be the first thing to get tossed overboard.

Wow! Very  8-)  =D&gt;

Not a request for enhancements, just a question relating to potential SkySpy Server-Service configuration(s)...

If you have more than one LCX station/gateways (1 remote - 1 local)...
would it be possible to create/run separate instances of the SSS-S, on the same physical server - each SSS-S instance
having a unique (configurable) service name and associated registry configuration entry?

e.g.
LCX1 Gateway at remote location.
LCX2 Gateway at local location.

Physical Server (running two SkySpy Server-Service instances to handle two LCX Gateways):

SSS-S instance 1 (for LCX1):
registry config = Service name: "SkySpyServer1", SSS-S listening IP: "192.168.2.100:6005", output to mySQL DB on IP: "192.168.2.100:3306", DB username: "SSSS_user1", pw="xxxxxx" Database Name: "LCX1_SkySpyServer1")

SSS-S instance 2 (for LCX2):
registry config = Service name: "SkySpyServer2", SSS-S listening IP: "192.168.2.100:5080", output to mySQL DB on IP: "192.168.2.100:3306", DB username: "SSSS_user2", pw="xxxxxx", Database Name: "LCX2_SkySpyServer2")

If unique service names & configs are a problem, we could always run two servers, (maybe 1 physical, 1 VM) each with their own SSS-S installations.


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on March 04, 2014, 02:18:18 PM

FYI, I've written parts of the server in PHP already, that allows you to register a gateway and a weather station to the php server.

Currently it doesn't do anything with the data, but it handles the packets and responds keeping the gateway and weather station happy.

I have a few more things to document, but  I post the source code in a day or two.  Probably save you a lot of time for you to run on your
linux box.

Skydvrz has the .php server code, but I don't know what he thinks of it.

-M





I started working on the SkySpy Server-Service (that is a mouthful!) tonight.

The plan (although things may change):

1) The service runs in the background and does all the non-visual stuff that ssServer does now - GW comm, WU Update, Mini-web site, DB update.


I just discovered this thread. Your program looks great, and I appreciate all the hard work that you've put into it. I have a new LCX station and was getting ready to connect it to their server, but really would like to keep the data local so I can access it whenever I want.

I would use your program, but I only use Linux. I would like to write a server like you've done, but using python in linux and have it write data to a local MySQL server. Then write some PHP scripts to view data and graphs with a web browser. I've done this sort of thing before in python and in C. Is it possible to get the source for your program, or perhaps the part that communicates with the gateway device and keeps it happily sending data.

I'm happy to help this project however I can, and of course make available any code I write.

Thanks a lot for your help.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on March 04, 2014, 03:10:40 PM

FYI, I've written parts of the server in PHP already, that allows you to register a gateway and a weather station to the php server.
Currently it doesn't do anything with the data, but it handles the packets and responds keeping the gateway and weather station happy.
I have a few more things to document, but  I post the source code in a day or two.  Probably save you a lot of time for you to run on your
linux box.
Skydvrz has the .php server code, but I don't know what he thinks of it.

Thanks, I'm sure that will save me a bundle of time. I look forward to seeing your code.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 04, 2014, 07:30:07 PM
If someone uses the local DB, will the monitor program still work if it's directed to the local DB?

No. A local database is usable by only one app at a time.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 04, 2014, 08:03:48 PM
Is it possible to get the source for your program, or perhaps the part that communicates with the gateway device and keeps it happily sending data.

I'd like to see more development on this, and I "speak" several computer languages.  I would love to see more research done on the unknown SDP fields, and the meanings of some of the other packet that are currently ignored. Please pass along anything you learn and I'll update ssServer  - with attribution if you like.

I did this in Delphi (Object Pascal) because I use it daily for other projects. So, if you can read Delphi then here is the source for you or anyone else that is interested.  Note that a lot of 3rd Party component libraries are not included, so you probably can't compile it even if own Embarcadero Delphi XE or higher.  They are up to XE5 right now.

https://www.dropbox.com/s/b7x31l6uutjgznq/ServerCloneSrc.rar (https://www.dropbox.com/s/b7x31l6uutjgznq/ServerCloneSrc.rar)

ssMainForm.pas handles the TCP/IP, HTML, database login, and the packet handshake logic side of things;

ssDataModule.pas handles SDP packet parsing, writing to the database, mini-web page HTML generation, raw sensor data to American units conversion, spoofed GW packet creation, Wunderground PWS updates, and other housekeeping methods.

This is just the server code - the SkySpy Monitor program is composed of some really expensive components that I cannot publish, so there wouldn't be much point in releasing the remaining 10% of non-copyright code for that.  If you want to see SQL queries to read the data back and graph it, then write me privately.  Most of the SQL queries will be self-evident when I get around to publishing the MySQL schemas.  That won't come until I ween everyone off my personal MySQL server.  :grin:

Also... This is a work in progress: The server software is in the process of having a massive rewrite and I won't be updating the version found in the RAR.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 04, 2014, 08:19:14 PM
If you have more than one LCX station/gateways (1 remote - 1 local)...
would it be possible to create/run separate instances of the SSS-S, on the same physical server - each SSS-S instance
having a unique (configurable) service name and associated registry configuration entry?

Define Local and Remote  :-)

If they are physically separated by a large distance, then you'd probably want to use separate PCs at each location to run the ssServer software and then have ssServer report readings to a common MySQL server - like we do now.

I could make the Gateway port configurable, so that you can run more than one instance of the ssServer, each listening to a different GW.  I'll have to think about how to get them to fly in formation though.  It shouldn't be a problem with the MySQL database to have multiple ssServer instances running. 

You would have to play port forwarding games on your router, and I am not sure how you could run a single server computer reliably if you don't have a static IP for it.  The remote GW needs a static IP to send its packets to.  I haven't tried putting a hostname into the GW Proxy IP field.  Might be interesting to see if it chokes.

Quote
If unique service names & configs are a problem, we could always run two servers, (maybe 1 physical, 1 VM) each with their own SSS-S installations.

At worst you would need a PC at each location to avoid DHCP issues.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 04, 2014, 08:23:20 PM
Skydvrz has the .php server code, but I don't know what he thinks of it.

It looks good, especially the clock set code.  I noticed that without a valid Real Time Clock (RTC) packet, my display has drifted by a couple minutes in the last month.  My RTC packet is currently ignored by the display.

I will take your static code and translate it to a dynamically updating one.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on March 05, 2014, 06:02:15 PM

I'd like to see more development on this, and I "speak" several computer languages.  I would love to see more research done on the unknown SDP fields, and the meanings of some of the other packet that are currently ignored. Please pass along anything you learn and I'll update ssServer  - with attribution if you like.

I did this in Delphi (Object Pascal) because I use it daily for other projects. So, if you can read Delphi then here is the source for you or anyone else that is interested.  Note that a lot of 3rd Party component libraries are not included, so you probably can't compile it even if own Embarcadero Delphi XE or higher.  They are up to XE5 right now.

ssMainForm.pas handles the TCP/IP, HTML, database login, and the packet handshake logic side of things;

ssDataModule.pas handles SDP packet parsing, writing to the database, mini-web page HTML generation, raw sensor data to American units conversion, spoofed GW packet creation, Wunderground PWS updates, and other housekeeping methods.

This is just the server code - the SkySpy Monitor program is composed of some really expensive components that I cannot publish, so there wouldn't be much point in releasing the remaining 10% of non-copyright code for that.  If you want to see SQL queries to read the data back and graph it, then write me privately.  Most of the SQL queries will be self-evident when I get around to publishing the MySQL schemas.  That won't come until I ween everyone off my personal MySQL server.  :grin:

Also... This is a work in progress: The server software is in the process of having a massive rewrite and I won't be updating the version found in the RAR.

Thanks a lot skydvrz. I have used Pascal, so understanding this is no problem. And I completely understand about the proprietary libraries.

I already have some personal weather data in MySQL, so I'll probably just add to that database once I get more up to speed on this. It may take me a bit of time to have something working. Other projects are taking up time as well.

I see in the readme the GW must be registered with Lacrosse. I assume there may be hashes exchanged to authenticate, maybe in both directions. Is the registration still necessary or does your code handle that now? And do you know if the GW requires the registration to be renewed after some period of time?

Does the GW only use HTTP to send data, or does it open other TCP/UDP connections to the server as well?

Thanks again for the code and for your time working on this. Anything more I find out on SDP fields or other mysteries I'll let you know.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 05, 2014, 09:37:38 PM
I see in the readme the GW must be registered with Lacrosse. I assume there may be hashes exchanged to authenticate, maybe in both directions. Is the registration still necessary or does your code handle that now? And do you know if the GW requires the registration to be renewed after some period of time?

Mycal has done some work on the registration process, so maybe at some point it could be incorporated into either your server or mine.  He did a working registration spoof in PHP.

It probably won't matter for most folks as you get a free trial registration in the shipping box.  Just use that.

Quote
Does the GW only use HTTP to send data, or does it open other TCP/UDP connections to the server as well?

The GW uses strictly UDP to communicate with the GAS utility.  It uses HTTP over TCP/IP for everything else. 

All HTTP packets have a header that identifies the user's account and (somewhat) the packet type.  The rest of the packet is a mishmash of BCD, little-endian integers, repeated fields and other mysteries that keep me awake at night.

Study TSDPRec in the datamodule for notes on all known packets and how the binary junk maps into their various fields.  I show zero-based hex offsets for all the fields in the comments.  I may update it with MyCal's findings as I get further into his code.

Any help you can offer would be greatly appreciated.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on March 06, 2014, 04:39:36 PM

Hi, its time to release the PHP code server.   Please read the file, as it is all text and explains what is going on.

One thing if someone could do that is sniffing the lacrosse alerts data is to set the packet update rate on the control panel and find out what
changes in the replys from the server.

https://www.dropbox.com/s/wmihsm2gh1d6obs/request.breq

Allthough this code will register a "new" never been registered weather station, I recommend you register it at lacrosse alerts first, since it
writes a serial number into the weather station, and so far I have not been able to reset this data once it has been written.

Let me know what you think

-M

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 06, 2014, 10:50:45 PM
Here is a link to the latest ssServer installer:

https://www.dropbox.com/s/r9d6b7axwe2ruwr/ssServerInstall.exe (https://www.dropbox.com/s/r9d6b7axwe2ruwr/ssServerInstall.exe)

Just install this over the top of your current installation.  If you have modified your mini-webpage HTML, then you probably want to back that up first.  See the enclosed Readme for installation instructions.

If you don't have a database login send me a private message and I will send you one.

Here is a list of bug fixes in this release:

Code: [Select]
ssServerSim - 1.1.0.14 (Released 2014-03-06) [ View Issues ]
============================================
- 0000123: [Bug] Web update fails to restart ssServer properly  - resolved.
- 0000121: [Bug] WU update is not sending software version  - resolved.
- 0000122: [Bug] WU Update is using 24 hour rainfall window  - resolved.
- 0000119: [General] Not receiving or sending data  - resolved.
- 0000117: [Bug] Setup form needs to deactivate server while it is visible  - resolved.

Back to work on the service version...


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 07, 2014, 12:58:39 AM
If you are having problems updating v1.1.0.7 to the latest version (v1.1.0.14) then use the link in the previous post and manually overwrite your existing installation directory.  You will have to shut your old ssServer software down during the update process.

v1.1.0.14 has some substantial bug fixes, including (I Hope!) a better automatic-update feature.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on March 09, 2014, 01:33:36 PM
I ran a network capture on the GW traffic for about 17 hours, starting before I powered up the GW for the first time. It should include all the registration traffic, and the transition to daylight savings time. It was run on my firewall/router, which is a pfsense machine and can do traffic captures, and included all traffic to/from box.weatherdirect.com. The capture should be compatible with Wireshark.

For security reasons I don't want to put this on a public site, but I'm happy to email. PM me if you want it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: slim83301 on March 10, 2014, 11:49:57 AM
I just want to give a BIG THANKS to everyone out there working on this project. I have been AWOL for the past month but   last week I got back on track and just a few min ago I was able to see our data on wunderground!

Skydvrz, thanks so much for all the hrs you have put in!



Will
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 12, 2014, 07:28:05 PM
WOW!  It sure has been quiet here lately.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 12, 2014, 10:45:28 PM
It has been quiet since I have been up to my eyeballs in the next generation of SkySpy - the Windows Service version of ssServer.  I have about 1/3 of the infrastructure for that written, but even that needs testing. 

If you want to see the project development roadmap and things I am working on, check out my bug/feature-request web site for SkySpy:

http://idk.serveftp.net/bugs/roadmap_page.php (http://idk.serveftp.net/bugs/roadmap_page.php) For the project roadmap

and

http://idk.serveftp.net/bugs/view_all_bug_page.php (http://idk.serveftp.net/bugs/view_all_bug_page.php) For a list of all bugs and feature requests.

You can request a new login account on my bugs web site and I will grant it.

Back to work!  ](*,)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 17, 2014, 09:13:39 PM
Work progresses - I have the SkySpy Service app framework done, along with storage of settings in the registry.  The service will run in the background on your PC.  If it sees that you have SkySpy Monitor running, it will send new sensor readings to it and Monitor will display them in real-time.  I got that part working pretty well.  Service keeps running, reporting new readings to Wunderground and the database even if Monitor is offline.

I've been ping-ponging back and forth between working on Service and Monitor, since they both have to fly in tight formation.

Stay tuned to my Mantis web site for progress reports and roadmap changes!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: slim83301 on March 20, 2014, 10:18:12 AM
I wish there was more I could do other than send in data.

We have 2 very grateful teachers here that thank each and everyone who has made this work!

We are still waiting on maintenance to install the wind and rain collectors on the roof. Maybe next week over spring break.

Happy Spring Everyone!

Will
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Mitch W4OA on March 20, 2014, 03:12:34 PM
Quick question, do the sensors communicate directly with the Gateway or communicate with the display unit then the display unit communicates with the Gateway?

I hope my question is clear.

Mitch
(sempra)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 20, 2014, 03:54:16 PM
It is a little more complicated than that  :-)

The C84612 anemometer/weather vane and rain gauge connect to the outside temp/humidity sensor over a 915 MHz RF link.  The outside temp/humidity sensor then forwards all of it to the LCD console, also at 915 MHz.  The LCD console then processes all the readings and forwards them to the optional Gateway at... You guessed it... 915 MHz.  :grin:

The GW1000U ERF-100 communicates with the LaCrosse cloud server (or SkySpy) via TCP/IP.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Mitch W4OA on March 20, 2014, 04:08:05 PM

Thanks!

I can see why they would do it that way, to make sure what is displayed on the console is the same as displayed from their "cloud". However, seems very inefficient and results in spectrum pollution from duplicate rf transmissions of the same data.

Mitch
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 20, 2014, 05:50:24 PM
I can see why they would do it that way, to make sure what is displayed on the console is the same as displayed from their "cloud". However, seems very inefficient and results in spectrum pollution from duplicate rf transmissions of the same data.

Not really. I am pretty sure they use poor-mans time domain multiplexing:  Only one unit gets to transmit at a time, and at miliwatt  power levels to conserve battery life.  I'd be surprised if the signal got out of my back yard strong enough to detect with another weather station.

I believe the LaCrosse design goal was to have an outside unit located close to the house (outside temp/humidity) to act as the hub, with sensors most likely in inaccessible/distant places (anemometer and rain gauge) as the spokes. 

The hub has C-Cell batteries for long life, since it probably has to marshal the other external sensors and chat with the LCD display located in the house.  The Gateway is typically located near your Internet connection which may be too far from the sensors (shielded by roof and walls), but relatively close to the LCD.

I think it is a good design compromise - but don't get me started on parts quality.  My rain gauge died :-(

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Mitch W4OA on March 21, 2014, 02:44:54 PM

OK, I'm signed up, I just pulled the trigger on a C84612 weather station.

Mitch
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 25, 2014, 10:10:41 PM
I got the Service version of SkySpy Server mostly working tonight!

It is busily receiving GW packets, updating Weather Underground and serving up the "mini web site" when requested.

I also made progress on the "embedded MySQL" database.  What that means is you can simply run the SkySpy service on your PC and it will create a tiny database server to store your readings.  You do not need to install the whole MySQL database server...  But you can't use the tiny server for anything else but the SkySpy service.  In a future release, I will add some dump commands that you can send to the service to create CSV files of your data.

Next I need to get the Service to chat with SkySpy Monitor - but that is for tomorrow night :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on March 26, 2014, 12:57:08 AM
I got the Service version of SkySpy Server mostly working tonight!

It is busily receiving GW packets, updating Weather Underground and serving up the "mini web site" when requested.

I just want to say that my hat is off to you. What you are doing is greatly appreciated.  =D&gt;
I wish I could be more help to you!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 26, 2014, 10:10:41 PM
I got SkySpy ssServer chatting with ssMonitor tonight, plus I cleaned up a few bugs.  You can check progress on my Mantis bug/feature request server - there is a link earlier in this thread.

ssMonitor is about 75% done for the next milestone;  ssServer is at about 57%.  The numbers may change if there are any added bugs that pop up.  Development should pick up speed a bit, now that I have all the infrastructure designed.

Thanks to the folks that are submitting bug reports on Mantis and weather data to my database!

I hope to have a new release out sometime in the next week or so.  Stay tuned!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 30, 2014, 10:01:10 PM
I made a lot of progress on SkySpy today.  I have the new ssServer nicely integrated with ssMonitor.  I have updated both ssServer and ssMonitor to work in both full-blown MySQL database mode as well as "embedded" mode that does not require installation of the MySQL database server.  There are some minor bugs that need fixing, but I am on target for a major release sometime in the next few days.

Added, but not finished is an alert system that compares sensor readings with user-settable min/max/range values.  If a sensor reading is outside the range, then ssServer will send you an email.  I have the min/max/range setting code done in ssMonitor and will implement the messaging system in a later release.

https://www.dropbox.com/s/xqj0w58rshou6y3/alerts.jpg (https://www.dropbox.com/s/xqj0w58rshou6y3/alerts.jpg)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on March 31, 2014, 02:52:52 PM
I made a lot of progress on SkySpy today.  I have the new ssServer nicely integrated with ssMonitor.  I have updated both ssServer and ssMonitor to work in both full-blown MySQL database mode as well as "embedded" mode that does not require installation of the MySQL database server.  There are some minor bugs that need fixing, but I am on target for a major release sometime in the next few days.

Added, but not finished is an alert system that compares sensor readings with user-settable min/max/range values.  If a sensor reading is outside the range, then ssServer will send you an email.  I have the min/max/range setting code done in ssMonitor and will implement the messaging system in a later release.

https://www.dropbox.com/s/xqj0w58rshou6y3/alerts.jpg (https://www.dropbox.com/s/xqj0w58rshou6y3/alerts.jpg)

You are REALLY going to put those moron software developers from Lacrosse to shame.  I can't wait to try it out.   Thanks Skydvrz!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on March 31, 2014, 03:50:17 PM
You are REALLY going to put those moron software developers from Lacrosse to shame.  I can't wait to try it out.   Thanks Skydvrz!

Yeah, that web site of theirs is a train wreck.  Be glad, because it was an inspiration for SkySpy  :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on March 31, 2014, 07:31:17 PM
You are REALLY going to put those moron software developers from Lacrosse to shame.  I can't wait to try it out.   Thanks Skydvrz!

Yeah, that web site of theirs is a train wreck.  Be glad, because it was an inspiration for SkySpy  :grin:

We all are Very Glad your work is greatly appreciated  :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 07, 2014, 08:45:58 PM
Ok,

I have been locked in mortal combat with the $%&*# installer, trying to get it to do a nice smooth installation of SkySpy on a variety of Windows versions.  I am not having much luck.   ](*,)

If you are technically savvy and a brave soul, then you can do a manual install.  It is tricky, since the new ssServer is now a Windows service.  Services do not install like normal applications.

So, would you rather wait until I figure out how to use this install builder and make a nice neat, professional looking install/deinstall, or would you prefer immediate gratification, a somewhat difficult install, and a really difficult de-install should you decide to back out of it?

I would like to get some beta testers up and running with the new version, but with the above caveats.

Let me know your thoughts...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 07, 2014, 09:31:40 PM
Ok,

I have been locked in mortal combat with the $%&*# installer, trying to get it to do a nice smooth installation of SkySpy on a variety of Windows versions.  I am not having much luck.   ](*,)

If you are technically savvy and a brave soul, then you can do a manual install.  It is tricky, since the new ssServer is now a Windows service.  Services do not install like normal applications.

So, would you rather wait until I figure out how to use this install builder and make a nice neat, professional looking install/deinstall, or would you prefer immediate gratification, a somewhat difficult install, and a really difficult de-install should you decide to back out of it?

I would like to get some beta testers up and running with the new version, but with the above caveats.

Let me know your thoughts...

I'm in!!!!!!!! =D>    8-)

I have both Win 7 & 8.1 to try it on if necessary.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on April 08, 2014, 12:05:01 AM
I'm willing to give it a try I would be installing on XP Home & XP Pro 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 09, 2014, 12:29:44 PM
I'm willing to give it a try I would be installing on XP Home & XP Pro

Some of the installer issues I am struggling with are specifically related to XP, so I am not sure I can support that platform.  SkySpy will work OK on XP, but the old installer software I am using is pretty clunky and balks at the differences between XP and Win7. 

I did make a little headway last night with the installer and mods to SkySpy to help it straddle Windows versions, so stay tuned.

I recommend you upgrade ASAP to Win7 for lots of reasons.  Win8 would not be my first choice in upgrades though - no Mr Gates, I don't want my desktop to look like my cell phone.  #-o
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 09, 2014, 02:17:07 PM

Some of the installer issues I am struggling with are specifically related to XP, so I am not sure I can support that platform.  SkySpy will work OK on XP, but the old installer software I am using is pretty clunky and balks at the differences between XP and Win7. 

I did make a little headway last night with the installer and mods to SkySpy to help it straddle Windows versions, so stay tuned.

I recommend you upgrade ASAP to Win7 for lots of reasons.  Win8 would not be my first choice in upgrades though - no Mr Gates, I don't want my desktop to look like my cell phone.  #-o


XP's life is about to come to an end in about a month.  That will leave your system open to any vulnerabilities.  As long as there are bad guys out there with computer skills, popular operating systems are open to new attacks.  Win 8 is great if you are using it on a touch tablet, but sucks on a normal computer.  Win 7 IS the way to go.  :-)   Problem is that if you've had your systems for several years, then the old hardware may not support the new software and it's additional bloat.   :sad:

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 09, 2014, 02:50:31 PM
XP's life is about to come to an end in about a month.  That will leave your system open to any vulnerabilities. 

Actually XP's end of life is today.  :shock:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 09, 2014, 04:17:09 PM


Actually XP's end of life is today.  :shock:

You're correct XP support is no more.  Well they did support it for almost 14 years.  I don't know why there are so many people complaining about support going away.  Shame on all the companies that have not prepared for this.  If XP were an automobile, OEM support would have been done away with years ago.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 09, 2014, 04:37:37 PM
Actually XP's end of life is today.  :shock:

A note to the lurkers that are still running XP:

If you absolutely must continue using XP, then immediately stop using the Internet Explorer (IE) web browser that comes with XP. Install either the Firefox or Chrome web browser and start using one of those instead.  Both Firefox and Chrome are free for the downloading.

Internet Explorer will no longer get malware prevention patches, so you are really hanging your you-know-what in the wind using it.  Not only that, but using IE identifies you as an XP user to every web site (good or bad) that you visit, effectively painting a big red target on your back as you surf the 'net. 

Alternate browsers will continue getting patched as new malware threats arise.

Next, make sure you have a good anti-virus program installed - every bad guy on the planet is going to be gunning for you and your XP box.  That said, even an anti-virus program will not protect you like a fully-supported operating system would.

BTW, most of the free anti-virus programs are not very good - you get what you pay for.

Now it is time for this physician "to go heal himself".  My wife is still running XP, so it is time for a new motherboard and Win7-Pro64 for her.  :lol:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: SlowModem on April 09, 2014, 04:50:50 PM
XP's life is about to come to an end in about a month.  That will leave your system open to any vulnerabilities. 

Actually XP's end of life is today.  :shock:

Actually, my computer did an update last night.  Probably the last one.  I'm surprised MS didn't put a kill program in the update.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 09, 2014, 05:38:06 PM


BTW, most of the free anti-virus programs are not very good - you get what you pay for.


What Security suite are you using.  I've been using Both Norton 360 and Norton Security Suite, both seem to be doing their job.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 09, 2014, 06:08:44 PM
I use Norton 360 at home and Symantec Endpoint Protection ($$$!) at work.  My wife uses the Norton 360 Backup, saving her files to our NAS.  It seems to work pretty well.

Both are pretty decent at detection and not whining about false-positive detections.

The hallmark of a bad AV is constant whining about viruses that don't exist.  They do it to make their competition look weak:  Oh look!  I just found 96 virus programs on your PC that  Norton missed!  Baaad Norton!  Then it proceeds to delete a bunch of non-virus, trivial files... or worse yet, half of your legitimate utility apps.

:grin:

I do use a freebie AV program that came with my NAS. I don't execute any programs there and everything passes through my desktop AV, so it is relatively safe.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 09, 2014, 09:55:48 PM
<shameless_plug>
I've been using Linux at home and work almost exclusively for years. I only run Windows (XP in a virtual machine under Linux) to run Labview (work) and watch Netflix streaming (home).

Here's a bit on Linux alternatives to Windows:
http://www.category5.tv/episodes/342.php/6:45

Karl
</shameless_plug>
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 10, 2014, 01:03:48 PM
I ran OS/2 for quite a while and really liked it.  I developed a bunch of shareware and commercial products for it over the years.  Sad to see it go, as I really liked the object-desktop approach.

I used UNIX on Sun and Apollo back in the day and did a lot of microwave circuitry design/simulation software development on that, but never got into Linux.

I have done extensive programming on LabWindows, mostly writing automated test software for the microwave and RF manufacturing industry and some laser precision-measurement systems.  One of my specialties is machine control using GPIB/IEEE-488.  LabWindows is a distant cousin to LabView. 

These days I program in Delphi on Wintel, but also program in PHP when doing web development.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 10, 2014, 05:33:12 PM
I purchased my La Crosse PWS C84612 last winter and finally with the snow gone and weather warming up I was able to get it installed yesterday!  I configured the gateway and got the online stuff working (lacrossealerts.com).  I also setup a wunderground acct to upload my data until I realized that I couldn't by default!  After a google search lead me hear and reading through this post I think I found exactly what I was looking for!  Amazing work SkyDvrz!  I see a lot of progress have been made from post 1 until today.  So if I want to start tinkering around should I just wait until your latest beta (from which it appears is almost ready) be released or proceed with downloading your server app from post #76 (page4)?   Thanks!  Can't Wait! :lol:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 10, 2014, 06:23:09 PM
So if I want to start tinkering around should I just wait until your latest beta (from which it appears is almost ready) be released or proceed with downloading your server app from post #76 (page4)?   Thanks!  Can't Wait! :lol:

That is up to you - I am working on the next release almost every night, but I am not sure how long it will take.  The old version that is still available for download works fine, but it won't restart after a computer shutdown or power failure.  The new version will restart as soon as the computer reboots.

Contact me via private mail and I will set up a database account login for you.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 10, 2014, 06:29:36 PM
The correct link to the OLD software is

https://www.dropbox.com/s/r9d6b7axwe2ruwr/ssServerInstall.exe (https://www.dropbox.com/s/r9d6b7axwe2ruwr/ssServerInstall.exe)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 11, 2014, 05:06:44 PM
Hi there.

Thanks skydvrz for setting up username/password to use your ssServer Dbase.   Unfortunately I'm running into troubles.   I've skimmed this thread a few times and verified my ssServer settings over and over but something isn't right.  I followed the instructions in the readme file a few times now trying to pinpoint the issue.   I configured the ssserver to idk.serveftp.net port 3307 and logged in with my acct from skydvrz.  I ran and configured the GAS as instructed but no data ever shows in the SSserver window fields.   

Also, I presume since I'm basically changing how/or where it is sending its data that I will no longer be submitting it to my lacrossalerts site.   What if I wanted to (even if temporarely) reconfigure it to submit the data to LacrosseAlerts ?  Would I use the GAS to do so or do I have to register it per the la crosse docs)

Thanks for any help!

Mark
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 11, 2014, 05:32:18 PM
Hi there.


Also, I presume since I'm basically changing how/or where it is sending its data that I will no longer be submitting it to my lacrossalerts site.   What if I wanted to (even if temporarely) reconfigure it to submit the data to LacrosseAlerts ?  Would I use the GAS to do so or do I have to register it per the la crosse docs)

Thanks for any help!

Mark

You are SURE that you used your local network address E.G. 192.168.1.xxx at the bottom of the GAS (just past the Use checkbox at the bottom) and there is a checkmark in that box.  The address you entered here is probably the same IP as the one listed at the top of GAS (PC-IP) minus the leading 1:    You then clicked on Set, waited and then clicked on Reboot.  Skydvrz may be able to add to this.


Once you do have ssServer set up, data no longer goes to LaCrosse.  To return to LaCrosse, just uncheck Use at the bottom of GAS, Hit Set and Reboot. Wait and data should be going back to their site.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 11, 2014, 05:42:52 PM
I've skimmed this thread a few times and verified my ssServer settings over and over but something isn't right.  I followed the instructions in the readme file a few times now trying to pinpoint the issue.   I configured the ssserver to idk.serveftp.net port 3307 and logged in with my acct from skydvrz. 

I am not seeing any MySQL traffic from you.  Were you able to log in successfully?  You should see an error dialog box or a progress bar appear when ssServer sends me a bug report upon login failure.  BTW, I don't have a "Mark" in my users list.  Did you use a different email name?  I base your login on your email address.

I had a bunch of Mantis errors recently with someone entering " idk.serveftp.net" <- note the leading space.  That is not a valid database hostname.  Could that have been you?  If so, remove the space and try again.  Also, if you leave the hostname blank, ssServer will try to connect to a local MySQL server (and fail) on the same PC.  Someone tried that recently too :-)

Quote
I ran and configured the GAS as instructed but no data ever shows in the SSserver window fields.   

You need to let it run for a few minutes.  The Gateway only sends Sensor Data Packets (SDP)  every 3-15 minutes.  Be patient :-)

Quote
Also, I presume since I'm basically changing how/or where it is sending its data that I will no longer be submitting it to my lacrossalerts site.   What if I wanted to (even if temporarely) reconfigure it to submit the data to LacrosseAlerts ?  Would I use the GAS to do so or do I have to register it per the la crosse docs)

You are simply changing where the Gateway sends its data.

Just shut off the proxy settings with GAS, click the save button (or whatever it says).  Wait a few seconds for the new settings to store, then click the Reboot button in GAS.

Any time you change settings in GAS, you have to do the save/pause/reboot dance or the settings won't "take".

After removing the proxy settings, your Gateway should begin reporting to La Crosse again within a few minutes - again, be patient.  La Crosse has a minimum update interval of 15 minutes, downhill, with the wind at your back (pardon the pun).  The La Crosse server can go to sleep for hours - as you might have noticed.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 11, 2014, 05:43:41 PM
10ACTony:  Good catch on the IP address thing.  I had forgotten about that.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 11, 2014, 05:53:57 PM
Hey! I just checked the MySQL server stats - we just went over 1 million individual weather sensor readings since the start (11 Jan 2014) of this project!  Don't worry - I have several Terabytes of disk space left.  8-)

Also, you guys will be hosting your own mini-databases in the next version - no more free rides :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 11, 2014, 06:11:43 PM
I just reran the setup from scratch all over again and we have a winner!  I'm going to enjoy skydvrz db free ride for now!  Thanks everyone!

And the upload to Weather Underground is working also!  SWEET!  This is sooooo cool!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 11, 2014, 07:11:19 PM

Also, you guys will be hosting your own mini-databases in the next version - no more free rides :-)

 =D&gt;      :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 11, 2014, 07:14:03 PM

And the upload to Weather Underground is working also!  SWEET!  This is sooooo cool!

We are SOOOOO lucky that skydvrz just happened to buy the same product and decided to show what real software can do.  Without him we'd be SOL.   :sad:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 12, 2014, 12:48:06 PM
Quick question, on the local web page launched from the ssServer Open Browser button is there a way to configure the time zone?  The page shows pacific time it appears and I'm in the central time zone.    Another one,  finally got some rain since I installed my PWS a few days ago.   The ssServer shows Rain Fall .45 (in).   The La Crosse PWS Display shows Total Rain Total of .44.   Is this just a cause of rounding or how it's done with the different displays (Wunderground vs SsServer vs PWS Display)?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 12, 2014, 02:05:55 PM
is there a way to configure the time zone? 

Not currently.  The next version will store everything in local time - actually the current time at the MySQL database server.  The database will likely be at your house, not mine.  :grin: 

The Wunderground updater converts local time on your PC to GMT before sending it to WU, so that one should be OK no matter where your database server is located.

Quote
The ssServer shows Rain Fall .45 (in).   The La Crosse PWS Display shows Total Rain Total of .44.   

La Crosse probably uses a different Metric to American units algorithm than I do.  They usually match what I calculate, but sometimes not. The RG actually reports in Millimeters, IIRC, so I have to scale it to Inches.

Here is the calculation:

Code: [Select]
FUNCTION TsssDm.GetRainfallIn(RF : TRainFallArray) : Double;
BEGIN
    Result := ((DisplayToDigit(RF[0]) * 10.0) + (DisplayToDigit(RF[1]) / 10.0) + (DisplayToDigit(RF[2]) / 1000.0)) * 0.0393701;
END;

RF is a 3 byte block of BCD-encoded binary in the SDP.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 12, 2014, 07:25:34 PM
I am finally making headway again!  I gave up on the &$#(*! InstallAware installer creation software I was using and switched to a newer one - Inno setup.  This is a brand new/steep learning curve that I have to crawl up, but I am finally able to complete most of an install and have it actually function.  InstallAware 7 (the only version I have) was screwing up the service install such that it would hang on start up.  The new version of InstallAware is $1000, so I think I will give it a miss.  Inno is open source / free.  I like free.  ;)

Inno is letting me get a lot farther, so I am highly optimistic.

It is not ready for prime time - there are a couple manual steps I have to do to finish a given install, but I hope to figure out how to do that with Inno code soon.
 
Other news:

I added a link to the SkySpy mini-web site page that points to your Wunderground graphs and tabular data.  If you click it, it jumps your browser to your PWS page.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 13, 2014, 09:03:21 PM
Can't wait SkyDrvz! 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 14, 2014, 10:51:46 PM
Mycal and Skydrvz, thank you both very much for all your hard work in decoding the packets going back and forth to the Lcx gateway. I set up an Apache server with Mycal's php code, and adapted from Skydrvz's decoding of the data block. Amazingly, I'm intercepting and saving data to a text file. Next step is the database.

One thing I noticed is that the BCD temperature outputs seem to be in tenths of degrees C. In the equation:
Result := (I * 0.18030) - 40.15
the 0.18030 may actually be a degree C-to-F conversion scaled to 0.1 deg (9/5 * 0.1). Also, 40.15 probably should be 40.0. It appears they avoid negative numbers in the BCD data by adding 40 deg C to temperatures before the gateway sends it out, then subtract it in software downstream. Anyway, I think (I / 10.0 - 40) will get degrees C. Then make the standard conversion to F.

I also found some additional date fields. Maybe you've already found these, Skydrvz. It appears some of the date fields span 6 bytes of output, and are offset right by 4 bits. So the first BCD digit of the year is in the lower 4 bits of the first byte. It goes on from there one BCD nybble at a time, with the last digit of the minute output in the upper nybble of the 6th byte. In some cases the next date field starts in the lower 4 bytes of the 6th byte of the previous date field. So there's overlap in bytes, but not in nybbles.

These are the offsets in the post data of the date fields I have so far, including ones from Skydrvz. The asterisk indicates ones that span 6 bytes and start with the lower nybble of the first byte. All the others span 5 bytes and start with the high nybble.

0x04* Max Indoor Temp
0x09* Min Indoor Temp
0x16* Max Outdoor Temp
0x1b* Min Outdoor Temp
0x28* Unknown
0x2d* Unknown
0x3A  Max Indoor Humidity
0x3F  Min Indoor Humidity
0x47  Max Outdoor Humidity
0x4C  Min Outdoor Humidity
0x6A  Unknown
0x75  Unknown
0x80* Last Rain Reset Time
0x89  Unknown at initial date/time? 1/1/2013 00:00
0x98  Time of Max Wind Gust
0xB6  Unknown
0xBB  Unknown
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 01:24:58 AM
Thanks KecKec!  I'll check your findings out in the morning - its been a long day!

If you can figure out how to set the clock on the LCD display reliably then that would be fabulous!  I still have not figured out how to do that and can use all the help I can get on this one step.

Back to SkySpy...

I got the SkySpy installer / deinstaller working finally!   \:D/

I will be doing a release of the new Windows service version probably tomorrow.  I am writing the manual now.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 12:38:42 PM
The transaction problem I'd really like to solve is the 5-byte send / RTC response packet.    This is the 38 byte 01:01 packet the server is supposed to send in response to the Gateway sending an 01:01 5-byte request.  The clock on my LCD display is slowly drifting, since it is not getting valid Real Time Clock (RTC) reset responses from SkySpy.  SkySpy puts the correct time in the RTC packet, but the LCD display ignores it since there is some sort of handshake problem.

Probably no big deal since clock drift is entirely cosmetic.  The time shown on the clock does not affect the timestamps on the data sent to WU.

I hate inaccurate clocks though. I even wear an atomic-clock wristwatch that syncs with WWVB every night  8-)

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 12:53:35 PM
One thing I noticed is that the BCD temperature outputs seem to be in tenths of degrees C. In the equation:
Result := (I * 0.18030) - 40.15
the 0.18030 may actually be a degree C-to-F conversion scaled to 0.1 deg (9/5 * 0.1). Also, 40.15 probably should be 40.0. It appears they avoid negative numbers in the BCD data by adding 40 deg C to temperatures before the gateway sends it out, then subtract it in software downstream. Anyway, I think (I / 10.0 - 40) will get degrees C. Then make the standard conversion to F.

Yes, I used the same initial calculation as you, but got some inaccurate numbers. 

The tweaks to the formula were arrived at empirically to make the calculated Fahrenheit temp match what the LCD display shows.  My numbers are typically +- 0.1 F of what my display says.  I drew graphs and re-jiggered the slope-intercept values for minimum error, measured against what the LCD says.  I even put the temp sensor in my freezer for an hour and later hit it with a blow dryer to check the cal curves.

Maybe I have simply recreated a bug in the LCD display code :-)

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 15, 2014, 01:37:05 PM
The transaction problem I'd really like to solve is the 5-byte send / RTC response packet.    This is the 38 byte 01:01 packet the server is supposed to send in response to the Gateway sending an 01:01 5-byte request.  The clock on my LCD display is slowly drifting, since it is not getting valid Real Time Clock (RTC) reset responses from SkySpy.  SkySpy puts the correct time in the RTC packet, but the LCD display ignores it since there is some sort of handshake problem.

Could this be a checksum problem? Does the gateway use any other data that you send it, which might confirm that it is happy with the checksum?

mycal appends a straight 8-bit sum when he sends packets to the gateway. Using that algorithm doesn't seem to match the checksum in the post data I receive from the gateway. I've also tried an 8-bit exclusive or but that doesn't agree either. I'll experiment with some others.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 01:51:40 PM
Could this be a checksum problem? Does the gateway use any other data that you send it, which might confirm that it is happy with the checksum?

I don't think the hardware actually checks checksums :-)  Checksums are a bit superfluous when using TCP/IP anyway.

It may be some sort of revenue protection feature, buggy firmware or clueless newbie programmers unaware of TCP/IP intricacies.

The 5-byte send that the GW does contains some random looking bytes at the end of the packet that don't seem to correspond to any sort of sensor readings and is probably not a checksum.  Might be the result of some previous step in a state-machine. 

Quote
mycal appends a straight 8-bit sum when he sends packets to the gateway. Using that algorithm doesn't seem to match the checksum in the post data I receive from the gateway. I've also tried an 8-bit exclusive or but that doesn't agree either. I'll experiment with some others.

I tried MyCal's method in SkySpy and noticed the same thing.  This needs more research. 

He also says he could adjust his LCD clock, but I have not been able to duplicate that result.  I'd like to see some code that uses dynamic values instead of hard-coded constants like MyCal's example.

Shoot me some PHP code if you make any progress.  I speak PHP.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 08:42:28 PM
Good news!

Here is a link to the next generation of SkySpy:

https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe (https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe)

Here is the user's manual, if you want to read about it before downloading:

https://www.dropbox.com/s/brnz7jesdvvbmg5/ReadMe.pdf (https://www.dropbox.com/s/brnz7jesdvvbmg5/ReadMe.pdf)

Oops! <edit>

I forgot to add the schema info to the installer for the folks that want to use their own MySQL database.  Here is a link to the MySQL script to create the schema and tables.  You do not need this if you use the embedded database!

https://www.dropbox.com/s/rlbdmp4gohnp1vx/weather.sql (https://www.dropbox.com/s/rlbdmp4gohnp1vx/weather.sql)

The schema allows multiple users to share the same tables, so you can buddy-up with someone willing to host the database server.  Any MySQL server can probably handle several hundred users of this app - given enough hard disk space. 

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 15, 2014, 08:44:07 PM
Long time lurker but loving what I am seeing. I have an Ubuntu server running 24/7 at home so would prefer a php solution but it might be good to try to use the same database schema as skydvrz so that any windows front end stuff can connect to the same database.
Here I am the freeloader giving instructions... sorry.

A big thanks to all who have contributed!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 15, 2014, 08:51:14 PM
Deleted

 #-o
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 08:53:36 PM
10ACTony: see my edited post. Let me know if you have any problems.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 15, 2014, 09:03:25 PM
I see I forgot to add the utility schema script for MySQL.  I will have to do a little more work on SkySpyService before you can use it on your own MySQL server.  Sorry about that!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 15, 2014, 09:58:34 PM
Beautiful work!   I could use my SQL Server Express local Dbase correct?

I have never used mySql but use MS SQL all the time at work and SQL Express.   The script file you provide won't work with these without some modificcation.  From my own research I found this

"I think, you can start mysql in ANSI mode and generate the scripts
with this standard which is ready to use in SQL server with no
modification. see the following page for more details:
http://dev.mysql.com/doc/mysql/en/ANSI_mode.html"

I know you are busy so please no hurry, but would you be able to provide this script so it's compatible for us with MS SQL ?   Thanks a ton.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 15, 2014, 11:01:48 PM
No Luck.   Closed previous running and working ssServer and installed the new SkySpy.   Opened the monitor and after 25 minutes the readings still haven't updated.   The Last Packet Type has shown everything from Ping Received, to DSP Received, to RTC Packet Received but after 25+ minutes still no updated weather readings.  Rebooted and same behavior.  Any suggestions?

Windows 7 Home Premium, 64 bit


---
I stopped the SkySpy Server Service and closed the monitor.  I launched the ssServer instead this time and within 5 minutes it was filled with data.  Hmmmmmmmmmmm.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 15, 2014, 11:59:12 PM
No Luck.   Closed previous running and working ssServer and installed the new SkySpy.   Opened the monitor and after 25 minutes the readings still haven't updated.   The Last Packet Type has shown everything from Ping Received, to DSP Received, to RTC Packet Received but after 25+ minutes still no updated weather readings.  Rebooted and same behavior.  Any suggestions?

Windows 7 Home Premium, 64 bit


---
I stopped the SkySpy Server Service and closed the monitor.  I launched the ssServer instead this time and within 5 minutes it was filled with data.  Hmmmmmmmmmmm.

Exactly the same here. Not hitting WU either. Last date of Update  is listed as Mar 26, 2014., but historical data is current up to the point the service was supposed to start logging.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 12:25:39 AM


http://dev.mysql.com/doc/mysql/en/ANSI_mode.html"


PAGE NOT FOUND
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 12:55:43 AM
Beautiful work!   I could use my SQL Server Express local Dbase correct?

I have never used mySql but use MS SQL all the time at work and SQL Express.   The script file you provide won't work with these without some modificcation.  From my own research I found this

"I think, you can start mysql in ANSI mode and generate the scripts
with this standard which is ready to use in SQL server with no
modification. see the following page for more details:
http://dev.mysql.com/doc/mysql/en/ANSI_mode.html"

I know you are busy so please no hurry, but would you be able to provide this script so it's compatible for us with MS SQL ?   Thanks a ton.

Sorry, this app is MySQL-only right now.  There will be subtle differences between MySQL and other databases that I cannot support.  You could rewrite the source and support other databases fairly easily though.  Unfortunately, I am out of bandwidth right now :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 01:04:59 AM
Beautiful work!   I could use my SQL Server Express local Dbase correct?

I have never used mySql but use MS SQL all the time at work and SQL Express.   The script file you provide won't work with these without some modificcation.  From my own research I found this

"I think, you can start mysql in ANSI mode and generate the scripts
with this standard which is ready to use in SQL server with no
modification. see the following page for more details:
http://dev.mysql.com/doc/mysql/en/ANSI_mode.html"

I know you are busy so please no hurry, but would you be able to provide this script so it's compatible for us with MS SQL ?   Thanks a ton.

I really don't know.  I am very familiar with MySQL but not other databases. Please note that there are some serious problems with the current release and external databases.  I advise using either my MySQL database server or Embedded mode right now.

I am working on a simplified version of SkySpyService that can work on a MySQL database schema that is not as complex as the original design.  The current schema is designed to be fairly hacker-proof and is not a good paradigm for wide distribution.

The embedded design is designed for wide distribution.  Use that mode for now,  and I promise great things for the more sophisticated users in the near future.

Sorry for the delay, but I wanted to get something useful out for wide distribution as soon as possible.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 01:09:57 AM
No Luck.   Closed previous running and working ssServer and installed the new SkySpy.   Opened the monitor and after 25 minutes the readings still haven't updated.   The Last Packet Type has shown everything from Ping Received, to DSP Received, to RTC Packet Received but after 25+ miWnutes still no updated weather readings.  Rebooted and same behavior.  Any suggestions?

Windows 7 Home Premium, 64 bit

Hmm.  It should start working within 5 minutes or so.  What wasn't updating?  WU?  You do need to run the setup in the new SkySpy monitor and configure your WU Station ID, Etc.  Did you do that?

If you are seeing packet updates in the new Monitor then things are working perfectly.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 02:52:33 AM


Hmm.  It should start working within 5 minutes or so.  What wasn't updating?  WU?  You do need to run the setup in the new SkySpy monitor and configure your WU Station ID, Etc.  Did you do that?

If you are seeing packet updates in the new Monitor then things are working perfectly.

No data at all either on the current data page or anything in the remote database. The last update for WU and Sensor never changes and is back in March on mine so it's not even the last data from the the old server program. But the data that was equivalent to the old debug data (showing different packets and times) did change.  Running on old server now.  Willing to try anything and get you whatever data you need,  real time if necessary.  I did do the setup in the service.

Tony
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 11:07:02 AM
I could use my SQL Server Express local Dbase correct?

Sorry... Almost certainly not.  SkySpy uses TCP/IP to communicate with MySQL in its native format. 

I don't use any sort of database-agnostic layer like ODBC, so you are pretty much stuck with either the built-in embedded server or a full up MySQL installation.  I did this for performance reasons as there can be several million records to muck through at a given time. ODBC or name-your-favorite "anydac" layer between the app and its data can really slow things down.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 11:32:54 AM
No Luck.   Closed previous running and working ssServer and installed the new SkySpy.   Opened the monitor and after 25 minutes the readings still haven't updated.   The Last Packet Type has shown everything from Ping Received, to DSP Received, to RTC Packet Received but after 25+ minutes still no updated weather readings.  Rebooted and same behavior.  Any suggestions?

Windows 7 Home Premium, 64 bit

So are you saying the sensor readings do not show up in the Monitor notebook tab, Weather Underground or both?  Have you configured the WU settings?  Have you tried using Embedded mode? 

If the Monitor shows "SDP Received" every 5 minutes or so in the Last Packet Type field, then SkySpyService is working fine.

SkySpy was built and tested on Win7-64 Pro.

You should see something like this:

(http://idk.serveftp.net/images/Captured003.jpg)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 11:43:57 AM
No data at all either on the current data page or anything in the remote database. The last update for WU and Sensor never changes and is back in March on mine so it's not even the last data from the the old server program. But the data that was equivalent to the old debug data (showing different packets and times) did change.  Running on old server now.  Willing to try anything and get you whatever data you need,  real time if necessary.  I did do the setup in the service.

Try opening SkySpy.ini, found in your install directory using Notepad.  Let me know if anything looks funny.  Send it to me via private mail and I will check it out.

What mode are you trying to use?  Embedded or Full database?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 12:56:57 PM
I re-installed the program this morning and re-entered all the info, and today it's working like a champ.  I'm glad it happened to someone else because I couldn't figure out where I went wrong.  Non-embedded and WU working like a champ today. :grin:   =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 02:34:03 PM
Boy am I the unlucky one :-) 

I uninstalled and rebooted and then re-installed. Same symptom.  Unfortunately for me the only fields that update under the Service Readings tab are the Last Packet Type and Last Packet Time.  They cycle through the Ping Received, SDP Received, and RTD Packer retrieved items.   The Last WX Data and Last WU Update Date fields have a static value from 3/26/2014.  Not sure where it gets this from as I only just installed my PWS last week and configured it for SServer shortly there after.   Even though the readings never update (besides the packet type and type fields as mentioned above) I did proceed to configure WU.  Yet,  WU never gets any updates.   Also of note the Use Embedded MsSQL box is checked and all the other values below it are the defaults.  Did look over the INI file but nothing looks out of wack to me anyways.   I'm stumped.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 03:06:03 PM
Boy am I the unlucky one :-) 

I uninstalled and rebooted and then re-installed. Same symptom.  Unfortunately for me the only fields that update under the Service Readings tab are the Last Packet Type and Last Packet Time.  They cycle through the Ping Received, SDP Received, and RTD Packer retrieved items.   The Last WX Data and Last WU Update Date fields have a static value from 3/26/2014.  Not sure where it gets this from as I only just installed my PWS last week and configured it for SServer shortly there after.   Even though the readings never update (besides the packet type and type fields as mentioned above) I did proceed to configure WU.  Yet,  WU never gets any updates.   Also of note the Use Embedded MsSQL box is checked and all the other values below it are the defaults.  Did look over the INI file but nothing looks out of wack to me anyways.   I'm stumped.

Check the in file in the install folder.  I uninstalled last night , then this AM I saw the message from skydvrz about the in file.  It was still in the folder even after it was uninstalled, and it still had the defaults for the server.  Make sure your login info is correct at the bottom of the file.  Today everything was there to support my installation.  Worth a check.

;-)

Tony
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 03:15:30 PM
Howdy!  thanks. Yep I even renamed that file  and stopped / started the service and restarted the sky monitor and it rebuilt this file from scratch.

In the most simple terms, I followed the instructions and installed the application.  I launched the monitor and it never populates the data even after 30 minutes of waiting. There is no configuration to mess with being the GW was configured and working with ss server so nothing had to be done there.   But yes I did proceed to configure the WU login info and it does show in the ini file but again technically the monitor should still be showing me data and it isn't which would also explain to me why WU isn't getting any data either.   Yet the Last Packet Type and Last Packet Time continue to show it's communicating with the gw to some degree.  I'm confused and need a drink  ](*,)  :lol:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 03:20:01 PM
Quote
The Last WX Data and Last WU Update Date fields have a static value from 3/26/2014.

That is the date/time that I wrote that code.  In a future release, I will blank those fields until some actual data comes in.

Quote
Even though the readings never update (besides the packet type and type fields as mentioned above) I did proceed to configure WU.

The way it is supposed to work is that the Last Packet Type and date should update every 30 seconds or so.  This is the Gateway and SkySpyService chatting together. 

If readings are not appearing in SkySpy Monitor after an SDP packet is detected, then something is probably wrong with your settings that causes an internal error in SkySpyService.

The SDP packet event should update your Monitor screen sensor values fields.

Note that if you are successfully updating Weather Underground, then you don't need to run Monitor at all.  All of the smarts are in SkySpyService - the part that runs behind the scenes in Windows.

Quote
Did look over the INI file but nothing looks out of wack to me anyways.

Send me your ini file and I will test it and get back to you.  You can change your Wunderground password afterwards if you are paranoid :-) 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 03:22:26 PM
I'll  get the ini to you shortly but note that besides the issue with monitor not updating WU is NOT receiving data / updating.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 03:28:36 PM
Howdy!  thanks. Yep I even renamed that file  and stopped / started the service and restarted the sky monitor and it rebuilt this file from scratch.
Watch out - SkySpy will fill in all default values for your settings if you delete this file.  Default settings WILL NOT WORK.  Settings are NOT imported from previous SkySpy versions!

So, go back and look at all of your settings - something is probably sideways. 
Quote

Quote
Yet the Last Packet Type and Last Packet Time continue to show it's communicating with the gw to some degree. 

Make sure you click the Ok button on the setup screen, not the Cancel or "red X".  Settings are only saved if you click OK.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 03:38:27 PM
What settings are we talking about?  WU settings?  Use Embedded MySql settings?  Something else?   I am using Use Embedded MySQL and all the values are their defaults and according to the docs not used while in this mode.   Besides this to get the monitor to show / receive data there aren't really any settings (the GW was already configured)

I just just ran the installer which installed the service and then ran the sky monitor.  It should just start receiving data to at least show me its working by 15_ minutes, as you know.  Which it isn't besides the Last Packet Type an Last Packet Times.  It  I did setup WU for beeps and giggles but it too isn't getting data. (sorry if i'm repeating myself)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 04:11:47 PM
You should step through all the settings and make sure they are appropriate for your system.

The WU settings will definitely not work when using defaults.  You must insure that your station ID is correct and that your WU password did not get munged by the SkySpy spell checker.

If you are using Embedded mode, then there are not too many ways to mess things up. In Full-MySQL there are lots of ways to die :-)

Embedded mode does not use Hostname, Username, Password or Port.  It does need the schema name set to "weather".

If you are using my MySQL server, make sure that your database password does not get spellchecked/"fixed" by SkySpy.  You have to "sneak up" on the password field to make sure the spellchecker does not capitalize the first letter, fix spelling, Etc.

I will fix the password spell check problem soon.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 04:23:58 PM
Important Note!

You cannot run both ssServer v1.1 and the new SkySpyService v1.2.0.X at the same time.  They will fight over TCP/IP port 8000 and stop working.

Make sure you nuke the old version before trying to install the new version because the installer launches SkySpyService immediately.  The service will never be able to grab port 8000 until you kill the old program and restart your PC (or manually restart the service).
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 04:39:08 PM
Important Note!

You cannot run both ssServer v1.1 and the new SkySpyService v1.2.0.X at the same time.  They will fight over TCP/IP port 8000 and stop working.

Make sure you nuke the old version before trying to install the new version because the installer launches SkySpyService immediately.  The service will never be able to grab port 8000 until you kill the old program and restart your PC (or manually restart the service).

Just exiting the old server should kill it right, no need to reboot before installing the new service.

I want to say that is what I did last night, but thinking about it makes me wonder if I didn't stop the server until just after installing the new service.  If so, would that be why the ini file still had your defaults in it this morning???  But I did see the last packet (ping RTC and SDP) displayed with the correct time, so the service must have been running, right??
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 04:47:06 PM
Howdy!  For the sake of troubleshooting I like to remove all the extra overhead.  Back to the basics.   I'm not worried about WU.   I'm also just using the embedded dbase.  I'm not concerned about any other configuration options..   So without those in the picture the question then remains is why after installation and no extra configuration does the monitor still not update the fields (besides the packet fields) - which would show me its working fully.   Without concerning the other funciontaility (wu and other dbases (mySql)) what other settings can I really check? 

The service is apparently getting the data because the monitor does show the last packet fields and.

From your PM.   No I am not running both versions at the same time.   I wish it was that easy :-)  When trying to use the new service I have never tried to run the ssServer also or vice versa.    WU might show the old ssServer still being used because last night after trying and failing to get the new version working I disabled the service and restarted SsServer.  SsServer continues to work (showing me the data with the ssServer app and pushing my data to WU).  But no when I've installed or tried to work with the new app ssServer has never been running. Hope that makes sense.   For now I have disabled the new service and closed the monitor and relaunched ssServer so I could get some data flowing back up to WU.  Rains moving in :-0
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 06:17:00 PM
Just exiting the old server should kill it right, no need to reboot before installing the new service.

That should do it, but if you have the old server in your startup folder, it will give you no end of grief.  ZIP or delete the install directory for the old server.

Quote
I want to say that is what I did last night, but thinking about it makes me wonder if I didn't stop the server until just after installing the new service.  If so, would that be why the ini file still had your defaults in it this morning??? 

Running the old server may have caused the Service to hang, since it gets launched immediately by the Installer.  I have to do that to complete the installation.

The Service creates a new, all-defaults INI if it can't find one in the install directory.  This default INI file puts you into embedded mode, shuts off both WU updates and the mini-web site.

I leave the SkySpy.INI file in the installation folder after a de-install, in case you want to retain your settings and reinstall later.  I also leave the embedded database files in place - it was too complicated deleting them after a de-install, but I may change that in a future release.

It is also possible that if the service hangs and you try to re-install or un-install, then it might not get unloaded properly and bad things might happen.  You can kill a hung Service with the process manager.  You can also manually uninstall Service by opening a command line box, CD'ing to the installation directory and entering:

SkySpyService -uninstall

You should see a little dialog box pop up that indicates success.

You can also manually install the service by doing the same steps as above, but entering

SkySpyService -install

You only need to do this if you don't want to run the entire installation EXE and the Service is not listed in the Windows Service manager.

It is much easier to just run the installer or deinstall from the Windows Programs screen.

Something strange is going on...  It is working fine for me here.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 06:22:46 PM
From your PM.   No I am not running both versions at the same time. 

Very weird.  What version of Windows are you running? Sorry if you told me before - its been a long day.

Quote
  Rains moving in :-0

I wish I had your weather!  It seems like it never stops raining here in the Pacific Northwest.  We get about 60 days of *dry* weather a year here.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 16, 2014, 06:25:47 PM

Just exiting the old server should kill it right, no need to reboot before installing the new service.

I want to say that is what I did last night, but thinking about it makes me wonder if I didn't stop the server until just after installing the new service.  If so, would that be why the ini file still had your defaults in it this morning???  But I did see the last packet (ping RTC and SDP) displayed with the correct time, so the service must have been running, right??

I tried the above, just to try to figure out what happened last night.  - I uninstalled the service, started ssServer, waited until it updated the first time then installed the service, and after it was installed stopped  ssServer. I did the configuration and saved it.  Nothing ever showed up in last packet type or time.

Manually started the service.  Now last packet time and type update but service readings and times are NOT updating just like last night.  I saw two SDP displayed in Type  as well as noticed the greem light blink on the gateway.  I looked at the skyspy.ini file and everything in there is the default, so they DID NOT save my configuration.  Stopped Monitor and restarted checked configuration in monitor, everything is correct, but the ini is still at the defaults.  I made a backup of the  the working ini file and now copied it to skyspy.ini.  Stopped the service and restarted.  Everything is now working normally again.

 =D&gt;    =D&gt;    =D&gt;    =D&gt;

 skydvrz must be keeping the configuration for monitor in the registery and the configuration for the service in the ini file.  Does that make sense K?

Clicking OK on the Setup screen doesn't always seem to save the changes if any.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 07:43:30 PM
Everything is now working normally again.

 =D&gt;    =D&gt;    =D&gt;    =D&gt;

I think you found it Tony - you can't run ssServer while installing the new version or bad things will happen.

Quote
skydvrz must be keeping the configuration for monitor in the registery and the configuration for the service in the ini file. 

No, they both share the INI file.  It might get locked open if you run both old/new programs at the same time - or it might be a directory rights issue with writing files into the Program Files folder tree.  I am admin on my Win7 box, you may not be on your computer.

Quote
Clicking OK on the Setup screen doesn't always seem to save the changes if any.

I have not seen that on a properly installed Service/Monitor setup.  Not that it couldn't happen :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 07:52:51 PM
I'm running Windows 7 64 bit home premium.   I have sys internals TCPView and ProcMonitor running to see if that can shed any light but nothing stands out.  If I didn't have / make any changes to setup I don't think the ini even comes into play.   When I started over, of course an INI is created on first launch of skyspy.   And without making any changes using the Setup button I should be getting data but I don't.. just the packets saying its talking to the gw!   

I've tried totally removing ssserver and it's related folder contents to a zip.  I've then ran the uninstaller to completely remove SkyService.  I manually deleted any left over files including those in the \programdata\skyspydata folder.  I 've reboot my pc and the gw.   And after reinstalling I once again see the monitor getting the pings, sdp, etc but nothing more.   I'm officially at not sure where to go next.  I've done computer support for over 15 years and have been around since my Vic-20 but this one I'm stumped!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 08:57:04 PM
Quote
I'm running Windows 7 64 bit home premium.

Do you have a plain vanilla installation?  No SSDD for a C: drive, Windows installed on C:, no VMs or tightened down user privileges?

SkySpy needs to write its INI file when you make changes to settings. If you have really strict user privs then it may not be able to save changes.

You might try running Monitor as Admin until I figure out what to do with the INI file.

I really really wanted to use the registry instead of an INI file, but it was not possible due to the service.  I would need to use the HKey Local Machine (HKLM) part of the registry and not everyone can write to that.  The service needs to use HKLM to be able to operate when nobody is logged into Windows, but the PC is still running.

SkySpy also needs access to C:\ProgramData\SkySpy

It will create that folder and some sub-directories if it does not exist (ala XP).  It keeps its embedded database files there.

Hmmm.  Maybe I'll put the INI file there for one-stop-shopping reasons! :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 09:10:40 PM
Yep no odd / non-standard configs.  I'm running it and logged in as myself with admin rights.   The ini file isn't locked down by any means and when I do make setup changes in the ssMonitor it does write these changes to the ini.   I was watching the files in the  C:\ProgramData\SkySpy and they do update (timestamp) especially the ibdata1 file which grows in size  as the ssmonitor / service is running.   ](*,)  this is a challenge but I'm sure will get it somehow, someway and soon.    Btw, 90 minutes north of my location received a nice late season snowstorm.  Glad we didn't get in on that because I'm done for winter this year! ha

btw in the ssMonitor under the Logging tab.  Anyway we can use this to get some log information that may help?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 16, 2014, 09:40:52 PM
Yep no odd / non-standard configs. 

There won't be much in the logs, but you can click the Help tab on the main ribbon in Monitor and then click on the Tech Support button.  Fill in the title: something like No Sensor Readings.  Fill in some sort of short description.

You can send me other debug files by dragging and dropping them on the Grid below the Description field.

You can select the priority of the issue using the radio buttons.  The appropriate priority would be Immediate in your case.

Click OK and SkySpy will zip all the files up and post the whole works as a bug report on my Mantis server. 

The progress bar will update as the report is transmitted.

Cool huh?!

You can do the same thing with feature requests by clicking the... Wait for it... The feature request button!

I am adding more debug code to log some behind-the-scenes stuff that may help with your problem.  I am beginning to think you have a problem with Named Pipes.  There is a separate pipe for the service to send sensor readings to the monitor display, but this should not affect WU updates even if it is broken.

Yet another named pipe sends debug information from Service to Monitor.  That one appears to be working.  Weird!

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 16, 2014, 09:48:42 PM
That is cool!  Wow!   I just submitted one.  I'll name my pipes whatever you want to get this working !   :-P
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 17, 2014, 08:49:11 PM
Good news!   I can never stop troubleshooting.   I decided to install mySQL and configured it with a root and user acct this evening on my home pc.    I have never used mySQL.   I use SQL Server 2008 at work a lot but have never had the need to use / learn mySQL even though they are similar in many ways.    I ran your script to build the tables and such.    I restarted the service and launched the monitor.  I configured it for Use Embedded mySQL but after about 10 minutes nothing was different.  After playing around with mySQL I'm sure I don't have the acct setup correctly yet.  I learn that later :-)   But anyways, I unchecked the Use Embedded mySQL checkbox and within minutes BOOM!  Data!!!!      So it appears just by having mySQL installed it fixed the issue even though I'm currently NOT using the Embedded mySQL option.  Whoo hooooooo
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 17, 2014, 09:58:04 PM
So it appears just by having mySQL installed it fixed the issue even though I'm currently NOT using the Embedded mySQL option.  Whoo hooooooo

I am glad you got it working, but I am a bit skeptical that the MySQL install was the fix.  There is probably a threading problem with the SkySpy code or maybe a problem with named pipes (a way to transfer information between apps running under windows).

Right now it is not possible to use any other MySQL server except mine due to some missing tables and records that are not part of the script I have been sending out.  A whole schema containing multiple tables is missing from the current script.

I am working on a sanitized MySQL script that has the minimum tables and records you would need to run your own database.  I hope to release that soon.  I also need to make some changes to the code to be able to work in this abbreviated MySQL environment.

I am going to have to duck out of here for a couple days.  I have step-daughter and grandkids visiting so I have to go play granddad to a couple little boys.  We are having big fun on my hobby farm, playing with ducklings, chicks and bunnies - not much software development going on :-)

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 18, 2014, 12:28:56 AM

I am going to have to duck out of here for a couple days.  I have step-daughter and grandkids visiting so I have to go play granddad to a couple little boys.  We are having big fun on my hobby farm, playing with ducklings, chicks and bunnies - not much software development going on :-)

Have a great Easter Weekend.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on April 19, 2014, 09:24:55 PM
Yes I hope that all are having a great Easter weekend. I know that I am going to have fun with the grandkids tomorrow.

Just thought that I would ad a note of what I have found in installing the new SkySpyService I have Installed on two machine at home a (XP home) machine and at my Vacation Home a (win 7 pro) machine and on both machines I could not get any reading in the SkySpy Monitor last packet type or last packet time windows. I kept reading the post here and installed and uninstalled made sure that the settings looked  right in the SkySpy config file. but no success finally I installed with window firewall turned off and bingo last packet type or last packet time windows started updating right away.

Running in embedded mode still no data in the current readings windows and no data on the mini web page. after 30 minutes. So Un-checked Use Embedded MySQL checkbox. Restated SkySpy Monitor running my accounts on Skydvrz's MySQL server and bingo current data updated in minutes. :grin: :grin:

If I turn window firewall back on it stops working so my question is what exceptions should I set in the firewall to be able to turn it back on.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 21, 2014, 10:15:51 PM
Here is a new SkySpy release:

https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe (https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe)

I have fixed a few bugs and improved the service debug viewer. 

I have also added two SQL files to the release for people that want to host their own (or buddy up with someone who does) MySQL database server.  You can find the SQL scripts in the install directory.  You need to run both the "weather" and "utility" schemas. 

If you want to host multiple SkySpy users, simply create a MySQL account for each person and give them external access using that account.  You also need to create multiple accounts if you want to store sensor readings from more than one physical location (home  + vacation cottage, Etc).

SkySpy will autocreate a station ID number for them, the first time they log in and submit sensor data.  The Station ID is used to separate different user's sensor readings from each other.

If you just want to host MySQL on your home LAN, then just use your existing account.

If you don't want to mess with MySQL, then configure SkySpy to use the embedded server, with limited features.

The install manual (unchanged since the last version) can be found here:

https://www.dropbox.com/s/brnz7jesdvvbmg5/ReadMe.pdf (https://www.dropbox.com/s/brnz7jesdvvbmg5/ReadMe.pdf)

I will be shutting down public access to my database server soon, so please test the database features and make sure they are working for you.

Enjoy!

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 22, 2014, 09:58:27 AM
Any suggestions if currently running the prior version before installing this version?   Do we need to uninstall it first?  Can we upgrade on top of the existing?  If so, should we stop the windows service?  Thanks again for the great work!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 22, 2014, 10:20:52 AM
Any suggestions if currently running the prior version before installing this version?   Do we need to uninstall it first?  Can we upgrade on top of the existing?  If so, should we stop the windows service?  Thanks again for the great work!

If you were running the previous "service" version, then you can just install on top of the old version.  You can also uninstall the old version first if you like.  Both the installer and deinstaller tell the service to stop prior to removing or replacing it.

For the lurkers:  If you are running the non-service, desktop version then please shut it down prior to installing the service version.  Do not try to run both the old desktop version and the service at the same time.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 22, 2014, 02:24:23 PM
 =D&gt;      =D&gt;      =D&gt;      =D&gt;      =D&gt;

Skyspy is now running totally on my system.

But . . . .

There still seems to be a problem with the ini file.  I did a complete uninstall of the previous version renamed the skyspy.ini file.  This was the only file in the folder.  Installed the latest version of the server and after installation there was no skyspy.ini file in the folder.

Monitor came up and already had my username and info about skudvrz server.  Logged in.  No data changed on display.  Checked the rest of setup  (The rest is from because it's running so nice now, I don't want to go back and mess with it).  External display info was default.  Mini web contained the settings that were in there from the previous version.  Clicked OK. and exited program.  Still no skyspy.ini.

Stopped service copy old ini to skyspy.ini. Restarted service and monitor.  Started to collect data.

Since there wasn't an ini file in the earlier non service versions, I'm guessing that username, server address and port may have been kept in the registery. I did a search of the registery and could not find the username although there were references to ssServer and ssMonitor.

I had the same problems getting it to report to my database rather than skydvrz.  Changed the address when monitor first stared up.  Checked in setup the address was still the remote server. Manually changed the server address and clicked OK. Checked ini, changes not included. Stopped server, manually changed ini file and saved it.  Restarted service.  Everything seems to be working now.

Whew!    :???:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 22, 2014, 04:01:56 PM
Sorry!  I forgot to mention that the SkySpy.ini file is now kept in

c:\programdata\SkySpy

along with all the files necessary for embedded mode.  This includes tables and internal schema files used by the simplified embedded MySQL server.  Many of the files found there are autocreated by MySQL the first time you use embedded mode.

The INI file is no longer stored in Program Files folder, as this is not an appropriate place for writable files if you don't have admin privileges.  It may or may not be deleted by the latest installer.  I will have to check on that.

The login screen probably still uses the registry, but all other information is kept in the INI.

Note that the service re-checks the INI file every minute or two.  It loads in all the new settings found there just before it does anything with the database, sends data to WU, displays the mini-web page, Etc.  It then promptly forgets them until the next time it needs to wake up.

This allows you to make changes in Monitor and have settings changes picked up by the service in near real-time.

You won't see any major changes until the service detects an SDP - that is when all the heavy lifting happens.  This type of packet occurs about once every 3-5 minutes.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 22, 2014, 04:44:30 PM
Sorry!  I forgot to mention that the SkySpy.ini file is now kept in

c:\programdata\SkySpy


Ahhhhhhh!

Now some of it makes sense.
 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 23, 2014, 11:57:46 AM
I have started working on the "GAS" feature of SkySpy Monitor.  It will replace the LaCrosse GAS utility, allowing you to switch between SkySpy and the LaCrosse web site (although I don't know why you would want to do that)  8-)

It will let new SkySpy users to do one-stop-shopping when configuring their weather station for use with SkySpy for the first time.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on April 23, 2014, 02:59:44 PM
I have started working on the "GAS" feature of SkySpy Monitor.  It will replace the LaCrosse GAS utility, allowing you to switch between SkySpy and the LaCrosse web site (although I don't know why you would want to do that)  8-)

It will let new SkySpy users to do one-stop-shopping when configuring their weather station for use with SkySpy for the first time.


Again I want to Thank you for all the hard work that you have put into this

I am new to MySql and need some help ](*,) I downloaded and installed mysql 5.6.17 and ran the "weather" and "utility" schemas (does it matter which one you run first) And then Installed the new skyspyserver and I kind of have it working the skyspy service it sending the data to my sql server but if I try to log into it with SkySpy Monitor it send a error report (not sure where it is storing these error reports) but finally it logs in but does not show anything in the in the service info window and no current sensor readings show up but if I go to my mini web page it shows my new station id and the current data that has been log for today. And Wu web site is updating also.

Any help that anyone can give me would be greatly appreciated.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 23, 2014, 04:12:12 PM
Were you using skydrvz server initially?  If so, when you start the Monitor program did you change the server address there? Click the button on the left (Advanced?). Change that to your MYSQL server address.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 23, 2014, 04:22:31 PM
Will it screw anything up to have more than one instance of the monitor program running.  If I wanted the monitor running on my desktop as well as/or rather than on the server, will it work?

Tried it and this is what I found out so far.

I just moved the skyspy program folder and ran the monitor program, logged in the first time but when I tried to change the setup everything resulted in an error to skydvrz.  Killed to program and move the skyspydata folder logged in the my server and this is what I have:

Service Readings and Service Debug on Monitor screen do not change, however Historical reading and Debug do update.  Does the server have to be running for service reading to update?

Going to try an install to see what happens.

That didn't work either.  Is this something that can be done??  Placing on debug site.


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on April 23, 2014, 05:03:13 PM
Were you using skydrvz server initially?  If so, when you start the Monitor program did you change the server address there? Click the button on the left (Advanced?). Change that to your MYSQL server address.
Yes I was using skydrvz server initially and yes I changed the server address to my server address by clicking the Advanced button. And that is when I get the error report on logging in.

I then closed the monitor and opened it again and changed back to skydvrz server and logged in and when it opens the ping data shows up and pretty soon the current data shows up and if I click on the graph or the history tab it load the data on skydrvrx server and that data stops at the time when I installed the new skyserver.

I then closed the monitor and opened it again and changed back to logging into my server and get the error report  and when it logs in still no ping data or current data but if I click on the graph tab it sends a error report but then load a graph of my data from when I installed the new skyserver but if I go to the history tab it only shows the error report and then no data.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on April 23, 2014, 05:12:09 PM


I then closed the monitor and opened it again and changed back to logging into my server and get the error report  and when it logs in still no ping data or current data but if I click on the graph tab it sends a error report but then load a graph of my data from when I installed the new skyserver but if I go to the history tab it only shows the error report and then no data.

You'll have to wait for skydvrz to to check the error log.  He may even be able to answer off the top of his head.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 23, 2014, 09:08:55 PM
There are some bugs in the login code that I need to fix - actually they are already fixed.  Unfortunately, I have the "hood up" on a big chunk of ssMonitor right now and can't ship the bug fix without finishing that part. 

That plus I have 60 baby chicks (I own a hobby farm) coming in the US Mail tomorrow, so I need to get ready for that - did I mention it is p*ssing down rain right now and I have to get outside chicken brooders set up tonight?

The quick fix is to switch back to my server for a day or two - or use embedded mode.  I have all the correct tables and record values for the utility schema that some of you don't have.  That is why ssMonitor works when connected to my server.

Anyway, I have a lot of mud to slog through tonight, so I better get running.  :-|
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 23, 2014, 10:30:13 PM
Ok, setting up brooders in the rain and mud took less time than I figured.    \:D/ 

Here is the next release (untested) that should address the issues I saw today in the automated bug reports.

https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe (https://www.dropbox.com/s/omy7kmg4huhlwe8/SkySpyInstall.exe)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 23, 2014, 11:52:28 PM
Good Evening!   Tried the latest version from late tonight but I couldn't get it to log into my mysql instance at all or provide service readings and service debug information. (using either mySQL instance or the embedded)  Reverted back to 1.1.0.71 and  it once again allowed me to login using my localhost instance of mysql and also populated the current senser readings once again.  WU continued to work though with either version installed.   Like what I'm seeing.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 24, 2014, 12:33:07 AM
Hmmm.  Back to the drawing board.  Too many interruptions from farm animals tonight, apparently.  I live a complicated life.

I will reassess in the morning.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on April 24, 2014, 02:11:47 AM
skydvrz, I hope you're having fun doing this because it sure seems hard :)

I still haven't got this working on my Win 8.1 box. The ss monitor never shows any data except pings, packet requests and so on. The mini web server doesn't show anything. However WU did get two updates earlier this evening. I've tried both the embedded server and mysql.

I noticed that I could only get ping requests if Windows Firewall was completely disabled. That didn't seem right so I took a look at the connection logs. First thing I noticed is that when the firewall was enabled, the listing for skyspy was garbled with chinese characters. Once I wrote a rule for it with the actual name, it started showing activity. Looking back over the past couple of hours, I see outbound traffic originating on a variety of ports but all headed to 127.0.0.1 port 1000. Inbound traffic is headed to port 8000 from the gateway.

Does that 127.0.0.1:1000 seem right to you?

Also when skyspy is running, the C84612 display doesn't show that it's connected to the internet.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 24, 2014, 10:33:11 AM
skydvrz, I hope you're having fun doing this because it sure seems hard :)

Harder than Chinese Calculus, I am here to tell ya.  I have been working on this since before Christmas in almost all of my free time.  Yes, it is fun.  :lol:

Quote
I still haven't got this working on my Win 8.1 box. The ss monitor never shows any data except pings, packet requests and so on. The mini web server doesn't show anything. However WU did get two updates earlier this evening. I've tried both the embedded server and mysql.

Don't forget that WU updates only happen every 4-15 minutes.  That is just how fast the GW goes.  Watch a couple of SDP packets go by before panicking.  Are you seeing WU events in Monitor?  They should occur right after SDP events.

Quote
I noticed that I could only get ping requests if Windows Firewall was completely disabled. That didn't seem right so I took a look at the connection logs. First thing I noticed is that when the firewall was enabled, the listing for skyspy was garbled with chinese characters. Once I wrote a rule for it with the actual name, it started showing activity. Looking back over the past couple of hours, I see outbound traffic originating on a variety of ports but all headed to 127.0.0.1 port 1000. Inbound traffic is headed to port 8000 from the gateway.

Does that 127.0.0.1:1000 seem right to you?

SkySpyService needs to communicate with the LaCrosse Gateway module over port 8000.  It sends and receives on this port.  It uses this same port for the mini-web site.  It also uses port 80 to send me automated bug reports and to send WU weather data.

SkySpy Monitor uses port 80 to send me automated bug reports.  It will use the Gateway Configuration port in the next release.

Both the service and monitor may use port 3306 or 3307 to communicate with MySQL, but only if you are not using Embedded mode.  Port 3307 is only used it you have an account on my MySQL database server.  Normally MySQL works over port 3306.

So...  Make sure you set up your firewall rules accordingly. 

I don't know what is going on with 127.0.0.1:1000.  127.0.0.1 is the generic address for "this computer" or "localhost".  Port 1000 is not used by SkySpy AFAIK.

Quote
Also when skyspy is running, the C84612 display doesn't show that it's connected to the internet.

Yes, I know.  It doesn't hurt anything though. 

Normally the GW asks the LaCrosse server for the current time using a special handshake that I have not figured out yet.  When the LCX server replies with the time, the GW turns on the Internet indicator on the C84612 and resets the clock to the correct time.  I am still researching the problem.

Your C84612 clock display will start to drift over time until I fix this.  SkySpy uses your computer clock to tag WU sensor readings with local time, so WU will not be affected.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: markmanxp on April 24, 2014, 11:56:25 AM
A few observations this morning.

Looking at the SkyMonitor it shows version 1.1.0.71
Looking at the Local Web Server it shows version 1.2.0.43.   Must not have downgraded after I reinstalled an earlier version last night.

It has been raining most of the night and morning.

SkyMonitor shows 0.00 for both Rainfall 1 hour and RainFall Today.   Weather Underground has been receiving correct data it appears, so that is GOOD!   Looking at the local web version it is showing the Total Rainfall field being populated and incrementing as the rain fell.

Looking at the tables in mySQL instance it is getting data but looking at the sensorvalues table there are no entries with Sid values between 10-13 which from what I can derive by looking at Sensors table are the rain values (9 - rainfall total, 10 - rainfall 1H, 11 - rainfall 24H, 12 - Rainfall Wk , 13 - rainfrall Mo).   Am I seeing this correctly or not understanding your table design.   Also loooking at the database 'weather' in MySQL Workbench is there a way to see the relationships between the different tables (primarykey - foreign key like in a design view?)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 24, 2014, 01:09:22 PM
A few observations this morning.

Looking at the SkyMonitor it shows version 1.1.0.71
Looking at the Local Web Server it shows version 1.2.0.43.   Must not have downgraded after I reinstalled an earlier version last night.

Those are the current versions of SkySpy Service (1.2.0.43) and SkySpy Monitor (1.1.0.71).  The Service portion runs in the background on your computer.  Monitor is used to watch and or configure the Service.  You can watch the service on WU just as well.  Once the service is configured, you really don't need to use ssMonitor any more.

It is the Service that talks to WU, so you will see the service version if you click on the "About this PWS" on your WU weather station page.

Quote
It has been raining most of the night and morning.

SkyMonitor shows 0.00 for both Rainfall 1 hour and RainFall Today.   Weather Underground has been receiving correct data it appears, so that is GOOD!   Looking at the local web version it is showing the Total Rainfall field being populated and incrementing as the rain fell.

I will have to take a look.  Unfortunately, my rain gauge died and I have not had a chance to do surgery on it.  It will be difficult to debug the software without some sensor inputs.  It has been working OK at other locations.

Quote
Looking at the tables in mySQL instance it is getting data but looking at the sensorvalues table there are no entries with Sid values between 10-13 which from what I can derive by looking at Sensors table are the rain values (9 - rainfall total, 10 - rainfall 1H, 11 - rainfall 24H, 12 - Rainfall Wk , 13 - rainfrall Mo).   Am I seeing this correctly or not understanding your table design.   Also loooking at the database 'weather' in MySQL Workbench is there a way to see the relationships between the different tables (primarykey - foreign key like in a design view?)

Rainfall measurement is a strange duck with the C84612.  It is measured as millimeters since the last reset.  I save the current value in inches in the DB, but calculate the rainfall per hour using a SQL query that calculates the height increase in the desired time period.  I may not be updating Monitor correctly, but I imagine that WU is correct.  I wrote the function listed below specifically for WU. 

Here is the SQL query method I use to access the rainfall records in MySQL.  You can manually enter the same query to calculate rainfall in a given period:

Code: [Select]
FUNCTION TsssDm.RainFall(Hours : Integer) : Single;
VAR
    Q : TQ;
BEGIN
    Q := TQ.Create(Self); // create a query component on the heap
    TRY
        Q.Add('SELECT max(value) - min(Value) as rf'); // build the query
        Q.Add('FROM packets p, sensorvalues sv');
        Q.Add('where p.stationid=:stationid and sv.id=p.id and sv.sid=:RFID and DATE_ADD(p.Timestamp, INTERVAL :hours HOUR) >= now()');
        Q.SVP('stationid', OprLoginId);     // set the SQL query parameters
        Q.SVP('rfid', ORD(SRainInTot));    // use the total rainfall sensorvalues records; rfid=9
        Q.SVP('hours', IntToStr(Hours));  // number of hours to use
        Q.Open; // execute the query
        Result := VDQ(Q, 'rf', 0); // get the calculated rainfall for the requested period. Use Zero inches if no matching records
    FINALLY
        Q.Free; // free up memory
    END;
END;

Note that this algorithm will break if you reset your rain inches total on the C84612 and then try to calculate across the cusp.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 24, 2014, 01:44:40 PM
I did a quick review of the code.  The only rainfall value passed to Monitor is the current inches of total rain since the last reset.  The 1Hr value shown in Monitor will always be zero, until I finish this code.

As I said before, SkySpy Service sends WU the correct rainfall / Hr information.

Also, all of the sensor dates (min/max reading times) are currently shut off.  They aren't very useful, so I don't want to spend a lot of time messing with them.  WU calculates all this stuff  for you from your raw data, so there is not much sense in reinventing the wheel.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 24, 2014, 03:15:29 PM
Since there's discussion on measuring rainfall, I figured it's probably about time to send my layout of fields in the SDP. I've converted the code to using nybbles, but the spreadsheet shows both nybbles and bytes. See the notes at the bottom of the spreadsheet for comments about rainfall measurements. There is still more data in the SDP that I haven't identified, but I'm working on that.

Also, I've been working on checksums, since the console may ignore date/time updates unless it sees the right checksum. The fact there are checksums at all on the TCP data probably means the gateway itself isn't manipulating the data, and the checksums are there for the wireless link between console and gateway. I've figured out the checksum for the SDP (a 16-bit crc). but I haven't cracked it for the other messages. Stay tuned.

The spreadsheet for the field layout is here:
http://www.keckec.com/files/sdp_layout.xls
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on April 24, 2014, 04:47:03 PM
I gave this another try and got the same result. After a while, WU would get one update and then stop. I left the firewall open overnight and there was only the one update.

I took a closer look at the connections log for Windows firewall. I found skyspyservice only connected outbound to these two ips:

129.0.0.1 port 1000
24.18.61.97 port 80 (skydvrz)

I looked for any traffic from skyspysrvice to WU (38.102.136.104) on any port and didn't see any. What would be more definitive is if I were to see a connection when there was a WU update, but my logs don't go back that far.

As far as inbound activity, for skyspyservice, most of it was from 192.168.1.119 (the gateway). There were two connections from 127.0.0.1 ports 52256 and 52260.

The only inbound traffic to port 8000 was from the gateway. The only outbound traffic to 127.0.0.1:8000 was from Chrome.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 24, 2014, 08:23:24 PM
If you are a SkySpy user that has been successfully storing weather data to a MySQL database under your control, or are using Embedded mode then please disregard this message.

For users that are trying to connect to their own MySQL database server and have problems logging in or actually storing readings: Please back up your MySQL server, shut ssMonitor down and then run this script:

https://www.dropbox.com/s/tj2ydir756a1sa8/utility.sql (https://www.dropbox.com/s/tj2ydir756a1sa8/utility.sql)

The script will create the missing Utility schema/tables on your MySQL server. If you already have this schema, the script will delete it and then recreate an empty one.  It should retain any readings stored with the previous Utility schema, but a backup is never a bad thing when making big changes to a database.




Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on April 25, 2014, 12:03:12 AM
Cool I am running ssMonitor v1.1.0.72 I just ran the utility.sql and now ssMonitor logs in fine and ping data shows up and so does current data  history tab and graghs now work Thanks Skydvrz =D&gt; =D&gt; =D&gt; =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 25, 2014, 12:33:26 PM
MtHighVs,

Good news!  I am rewriting both the service and monitor to autocreate missing schemas.  That should make things simpler for new users.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on April 25, 2014, 12:43:20 PM
The new script worked like magic. Thanks!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 26, 2014, 10:56:29 PM
The new script worked like magic. Thanks!

I have noticed a distinct lack of bug reports on my Mantis server, so I guess the new script fixed most of the problems.  Phew!   :grin:

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 26, 2014, 11:28:10 PM
Success!!! Finally got the time to sync and the Internet annunciator lit on the console display.  \:D/

All it took (I think) is the correct checksum in the last 2 bytes of the 38-byte 14:01 packet. I initially had the 8-byte station id wrong. Once that was right, the console beeped once and its clock was synced.

The console/gateway has now gone into another mode, sending 210-byte packets about every 2 seconds, along with the 197-byte SDP once every 4 or so minutes as before. I saw this in my captures when it was talking to the LCX servers. Haven't tried to decode this 210-byte packet yet. I see what look like dates and times that are advancing each packet. It like might be sending historical data, to bring the server up to date?

More decoding to do...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 27, 2014, 12:41:05 PM
All it took (I think) is the correct checksum in the last 2 bytes of the 38-byte 14:01 packet. I initially had the 8-byte station id wrong. Once that was right, the console beeped once and its clock was synced.

Station ID?  What fields/offsets are you talking about?

I have updated SkySpy Service to use a 16 bit additive checksum.  I assume you meant additive and not CRC16.

Quote
The console/gateway has now gone into another mode, sending 210-byte packets about every 2 seconds, along with the 197-byte SDP once every 4 or so minutes as before. I saw this in my captures when it was talking to the LCX servers. Haven't tried to decode this 210-byte packet yet. I see what look like dates and times that are advancing each packet. It like might be sending historical data, to bring the server up to date?

This may be alarm packets sent by the console.  You can configure the console to trigger on various min/max sensor values.  I shut all of mine off, since the packets were not very useful (at the time).
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 27, 2014, 02:28:41 PM
Station ID?  What fields/offsets are you talking about?

This is the 8-byte field that starts at byte 1 (counting from 0) in the 38-byte 14:01 packet type that sets the clock. The ID starts with 0x7f 0xff as the first two bytes. From Mycal's instructions it has to match the ID of the gateway/console. I copied my ID from a capture of data coming from the weatherdirect servers. According to Mycal, it's an ID field that gets stored in the console/gateway after being sent from the server during the registration process. Once set, I'm not sure if there is a way to reset or change it.

Quote
I have updated SkySpy Service to use a 16 bit additive checksum.  I assume you meant additive and not CRC16.

Yes. It is a 16-bit sum (not crc) of the first 36 bytes in the packet, and is placed big-endian at the end of the packet to make 38 bytes.

Quote
This may be alarm packets sent by the console.  You can configure the console to trigger on various min/max sensor values.  I shut all of mine off, since the packets were not very useful (at the time).
My alarms are all off as well, so I don't think it's that. In the capture while connected to weatherdirect, it sent these packets for about 15 minutes right after initial registration, then never again.

I'm a little concerned all this RF traffic might decrease battery life of the console, since it's sending data every 2 seconds instead of 4 minutes. I'll look to see if there is something else in the original capture that might convince the console to give it a rest.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 27, 2014, 04:40:23 PM
This is the 8-byte field that starts at byte 1 (counting from 0) in the 38-byte 14:01 packet type that sets the clock. The ID starts with 0x7f 0xff as the first two bytes. From Mycal's instructions it has to match the ID of the gateway/console. I copied my ID from a capture of data coming from the weatherdirect servers. According to Mycal, it's an ID field that gets stored in the console/gateway after being sent from the server during the registration process. Once set, I'm not sure if there is a way to reset or change it.

Ok, I implemented that.  I just parrot back the account ID from the HTML header after replacing the 2 leading bytes with 7FFF. 

Quote
Yes. It is a 16-bit sum (not crc) of the first 36 bytes in the packet, and is placed big-endian at the end of the packet to make 38 bytes.

Did that....  Its not working though.

Here is my current RTC response packet structure:

Code: [Select]
    TDateArray       = ARRAY[0..2] OF Byte;  // YMD
    TServerTimeOfDay = ARRAY[0..2] OF Byte;  // includes seconds
    TTimeOfDay       = ARRAY[0..1] OF Byte;  // HM

    // server sends local date/time to GW in this format
    TRTCRec = PACKED RECORD
        ServerTime: TServerTimeOfDay;
        ServerDate: TDateArray;
    END;

    TResp0100Packet = PACKED RECORD          // 38 byte reply from server
        UnknownByte:    Byte;                // $0      either $1 $2
        AcctID:         UInt64;              // $1-$8   some sort of registration ID?
        Unknown4:       ARRAY[1..7] OF Byte; // $9-$F   invariant in all the dumps I have seen
        Unknown5:       ARRAY[1..6] OF Byte; // $10-$15 invariant
        Epoch:          word;                // $16-$17
        ServerDateTime: TRTCRec;             // $18-$1D backwards from normal date/time arrangement
        Unknown6:       Word;                // $1E-$1F $5407 on mine, different on other gateways
        Unknown7:       LongWord;            // $20-$23
        Checksum:       word;                // $24-$25
    END;

Are you doing anything with the epoch bytes?  Epoch increments slowly over several weeks/months until it wraps at 7FFF

Quote
This may be alarm packets sent by the console.  You can configure the console to trigger on various min/max sensor values.  I shut all of mine off, since the packets were not very useful (at the time).

Some of them are not so obvious - you have to go through the full settings on the LCD monitor to disable them all.

Quote
My alarms are all off as well, so I don't think it's that. In the capture while connected to weatherdirect, it sent these packets for about 15 minutes right after initial registration, then never again.

I don't know.  The extra packets are probably not useful to the project, so I don't think it is worth taking up a lot of time analyzing them.

Quote
I'm a little concerned all this RF traffic might decrease battery life of the console, since it's sending data every 2 seconds instead of 4 minutes. I'll look to see if there is something else in the original capture that might convince the console to give it a rest.

If the packet storm only happens on initial Internet connection or after a registration, then it is probably no big deal.  That said, I had to replace my outside temp/humidity sensor batteries today, after only 4 months of use.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 27, 2014, 05:06:37 PM
BTW, Epoch increments by 18 (base 10, 0x12 hex) every 2 hours.  I have no idea what it does, but maybe if you don't increment it properly, the Internet indicator shuts off?

Does your RTC update work past a 2 hour timeframe?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 28, 2014, 03:18:34 PM
Ok, I implemented that.  I just parrot back the account ID from the HTML header after replacing the 2 leading bytes with 7FFF. 
I tried that but it didn't work for me. The value that did work was contained in the 38-byte 14:01 clock-set packets from weatherdirect. It was also contained in a single 14:00 packet that came from weatherdirect before any of the 14:01 packets. Mycal thinks this is what actually sets that ID in the console/gateway. That makes some sense because after the 14:00 packet, the GW immediately responded with a 01:14 packet (confirmation?) containing that same ID. Then the server answered with a 1C:00 packet. Mycal thinks the 1C:00 packet is to "seal the deal" on registration.

Other than the above packets, I don't see the ID that worked anywhere else in my captures. This may be a problem if you (or your users) don't have any captures from which to grab that ID. What may work is to somehow reset the ID in the console, then either go through the registration process again with weatherdirect while capturing packets. You could make Skyspy into a full proxy, then go through the registration process again and grab the ID as it goes by. Or you could reset your console and see if you can force an ID code into the console from Skyspy. Maybe the "factory reset" described in the manual would reset the console's ID.

Quote
Did that....  Its not working though.
Here is my current RTC response packet structure:

Here's mine:
Code: [Select]
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d|0e|0f|10|11|12|13|14|15|16|17|18|19|1a|1b|1c|1d|1e|1f
00 - |01|7f|ff|14|76|3c|3b|5b|61|00|32|00|0b|00|00|00|0f|00|00|00|03|00|3e|de|09|47|11|28|04|14|53|07
20 - |04|00|00|00|05|ad

Quote
Are you doing anything with the epoch bytes?  Epoch increments slowly over several weeks/months until it wraps at 7FFF

I haven't yet but I just had a look at what they were in the captures when connected to weatherdirect. I do see them incrementing by 0x12 (hex, not decimal), sometimes every 2 hours, but not at other times. And in the first 15 minutes after registration, these two bytes jumped a lot. Always increasing, but by large values, and always a multiple of 0x12. Once it increased by 0x2520 in 3 minutes time. The capture spans the change to DST, so that may add oddities. I think time zone and DST info are probably embedded in there somewhere.

Right after registration the GW sent out 01:00 (time request) packets every couple minutes (8 over 15 minutes), and the server responded with 14:01 packets. I think this makes sense. Unless they implement something like NTP that can deal with latency, and packet loss and retransmission, they'd have to get a number of clock-set packets to be sure they have the time right. After the first 15 minutes, it settled down to 3/hour or so.

That 15-minutes coincided with the storm of 210-byte 01:01 packets every 2 seconds.

Quote
If the packet storm only happens on initial Internet connection or after a registration, then it is probably no big deal.  That said, I had to replace my outside temp/humidity sensor batteries today, after only 4 months of use.
Trouble is, the 01:01 packets every 2 seconds hasn't stopped in 4 days. I think something must not be happy with what I'm sending it in the 38-byte clock-set packets, or that it's not getting a proper confirmation somewhere.

Here is a link to my two capture files, which starts when I first plugged the GW into the LAN, and goes through the registration process. There is a gap between the files when I accidentally stopped the capture and restarted it again. The time change to DST occurs during the second capture file, and there's some funny business in the times around there that I haven't figured out yet.
http://www.keckec.com/weather/files/capture_2014-03-08.zip

Here is the PHP file as I have it now. Much of it is from mycal. This is a work in progress, and a lot of clean-up is needed. This is my first time doing any PHP code, so go easy.  :-)
http://www.keckec.com/weather/request.breq
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 29, 2014, 12:38:44 AM
Ok, I implemented that.  I just parrot back the account ID from the HTML header after replacing the 2 leading bytes with 7FFF. 
I tried that but it didn't work for me. The value that did work was contained in the 38-byte 14:01 clock-set packets from weatherdirect. It was also contained in a single 14:00 packet that came from weatherdirect before any of the 14:01 packets. Mycal thinks this is what actually sets that ID in the console/gateway.

Hmmm.  Something funny is going on.  I implemented your 38 byte packet, letter perfect (using my LCX-assigned StationID) and it does not reset the RTC or light up the Internet indicator on my console.  I even checked back in to the La Crosse server to capture some authentic packets and compared SkySpy packets to LCX.  They are identical....  There is something else that we are missing.

Those 210-byte packets on initial connection to the LCX server are really scary.

Anyway, as a sanity check, I have been capturing SkySpy traffic to the GW traffic to the LCX server, just to make sure there is no corruption.  AFAICT, there is none.

My guess is the 5 byte packet, the ping response or something else modifies a state machine in the GW.

I have not reviewed your PHP code - I was busy with the 38 byte packet tonight - Maybe there are some clues there.  I will try to devote some time to it tomorrow.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 29, 2014, 02:20:18 AM
Hmmm.  Something funny is going on.  I implemented your 38 byte packet, letter perfect (using my LCX-assigned StationID) and it does not reset the RTC or light up the Internet indicator on my console.  I even checked back in to the La Crosse server to capture some authentic packets and compared SkySpy packets to LCX.  They are identical....  There is something else that we are missing.
That is odd. I implemented the epoch code, but it didn't seem to make a difference. I verified that the console's time is being set from the 38-byte packets -- if I change the console time with the SET button, it changes back after a few seconds. Then I changed what time is being sent in the 38-byte packet, and the console time followed it.

Quote
My guess is the 5 byte packet, the ping response or something else modifies a state machine in the GW.
I've looked at that, but I don't see anything significant. Originally, mycal's php code inserted the http 'content-type' header on all outgoing packets. I disabled that when there is no content, to match what I saw from LCX, but it didn't seem to matter.

Quote
I have not reviewed your PHP code - I was busy with the 38 byte packet tonight - Maybe there are some clues there.  I will try to devote some time to it tomorrow.
Thanks for looking at that. I'm going to see if I can find anything significant in the 210-byte packets. Maybe there's clue there.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 29, 2014, 12:28:29 PM
Keckec,

I read your PHP code, but have not tried executing it yet.

Aha!

When describing your process you left out the part about offsetting the 38 byte packet checksum by 7.  I will try that tonight when I get home. 

I guess they seed the checksum generator with 7, either by bug or design.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 29, 2014, 03:32:11 PM
Aha!

When describing your process you left out the part about offsetting the 38 byte packet checksum by 7.  I will try that tonight when I get home. 

Oops. Sorry for giving you bad data. Didn't mean to waste your time on that.
k
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 29, 2014, 09:16:50 PM
Finally getting around to connect the gateway for the first time and of course I have misplaced my activation code. LaCrosse support will not give me a new one, instead I should purchase a subscription.
I plan on registering the gateway and weather station using Mycal and keckec's php file from yesterdays post. Any advice or gotchas I should look out for?
Thoughts about setting a serial number? I am not worried about never using the LaCrosse services.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 29, 2014, 10:47:33 PM
Finally getting around to connect the gateway for the first time and of course I have misplaced my activation code. LaCrosse support will not give me a new one, instead I should purchase a subscription.
I plan on registering the gateway and weather station using Mycal and keckec's php file from yesterdays post. Any advice or gotchas I should look out for?
Thoughts about setting a serial number? I am not worried about never using the LaCrosse services.
The PHP code has a place for the serial number, near the top. The gateway sends a 7F:10 packet (http header), and the php code (if enabled) sends a 14:00 packet, which contains the serial number, date & time, and some other stuff. The gateway sends a 00:14 packet, and the php code returns a 1C:00 packet. There looks like a place in the PHP code to enable this, but it's untested by me. All this is derived from captures of the registration process with LCX, and I don't think any of us have done it ourselves. You can have a look at this in my capture file here:
http://www.keckec.com/weather/files/capture_2014-03-08.zip
This file is viewable in Wireshark, available for a number of platforms. The serial number exchange starts around packet number 32.
It's possible you can make up your own serial number. It seems it has to start with 7FFF. I don't think any of us have tried making it up, and I'm not sure that just any 8-byte code will work.

You'll also have to add the following checksum code to the php code where it generates the 14:00 packet. Remove the code for the fixed checksum that's there now. Note that the offset of this checksum is different than in the 14:01 packet, which is similar.
Code: [Select]
$csum=checksum16($reply)+0x17;
$reply.=chr($csum>>8).chr($csum&0xff);

We're all very interested in your results with this.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 29, 2014, 11:45:56 PM
Perhaps I am doing something wrong, I have changed the following:

Code: [Select]
$station_serial=pack("H*" , "7fff201404292594");   
$register_this_serial=1;

Line 841 removed the comment:
Code: [Select]
if($register_this_serial)
    {
         // Do you really want to do this?
         $do_reply=1;
    }

Line 894, added comments and 2 lines:
Code: [Select]
// Checksum
//$reply=$reply.chr(checksum8($reply));
$csum=checksum16($reply)+0x17;
$reply.=chr($csum>>8).chr($csum&0xff);

Initially on the Gateway the red LED was flashing. I pushed the button after using the Gateway Advanced Setup and after a while the red LED went solid.

dump.log wrote out 3 entries:
2014-04-29 21:18:47   Packet Type 00:10 Length: 0 bytes
     No Data
2014-04-29 21:19:22   Packet Type 00:20 Length: 0 bytes
     No Data
2014-04-29 21:19:28   Packet Type 00:30 Length: 0 bytes
     No Data

I then held the Rain button on the weather station until it beeped.
REG flashed for a few seconds the there were 2 beeps and INTERNET flashed.

dump.log now has:
2014-04-29 21:19:38   Packet Type 00:70 Length: 0 bytes
     No Data

No matter what I do I only get packet types 00:70.

Stupid me, forgot to press the button on the gateway while REG is flashing on the weather station.
dump.log now also has:
2014-04-29 22:29:47   Packet Type 7F:10 Length: 13 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c
00 - |01|02|03|04|05|06|07|08|02|00|22|73|4d

Still doesn't seem to set the serial number and still stuck and looking for ideas?

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 30, 2014, 12:40:14 AM
Initially on the Gateway the red LED was flashing. I pushed the button after using the Gateway Advanced Setup and after a while the red LED went solid.

dump.log wrote out 3 entries:
2014-04-29 21:18:47   Packet Type 00:10 Length: 0 bytes
     No Data
2014-04-29 21:19:22   Packet Type 00:20 Length: 0 bytes
     No Data
2014-04-29 21:19:28   Packet Type 00:30 Length: 0 bytes
     No Data

I then held the Rain button on the weather station until it beeped.
REG flashed for a few seconds the there were 2 beeps and INTERNET flashed.

dump.log now has:
2014-04-29 21:19:38   Packet Type 00:70 Length: 0 bytes
     No Data

No matter what I do I only get packet types 00:70.

Stuck and looking for ideas?
You might try adding the following code right before the comment that says "All output created here", near line 1128 on mine but our line numbers probably differ at this point. You'll have to adjust the curly braces. This should log any other packets the gateway sends along.
Code: [Select]
    } else {
        // Log any other packets that come along
        $postdata = file_get_contents("php://input");
        write_dump_log($postdata,$id1.":".$id2);
    }
Maybe it isn't happy with the serial number you're using. You could try the serial number I used, 7fff14763c3b5b61. It's quite possible this number is derived from some other part of my initial exchange and won't work for you. Do you have Wireshark or tcpdump (Linux) available? If so you could capture the actual exchange. This is all new territory, and maybe there's something else we don't know about. Also, check the logs from your web server to make sure there isn't some other problem.

I'm out of ideas but will post back if I think of more.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 30, 2014, 12:59:28 AM
Finally got it to work:

Started off by adding $do_reply=1 above this section because I already knew I wanted to send a 14:00 packet:
Code: [Select]
$do_reply=1;
                if($do_reply)
                {
                        // Reply with 14:00 and 38 bytes of data.  This data is likely configuation of the WS
                        header('HTTP_FLAGS: 14:00');

After this it still didn't work so I left the above and set the serial manually here:
Code: [Select]
//      $reply.=$station_serial;
$reply.=chr(0x7f).chr(0xff);                                                           
$reply.=chr(0x20).chr(0x14).chr(0x04).chr(0x29).chr(0x25).chr(0x94);

Still no go so I changed the checksum back to the original:
Code: [Select]
// Checksum
    $reply=$reply.chr(checksum8($reply));
     //      $csum=checksum16($reply)+0x17;
     //      $reply.=chr($csum>>8).chr($csum&0xff);

Looks like it registered and INTERNET is showing on the WS. It also correctly set my date and time which I manually messed up first.

Contents of dump.log
Code: [Select]
2014-04-29 22:47:50   Packet Type 00:70 Length: 0 bytes
     No Data
2014-04-29 22:49:02   Packet Type 7F:10 Length: 13 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c
00 - |01|02|03|04|05|06|07|08|02|00|22|73|4d
2014-04-29 22:49:03   Packet Type 01:14 Length: 14 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d
00 - |7f|ff|20|14|04|29|25|94|02|00|22|14|5e|63
2014-04-29 22:49:06   Packet Type 01:01 Length: 197 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d|0e|0f|10|11|12|13|14|15|16|17|18|19|1a|1b|1c|1d|1e|1f
00 - |01|64|14|01|01|30|10|10|03|51|30|10|10|01|06|18|00|61|20|06|18|00|01|30|10|10|00|11|30|10|10|00
20 - |16|11|00|61|10|0a|a1|00|01|30|10|10|00|11|30|10|10|00|16|11|00|61|10|0a|aa|a0|13|01|01|00|01|13
40 - |01|01|00|35|36|29|29|13|01|01|00|01|13|01|01|00|01|32|32|aa|00|00|00|00|00|00|00|00|00|00|00|00
60 - |00|00|00|00|00|00|00|00|00|00|13|01|01|00|01|00|00|00|00|00|00|13|01|01|00|32|00|00|00|00|00|00
80 - |00|00|00|00|00|a0|00|00|00|13|01|01|00|00|00|00|00|00|00|00|55|00|bb|bb|13|01|01|00|01|00|00|00
a0 - |00|00|00|15|00|bb|bb|02|99|21|01|32|02|99|11|01|28|02|99|31|01|35|13|01|01|00|01|13|01|01|00|31
c0 - |05|53|07|e9|1f
2014-04-29 22:49:15   Packet Type 01:00 Length: 5 bytes
     |00|01|02|03|04
00 - |41|64|15|c0|f9
2014-04-29 22:49:15   Packet Type 14:01 Length: 38 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d|0e|0f|10|11|12|13|14|15|16|17|18|19|1a|1b|1c|1d|1e|1f
00 - |01|7f|ff|20|14|04|29|25|94|00|32|00|0b|00|00|00|0f|00|00|00|03|00|3e|de|10|49|15|29|04|14|53|07
20 - |04|00|00|00|05|18
2014-04-29 22:49:17   Packet Type 00:70 Length: 0 bytes
     No Data
2014-04-29 22:53:18   Packet Type 00:70 Length: 0 bytes
     No Data
2014-04-29 22:53:21   Packet Type 01:01 Length: 197 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d|0e|0f|10|11|12|13|14|15|16|17|18|19|1a|1b|1c|1d|1e|1f
00 - |01|64|14|01|01|40|42|91|05|31|30|10|10|01|06|18|00|61|20|06|17|00|01|30|10|10|00|11|30|10|10|00
20 - |16|11|00|61|10|0a|a1|00|01|30|10|10|00|11|30|10|10|00|16|11|00|61|10|0a|aa|a0|13|01|01|00|01|14
40 - |04|29|10|53|36|29|29|13|01|01|00|01|13|01|01|00|01|32|32|aa|00|00|00|00|00|00|00|00|00|00|00|00
60 - |00|00|00|00|00|00|00|00|00|00|13|01|01|00|01|00|00|00|00|00|00|14|04|29|10|52|00|00|00|00|00|00
80 - |00|00|00|00|00|a0|00|00|00|13|01|01|00|00|00|00|00|00|00|00|55|00|bb|bb|13|01|01|00|01|00|00|00
a0 - |00|00|00|15|00|bb|bb|02|99|31|01|35|02|99|11|01|28|02|99|31|01|35|13|01|01|00|01|14|04|29|10|53
c0 - |04|53|07|a4|bc
2014-04-29 22:57:19   Packet Type 00:70 Length: 0 bytes
     No Data
2014-04-29 22:57:35   Packet Type 01:01 Length: 197 bytes
     |00|01|02|03|04|05|06|07|08|09|0a|0b|0c|0d|0e|0f|10|11|12|13|14|15|16|17|18|19|1a|1b|1c|1d|1e|1f
00 - |01|64|14|01|01|40|42|91|05|41|30|10|10|01|06|18|00|61|20|06|17|00|01|30|10|10|00|11|30|10|10|00
20 - |16|11|00|61|10|0a|a1|00|01|30|10|10|00|11|30|10|10|00|16|11|00|61|10|0a|aa|a0|13|01|01|00|01|14
40 - |04|29|10|57|36|29|29|13|01|01|00|01|13|01|01|00|01|32|32|aa|00|00|00|00|00|00|00|00|00|00|00|00
60 - |00|00|00|00|00|00|00|00|00|00|13|01|01|00|01|00|00|00|00|00|00|14|04|29|10|56|00|00|00|00|00|00
80 - |00|00|00|00|00|a0|00|00|00|13|01|01|00|00|00|00|00|00|00|00|55|00|bb|bb|13|01|01|00|01|00|00|00
a0 - |00|00|00|15|00|bb|bb|02|99|31|01|35|02|99|11|01|28|02|99|31|01|35|13|01|01|00|01|14|04|29|10|57
c0 - |04|53|07|f1|8f

Thanks to everyone who has contributed so far!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 30, 2014, 08:58:56 AM
Now that everything is registered I changed the gateway to point to my windows box and to the SkySpy service so it starts collecting some data to my MySQL database.
Hopefully keckec is working on getting the php to write data to the same database.  :grin:

One thing to note for those running SkySpy in win7. When you run SkySpy Monitor right-click and run as Administrator otherwise any changes to the database settings tab will not 'stick'.

Once again a big thanks to everyone who has contributed code, test data or other knowledge to this endeavor!

 =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 30, 2014, 10:17:38 AM
Finally got it to work:
Outstanding work!!!   =D&gt;

That's a big hurdle in this process you've overcome.
Quote
After this it still didn't work so I left the above and set the serial manually here:
Code: [Select]
//      $reply.=$station_serial;
$reply.=chr(0x7f).chr(0xff);                                                           
$reply.=chr(0x20).chr(0x14).chr(0x04).chr(0x29).chr(0x25).chr(0x94);
Where did you get the serial number? Did you just make it up?

Quote
Looks like it registered and INTERNET is showing on the WS. It also correctly set my date and time which I manually messed up first.
Congratulations. Looks like you're set.

I'm curious if your gateway is sending out 210-byte 01:01 packets. I think the GW is supposed to send these out for about 15 minutes, then stop. My gateway is still sending these, about every 2 seconds, with occasional breaks for a minute or two. I turned off logging of those packets in the php file, but it might still be writing the binary contents to a file postdata210.bin. Do you see this file in the same directory as the php file? If so, does its last modification time show that it's still being written to, or was it only written to right after you registered it.

Yes, my plan is to have the php code dump data into a mysql database, and have other php code to display historical data, graphs, etc. That might be down the line a bit as I have some other projects that need attention.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on April 30, 2014, 07:42:45 PM
Quote
Where did you get the serial number? Did you just make it up?
I made it up - 7fff201404292594
7fff - Someone said they all started with this
20140429 - Yesterdays date
2594 - My address  :-)

Quote
I'm curious if your gateway is sending out 210-byte 01:01 packets.
Yes it is, in less than 10 hours the postdata210.bin file has grown to 2.2 MB
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on April 30, 2014, 08:01:35 PM
Quote
Where did you get the serial number? Did you just make it up?
I made it up - 7fff201404292594
7fff - Someone said they all started with this
20140429 - Yesterdays date
2594 - My address  :-)
That's great news. I was hoping we could choose anything. The numbers we've seen from LCX all start with 7FFF, so that may be important.
Quote
Quote
I'm curious if your gateway is sending out 210-byte 01:01 packets.
Yes it is, in less than 10 hours the postdata210.bin file has grown to 2.2 MB
You can disable saving that data if you want. It probably won't do you any good unless you want to analyze it. What it writes is the system date as a 4-byte integer, followed by the 210-byte payload from the packet. Each time your GW sends one of these it adds another 214 bytes to the file. You can turn this off by commenting out the following line near the bottom of the file:
Code: [Select]
file_put_contents($filename,$packtime.$postdata,FILE_APPEND);That line saves all the packets we haven't really figured out yet.
That the GW sends these packets really doesn't matter for the PHP code, but I think it may significantly shorten battery life of the console. I see the Internet annunciator on the console blink when it sends one of these, so I think it's probably firing up its RF transmitter to send the data to the GW. Doing that every 2 seconds instead of every 4 minutes is bound to take more power. Skydvr doesn't see this on his, so there's bound to be something that will stop it. I'm working on that.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on April 30, 2014, 10:11:35 PM
I'm still struggling with the 38 byte RTC packet.  Everything is sent correctly in the content part of the packet and I am now creating the checksum  using the 7-offset.  Still no clock reset / Internet indicator.  I even hand-calculated / double-checked the checksum on several packets.  Everything is perfect in the content.

Not so, in the HTML header.  #-o

I am beginning to think the GW HTML parser firmware is pretty dumb and can't grok header lines received out of order.  Using the HTML library I have, I don't have full control over which line each header value appears on.  All the lines are in there just fine, just not in the normal IIS order.

I can reconnect to LCX and the clock resets within a minute or two.  Using SkySpy, the Internet indicator shuts off and the clock begins to drift (very slowly).

I will look into alternate setups of my TCP/IP & HTML libraries (Internet Direct, aka "Indy").  Maybe I can better simulate IIS  ](*,)

For the record, SkySpy sends this in each header (I may have not capitalized the text correctly - I was working from hand-written notes):

Code: [Select]
Connection: close
Content-Type: application/octet-stream
Content-Length: 38
Cache-Control: private
Date: Thu, 01 May 2014 00:45:15 GMT
X-Powered-By: ASP.NET
X-Aspnet-Version: 2.0.50727
HTTP_FLAGS: 14:01
Server: Microsoft-IIS/8.0

What IIS Sends:

Code: [Select]
Cache-Control: private
Content-Length: 38
Contenty-Type: application/octet-stream
Server: Microsoft-IIS/8.0
X-Aspnet-Version: 2.0.50727
HTTP_FLAGS: 14:01
X-Powered-By: ASP.NET
Date: Thu, 01 May 2014 00:45:15 GMT
Connection: close

Can somebody check if an out-of-order header causes the GW to ignore the 38 byte packet?

I have not had enough time to set up my Apache server (Windows) to enable PUTs and run funny-named PHP files.  It is a working, public server, so I can't make big changes on it without knocking my users off the air.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 01, 2014, 10:32:37 AM
For the record, SkySpy sends this in each header (I may have not capitalized the text correctly - I was working from hand-written notes):
Code: [Select]
Connection: close
Content-Type: application/octet-stream
Content-Length: 38
Cache-Control: private
Date: Thu, 01 May 2014 00:45:15 GMT
X-Powered-By: ASP.NET
X-Aspnet-Version: 2.0.50727
HTTP_FLAGS: 14:01
Server: Microsoft-IIS/8.0

What IIS Sends:

Code: [Select]
Cache-Control: private
Content-Length: 38
Contenty-Type: application/octet-stream
Server: Microsoft-IIS/8.0
X-Aspnet-Version: 2.0.50727
HTTP_FLAGS: 14:01
X-Powered-By: ASP.NET
Date: Thu, 01 May 2014 00:45:15 GMT
Connection: close
Can somebody check if an out-of-order header causes the GW to ignore the 38 byte packet?
Here are the headers, in order, that my php code is sending out:
Code: [Select]
Date: Thu, 01 May 2014 14:05:12 GMT
Server: Apache/2.2.22.(Ubuntu)
X-Powered-By: ASP.NET
HTTP_FLAGS: 14:01
X-ApsNet-Version: 2.0.50727
Cache-Control: private
Content-Length: 38
Connection: close
Content-Type: application/octet-stream..
This is setting the clock, but obviously something is wrong since it keeps sending the 210-byte packets out. I tried changing the php code to make the headers just like IIS, but failed miserably. From what I read, PHP doesn't have full control of headers either. Apache puts in its own before sending the packet out, and it removes any conflicting ones from PHP.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 01, 2014, 12:31:58 PM
There are two people trying to use a private MySQL database that are filling up my bug reporting system with error reports.

Their names are "Scott" and "Joea".

Please run the MySQL utility script found earlier in this thread.  You will not be able to successfully run SkySpy until you do. 

Either that, or run SkySpy in embedded mode.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 02, 2014, 12:52:43 PM
This is setting the clock, but obviously something is wrong since it keeps sending the 210-byte packets out.

Are you ACKing the 210 byte packet?  If you never reply, then the GW may keep trying.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 03, 2014, 06:59:06 PM
Ok, I figured out what the problem was with my 38 byte packet generator.  The HTML/TCP/IP library I am using was encoding the bytes as UTF-8 instead of raw binary.  This tweaked exactly one byte - right in the middle of my serial number.  Not only did this invalidate my serial number, but it also affected the checksum since the byte was tweaked after it was calculated.  I only caught it by running Wireshark and snooping my own packets.

I started using streams instead of a string for a buffer for the 38 byte payload and things started to work - this bypasses the encoder.  I got a beep, the display contrast changed, the clock reset, and the Internet indicator turned on.  Yaaay!   =D&gt;

Unfortunately, this triggered a storm of 210 byte packets.  Boooo!  #-o

Sound familiar?  Read on for some good news:

I was able to stop the GW from jabbering by connecting to the LCX server - it would jabber long SDP packets for anywhere from 10-50 times and then mysteriously quit after the usual handshakes and LCX sending a 38 byte Real Time Clock (RTC) packet.

After analyzing what was going on, I found that the Epoch word in the RTC packet is the key.  It must match what the GW expects or it goes into jabber mode.  You can't set it to a constant value, unless you also trash your "serial number" - but then the Internet indicator goes off and clock updates cease.  Don't forget that the Epoch word changes every two hours.  The current version of SkySpy uses a bad SN (and my personal Epoch number), so you don't get jabbering.

Once I set the Epoch correctly, everything started to work!!!!  \:D/  No more jabbering!

Unfortunately, everyone's Epoch value is different.  I think it depends on the date and time that you register the GW.  You can figure it out by looking at what LCX sends you in the 38 byte packet - note the current local time/date and current Epoch value.  Then you can use this algorithm to simulate it in your own 38 byte RTC packet:

Code: [Select]
    SeedDate         := EncodeDate(2014, 5, 3) + EncodeTime(15, 10, 00, 0);
    StartingEpoch    := $7da8; // my epoch value on 3 May 2014 at 15:10:00 local time

    ElapsedTime   := (Now() - SeedDate);  // calc integer days plus fraction of 24 hours
    ElapsedEpochs := TRUNC(ElapsedTime / (2.0 / 24.0)); // Epoch updates every 2 hours
    RawEpoch      := ElapsedEpochs * $12;  // Epoch increases by 0x12 each period
    RawEpoch      := RawEpoch + StartingEpoch; // calc new Epoch
    RawEpoch      := RawEpoch MOD $7FFF; // Epoch rolls over at $7FFF so use modulus

    R.Epoch := SwapWordBytes(RawEpoch); // Make it big-endian and write it to offset 0x16..0x17 in the RTC payload

I know you PHP guys have figured out a way to simulate a registration.  Does this initialize the Epoch to zero, or does it retain the original value?

If it set the Epoch to zero, then we are in Fat City - because everyone can use the same StartingEpoch.  If not, then something else will have to be done.  I would like to remove all dependency on the LCX cloud server.


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 04, 2014, 01:27:53 PM
It turns out that the GW squawks its own Epoch value in the 30 byte 01:01 packet at offset 0x1A. Maybe it does not need to be calculated after all.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Scott_A on May 04, 2014, 05:00:33 PM
I have begun working on a web based presentation layer. I won't be able to spend much time on it this week so I thought I would give you a sneak peak, and perhaps some of you could improve on it.  :grin:

You will need your data in the MySQL database with the schema from skydvrz and you will also need a php webserver.
Extract the files to the root of the web server or virtual directory. Edit data.php and insert appropriate MySQL credentials. Browse to the page.

Download the code here:
https://www.dropbox.com/s/0xrf1vgn76s5bf9/Weather.zip (https://www.dropbox.com/s/0xrf1vgn76s5bf9/Weather.zip)

Some of my earlier data isn't very pretty as I was jumping my gateway between skydvrz windows service and the php gateway code. (The windows service is currently capturing the data.)

Note: The chart library is licensed free for non commercial use.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 05, 2014, 12:45:13 AM
Ok, I figured out what the problem was with my 38 byte packet generator.  The HTML/TCP/IP library I am using was encoding the bytes as UTF-8 instead of raw binary.  This tweaked exactly one byte - right in the middle of my serial number.  Not only did this invalidate my serial number, but it also affected the checksum since the byte was tweaked after it was calculated.  I only caught it by running Wireshark and snooping my own packets.

I started using streams instead of a string for a buffer for the 38 byte payload and things started to work - this bypasses the encoder.  I got a beep, the display contrast changed, the clock reset, and the Internet indicator turned on.  Yaaay!   =D&gt;

Unfortunately, this triggered a storm of 210 byte packets.  Boooo!  #-o
Contratulations!!!!  And welcome to the club  :-|

Quote
The current version of SkySpy uses a bad SN (and my personal Epoch number), so you don't get jabbering.
I found that giving it an incorrect checksum would do the same thing.
Quote
Code: [Select]
    RawEpoch      := RawEpoch MOD $7FFF; // Epoch rolls over at $7FFF so use modulus
Should that be RawEpoch MOD $8000
Quote
I know you PHP guys have figured out a way to simulate a registration.  Does this initialize the Epoch to zero, or does it retain the original value?
I haven't actually done this. I think to do this the GW must be reset and maybe the console too.
I found instructions to do the GW at
http://weatherdirect.com/help/gatewayhelp4.aspx
and for the console on page 3 of the manual.
Quote
If it set the Epoch to zero, then we are in Fat City - because everyone can use the same StartingEpoch.  If not, then something else will have to be done.  I would like to remove all dependency on the LCX cloud server.
I'm not sure about the 2-hour period for incrementing the epoch. Looking at the 38-byte packets in my capture from LCX cloud, it seems to match for a time, then something changes and the next group of packets match a slightly different StartingEpoch. It seems like the time might be something slightly different than 2 hours. I only have about 60 captured 38-byte packets from LCX, so it's hard to tell.

My capture spans across the change to daylight savings time. The epoch didn't change across that boundary, so probably the Now() and SeedDate used to find the ElapsedTime should be in GMT.

Also, I think the 01:01 30-byte packets are the tail end of the history data sent out in the 210-byte 01:01 packets. Data seems to be sent out in 210-byte packets as long as there is enough data to fill 210 bytes, then a short packet at the end to finish up. I've seen various length packets for this. Any 01:01 packet that starts with 0x21 0x64 is one of these. The "epoch" in these doesn't look like the same epoch in the 38-byte RTC packets. In the 0x21 0x64 packets, one epoch is sent in bytes 6 and 7, and a different one in the two bytes right before the 2-byte crc at the end, regardless of the overall packet length. These epochs are relative to the date/time of the data in the packet, not the current time. And, these epochs seem to increment by 0x12 every 15 minutes. Also, the data records in these packets are mostly 18 (0x12) bytes long. Maybe more than coincidence that that's the amount the epoch increments. I am beginning to think these are not strictly time-related, but maybe a database index, or ???
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 05, 2014, 11:28:08 AM
I have begun working on a web based presentation layer. I won't be able to spend much time on it this week so I thought I would give you a sneak peak, and perhaps some of you could improve on it.  :grin:

Very cool, Scott_A  =D&gt;
I haven't installed Skydvr's schema yet but I'll try it and your code out on my system.

Have you tried Skydvr's epoch calculations to stop the GW from sending lots of 210-byte data? So far I it hasn't worked for me.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on May 05, 2014, 01:32:10 PM

Hi guys, been out of pocket for a while,  Good work continuing it looks like.  I looked at keckec's new reqeust.breq great that it keeps evolving.

I've never had any problem with the internet indicator, mine has been solid for the last month on my PHP code,  I don't think I do anything on the epoch.   I do see the 210 byte packets on occasion, but no storm.  I'll have too look at the new data to see what I set the epoch on registration too.  I have not found a way to reset the weather station SN after it has been set FYI.  I have reset the weather station and the gateway and registered them both but I had to use the original weather station SN (which I set to all zeros) to re-register it.

I did also figure out the frequency setting to control how often the gateway pings the service, but I have to dig out those notes.

Anyway I will try to get some time to play with some of this new information this week.


-M


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on May 05, 2014, 02:57:56 PM

I tried MyCal's method in SkySpy and noticed the same thing.  This needs more research. 

He also says he could adjust his LCD clock, but I have not been able to duplicate that result.  I'd like to see some code that uses dynamic values instead of hard-coded constants like MyCal's example.

Shoot me some PHP code if you make any progress.  I speak PHP.

I will have to check my PHP code on dropbox against my server, since it is all dynamic on the server and has been keeping time for over a month perfectly.

-M


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 05, 2014, 03:13:08 PM
Hi MyCal, great to see you back.

Has anyone looked at the 18-byte 70:00 packets sent from LCX in response to the 00:70 packets from the GW? I see several kinds of data in them, which looks like possibly a bit mask. Maybe a way to remotely reset stuff on the GW or console?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on May 05, 2014, 04:04:25 PM
Hi MyCal, great to see you back.

Has anyone looked at the 18-byte 70:00 packets sent from LCX in response to the 00:70 packets from the GW? I see several kinds of data in them, which looks like possibly a bit mask. Maybe a way to remotely reset stuff on the GW or console?

Yes I captured these packets while talking to lacross alerts and changing the interval time on their website and I have the data,  I need to get those packets decoded and posted.  It was not what I expected, but I think I made sense of it.  I need to find my notes on this.

-M
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 07, 2014, 10:57:59 PM
I am successfully setting the clock, lighting up the Internet indicator and preventing jabbering of large packets.  The trick is to let it jabber (from a bad RTC packet/invalid epoch) and then parse the 210 byte packet.  It contains several copies of the Epoch value.  Grab one and save it to non-volatile storage (I use the DB). Also note the date and time - save that too.  Each time the unit requests a 38 byte RTC packet, then use the DB values to calculate the correct Epoch - remember, it changes every 2 hours.  Send that value and the jabbering stops completely.

If it ever restarts jabbering, the state machine mentioned above will stop it.  Also, this algorithm should work on anyone's unit, no matter where in the Epoch cycle they are.

Code: [Select]
    T210ByteSquawk = PACKED RECORD
        Unknown1:     word;  // 0-1
        Unknown2:     word;  // 2-3
        CurrentEpoch: word;  // 4-5
        Unknown3:     ARRAY[1..204] OF Byte; // dont care
    END;

Code: [Select]
FUNCTION TsssDm.CalcEpoch : word;
VAR
    ElapsedTime :   TDateTime;
    C,
    ElapsedEpochs : Integer;
    StartingEpoch,
    RawEpoch :      word;
    SeedDate :      TDateTime;
    Strg :          STRING;
BEGIN
    Strg     := VarDefault(GlobalSetting['EpochCusp'], '');
    SeedDate := TD24StrToDateTime(Strg);
    Strg     := VarDefault(GlobalSetting['CurrentEpoch'], '$0');
    Val('$' + Strg, StartingEpoch, C);

    ElapsedTime   := (Now() - SeedDate);  // days plus fraction of 24 hours
    ElapsedEpochs := TRUNC(ElapsedTime / (2.0 / 24.0));
    RawEpoch      := ElapsedEpochs * $12;
    RawEpoch      := RawEpoch + StartingEpoch;
    RawEpoch      := RawEpoch MOD $8000;
    Result           := RawEpoch;
END;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 08, 2014, 11:03:31 AM
I am successfully setting the clock, lighting up the Internet indicator and preventing jabbering of large packets.  The trick is to let it jabber (from a bad RTC packet/invalid epoch) and then parse the 210 byte packet.  It contains several copies of the Epoch value.  Grab one and save it to non-volatile storage (I use the DB). Also note the date and time - save that too.  Each time the unit requests a 38 byte RTC packet, then use the DB values to calculate the correct Epoch - remember, it changes every 2 hours.  Send that value and the jabbering stops completely.

If it ever restarts jabbering, the state machine mentioned above will stop it.  Also, this algorithm should work on anyone's unit, no matter where in the Epoch cycle they are.

YOU DID IT!!! Thank you! Thank you! My chatterbox has stopped blabbering!!!  =D&gt;

I suspect the epoch value is actually a memory address. And it seems to wrap to 0x01a6, not zero. I think the 210-byte packets (and many other sizes, which all start with 0x2164) are dumps of history memory. I have about 60MB of this stuff to examine, when I get time. Thanks again, Skydvrz.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 08, 2014, 03:54:04 PM
Skydvrz, what do you have your history interval set to on LCX Alerts, on the Edit Device Page? Is it 2 hours?

I think this sets the time period over which the epoch increments by 0x12. Mine is incrementing every 15 minutes, which is what I have the history interval set to on LCX Alerts page. I'm not sure how LCX sets this in the GW, but the epoch in the 38-byte packets sent by LCX changed by 0x24 in 2 minutes when I changed the history interval from 15 minutes to 30 minutes on the LCX Alerts web site.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 08, 2014, 04:02:26 PM
If you are talking about alerts/alarms on the LCD panel, then I think I have them shut off. 

If you are talking about the LCX web site, then I don't know.  That was the major reason I dumped them and started SkySpy - their alerts system is awful.

I will check both when I get home from work.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 08, 2014, 04:09:05 PM
I just checked the LCX web site - my "history interval" is 15 Minutes.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 08, 2014, 04:11:07 PM
Hmm. Yes, that's the setting I was thinking of. I don't know why the difference in the epoch update interval. More to learn...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on May 08, 2014, 04:33:30 PM
FYI here is the output of the 00:70 packet when adjusting the history interval.

I do know that this packet does affect the sending of the 00:70 packet, I was pretty sure at one time that it was in seconds, IE:
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 00 f0

caused a 00:70 packet to be sent every 240 seconds,  I know I did adjust it down to 5 seconds and it worked.   I cannot find my notes on this but as far as I could tell it only affected this packet, but I did not play with the fields or set the upper bytes to FF FF ...

-M



All these assume
Request:
HTTP_IDENTIFY: xxx:00:xxx:70


15 Min:
------------------------------------------------------
Reply from server:

HTTP_FLAGS:  70:00
00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00 00 f0
 

30 min:
------------------------------------------------------
Reply from server:

HTTP_FLAGS:  70:00
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 04 d7 


1hr:
------------------------------------------------------
Reply from server:

HTTP_FLAGS:  70:00
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 04 b8 


2hr:
------------------------------------------------------
Reply from server:

HTTP_FLAGS:  70:00
ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff ff 04 c5 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 08, 2014, 05:28:36 PM
Epoch information is also sent by the GW in the 30 byte 0101 packet.  It should be fairly easy to derive the interval from that.  The 30 byte packet is not sent very often though.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 08, 2014, 07:38:29 PM
Epoch information is also sent by the GW in the 30 byte 0101 packet.  It should be fairly easy to derive the interval from that.  The 30 byte packet is not sent very often though.
Following is my take on what some of the various length packets are about, including the 30-byte packets. I think the 30-byte packets are the last packet of a longer dump of history data, which starts with a 210-byte packet if there is enough data to fill one. The 30-byte packets start with 0x2164, same as the 210-byte ones, which I think this is a data type indicator. In each of these the first 8 bytes are a header. Bytes 4 & 5 of that are the epoch, which I think might actually be the address in memory where the next piece of history data will be written.

Bytes 6 & 7 are an address 0x12 higher than the address of the newest history record in the packet. History in the packet is chronologically reversed, so the most recent is sent as the first record in the packet. The address of the oldest record in the packet is in the 2 bytes before the 2-byte crc at the end of the packet, regardless of the overall packet size. The oldest address of a packet is the same as that in bytes 6 & 7 of the previous packet. Records seem to be 18 bytes long (0x12 == the increment value of the address, or epoch), and 11 records fit into a 210-byte packet: 8 bytes header, + (11 records * 18 bytes/record) + 2 bytes starting address + 2 bytes crc = 210 bytes total. Other numbers of records fit in various length packets: 30,48, 66, 84, 102, 120, 138, 156, 174, 192, and 210 bytes, corresponding to 1 through 11 history records each. I've seen all these length packets come from the GW.

If a 0101 0x2164 packet is the final one in the dump, the address in bytes 6 & 7 is the same as the address in bytes 4 & 5. I think bytes 4 & 5 contain the address of the next history record to be dumped, 0x12 bytes higher than the most recent one in the packet, and bytes 6 & 7 contain the address of the next place history should be saved. If the last history record dumped is the last one, these two addresses will be the same. I've found this is always the case for 0101 packets starting with 0x2164 that are less than 210 bytes long. It may be the case of some of the 210-byte packets as well, if the amount of history being dumped happens to just fit in 210-byte packets, but I haven't checked this.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 08, 2014, 10:06:46 PM
I think that in the context of a single user system with either an embedded database or a local MySQL DB engine, a server app will not have the same problems LCX does.  Things like oversubscription and flaky software.  Is it worth putting a lot of effort into parsing the historical data when we have everything we need in the 197 byte 0101 packet?

The only people that would need historical data would be the folks that want to shut their computers off at night.  That might be dangerous to your data if you forget to turn it back on in the morning and overrun your history buffer.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 09, 2014, 12:40:24 AM
I think that in the context of a single user system with either an embedded database or a local MySQL DB engine, a server app will not have the same problems LCX does.  Things like oversubscription and flaky software.  Is it worth putting a lot of effort into parsing the historical data when we have everything we need in the 197 byte 0101 packet?
I agree. I wasn't suggesting to include it in anything you're doing. It was mainly to share what I've found, in case it would help anyone.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on May 14, 2014, 05:29:10 PM
WOW it's quiet again.  4 days without a peep.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 14, 2014, 06:29:02 PM
I am busy with the next release!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on May 15, 2014, 11:19:55 AM
I am busy with the next release!

 =D&gt;
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 20, 2014, 09:38:22 PM
Update:

I have improved my packet dissector so that it can handle nybble offsets into packets.  The old one could not handle some fields that did not start on byte boundaries. This hid some of the more obscure data fields.  While none of the new fields are needed by WU, they may be useful if you are saving data to your own MySQL database for later analysis.

The new SDP packet dissector:

Code: [Select]
UNIT sssSDP;

INTERFACE

TYPE
    TDTBytes =
        (
        DTYr,
        DTMo,
        DTD,
        DTH,
        DTM
        );

    TBCDDateTime = ARRAY[TDTBytes] OF Byte;
    TRainArray   = ARRAY[0..11] OF Byte;
    TWindHist    = ARRAY[0..3] OF Byte;
    TKWord       = ARRAY[0..1] OF Byte;
    TRainElement = ARRAY[0..3] OF Byte;
    TBP          = ARRAY[0..1] OF Byte;
 
    // self-reading record - conversion done elsewhere
    TSDPRecA = PACKED RECORD
    PRIVATE
        Flags: ARRAY[0..196] OF Byte;  // payload goes here
        FUNCTION GetBCDDateTime(CONST Index : Integer) : TBCDDateTime;
        FUNCTION GetBP(CONST Index : Integer) : TBP;
        FUNCTION GetByte(CONST Index : Integer) : Byte;
        FUNCTION GetRainArray(CONST Index : Integer) : TRainArray;
        FUNCTION GetRainElement(CONST Index : Integer) : TRainElement;
        FUNCTION GetWindHist(CONST Index : Integer) : TWindHist;
        FUNCTION GetWord(CONST Index : Integer) : TKWord;
        PROCEDURE FillResult(VAR R; CONST BuffSize, aIndex : Integer);
    PUBLIC
        PROCEDURE Dump;  // debug only
        PROCEDURE WriteProp(VAR F : TextFile; CONST PropName : STRING; VAR Prop; PropSize : Byte); // debug only
        // index is $OOOOOOLL where O=field offset in nybbles into payload and LL= length in nybbles of the field
        // Property getters shift all nybbles in a field to the rightmost position (on odd length fields)
        PROPERTY PacketID: Byte index $0002 Read GetByte;
        PROPERTY Status: TKWord index $0603 Read GetWord;
        PROPERTY DTMaxIT: TBCDDateTime index $090a Read GetBCDDateTime;
        PROPERTY DtMinIT: TBCDDateTime index $130a Read GetBCDDateTime;
        PROPERTY MaxIT: TKWord index $1d03 Read GetWord;
        PROPERTY MinIT: TKWord index $2203 Read GetWord;
        PROPERTY CurrIT: TKWord index $2703 Read GetWord;
        PROPERTY DtMaxOT: TBCDDateTime index $2d0a Read GetBCDDateTime;
        PROPERTY DtMinOT: TBCDDateTime index $370a Read GetBCDDateTime;
        PROPERTY MaxOT: TKWord index $4103 Read GetWord;
        PROPERTY MinOT: TKWord index $4603 Read GetWord;
        PROPERTY CurrOT: TKWord index $4b03 Read GetWord;
        PROPERTY DtUnknown1: TBCDDateTime index $510a Read GetBCDDateTime;
        PROPERTY DtUnknown2: TBCDDateTime index $5b0a Read GetBCDDateTime;
        PROPERTY DtUnknown3: TBCDDateTime index $650a Read GetBCDDateTime;  // invalid date in test case
        PROPERTY CurrOT2: TKWord index $6403 Read GetWord;
        PROPERTY Stat2: Byte index $7202 Read GetByte;
        PROPERTY DtMaxIH: TBCDDateTime index $740a Read GetBCDDateTime;
        PROPERTY DtMinIH: TBCDDateTime index $7e0a Read GetBCDDateTime;
        PROPERTY MaxIH: Byte index $8802 Read GetByte;
        PROPERTY MinIH: Byte index $8a02 Read GetByte;
        PROPERTY CurIH: Byte index $8c02 Read GetByte;
        PROPERTY DtMaxOH: TBCDDateTime index $8e0a Read GetBCDDateTime;
        PROPERTY DtMinOH: TBCDDateTime index $980a Read GetBCDDateTime; // this may have wrong offset - invalid date in test case
        PROPERTY MaxOH: Byte index $a202 Read GetByte;
        PROPERTY MinOH: Byte index $a402 Read GetByte;
        PROPERTY CurrOH: Byte index $a602 Read GetByte;
        PROPERTY DtUnknown4: TBCDDateTime index $d40a Read GetBCDDateTime;
        PROPERTY DtLast1HrRain: TBCDDateTime index $ea0a Read GetBCDDateTime;
        PROPERTY DtLastRainReset: TBCDDateTime index $1010a Read GetBCDDateTime;
        PROPERTY RainHistory: TRainArray index $10b17 Read GetRainArray; // break elements into separate properties?
        PROPERTY RFTotal: TRainElement index $11208 Read GetRainElement;
        PROPERTY AvgWind: TKWord index $12204 Read GetWord;
        PROPERTY WindDirHist: TWindHist index $12a06 Read GetWindHist;
        PROPERTY DtMaxGust: TBCDDateTime index $1300a Read GetBCDDateTime;
        PROPERTY MaxGust: TKWord index $13a04 Read GetWord;
        PROPERTY CurrGust: TKWord index $14004 Read GetWord;
        PROPERTY WindStat: TKWord index $14404 Read GetWord;
        PROPERTY WindDirHist2: TWindHist index $14806 Read GetWindHist;
        PROPERTY CurrBP: TBP index $14f04 Read GetBP;
        PROPERTY BPDelta: TWindHist index $15306 Read GetWindHist; // this is a guess
        PROPERTY MinBp: TBP index $15904 Read GetBP;
        PROPERTY MaxBp: TBP index $16304 Read GetBP;
        PROPERTY DtUnknown5: TBCDDateTime index $16c0a Read GetBCDDateTime; // bp related?
        PROPERTY DtUnknown6: TBCDDateTime index $1760a Read GetBCDDateTime; // bp related?
        PROPERTY CheckSum: TKword index $18604 Read GetWord;
    END;

IMPLEMENTATION

USES
    dgLib,
    SysUtils;

{ TSDPRecA }

// debug only - not typically used
PROCEDURE TSDPRecA.Dump;
VAR
    Dt : TBCDDateTime;
    R :  TRainArray;
    Wh : TWindHist;
    W :  TKWord;
    B :  Byte;
    F :  TextFile;
    BP : TBP;
BEGIN
    AssignFile(F, 'junk.txt');
    Rewrite(F);

    B := PacketID;
    WriteProp(F, 'PacketID', B, SizeOf(Byte));   // Byte

    W := Status;
    WriteProp(F, 'Status', W, SizeOf(TKWord));   // TTemp

    Dt := DTMaxIT;
    WriteProp(F, 'DTMaxIT', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtMinIT;
    WriteProp(F, 'DtMinIT', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    W := MaxIT;
    WriteProp(F, 'MaxIT', W, SizeOf(TKWord));   // TKWord

    W := MinIT;
    WriteProp(F, 'MinIT', W, SizeOf(TKWord));   // TKWord

    W := CurrIT;
    WriteProp(F, 'CurrIT', W, SizeOf(TKWord));   // TKWord

    Dt := DtMinOT;
    WriteProp(F, 'DtMinOT', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    W := MaxOT;
    WriteProp(F, 'MaxOT', W, SizeOf(TKWord));   // TKWord

    W := MinOT;
    WriteProp(F, 'MinOT', W, SizeOf(TKWord));   // TKWord

    W := CurrOT;
    WriteProp(F, 'CurrOT', W, SizeOf(TKWord));   // TKWord

    Dt := DtUnknown1;
    WriteProp(F, 'DtUnknown1', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtUnknown2;
    WriteProp(F, 'DtUnknown2', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtUnknown3;
    WriteProp(F, 'DtUnknown3', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    W := CurrOT2;
    WriteProp(F, 'CurrOT2', W, SizeOf(TKWord));   // TKWord

    B := Stat2;
    WriteProp(F, 'Stat2', B, SizeOf(Byte));   // Byte

    Dt := DtMaxIH;
    WriteProp(F, 'DtMaxIH', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtMinIH;
    WriteProp(F, 'DtMinIH', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    B := MaxIH;
    WriteProp(F, 'MaxIH', B, SizeOf(Byte));   // Byte

    B := MinIH;
    WriteProp(F, 'MinIH', B, SizeOf(Byte));   // Byte

    Dt := DtMaxOH;
    WriteProp(F, 'DtMaxOH', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtMinOH;
    WriteProp(F, 'DtMinOH', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    B := MaxOH;
    WriteProp(F, 'MaxOH', B, SizeOf(Byte));   // Byte

    B := MinOH;
    WriteProp(F, 'MinOH', B, SizeOf(Byte));   // Byte

    B := CurrOH;
    WriteProp(F, 'CurrOH', B, SizeOf(Byte));   // Byte

    Dt := DtUnknown4;
    WriteProp(F, 'DtUnknown4', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtLast1HrRain;
    WriteProp(F, 'DtLast1HrRain', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtLastRainReset;
    WriteProp(F, 'DtLastRainReset', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    R := RainHistory;
    WriteProp(F, 'RainHistory', R, SizeOf(TRainArray));   // TRainArray

    W := AvgWind;
    WriteProp(F, 'AvgWind', W, SizeOf(TKWord));   // TKWord

    Wh := WindDirHist;
    WriteProp(F, 'WindDirHist', Wh, SizeOf(TWindHist));   // TWindHist

    Dt := DtMaxGust;
    WriteProp(F, 'DtMaxGust', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    W := MaxGust;
    WriteProp(F, 'MaxGust', W, SizeOf(TKWord));   // TKWord

    W := CurrGust;
    WriteProp(F, 'CurrGust', W, SizeOf(TKWord));   // TKWord

    W := WindStat;
    WriteProp(F, 'WindStat', W, SizeOf(TKWord));   // TKWord

    Wh := WindDirHist2;
    WriteProp(F, 'WindDirHist2', Wh, SizeOf(TWindHist));   // TWindHist

    BP := CurrBP;
    WriteProp(F, 'CurrBP', BP, SizeOf(TBP));   // TKWord

    Wh := BPDelta;
    WriteProp(F, 'BPDelta', Wh, SizeOf(TWindHist));   // TWindHist

    BP := MinBp;
    WriteProp(F, 'MinBp', BP, SizeOf(TBP));   // TKWord

    BP := MaxBp;
    WriteProp(F, 'MaxBp', BP, SizeOf(TBP));   // TBP

    Dt := DtUnknown5;
    WriteProp(F, 'DtUnknown5', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    Dt := DtUnknown6;
    WriteProp(F, 'DtUnknown6', Dt, SizeOf(TBCDDateTime));   // TBCDDateTime

    W := CheckSum;
    WriteProp(F, 'CheckSum', W, SizeOf(TKword));   // TKword

    CloseFile(F);
END;

// fill output buffer R with the contents of the payload, at the given offset
// and expected field length
PROCEDURE TSDPRecA.FillResult(VAR R; CONST BuffSize, aIndex : Integer);
VAR
    I,
    NybOffset : Integer;
    NrNybs :    Integer;
    Mask :      Integer;
    INyb :      Integer;
    Nt,
    B :         Byte;
    Buff :      TByteArray Absolute R;
    IBuff :     Integer;
BEGIN
    NrNybs    := aIndex AND $FF;
    NybOffset := aIndex SHR 8;

    IF (NrNybs SHR 1) > Buffsize THEN
        RAISE Exception.Create('Buffer Overrun Detected');

    FillChar(Buff, BuffSize, 0); // clear the buffer - set it to all zeros

                                 // field is located completely on natural byte boundary, so just copy bytes
    IF (NOT Odd(NybOffset)) AND (NOT Odd(NrNybs)) THEN BEGIN
        FOR I := 0 TO (NrNybs DIV 2) - 1 DO
            Buff[I] := Flags[(NybOffset DIV 2) + I];
        EXIT;
    END;

    // field was not on byte boundary, so do nybble copy
    FOR INyb := NybOffset TO NybOffset + NrNybs - 1 DO BEGIN
        // get the nybble we want from the flag array and put it into Nt
        B := Flags[INyb DIV 2];

        // calculate where we want to put the nybbles in the output buffer
        IBuff := (INyb - NybOffset) DIV 2;

        IF Odd(INyb) THEN BEGIN
            Nt          := (B AND $F);                         // get the right nybble
            Buff[IBuff] := (Buff[IBuff] AND $F) OR (Nt SHL 4); // or it into the proper place in the output buffer
        END
        ELSE BEGIN
            Nt          := (B AND $F0) SHR 4; // get the left nybble
            Buff[IBuff] := (Buff[IBuff] AND $F0) OR Nt;
        END;
    END;

    // shift all nybbles 4 bits to right, if there is dead space on the far right of the output buffer
    IF Odd(NrNybs) THEN BEGIN
        // go backwards through the output buffer
        FOR INyb := NrNybs DIV 2 DOWNTO 1 DO BEGIN
            Buff[INyb]     := Buff[INyb] SHR 4;               // shift it right one nybble
            B              := (Buff[INyb - 1] AND $0F) SHL 4; // get neighbor low nybble on left and shift it to high nybble
            Buff[INyb]     := Buff[INyb] OR B;                // move it to current byte
            Buff[INyb - 1] := Buff[INyb - 1] SHR 4;           // shift neighbors upper nybble into lower nybble
        END; // shampoo, rinse and repeat
    END;
END;

FUNCTION TSDPRecA.GetBCDDateTime(CONST Index : Integer) : TBCDDateTime;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetBP(CONST Index : Integer) : TBP;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetRainArray(CONST Index : Integer) : TRainArray;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetRainElement(CONST Index : Integer) : TRainElement;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetByte(CONST Index : Integer) : Byte;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetWindHist(CONST Index : Integer) : TWindHist;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

FUNCTION TSDPRecA.GetWord(CONST Index : Integer) : TKword;
BEGIN
    FillResult(Result, SizeOf(Result), Index);
END;

PROCEDURE TSDPRecA.WriteProp(VAR F : TextFile; CONST PropName : STRING; VAR Prop; PropSize : Byte);
VAR
    B : TByteArray Absolute Prop;
    I : Integer;
BEGIN
    Write(F, PadTrim(PropName, 20));
    FOR I := 0 TO PropSize - 1 DO
        Write(F, IntToHex(B[I], 2) + ' ');
    WRITELN(F);
END;

END.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 21, 2014, 03:25:25 PM
Here is my current layout for the 197-byte data packets. I have a few more experiments to run to look for more, and may get to that this weekend. There is likely nothing new here that is important for WU, but I may use a few for my stuff.

http://www.keckec.com/weather/LCX_Data_Layout.pdf

Several data fields are followed in the packet by fields that are almost always 0. Some of these I have verified contain data valid flags. I suspect some of the others may as well. Ones connected by RF links are relatively easy to test, but the ones which aren't (inside temp/humidity, for instance), may be impossible to verify without unsoldering some parts.

There are some I've listed as all-time max and min values. They may be in fact be automatically reset on some schedule such as yearly, or maybe Jun 30 -> July 1. I did test for reset over a new-year transition by setting the clock ahead to Dec 31, but nothing was reset. There may be a way to remotely reset these, but I haven't tried.

Corrections & comments welcome
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 22, 2014, 08:33:23 PM
Here is my current layout for the 197-byte data packets.
http://www.keckec.com/weather/LCX_Data_Layout.pdf

Do you have a spreadsheet version of that PDF?  A CDF or other machine-readable file would be fine too.

I need to make a bunch of notes and the PDF is a bit of a pain since I have to write in the margins of a printout  :?

TIA!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 22, 2014, 10:00:21 PM
Keckec:

There may be a problem with Min Outside Temp at nybble-offset $46.  Mine calculates out to -15F.  The display says my min OT was 35F.  All the other temps seem OK so far.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 23, 2014, 10:39:24 AM
Skydvrz,

Thanks for looking this over.

Do you have a spreadsheet version of that PDF?
The spreadsheet is here:
http://www.keckec.com/weather/LcxLayout.ods
The notes are in a different file, which grabs the spreadsheet from disk to make the pdf. You'll probably have to tell it where the spreadsheet is on your system. Anyway, it's here:
http://www.keckec.com/weather/DataLayoutNotes.odt

There may be a problem with Min Outside Temp at nybble-offset $46.  Mine calculates out to -15F.  The display says my min OT was 35F.  All the other temps seem OK so far.
Hmmm.  It seems to be working here. My raw data right now is 0x542, which is coming out to 57.56F. Here is the php code that converts to deg F.
Code: [Select]
function bcd2int(&$nybs,$startn,$len)
{
    $retval = 0;
    for ($i = 0; $i < $len; $i++)
    { 
        if ($nybs[$startn+$i] > 9)
        { 
            return -1;
        }
        $retval = $retval * 10 + $nybs[$startn+$i];
    }
    return $retval;
}

function temp2str(&$nybs,$startn)
{
    $retstr = "";
    $val = bcd2int($nybs,$startn,3);
    if ($val < 0)
    {
        return "  N/A ";
    }
    $otemp = $val / 10.0 - 40.0;
    $otemp = $otemp * 9.0 / 5 + 32.0;
    $retstr .= sprintf("%6.2f",$otemp);
    return $retstr;
}
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 23, 2014, 11:11:04 AM
I use a similar algorithm to calculate Fahrenheit, plus I use the same function for all temperature fields.  All the other ones are OK.

Maybe the value found at that offset (Nybble $46) is not what you think?

I will check the raw numbers tonight try to get a handle on what is going on.

I am successfully storing all the "new" sensor fields in MySQL.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 26, 2014, 10:28:51 AM
I made a correction to the layout. The all-time minimum outside temperature should be at nybble 0x6a, not 0x181, and it's actually wind chill. I don't know what the 3-nybble field at 0x181 is, but it still looks like a temperature. I've updated the spreadsheet:
http://www.keckec.com/weather/LcxLayout.ods
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on May 26, 2014, 05:42:38 PM
I made a correction to the layout. The all-time minimum outside temperature should be at nybble 0x6a, not 0x181, and it's actually wind chill. I don't know what the 3-nybble field at 0x181 is, but it still looks like a temperature. I've updated the spreadsheet:
http://www.keckec.com/weather/LcxLayout.ods

Thanks for the info, but you do know that the spreadsheet errors when opening because it can't access your note file.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on May 26, 2014, 09:11:48 PM
I got the new improved SDP packet dissector storing data to MySQL.  I also updated ssMonitor to display the new fields. 

There are a few minor problems with some of the new fields.  The one I mentioned before remains, plus I noticed a couple others.  I have circled them on the screen shot of the new SkySpy tab found below.  OT Min is at (nybble) offset $46.  OT All time low is at offset $6a. Unknown date is at offset $112.  OH Min Date is at offset $98

(http://idk.serveftp.net/images/badfields.jpg)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 27, 2014, 10:44:35 AM
There are a few minor problems with some of the new fields.  The one I mentioned before remains, plus I noticed a couple others.  I have circled them on the screen shot of the new SkySpy tab found below.  OT Min is at (nybble) offset $46.  OT All time low is at offset $6a. Unknown date is at offset $112.  OH Min Date is at offset $98
The unknown date is the same in your data as mine. Somewhere I saw in LCX documentation that the initial value for dates is 1/1/2013. I assume this date is unused.
The difference between our OT min and OT all-time low fields is odd. Don't know where the differences lie.

Here is a hex dump of the nybbles from my station this morning. If you have something similar, could you post it. Maybe there's more than one format, maybe a firmware version difference. Maybe there's a clue in the unidentified nybbles.
Code: [Select]
by nyb |0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1|1|1|1|1|1|1|1|1|1|1|1|1|1|1|1|2|2|2|2|2|2|2|2|2|2|2|2|2|2|2|2|3|3|3|3|3|3|3|3|3|3|3|3|3|3|3|3
te ble |0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f|0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f|0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f|0|1|2|3|4|5|6|7|8|9|a|b|c|d|e|f
00 000 |0|1|6|4|1|0|2|1|0|1|4|0|5|1|6|1|7|3|5|1|4|0|5|1|7|0|7|3|0|7|0|7|0|0|5|8|9|0|0|6|4|5|0|0|0|1|4|0|5|2|6|1|7|3|8|1|4|0|5|2|6|2|3|0
20 040 |1|6|6|6|0|0|5|5|4|0|0|5|8|8|0|0|4|1|4|0|5|1|5|1|2|2|1|1|4|0|5|2|5|1|8|1|1|8|3|6|0|0|4|3|6|0|0|5|8|8|0|0|1|4|0|5|2|5|2|2|3|8|1|4
40 080 |0|5|1|7|0|7|2|9|5|7|2|0|5|1|1|4|0|5|2|7|0|6|3|1|1|4|0|5|2|6|1|7|4|1|8|7|5|1|8|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0
60 0c0 |0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|0|1|4|0|3|0|1|0|3|5|9|0|0|2|6|0|5|0|0|0|0|0|0|1|4|0|4|1|9|0|9|0|3|0|0|1|0|0|6|0|0|0|0|0|0
80 100 |0|1|4|0|5|0|1|1|7|3|1|0|0|0|0|0|0|0|1|3|0|1|0|1|0|0|0|0|0|0|0|0|0|0|0|0|f|c|0|0|5|0|2|4|4|4|4|3|1|4|0|5|2|7|0|6|2|6|0|4|5|c|0|0
a0 140 |0|1|b|0|0|0|1|0|2|4|4|4|4|3|0|2|9|9|5|1|0|1|4|2|0|2|9|6|9|1|0|0|5|5|0|3|0|2|9|1|0|2|5|8|1|4|0|2|2|8|1|6|2|4|1|4|0|1|1|3|1|0|1|9
c0 180 |0|4|5|3|0|3|7|2|c|f
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 27, 2014, 03:15:12 PM
Thanks for the info, but you do know that the spreadsheet errors when opening because it can't access your note file.
Sorry, I renamed one of the files but didn't update the one on the web site. I think this should work...
http://www.keckec.com/weather/LcxLayout.ods
http://www.keckec.com/weather/LCX_Data_Layout.odt
Downloading both files to the same directory should work.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on May 30, 2014, 01:28:32 AM
It's not in nybbles, but here's one of my SDP packet.


00000000h: 01 64 20 21 01 40 20 21 33 61 40 41 51 10 06 83
00000010h: 00 58 70 06 58 00 01 40 52 61 44 11 40 30 30 74
00000020h: 67 49 00 30 20 06 52 00 11 40 52 61 44 11 40 30
00000030h: 30 70 37 49 00 22 50 06 52 00 14 04 27 17 14 14
00000040h: 02 27 08 00 65 19 54 14 03 06 05 06 14 03 26 14
00000050h: 09 91 23 67 00 00 00 00 00 00 00 00 01 34 93 00
00000060h: 00 00 00 00 00 00 00 00 15 22 14 03 03 10 59 00
00000070h: 73 01 00 00 00 14 03 28 17 51 00 27 86 00 00 00
00000080h: 00 00 00 00 00 A0 46 93 02 13 01 01 00 00 00 00
00000090h: 00 00 B4 00 50 44 55 35 14 04 04 01 59 10 50 00
000000a0h: 01 B0 00 10 44 55 35 02 99 51 01 42 02 95 10 99
000000b0h: 93 03 05 81 03 55 14 04 28 16 20 14 02 06 09 50
000000c0h: 05 84 07 6A F6
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on May 30, 2014, 07:18:24 PM
10ACTony,

Thanks for sending the data packet. Here's what I see in it for the fields Skydvrz found to be out of range.
Min outside temp at nybble 0x46: raw=0x302 -> 14.4F (at 3/3/14 07:46)
All-time Low OT w/wind chill at nybble 0x6a: raw=0x225 -> +0.5F  (at 3/3/14 07:03)
Date/time of min outside humidity at nybble 0x98: 3/26/14 14:09

I don't know where you are. Do these values look at all reasonable?

Skydvrz's data packet had some odd values for those fields.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on June 01, 2014, 12:49:17 AM
10ACTony,

Thanks for sending the data packet. Here's what I see in it for the fields Skydvrz found to be out of range.
Min outside temp at nybble 0x46: raw=0x302 -> 14.4F (at 3/3/14 07:46)
All-time Low OT w/wind chill at nybble 0x6a: raw=0x225 -> +0.5F  (at 3/3/14 07:03)
Date/time of min outside humidity at nybble 0x98: 3/26/14 14:09

I don't know where you are. Do these values look at all reasonable?

Skydvrz's data packet had some odd values for those fields.

The min outside temp, date and time are exactly whats on my station, as is the date time of the min outside humidity.  Looking at the wind chill it's also very possible.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on June 06, 2014, 03:22:42 PM
Here's more info on setting the time interval to save history...

Two bytes control how often history is saved. Each time it is saved, the address to write the next history record (aka epoch) is incremented by 0x12. This is controlled by bytes 0x14 and 0x1f of the 38-byte 14:01 packet (which also sets console time). The "cloud" servers always set both bytes to the same value, and I've only done it that way. Intervals below 10 minutes also seem to speed up how often the SDP is sent. Here are the intervals for various values of these bytes:
Code: [Select]
Val  Interval
 0   1 minute
 1   5 minutes
 2   10 minutes
 3   15 minutes
 4   20 minutes
 5   30 minutes
 6   1 hour
 7   2 hours (saves on even hours, local time)
There may be valid higher values for these bytes, and there may be a value which disables saving history altogether. This might stop the epoch from incrementing. I haven't looked for that.

I have the history data format figured out, but have a few more details to work out on how to tell the console/gateway what to send. I'll post here when I get more.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on June 11, 2014, 12:32:53 AM

I had a few hrs to work on the code today,  I took a snapshot that keckec has updated and modified it to upload to weather underground.

It doesn't have the fix to stop sending 210 byte packets or anything, but it does seem to work well:

http://www.wunderground.com/personal-weather-station/dashboard?ID=KCAPETAL29

Code Snapshot is here:

https://github.com/lowerpower/LaCrosse


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on June 12, 2014, 02:35:22 PM

FYI you need to create a file on your webserver called php.log and make sure the webserver owns it or it is chmod 777

If you don't you will get an error and it the software will not work.

I added a second weather station using this software to Weather Underground.

http://www.wunderground.com/personal-weather-station/dashboard?ID=KCACLEAR4

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: benmargolin on June 12, 2014, 02:46:49 PM
I added a second weather station using this software to Weather Underground.
http://www.wunderground.com/personal-weather-station/dashboard?ID=KCACLEAR4

This is great stuff! I will try to finally give this a try, this weekend. Definitely time to stop "throwing away" my data at Lacrosse's servers...
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on June 20, 2014, 12:26:05 AM
Still working on this project, although a bit slower than before. Life, art, paychecks and grand-kids take precedence - and not necessarily in that order...  :-)

I am currently adding a feature where you can click a button and recreate all of your sensor readings using new (more accurate) algorithms.  This was a big change and broke a lot of stuff, so bear with me.  Those of you subscribing to my Mantis bug-reporting site will see all of my boo-boos as I write new code.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on June 26, 2014, 11:50:08 AM

I had a few hrs to work on the code today,  I took a snapshot that keckec has updated and modified it to upload to weather underground.

It doesn't have the fix to stop sending 210 byte packets or anything, but it does seem to work well:

http://www.wunderground.com/personal-weather-station/dashboard?ID=KCAPETAL29

Code Snapshot is here:

https://github.com/lowerpower/LaCrosse

Thank you for this!  It is working well for moving my data to:
http://www.wunderground.com/personal-weather-station/dashboard?ID=KGALOOKO3 (http://www.wunderground.com/personal-weather-station/dashboard?ID=KGALOOKO3)

David
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: kp4djt on July 07, 2014, 02:59:19 PM
I just moved to a remote location (well Internet wise) here ins S.W. Washington. At my previous location I had a nice cable connection ($$ but I could have had dialup) now I am stuck with HughesNet (B-[)

I had looked into trying to keep my data local back sometime ago, that was I believe
when this thread first started, I saw where it was doing data scrapes and whatnot,
I was more interested in local capture, but since I had all of that BW it was not high
on my list, NOW with Hughesnet as the ISP, I have to keep the bit count down
(no more Roku, alas... a single movie eats up 7/8ths of my monthly allocation, and
I am not paying them more for a half a movie more of BW.

Reviewing what you guys  have done, it has really come along. I am interested in
the Linux side of the house (only run Win7 in a VM every once in a while), indeed
I want to see if I can cram enough of it into a Raspberry PI to operate as the back
end and have a larger machine pull off the data every so often, or just try to set it
up to where the RPI does the updates. If I need to capture more history than the
RPI can hold I can just dump it to a db on a big machine and be done with it. I
have a RPI ordered and this will probably be the first thing I put on the device.

I will be interested in doing any testing that you guys want.

I run Ubuntu and a bit of CentOS.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 07, 2014, 05:23:10 PM
De-lurking for a bit :grin:
Thanks to what I've read here from mycal and keckec, and by sniffing the output of SkyDvr's program, I've been working on my own scripts for my Synology Diskstation to capture the packets, save them to my local version of SkyDvr's database, and relay them to weatherdirect.com for comparison of the replies.

I have multiple issues to discuss, so I'll post them separately to keep the conversation focused.

I'll start with the 210-byte history packets:
I'm seeing more variation in the 8-byte header than SkyDvr reported (e.g. they don't all start with 0x2164) but I haven't really figured out what the variations are.
I'm also slightly more variation in the 4-byte trailer than SkyDvr, also undeciphered so far.

I've mostly figured out the 18-byte data packets, the offsets below are in nybbles:
All values except wind direction are BCD.
Values except wind direction and speed are BCD.  Windspeeds are 3 nybbles in units of .01 km/h
StartEndDescription
02Maybe rainfall, ranges from 0x000 to 0x003 in my data
33A wind gust direction? varies from 0x0-0xf, but varies little from one to the next
44Wind direction: varies from 0x0-0xf, often greatly from one to the next
57Wind gust speed; 3 nybbles in .01 kph
810Wind speed; 3 nybbles in .01 kph
1112Outside relative humidity(%)
1314Inside relative humidity(%)
1519Barometer in .1 mbar
2022Outside T in .1 degC + 400
2325Inside T in .1 degC + 400
2635Date as ymdhi

Now, I have one question about these: How do I shut them up?  Every time my gateway sends a full current data packet, it retransmits the entire history again.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 07, 2014, 05:33:07 PM
I forgot to mention that with a big head start from mycal's request script, I am also successfully transmitting my data to weatherunderground.com:
http://www.wunderground.com/personal-weather-station/dashboard?ID=KFLORLAN112#history (http://www.wunderground.com/personal-weather-station/dashboard?ID=KFLORLAN112#history)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 07, 2014, 05:47:10 PM
Has anyone completely decoded the response to the 00:70 packets?

My response is 16 bytes of 00s followed by 2 bytes of seconds per data packet.  weatherdirect has many of the same response.

However, weatherdirect less often responds with 16 bytes of ff followed by 2 bytes 0x04xx, where xx in my logs is always > af, but is otherwise fairly random.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 08, 2014, 02:27:01 AM
Hi KennKong,

I just uploaded some stuff that might help. There are four files here: two describing the packets, the php code I use to grab the data from the gateway and feed it to the database, and the database schema.
http://www.keckec.com/weather/WxStuff.tgz

I apologize for the rough look of the .sql file, I'm in between modifications of the database schema and the php code, so they're out of sync in my source right now. What's in the tgz file is a dump of the running database (minus data) so it should work with request.breq (php code). To get it to stop sending history packets, you have to send it the address of the memory location that is to receive the next history record. It sends this address out in bytes 4 & 5 (big endian) of every type 0x21 packet (the first byte = 0x21). The request.breq file sends that out when asked, so every time there is a new history record saved by the console, it sends out of single history record in a 30-byte packet. The writeup should explain some of that.

Please ask any questions you might have about this stuff. I know I'm not the best at explaining these things.

I'm currently working on the data retrieval side of things, but it's very preliminary.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 08, 2014, 03:33:20 AM
I just uploaded some stuff that might help.

The layout looks like a slightly improved version of what I already got from your earlier posts.  I'll dig into it more tomorrow.

If you want data extraction, I have a page which reads the (local) weather database (a la SkyDvr) and displays the current conditions plus nice graphs of the history (thanks to Scott_A.)  It doesn't look pretty (OK, it looks pretty ugly) but that's because I've never done HTML, PHP, JScript or anything else web page related before.  Total newb, here.  I don't know if the link will work (I only use my webserver locally) but you will need IPv6 connectivity at a minimum (I don't publish my IPv4 address).
http://www.kennkong.info/Weather.htm (http://www.kennkong.info/Weather.htm)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 09, 2014, 01:51:12 AM
keckec, your info on the 14:01 reply packet was quite helpful.  I'm still struggling with the next history memory address part.  My logs show that while it almost always advances by 0x12 every 15 minutes (my history interval), occasionally it won't advance for longer than that, rarely much longer (i.e 2 hours).

Nonetheless, by getting the next history address closer to the last history address, it has finally gotten the long history recaps to stop.

BTW, any thoughts on the 5-byte packet that triggers this?  The first byte is always 0x41 (clearly, the packet type).
The third byte has always been one of (10, 11, 20, 21, 30, 31); may it be some kind of forecast value?  The other three bytes bounce around quite a bit, with no clear trends.  Also, the number of different values is much less than the numbers of samples, so they have some non-random meaning.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on July 09, 2014, 01:02:54 PM

I don't see why you cannot do everything on the pi, its plenty powerful accept and log the data and write it to its own database, and even display it on a web server.

Uploading to WU is mouse nuts bandwidth compared to streaming a movie, and probably wouldn't be noticeable on your bandwidth. (figure 500 bytes every 5 min)

-M



Reviewing what you guys  have done, it has really come along. I am interested in
the Linux side of the house (only run Win7 in a VM every once in a while), indeed
I want to see if I can cram enough of it into a Raspberry PI to operate as the back
end and have a larger machine pull off the data every so often, or just try to set it
up to where the RPI does the updates. If I need to capture more history than the
RPI can hold I can just dump it to a db on a big machine and be done with it. I
have a RPI ordered and this will probably be the first thing I put on the device.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 10, 2014, 02:41:23 AM
I finally got the history packets to dry up, by saving the last memory address from the history packets and sending it back in the 14:01 reply.  However, due to some fat fingering, I sent a bad address, and the thing has gone loopy.

Since the bad 14:01 packet, the system is sending the 00:70 packets non-stop.  Interestingly, the weatherdirect.com response to those changed from padding the first 14 bytes with 0x00 to 0x2e.  Also, the weatherdirect.com response to the 01:00 packet is also filled with 0x2e, except for the memory address bytes and the checksum.  Not even the leading 01 packet type byte is there.

I would really like to stop the flood of 00:70 packets.  I let the system talk directly to weatherdirect (without my relay) for a couple hours, and it's still doing it.  I've rebooted the gateway multiple times, and reset the weather station itself once.  No luck.  Any thoughts?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 10, 2014, 10:42:51 AM
I'm still struggling with the next history memory address part.  My logs show that while it almost always advances by 0x12 every 15 minutes (my history interval), occasionally it won't advance for longer than that, rarely much longer (i.e 2 hours).

Nonetheless, by getting the next history address closer to the last history address, it has finally gotten the long history recaps to stop.
Glad your history blabbermouth settled down. At one time I was trying to anticipate what the next history address will be, and that worked most of the time to keep it from sending history packets. But it didn't always work, and that might be what you're seeing too. At this point I just repeat the address it last sent out, so when the address changes what I send is behind by one history record length (0x12 bytes). So it then sends a 1-record history packet, I save the address from bytes 4&5, and am back in sync with it again. That works out since I save that history record. That way I have the complete history. I think this may be what weatherdirect does too.

BTW, any thoughts on the 5-byte packet that triggers this?  The first byte is always 0x41 (clearly, the packet type).
The third byte has always been one of (10, 11, 20, 21, 30, 31); may it be some kind of forecast value?  The other three bytes bounce around quite a bit, with no clear trends.  Also, the number of different values is much less than the numbers of samples, so they have some non-random meaning.
I think the second byte is RF signal strength, as a binary value in percent, with 5% granularity. This value is also in the same byte in the history (type 0x21) and data (type 0x01) packets. I don't know what the other bytes are. When mine was communicating with weatherdirect, the third byte contained same values as you, and a 35 once. I don't have a clue about the last two bytes.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 10, 2014, 11:14:21 AM
I finally got the history packets to dry up, by saving the last memory address from the history packets and sending it back in the 14:01 reply.  However, due to some fat fingering, I sent a bad address, and the thing has gone loopy.

Since the bad 14:01 packet, the system is sending the 00:70 packets non-stop.  Interestingly, the weatherdirect.com response to those changed from padding the first 14 bytes with 0x00 to 0x2e.  Also, the weatherdirect.com response to the 01:00 packet is also filled with 0x2e, except for the memory address bytes and the checksum.  Not even the leading 01 packet type byte is there.

I would really like to stop the flood of 00:70 packets.  I let the system talk directly to weatherdirect (without my relay) for a couple hours, and it's still doing it.  I've rebooted the gateway multiple times, and reset the weather station itself once.  No luck.  Any thoughts?

Are you sure the only problem was the address in the 14:01 packet? I've sent lots of bogus addresses while messing around and it always recovered when sent the right address. You might try having the gateway reset its connection to the weatherdirect servers. Have a look at this pdf for some reset info. I think it may be for a different model, but might work here.
http://www.weatherdirect.com/pdfs/help/Lost%20Forecast.pdf
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 10, 2014, 04:05:19 PM

At this point I just repeat the address it last sent out, so when the address changes what I send is behind by one history record length (0x12 bytes). So it then sends a 1-record history packet, I save the address from bytes 4&5, and am back in sync with it again. That works out since I save that history record. That way I have the complete history. I think this may be what weatherdirect does too.


I think the second byte is RF signal strength, as a binary value in percent, with 5% granularity.
Hmm! I've been saving the address from bytes 6 and 7.  Maybe that's why mine has stopped sending history packets completely now.  Like you, I had been getting the one record response every 15 minutes, but I haven't seen one since I sent the bad address packet.

I concur with the 01:00 byte 2 being RF signal strength.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 12, 2014, 02:24:23 PM
Now, I have one question about these: How do I shut them up?  Every time my gateway sends a full current data packet, it retransmits the entire history again.

You need to send a properly formatted 0010 reply to the GW after it requests the current time in a 0100 5-byte packet. 

The reply must contain your PWS serial number issued by LCX - the 7FFFxxxxxxxxxxxxxx hex number. 

It must also contain the correct local time/date (if you want your LCD display to have an accurate clock).

Last, and there is some controversy about this, you probably need to update the "epoch" field in the reply packet to match the value that the GW expects.  It is my opinion that if the epoch value is not set correctly, then the GW will interpret this as an internet interruption and start sending historical packets again.

I have this handshake working perfectly in the latest (unreleased) version of SkySpy.  The only time I see 210-byte packets is when I stop or pause the SkySpy service long enough for the GW to time out.

I have been pretty busy lately doing other stuff besides software development, but I have been working on the user manual this week.  I am up to 30 pages so far :-) 

I will need a couple brave souls to test the next version, sometime in the next week or two.  It supports both embedded MySQL as well as full-up MySQL.  It will autocreate the necessary MySQL schemas.  I have changed the original schemas slightly - enough to make them incompatible.  It should be relatively easy to migrate old schema data to the new format, but I have not gotten around to that.

Back to your problem:

The long 210-byte packets are a bit of a red herring.  I believe they are sent to LCX to catch up with historical sensor readings after a prolonged internet outage.  You don't need to deal with them if you are consistently logging/parsing the 197-byte Sensor Data Packets (SDP).  The SDP contains a single set of the most recent sensor readings. 

If you fail to properly handshake the 5-byte packet the GW sends out, the GW interprets this as a loss of connectivity.  The GW then starts sending historical data over-and-over-and-over until it gets a valid handshake from the "LCX server".  In this case the server is the SkySpy (or whatever) server.

Since you are not handshaking properly, most of your traffic will be useless historical data packets.

SkySpy ignores any 210 byte packets that pass by.  Typically your GW will sync up with SkySpy after a couple packets and stop sending them.  The only time they return is if you stop the SkySpy service long enough for the "internet" indicator to extinguish on the LCD panel.  My guess it that it times out after 5-10 minutes, but I have not measured it.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 12, 2014, 07:20:41 PM
You need to send a properly formatted 0010 reply to the GW after it requests the current time in a 0100 5-byte packet. 

The reply must contain your PWS serial number issued by LCX - the 7FFFxxxxxxxxxxxxxx hex number. 

It must also contain the correct local time/date (if you want your LCD display to have an accurate clock).

Last, and there is some controversy about this, you probably need to update the "epoch" field in the reply packet to match the value that the GW expects.  It is my opinion that if the epoch value is not set correctly, then the GW will interpret this as an internet interruption and start sending historical packets again.

I have this handshake working perfectly in the latest (unreleased) version of SkySpy.  The only time I see 210-byte packets is when I stop or pause the SkySpy service long enough for the GW to time out.

Instead of calling it "epoch", I call it "last history memory address".  When you get an history packet (an 01:01 packet with a first byte = 0x21, the last history address is in bytes 4-5.  I've been putting this in bytes 22-23 of my 14:01 replies to the 01:00 packets, and I've been getting one 30-byte history packet every 15 minutes or so.

I've stopped using your SkySpy service now that I've got my own web server doing the job, but it was very helpful in the early stages being able to snoop the traffic to and from it.  One comment though: it kept filling up the sensordatavalues table with old timestamps, never with any apparently valid data.

I'm still wrestling with what to do with the historical data myself.  I can't see how storing the values coming from the unit itself will be helpful, as I can always look up the same data from my database anytime.  I might store frequently required values in a history table to relieve my database from looking up the same thing over and over again, but I still don't think I'd populate that table with data from the weather station.  Maybe there's something there that I can't get from the db, but I can't see it.

P.S. I didn't emphasize that SkySpy was very helpful.  =D&gt; to you.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 12, 2014, 11:40:21 PM
I'm still wrestling with what to do with the historical data myself.  I can't see how storing the values coming from the unit itself will be helpful, as I can always look up the same data from my database anytime.  I might store frequently required values in a history table to relieve my database from looking up the same thing over and over again, but I still don't think I'd populate that table with data from the weather station.  Maybe there's something there that I can't get from the db, but I can't see it.

I think historical data might be useful if the server is down for some time. I don't try to achieve good reliability with UPSes, etc., and times when the power is out might be the times I later wish I had the data. I suspect the console/gateway will save history until the server is back up, at which time the server will store whatever was missed.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 13, 2014, 07:43:47 PM
I think historical data might be useful if the server is down for some time. I don't try to achieve good reliability with UPSes, etc., and times when the power is out might be the times I later wish I had the data. I suspect the console/gateway will save history until the server is back up, at which time the server will store whatever was missed.
Good point.  Saving the history as extra data points in sensorvalues certainly won't hurt.  I think the only problem is to be careful with the units, as some of those in the history packets seem to be in different formats than the current data packets.  Since my algorithm for saving the last memory address occurs after the history packet is sent, it should catch me up after an outage.

My versions of request.breq, Weather.htm (which displays my data) and wsdata.php (which retrieves the data) are almost working well enough to post.  I need to document a few tiny modifications to SkyDvr database schema that I made.  I also want it to work for multiple weather stations.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 13, 2014, 08:37:05 PM
Has anybody else noticed that the max gust per cycle (if that's what it really is) in bytes 160-161 of the sensor data packets are all multiples of 0xd8?  The current wind speeds in bytes 145-146 are all multiples of 0x24  (1/6 of the intervals for gusts) as well.  I've been interpreting both values as unsigned integers of 0.01 km/hr, meaning I'm getting a granularity of .36 kph and 2.16 kph for wind and gusts respectively.

The values I'm getting seem rather low to me.  I get daily thunderstorms where I live, with gusts often over 40 mph, but the highest wind values (current or gust) that I've seen are 0x534 (13.32 kph or 8.33mph).  I'm suspecting the units might be 0.1 km/h, not 0.01.

BTW, I pointed my leaf blower at the anemometer for a while, so a top speed of 83 mph is not ridiculous.

Am I way off?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 14, 2014, 04:10:04 PM
Has anybody else noticed that the max gust per cycle (if that's what it really is) in bytes 160-161 of the sensor data packets are all multiples of 0xd8?  The current wind speeds in bytes 145-146 are all multiples of 0x24  (1/6 of the intervals for gusts) as well.  I've been interpreting both values as unsigned integers of 0.01 km/hr, meaning I'm getting a granularity of .36 kph and 2.16 kph for wind and gusts respectively.

The values I'm getting seem rather low to me.  I get daily thunderstorms where I live, with gusts often over 40 mph, but the highest wind values (current or gust) that I've seen are 0x534 (13.32 kph or 8.33mph).  I'm suspecting the units might be 0.1 km/h, not 0.01.
I use the same decoding algorithm for all three wind values, at bytes 145, 157, and 160, and they seem to correlate ok. One check is the max gust value at byte 157. This should always match the console. I divide the 4-nybble binary value by 161.9 (there is a typo in my writeup, BTW). Are you using four nybbles for each value?

I did a histogram of wind gust values a while back. They only come in discrete values, but mine aren't always multiples of 0xd8. Most of the values seen differed by 0xd8, but a couple of values differed by 0xfc. I think this may be to compensate for inaccuracies in the anemometer.

I think the 6:1 difference in granularity between the wind speed and gust is because the speed is an average of 6 gust values. But I think the scaling is the same for both. 6 also corresponds to the number of wind direction nybbles in the SDP, at byte 149. This may be a function of how often SDPs are sent out, but I haven't done any testing.

Quote
BTW, I pointed my leaf blower at the anemometer for a while, so a top speed of 83 mph is not ridiculous.
My leaf blower must be a bit wimpy. When I did that, it only went up to about 35 mph  :-(
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 14, 2014, 08:12:28 PM
Here's a little snippet of PHP I use to calculate the heat index.  I store this value in the wind chill sensor, which makes that sensor more of a "Feels Like" temperature.  Note that if it is actually cold enough for the wind chill to be a factor, this function won't change it.
Code: [Select]
function heatindex($t, $rh) { // T in deg F
//Valid for T>80 F and RH>40%
if ($t < 80 || $rh < 40) return $t;
//{HI} = c_1 + c_2 T + c_3 R + c_4 T R
// + c_5 T^2 + c_6 R^2 + c_7 T^2R + c_8 T R^2 + c_9 T^2 R^2\ \,
//Constants from NOAA
$c = [ 0, -42.379, 2.04901523, 10.14333127, -0.22475541,
-6.83783E-3, -5.481717E-2, 1.22874E-3, 8.5282E-4, -1.99E-6 ];
$t2 = pow($t,2);
$rh2 = pow($rh,2);
$hi = $c[1] + $c[2]*$t + $c[3]*$rh + $c[4]*$t*$rh
+ $c[5]*$t2 + $c[6]*$rh2 + $c[7]*$t2*$rh + $c[8]*$t*$rh2 + $c[9]*$t2*$rh2;
return round($hi,1);
}
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 17, 2014, 02:53:49 AM
Can anyone make more sense of the rainfall data than I am?

According to keckec, the 7-nyb value at nyb 184 is the previous month rain.
Likewise, the 6-nyb value at 206 is the previous week rain.

My server has only been collecting data reliably since July 8, so I can't say anything about the monthly rainfall value except that in my case, it has been constant at 181.9 mm since I started collecting.

However, I definitely have more than a week of data, but my weekly rainfall value has been almost constant at 63.2 mm, the only variation being a few values of 75.0 on July 4 (before reliable collection began).

The range of the values seems appropriate for monthly and weekly rainfalls, by why aren't they changing?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 17, 2014, 09:31:39 AM
Can anyone make more sense of the rainfall data than I am?

According to keckec, the 7-nyb value at nyb 184 is the previous month rain.
Likewise, the 6-nyb value at 206 is the previous week rain.

My server has only been collecting data reliably since July 8, so I can't say anything about the monthly rainfall value except that in my case, it has been constant at 181.9 mm since I started collecting.

However, I definitely have more than a week of data, but my weekly rainfall value has been almost constant at 63.2 mm, the only variation being a few values of 75.0 on July 4 (before reliable collection began).

The range of the values seems appropriate for monthly and weekly rainfalls, by why aren't they changing?
As I understand it, the previous month's rain only changes on the first day of the month. Likewise the previous week's rain only changes when the week changes. The start of the week is defined as midnight of the day prior to the day of the week when batteries were last installed in the console. I think 1-hr and 24-hr rain are moving windows. They may put a timestamp on each bucket tip, and tally how many tips fall within each moving window.
Check out page 21 of the manual here:
http://www.lacrossetechnology.com/c84612/manual.pdf
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 17, 2014, 11:56:56 AM
Likewise the previous week's rain only changes when the week changes. The start of the week is defined as midnight of the day prior to the day of the week when batteries were last installed in the console.
Yup! That's what the manual says.  But assuming my console's week ended July 4, I would have expected the data to change again on July 12, but it didn't.  It would be an amazing coincidence if the rainfall from one week to the next was the same down to the last .01 mm!  I'll see what happens July 19, I guess.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 17, 2014, 03:23:51 PM
On the subject of rainfall, how are you guys detecting bucket tips from sensor and/or history data packets? I think the data available from the gateway is:
1-hour amount, 24-hour amount, total since reset, last reset time, and rainfall during last history period.
Ideally, I'd like to save a timestamp for each tip of the bucket, but that doesn't seem to be easily done. Any ideas?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 18, 2014, 04:11:51 PM
how are you guys detecting bucket tips from sensor and/or history data packets?

I just use the difference between the current rain since reset value from the last packet.  Sounds simple, but my code must take account of the possibility that a reset occurred between packets.  I also use this data to calculate running weekly and 30 day rain totals, since the given definition of weekly and monthly data doesn't work for me.  I can show you my code, if you're interested.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 19, 2014, 03:14:12 PM
I just use the difference between the current rain since reset value from the last packet.  Sounds simple, but my code must take account of the possibility that a reset occurred between packets.  I also use this data to calculate running weekly and 30 day rain totals, since the given definition of weekly and monthly data doesn't work for me.
Thanks for the input. The user reset does add complexity. I keep thinking there might be a way to remotely reset things via one of the packets sent to the gateway.

Something I might try is to use the rainfall value from the history records to tally rainfall. The php code could monitor the 1-hour rain value in the SDP, and when it first registers rain shorten the history interval to 5 minutes or even 1 minute. Maybe even adjust the history interval according to how hard it's raining. When the 24-hour rain in the SDP goes to zero, set the history interval back to its normal value.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 19, 2014, 05:06:36 PM
Something I might try is to use the rainfall value from the history records to tally rainfall.
I have a problem with the rainfall data in the history packets, in that it doesn't agree with the value from the SDPs.  For example, this morning we had a brief light rain.  The SDP reported 0.25 mm, but the history reported 0.125 mm.  Because I'm currently using data from both in my rainfall graphs, it introduces noise.  Look at the rainfall graph today on my weather page, page removed.  (Please don't sit on that page, it refreshes every 5 minutes and really whacks the database for about 15 seconds.)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 21, 2014, 11:01:17 AM
I have a problem with the rainfall data in the history packets, in that it doesn't agree with the value from the SDPs.  For example, this morning we had a brief light rain.  The SDP reported 0.25 mm, but the history reported 0.125 mm.  Because I'm currently using data from both in my rainfall graphs, it introduces noise.
Are you sure about the history scaling? I think the history rainfall field is in bucket tips, with each tip being about 0.01", or 0.25mm.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 21, 2014, 03:23:17 PM
I came across a way to change the interval for sending SDP packets, from the 14:01 packet sent in response to the 0100 packet. In reply 322 of this thread, I wrote that bytes 0x14 and 0x1f should both be set to the same values. That was wrong. It turns out that byte 0x14 controls the SDP interval and 0x1f controls the history record interval. The description of the history interval in reply 322 still holds for byte 0x1f.
Byte 0x14 sets the approximate maximum interval between SDPs to one minute greater than the value of the byte. So the default value of 3 sets the SDP interval to 4 minutes, and a value of 0 sets a 1-minute interval. If this value is set more than about 15 minutes, the GW often sends additional SDPs in between. It also appears that a history packet is not sent unless an SDP is sent first. So for long SDP intervals and short history intervals, history packets will contain more than one history record. There's probably more interaction with other things yet to be discovered.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 21, 2014, 06:59:20 PM
I came across a way to change the interval for sending SDP packets, from the 14:01 packet sent in response to the 0100 packet.
I've been controlling the SDP frequency by specifying the number of seconds  between packets (currently using 300) in the last two bytes of the 18-byte 70:00 reply to the 00:70 packets.  I've been setting 0x3 in both bytes 0x14 and 0x1f of the 14:01 packets.

Therefore, according to your analysis, I'm telling the gateway to send SDP packets every 4 minutes "your" way, and every 5 minutes "my" way (which was your way, since I copied your code to start with).  And I'm getting SDPs every five minutes, and history every 15.  So apparently, the 70:00 reply is controlling?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 21, 2014, 07:24:28 PM
Are you sure about the history scaling? I think the history rainfall field is in bucket tips, with each tip being about 0.01", or 0.25mm.
It's been raining here for an half-hour or so:  These are my data
Code: [Select]
Time  Ptype Stype Value
19:09 2 10 6.19
19:05 2 10 4.9
19:01 2 10 4.38
19:00 9 10 0
18:58 2 10 4.12
18:57 2 10 3.87
18:53 2 10 3.35
18:49 2 10 2.58
18:45 2 10 2.32
18:45 9 10 0
18:42 2 10 2.32
18:41 2 10 2.06
18:30 9 10 0.01
18:21 2 10 0
18:17 2 10 0
18:15 9 10 0.01
18:13 2 10 0
18:09 2 10 0
18:05 2 10 0
18:01 2 10 0
17:49 2 10 0
17:45 2 10 0
Packet type 2 are SDPs, and type 9 are history values.  All the sensor types are current rain hour on my system.  The values are as reported by the weather station, no scaling except for the decimal points (i.e. whatever units they are, I not converting them to anything).
As you can see, the history packets were reporting something before the SDPs did.  And while the SDP values continued to rise, the history values were 0's.  I only see history values of 0, 1, 2 and 3 in my data, so clearly these are not bucket tips.

Update: until we can figure these out, I'm removing them from my datasets.
Update 2: I only see values 0, 1, and 2.  There may have been history packets before I started saving them that had values of 3, but I can't prove that now.

Update 3: Based upon a review of the rainfall values I'm getting vs. the readings from a standard rain guage, I think a much closer interpretation of the rainfall values is that they are in thousandths of an inch.

Update 4: Based upon a comparison of the current rain since reset value of 15.996 in the SDP vs. 0.62 in. on the display, it seems that that the rain values are indeed in units of 0.01mm.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 21, 2014, 11:49:18 PM
Very interesting reading.

Thanks to your hard work I was able to tweek my system into sending data to WU via the a Gateway-sending-to-a-Proxy which is my Oracle Linux VM running the PHP Request code from: https://github.com/lowerpower/LaCrosse (https://github.com/lowerpower/LaCrosse)
and it works very well.

Based on your recent posts, I tried playing with the settings for the interval it sends in Weather data...not much luck.  Still runs around 5 minute intervals.

I am very interested in testing new settings, if you need someone to try them out.

Dave
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 22, 2014, 01:49:12 AM
Based on your recent posts, I tried playing with the settings for the interval it sends in Weather data...not much luck.  Still runs around 5 minute intervals.

I am very interested in testing new settings, if you need someone to try them out.
If you want to try something, set byte 0x14 of the 14:01 reply packet to 0. On mine, that changes the interval of the sensor data packet to 1 minute. For reference, the history interval setting in byte 0x1f of the same packet is 0x03 (15 minutes), and byte 0x11 of the 70:00 packet is 0xf0 (240 seconds). I believe this value sets the interval of the 00:70 "ping" packet, which I think is used as a keep-alive to maintain the connection.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 22, 2014, 09:52:58 AM
I have changed these settings, again this morning.  Do you have to reset the Gateway after changing these to get the 1 minute interval to work?

I noticed that the access.log shows the change of the 240 seconds.  I played with [byte 0x11 of the 70:00] and moved it to 20 seconds, the log changed, then 30 seconds and the logging reflected that change to.  Log keep alive length is 18.  The 38 byte log entry is the Weather data packet.

This below is a sample of the size...not the time changes.

Code: [Select]
192.168.2.206 - - [22/Jul/2014:09:34:41 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:34:42 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:34:44 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:36:05 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:37:26 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:38:34 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:09:38:43 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:38:46 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:38:47 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:38:49 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"

So the changes are in place in the PHP code, and I will post later what the results are.

Dave
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 22, 2014, 10:06:17 AM
Intervals are more frequent but seems like some duplication maybe, access log and php log below:

Code: [Select]
192.168.2.206 - - [22/Jul/2014:09:49:51 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:09:49:59 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:50:02 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:50:03 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:50:05 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:53:56 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:09:54:04 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:54:07 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:54:08 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:54:10 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:55:03 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:09:55:12 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:55:15 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:55:16 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:55:18 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:58:10 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:09:58:18 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:58:21 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:58:22 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:09:58:24 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:10:02:15 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:10:02:23 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:10:02:26 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:10:02:27 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:10:02:29 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"


PHP log below:
Code: [Select]
write weather underground : 2014-07-22 09:49:51 : &tempf=74.12&humidity=83&dewptf=69.77&windspeedmph=5.34&windgustmph=6.89&winddir=154&baromin=28.12&rainin=0.00&dailyrainin=0.03&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 09:53:56 : &tempf=74.12&humidity=83&dewptf=69.77&windspeedmph=2.45&windgustmph=6.89&winddir=115&baromin=28.12&rainin=0.00&dailyrainin=0.03&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 09:55:03 : &tempf=74.30&humidity=83&dewptf=69.95&windspeedmph=1.56&windgustmph=2.67&winddir=187&baromin=28.12&rainin=0.00&dailyrainin=0.03&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 09:58:10 : &tempf=74.48&humidity=82&dewptf=69.84&windspeedmph=3.34&windgustmph=8.23&winddir=127&baromin=28.11&rainin=0.00&dailyrainin=0.03&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 10:02:15 : &tempf=74.30&humidity=82&dewptf=69.67&windspeedmph=4.00&windgustmph=6.89&winddir=153&baromin=28.12&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 22, 2014, 10:59:22 AM
Interesting. Here is my log with byte 0x14 set to 0. Doesn't look like duplicates, the data is different in each SDP.

Code: [Select]
192.168.1.16 - - [22/Jul/2014:07:49:17 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:49:20 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 0 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:49:29 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:49:30 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:50:38 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 0 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:50:47 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:50:48 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:51:56 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 0 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:52:05 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:52:06 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:53:14 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 0 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:53:23 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:53:24 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:54:32 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 0 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:54:41 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.1.16 - - [22/Jul/2014:07:54:42 -0700] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"

I wonder if it could be different firmware versions in the station. I think there is a way to get the version, but don't remember.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 22, 2014, 11:09:58 AM
Maybe my PHP code is old?  I downloaded it from the GITHUB lowerpower and the code is about a month old.

https://github.com/lowerpower/LaCrosse (https://github.com/lowerpower/LaCrosse)

Are there updates available?


Dave
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 22, 2014, 01:00:30 PM
Could this be resources related on the host executing the PHP code?
 - I could add another virtual CPU if you think it would help.

The latest access log:
Code: [Select]
192.168.2.206 - - [22/Jul/2014:12:30:27 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:30:35 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:30:38 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:30:39 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:30:41 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:34:32 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:34:40 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:34:43 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:34:44 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:34:46 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:38:37 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:38:45 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:38:48 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:38:49 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:38:51 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:42:42 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:42:50 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:42:53 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:42:54 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:42:56 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:45:48 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:45:56 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:45:59 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:46:00 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:46:02 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:46:56 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:47:04 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:47:07 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:47:08 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:47:10 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:50:02 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:50:10 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:50:13 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:50:14 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:50:16 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:54:07 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 - "-" "-"
192.168.2.206 - - [22/Jul/2014:12:54:15 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:54:18 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:54:19 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 38 "-" "-"
192.168.2.206 - - [22/Jul/2014:12:54:21 -0400] "PUT http://box.weatherdirect.com/request.breq HTTP/1.1" 200 18 "-" "-"

Here is the PHP Log:

Code: [Select]
write weather underground : 2014-07-22 11:42:26 : &tempf=75.92&humidity=79&dewptf=70.38&windspeedmph=1.78&windgustmph=4.00&winddir=97&baromin=28.11&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 11:45:32 : &tempf=75.92&humidity=81&dewptf=70.96&windspeedmph=1.78&windgustmph=4.00&winddir=217&baromin=28.11&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 11:50:35 : &tempf=76.28&humidity=81&dewptf=71.3&windspeedmph=2.45&windgustmph=4.00&winddir=148&baromin=28.11&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 11:54:41 : &tempf=76.64&humidity=81&dewptf=71.65&windspeedmph=0.89&windgustmph=5.34&winddir=205&baromin=28.11&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 11:57:47 : &tempf=77.36&humidity=77&dewptf=71.17&windspeedmph=4.22&windgustmph=5.34&winddir=232&baromin=28.11&rainin=0.00&dailyrainin=0.01&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:01:52 : &tempf=77.54&humidity=78&dewptf=71.64&windspeedmph=3.34&windgustmph=5.34&winddir=176&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:05:57 : &tempf=77.72&humidity=78&dewptf=71.82&windspeedmph=0.67&windgustmph=1.33&winddir=165&baromin=28.11&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:10:02 : &tempf=78.26&humidity=79&dewptf=72.63&windspeedmph=1.56&windgustmph=2.67&winddir=183&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:14:07 : &tempf=78.44&humidity=78&dewptf=72.51&windspeedmph=0.89&windgustmph=1.33&winddir=226&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:18:12 : &tempf=78.44&humidity=78&dewptf=72.51&windspeedmph=1.33&windgustmph=1.33&winddir=308&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:22:17 : &tempf=78.62&humidity=79&dewptf=72.98&windspeedmph=3.11&windgustmph=4.00&winddir=222&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:26:22 : &tempf=78.80&humidity=78&dewptf=72.86&windspeedmph=3.34&windgustmph=5.34&winddir=247&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:30:27 : &tempf=78.80&humidity=78&dewptf=72.86&windspeedmph=1.33&windgustmph=1.33&winddir=180&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:34:32 : &tempf=78.98&humidity=77&dewptf=72.73&windspeedmph=3.11&windgustmph=4.00&winddir=105&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:38:37 : &tempf=79.16&humidity=77&dewptf=72.9&windspeedmph=0.67&windgustmph=1.33&winddir=253&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:42:42 : &tempf=79.34&humidity=75&dewptf=72.45&windspeedmph=0.67&windgustmph=1.33&winddir=95&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:45:48 : &tempf=79.16&humidity=76&dewptf=72.59&windspeedmph=0.44&windgustmph=1.33&winddir=211&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:46:56 : &tempf=79.16&humidity=76&dewptf=72.59&windspeedmph=2.45&windgustmph=4.00&winddir=207&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:50:02 : &tempf=79.16&humidity=76&dewptf=72.59&windspeedmph=1.33&windgustmph=2.67&winddir=281&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
write weather underground : 2014-07-22 12:54:07 : &tempf=78.62&humidity=75&dewptf=71.77&windspeedmph=2.45&windgustmph=6.89&winddir=97&baromin=28.10&rainin=0.00&dailyrainin=0.00&softwaretype=phpLaxWeatherBro : returned:success
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 22, 2014, 03:29:07 PM
Hmmm. I hadn't seen the GITHUB page. The code is older than what I'm using. I just posted a newer version of my request.breq at
http://www.keckec.com/weather/request.breq
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 22, 2014, 04:09:23 PM
If you want to try something, set byte 0x14 of the 14:01 reply packet to 0. On mine, that changes the interval of the sensor data packet to 1 minute. For reference, the history interval setting in byte 0x1f of the same packet is 0x03 (15 minutes), and byte 0x11 of the 70:00 packet is 0xf0 (240 seconds). I believe this value sets the interval of the 00:70 "ping" packet, which I think is used as a keep-alive to maintain the connection.
I can confirm the meaning of byte 0x14 in the 14:01 packet.  I set mine to 1, 3 and 4 and got sensor data packets approximately every 2, 4 and 5 minutes.
Now I'm going to test bytes 0x10-0x11 in the 70:00 packets.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 22, 2014, 05:20:20 PM
I've decided to make my versions of the various files I'm using available for inspection. DO NOT USE THESE ON YOUR SYSTEM WITHOUT SCRUTINIZING THEM CAREFULLY
weather.tgz (https://onedrive.live.com/redir?resid=983E422818223F10!1465&authkey=!AJDMd46gjobUCZI&ithint=file%2c.tgz)

Also, I've been experimenting with the bytes at the end of the 70:00 packet, and they don't seem to have any effect on any of the incoming packets, so far.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: mycal on July 22, 2014, 11:00:52 PM

The Lowerpower github is my archive, I have not added the history stuff that the others have been doing,
I just took the modifications that keckec did and added them and the weather underground push code.

At some point I will update it, but I've got pay projects I need to get done first....

From what I know the 70:00 Packet only controls the gateway ping, not the weather station data flow.  So you will get more packets, but not any more SDP packets with this setting.

I think kennkong is right about the SDP flow is based on the 14:01 packet.

-M




Maybe my PHP code is old?  I downloaded it from the GITHUB lowerpower and the code is about a month old.

https://github.com/lowerpower/LaCrosse (https://github.com/lowerpower/LaCrosse)

Are there updates available?


Dave
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 23, 2014, 12:43:48 AM
From what I know the 70:00 Packet only controls the gateway ping, not the weather station data flow.  So you will get more packets, but not any more SDP packets with this setting.

I think kennkong is right about the SDP flow is based on the 14:01 packet.
First, give credit where credit is due; keckec discovered that the 14:01 packet byte 0x14 controls the SDP flow.

I can now confirm that the last two bytes of the 70:00 packet control the frequency of the 00:70 packets, except that other packets seem to trigger them, too.  So setting my value to 900 (15 minutes) meant I was still getting one after every 01:00 packet, which seem to happen after every SDP.  So you can speed them up some, but there's a lower limit to how few you get.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on July 23, 2014, 12:53:03 AM
I'm refactoring my code to handle multiple stations.  Why am I doing this when I have just one?  Maybe I'm a masochist, but probably just a perfectionist.  Whatever!

Would someone care to arrange to send their data my way for a limited time?  At first, I'd like to test getting data from an unsolicited station, to make sure that my code rejects it.  Then I'd need to get some information (like the MAC address and serial number data that Lacrosse sends) so that I could set it up to accept data.

I can forward the packets back to you, so that you don't lose any data.  Any such test would only be for an hour or so, anyway.  Any volunteers?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 23, 2014, 02:53:19 PM
Hmmm. I hadn't seen the GITHUB page. The code is older than what I'm using. I just posted a newer version of my request.breq at
http://www.keckec.com/weather/request.breq

Excellent, I downloaded it and looked it over in the few free minutes I had this afternoon.

-- Wish I could start testing with it now.  Unfortunately, I need to focus on my billable work for a while.

Dave
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 25, 2014, 07:22:44 PM
I need some brave souls to download, install and test the next version of SkySpy.  Feedback is welcome!

Make sure you back up any existing installation and data before installing the new version.

Here is a link to the users manual.  I recommend reading it prior to starting the installation.  As usual, the installer will put another copy in your installation folder.

https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf (https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf)

Here is a link to the installer:

https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe (https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on July 26, 2014, 12:48:29 AM
Would someone care to arrange to send their data my way for a limited time?  At first, I'd like to test getting data from an unsolicited station, to make sure that my code rejects it.  Then I'd need to get some information (like the MAC address and serial number data that Lacrosse sends) so that I could set it up to accept data.

I can forward the packets back to you, so that you don't lose any data.  Any such test would only be for an hour or so, anyway.  Any volunteers?
OK, I'll bite. I sent you a PM.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 29, 2014, 02:15:10 PM
I need some brave souls to download, install and test the next version of SkySpy.  Feedback is welcome!

Make sure you back up any existing installation and data before installing the new version.

Here is a link to the users manual.  I recommend reading it prior to starting the installation.  As usual, the installer will put another copy in your installation folder.

https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf (https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf)

Here is a link to the installer:

https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe (https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe)

I downloaded it (never used it before)  installed MYSQL DB first...

Not enough info provided in the Readme to get it to work.

Found the utility from a few months ago.  ran it, didn't work.

Found the SQL statements in the install directory, didn't work.

uninstalled.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 29, 2014, 02:25:21 PM
I recommend using the embedded database for now, if you want to try out the program.

The full-up MySQL install is fairly complex, and not well tested.  That is why I don't cover it in the manual.

I could send you MySQL scripts to create the two schemas that are required.  I saw from your error messages that you had table and DB hostname issues.  You may not have given yourself sufficient privileges on MySQL to create and modify new tables.

Before you overwrite them, could you send me a backup of the two schemas (if any) on your MySQL server box:

weather2
utility2

I'd like to see what got created, so I can debug/update the internal MySQL creation scripts that are supposed to fire on initial installation.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dld121 on July 29, 2014, 02:32:31 PM
I am happy to send you what it built, but I am not a MYSQL expert.  Can you tell me what to do to back it up?

I was using MYSQL on  port 3308 and set your app to do the same.

The DB settings are:
My INI settings are:
Code: [Select]
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you
# *** upgrade to a newer version of MySQL.
[client]
port=3308

[mysqld]

# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
innodb_buffer_pool_size = 128M

# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin

# These are commonly set, remove the # and set as required.
basedir = "C:\\mysql56\\"
datadir = "C:\\mysql56\\data\\"
port = 3308
#server_id = ....


# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
join_buffer_size = 128M
sort_buffer_size = 2M
read_rnd_buffer_size = 2M

sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on July 29, 2014, 02:48:58 PM
I need some brave souls to download, install and test the next version of SkySpy.  Feedback is welcome!

Make sure you back up any existing installation and data before installing the new version.

Here is a link to the users manual.  I recommend reading it prior to starting the installation.  As usual, the installer will put another copy in your installation folder.

https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf (https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf)

Here is a link to the installer:

https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe (https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe)


After thinking about it for a bit, I'm guessing I should have Uninstalled the old version first.  I guess I'm just too used to just over writing the old version.  Maybe it would be a good idea to put a note in the quick setup reminding us to uninstall first.  Will try uninstalling old and do a clean install of the new version. When I get back home.

After recovering from my brain fart the installation went smoothly without any issues.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 29, 2014, 07:21:22 PM
After recovering from my brain fart the installation went smoothly without any issues.

Great!  Which mode are you using?  Embedded or Full MySQL?  I need to shut my public MySQL server off soon, before I get in Dutch with my ISP.  I am up to a 319 gigabytes of Internet usage this month.  Eeek!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 29, 2014, 08:09:31 PM
I am happy to send you what it built, but I am not a MYSQL expert. 

You are in for a spot of trouble then.  If you are a intermediate or better programmer and a network person, then you can dive right in and muddle your way through.  Otherwise, bad things will happen.  Setting up any sort of database engine is non-trivial.

Quote
Can you tell me what to do to back it up?

Download, configure and run MySQL Workbench. The Community version is free.

Use the Workbench Data Export feature to back up your schemas and data.  Zip or RAR the resulting SQL files and send them to me.

Quote
I was using MYSQL on  port 3308 and set your app to do the same.

If you are going to run your own server, then you should probably leave the port set to the default 3306.  I used 3308 for my public server since I run more than one MySQL server on my LAN and need to keep them separate and also allow for port forwarding through my router-firewall.

Quote
The DB settings are:

??? Nothing came through.

Quote
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you

As the file header says, it is just a template for initial installation. You need to use the Workbench to actually configure MySQL.  There are hundreds of settings.  Most of the defaults will work, but there are a few that probably need tweaking, depending on your computer hardware and needs.  This is where the programming and network skills come in.

You must run the Workbench utility on the same PC as the database engine if you want to adjust settings.  Once you have MySQL configured, you can monitor it from any workstation on your LAN using the same utility.

You need to restart MySQL to make any settings changes "take".


Here is a typical MySQL settings file (mysql.ini) for a v5.6.19 installation on Windows Server 2008, 64-bit.  Do NOT use the settings here as they are specific to the server hardware used.  These are not the settings I use on my server, since I can't access them just now.

Code: [Select]
# Other default tuning values
# MySQL Server Instance Configuration File
# ----------------------------------------------------------------------
# Generated by the MySQL Server Instance Configuration Wizard
#
#
# Installation Instructions
# ----------------------------------------------------------------------
#
# On Linux you can copy this file to /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options
# (@localstatedir@ for this installation) or to
# ~/.my.cnf to set user-specific options.
#
# On Windows you should keep this file in the installation directory
# of your server (e.g. C:\Program Files\MySQL\MySQL Server X.Y). To
# make sure the server reads the config file use the startup option
# "--defaults-file".
#
# To run run the server from the command line, execute this in a
# command line shell, e.g.
# mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# To install the server as a Windows service manually, execute this in a
# command line shell, e.g.
# mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# And then execute this in a command line shell to start the server, e.g.
# net start MySQLXY
#
#
# Guildlines for editing this file
# ----------------------------------------------------------------------
#
# In this file, you can use all long options that the program supports.
# If you want to know the options a program supports, start the program
# with the "--help" option.
#
# More detailed information about the individual options can also be
# found in the manual.
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
#
#
# CLIENT SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by MySQL client applications.
# Note that only client applications shipped by MySQL are guaranteed
# to read this section. If you want your own MySQL client program to
# honor these values, you need to specify it as an option during the
# MySQL client library initialization.
#
[client]
no-beep

# pipe
# socket=mysql
port=3306

[mysql]

default-character-set=utf8


# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this
# file.
#
# server_type=3
[mysqld]

# The next three options are mutually exclusive to SERVER_PORT below.
# skip-networking
# enable-named-pipe
# The Pipe the MySQL Server will use
# socket=mysql

# The TCP/IP Port the MySQL Server will listen on
port=3306

# Path to installation directory. All paths are usually resolved relative to this.
# basedir="C:/Program Files/MySQL/MySQL Server 5.6/"

# Path to the database root
datadir="C:/ProgramData/MySQL/MySQL Server 5.6/data\"

# The default character set that will be used when a new schema or table is
# created and no character set is defined


# The default storage engine that will be used when create new tables when
default-storage-engine=INNODB

# Set the SQL mode to strict
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

# Enable Windows Authentication
# plugin-load=authentication_windows.dll

# General and Slow logging.
log-output=NONE
general-log=0
general_log_file="WOLVERINE.log"
slow-query-log=0



# Binary Logging.
# log-bin

# Error Logging.
log-error="WOLVERINE.err"

# The maximum amount of concurrent sessions the MySQL server will
# allow. One of these connections will be reserved for a user with
# SUPER privileges to allow the administrator to login even if the
# connection limit has been reached.
max_connections=151

# Query cache is used to cache SELECT results and later return them
# without actual executing the same query once again. Having the query
# cache enabled may result in significant speed improvements, if your
# have a lot of identical queries and rarely changing tables. See the
# "Qcache_lowmem_prunes" status variable to check if the current value
# is high enough for your load.
# Note: In case your tables change very often or if your queries are
# textually different every time, the query cache may result in a
# slowdown instead of a performance improvement.
query_cache_size=0

# The number of open tables for all threads. Increasing this value
# increases the number of file descriptors that mysqld requires.
# Therefore you have to make sure to set the amount of open files
# allowed to at least 4096 in the variable "open-files-limit" in
# section [mysqld_safe]
table_open_cache=2000

# Maximum size for internal (in-memory) temporary tables. If a table
# grows larger than this value, it is automatically converted to disk
# based table This limitation is for a single table. There can be many
# of them.
tmp_table_size=118M

# How many threads we should keep in a cache for reuse. When a client
# disconnects, the client's threads are put in the cache if there aren't
# more than thread_cache_size threads from before.  This greatly reduces
# the amount of thread creations needed if you have a lot of new
# connections. (Normally this doesn't give a notable performance
# improvement if you have a good thread implementation.)
thread_cache_size=10

#*** MyISAM Specific options
# The maximum size of the temporary file MySQL is allowed to use while
# recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE.
# If the file-size would be bigger than this, the index will be created
# through the key cache (which is slower).
myisam_max_sort_file_size=100G

# If the temporary file used for fast index creation would be bigger
# than using the key cache by the amount specified here, then prefer the
# key cache method.  This is mainly used to force long character keys in
# large tables to use the slower key cache method to create the index.
myisam_sort_buffer_size=226M

# Size of the Key Buffer, used to cache index blocks for MyISAM tables.
# Do not set it larger than 30% of your available memory, as some memory
# is also required by the OS to cache rows. Even if you're not using
# MyISAM tables, you should still set it to 8-64M as it will also be
# used for internal temporary disk tables.
key_buffer_size = 25M

# Size of the buffer used for doing full table scans of MyISAM tables.
# Allocated per thread, if a full scan is needed.
read_buffer_size=64K
read_rnd_buffer_size=256K

# This buffer is allocated when MySQL needs to rebuild the index in
# REPAIR, OPTIMZE, ALTER table statements as well as in LOAD DATA INFILE
# into an empty table. It is allocated per thread so be careful with
# large settings.
sort_buffer_size=256K

#*** INNODB Specific options ***
# innodb_data_home_dir=0.0

# Use this option if you have a MySQL server with InnoDB support enabled
# but you do not plan to use it. This will save memory and disk space
# and speed up some things.
# skip-innodb

# Additional memory pool that is used by InnoDB to store metadata
# information.  If InnoDB requires more memory for this purpose it will
# start to allocate it from the OS.  As this is fast enough on most
# recent operating systems, you normally do not need to change this
# value. SHOW INNODB STATUS will display the current amount used.
innodb_additional_mem_pool_size = 47M

# If set to 1, InnoDB will flush (fsync) the transaction logs to the
# disk at each commit, which offers full ACID behavior. If you are
# willing to compromise this safety, and you are running small
# transactions, you may set this to 0 or 2 to reduce disk I/O to the
# logs. Value 0 means that the log is only written to the log file and
# the log file flushed to disk approximately once per second. Value 2
# means the log is written to the log file at each commit, but the log
# file is only flushed to disk approximately once per second.
innodb_flush_log_at_trx_commit=1

# The size of the buffer InnoDB uses for buffering log data. As soon as
# it is full, InnoDB will have to flush it to disk. As it is flushed
# once per second anyway, it does not make sense to have it very large
# (even with long transactions).
innodb_log_buffer_size=9M

# InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and
# row data. The bigger you set this the less disk I/O is needed to
# access data in tables. On a dedicated database server you may set this
# parameter up to 80% of the machine physical memory size. Do not set it
# too large, though, because competition of the physical memory may
# cause paging in the operating system.  Note that on 32bit systems you
# might be limited to 2-3.5G of user level memory per process, so do not
# set it too high.
innodb_buffer_pool_size=755M

# Size of each log file in a log group. You should set the combined size
# of log files to about 25%-100% of your buffer pool size to avoid
# unneeded buffer pool flush activity on log file overwrite. However,
# note that a larger logfile size will increase the time needed for the
# recovery process.
innodb_log_file_size=48M

# Number of threads allowed inside the InnoDB kernel. The optimal value
# depends highly on the application, hardware as well as the OS
# scheduler properties. A too high value may lead to thread thrashing.
innodb_thread_concurrency = 10

# The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full.
innodb_autoextend_increment=64

# The number of regions that the InnoDB buffer pool is divided into.
# For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency,
# by reducing contention as different threads read and write to cached pages.
innodb_buffer_pool_instances=8

# Determines the number of threads that can enter InnoDB concurrently.


# Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before
# it can be moved to the new sublist.
innodb_old_blocks_time=1000

# It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10.
innodb_open_files=300

# When this variable is enabled, InnoDB updates statistics during metadata statements.
innodb_stats_on_metadata

# When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table
# in a separate .ibd file, rather than in the system tablespace.
innodb_file_per_table=1

# Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none.
innodb_checksum_algorithm=0

# The number of outstanding connection requests MySQL can have.
# This option is useful when the main MySQL thread gets many connection requests in a very short time.
# It then takes some time (although very little) for the main thread to check the connection and start a new thread.
# The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily
# stops answering new requests.
# You need to increase this only if you expect a large number of connections in a short period of time.


# If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and
# synchronize unflushed data to disk.
# This option is best used only on systems with minimal resources.


# The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use
# indexes and thus perform full table scans.
join_buffer_size=256K

# The maximum size of one packet or any generated or intermediate string, or any parameter sent by the
# mysql_stmt_send_long_data() C API function.
max_allowed_packet = 160M

# If more than this many successive connection requests from a host are interrupted without a successful connection,
# the server blocks that host from performing further connections.


# Changes the number of file descriptors available to mysqld.
# You should try increasing the value of this option if mysqld gives you the error "Too many open files".
open_files_limit=4161

# Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND.
query_cache_type=0

# If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the
# sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization
# or improved indexing.
sort_buffer_size=256K

# The number of table definitions (from .frm files) that can be stored in the definition cache.
# If you use a large number of tables, you can create a large table definition cache to speed up opening of tables.
# The table definition cache takes less space and does not use file descriptors, unlike the normal table cache.
# The minimum and default values are both 400.
table_definition_cache=1400

# Specify the maximum size of a row-based binary log event, in bytes.
# Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256.
binlog_row_event_max_size=8K

# If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk.
# (using fdatasync()) after every sync_master_info events.


# If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk.
# (using fdatasync()) after every sync_relay_log writes to the relay log.


# If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk.
# (using fdatasync()) after every sync_relay_log_info transactions.


secure-auth = off
net_buffer_length = 2M
net_write_timeout = 1800
wait_timeout = 1800
basedir = "C:/Program Files/MySQL/MySQL Server 5.6"
interactive_timeout = 28800
innodb_lock_wait_timeout = 10
net_read_timeout = 1800

performance_schema = off


After you have a functional MySQL installation, you need to set up some user accounts.  You should probably create an account for SkySpy that only has access to the two necessary schemas and limit it to logging in from a specific IP address or subnet on your LAN if you use DHCP. This will keep the bad guys out of your server. You use the Workbench to create MySQL accounts and give them privileges.

So...  Do you see why I suggest using Embedded mode?  It is a heckuva lot simpler than full up MySQL.   :grin:


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on July 30, 2014, 01:10:32 AM
skydvrz

Anyway to get the data from the old database (weather) into the new (weather2) ??
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on July 30, 2014, 10:19:23 AM
Anyway to get the data from the old database (weather) into the new (weather2) ??

If you are using a full-up MySQL database engine, then yes.  Back it up and send it to me and I will translate it to the new format.  Are you still on my server?

Otherwise you can view/download your historical data from Weatherunderground.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on July 30, 2014, 12:21:26 PM
Anyway to get the data from the old database (weather) into the new (weather2) ??

If you are using a full-up MySQL database engine, then yes.  Back it up and send it to me and I will translate it to the new format.  Are you still on my server?

Otherwise you can view/download your historical data from Weatherunderground.
I'm on my own server, I'll send you the backup later today. Everything seems to be working great since I remembered to uninstall first.

Thanks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: benmargolin on July 31, 2014, 07:57:12 AM
I finally got around to trying some of this code, and used what I think was the latest snapshot of keckec's as well as kennkong. I couldn't get either one to work out of the box :( but after some cajoling I got the keckec stuff to mostly work -- I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!

I also ended up using this for determining my serial number: running mitmdump (http://mitmproxy.org/doc/mitmdump.html), which made it really easy.

I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(

I really appreciate all the work everyone's put into this effort! I wish I had more time to contribute.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on August 01, 2014, 11:01:20 AM
I finally got around to trying some of this code, and used what I think was the latest snapshot of keckec's as well as kennkong. I couldn't get either one to work out of the box :( but after some cajoling I got the keckec stuff to mostly work -- I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!
Sorry my php/schema isn't fully working. I'm sort-of feeling my way along, as this is the first time I've worked with PHP or SQL. The db schema has changed substantially several times, as I learn more.

I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(
I don't mind using github, and I currently use git with my own repo. So far my code is rough, and as you found out a lot of it doesn't work very well yet. I don't plan to send to WU, but store data for myself and a few friends, and provide web pages to view graphs, etc. I've been saving captures of all the packets to and from the station, and so far have about 500K packets. Right now I'm working on php code to import it into the db, which I doubt would be of much interest to others.

I'm still not sure the best way to gather rain data from the station. There are quite a few fields transmitted from the station that have to do with rain, but none that I see directly indicates bucket tips except the history packets. Even with history there is a compromise between storage depth and bucket-tip time granularity. And the history record may lose bucket tips when the history interval is changed. I'm still working on it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 01, 2014, 04:48:26 PM
I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!
I gather from context that you're talking about keckec's version of historical values, but it does bring up a point about my version that may not be clear.  My `records` table must be prepopulated with the records you want to keep, the request script only updates existing records.  It doesn't create missing ones.  This is because certain records like minimum wind and rainfall are useless, so why keep them?
Quote
I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(

I really appreciate all the work everyone's put into this effort! I wish I had more time to contribute.
I've never used github or any other repository before, but I'm not opposed to using any particular one.  In the meantime, I've updated my code weather.tgz (http://1drv.ms/1kbzfFU)  I've reconfigured everything to reside in one directory, which on my system I've set up as a virtual host.  These should work in the root directory of a web server, too.  As I've said before, I'm a total newb at web services, so these are hardly shining examples of best practices.  Any suggestions for improvement are welcome.

Update: I've created a github repository for this project Weather Gateway (https://github.com/kennkong/Weather-ERF-Gateway-1000U)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: benmargolin on August 02, 2014, 06:41:57 AM
Kenn, thanks a lot for this, I'm making some progress getting your stuff to work for me.
However, PLEASE include the data dump for the 'sensors' table, and any others that have "generic" data! Pretty tricky to figure out what is supposed to populate those...

Edit: fixuprain() still not working properly for me; commented out the call to it, for now. I also didn't have an admin@localhost user on my mysql install, so the views didn't work; I adjusted them to be the right user but maybe setting SQL SECURITY INVOKER would be a better solution. Also had to change the HTTP-IDENTIFY to be HTTP_IDENTIFY (underscore vs hyphen). Not sure why on your GW it (apparently!) sends a hyphen...?

You can see tweaks I've made so far in my fork at https://github.com/benmargolin/Weather-ERF-Gateway-1000U/compare/kennkong:master...master (https://github.com/benmargolin/Weather-ERF-Gateway-1000U/compare/kennkong:master...master) (really not much, mostly rearranging some stuff; no SQL changes documented, sadly.)

Anyhow, thanks again!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 02, 2014, 05:28:42 PM
Kenn, thanks a lot for this, I'm making some progress getting your stuff to work for me.
However, PLEASE include the data dump for the 'sensors' table, and any others that have "generic" data! Pretty tricky to figure out what is supposed to populate those...
I updated weather.sql to pre-populate the sensors table.  I added a file records.sql which has a script to populate the records table after you have at least one sensor reading
Quote
Edit: fixuprain() still not working properly for me; commented out the call to it, for now.
Looking more carefully at the code, fixuprain() might need an existing record for the last rain reset date in the sensordatevalues table to work.
Quote
I also didn't have an admin@localhost user on my mysql install, so the views didn't work; I adjusted them to be the right user but maybe setting SQL SECURITY INVOKER would be a better solution.
I didn't check the generated schema script closely enough.
Quote
Also had to change the HTTP-IDENTIFY to be HTTP_IDENTIFY (underscore vs hyphen). Not sure why on your GW it (apparently!) sends a hyphen...?
The problem is not with my gateway, it's with my webserver.  It doesn't have apache_request_headers() built-in, so I borrowed some code to create one.  That code changes the underscores to hyphens for some reason.  If your server has apache_request_headers() that leaves the underscores unchanged, well, how am I supposed to know that?  Remember, total webserver newb here.

Anyway, I pushed a few changes to the repository.  Thanks for your comments, I have a lot to learn.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 04, 2014, 12:24:13 AM
Thanks to keckec, I've tested my code with another station.  This helped me debug the fixuprain issue, which has been fixed and pushed to the repository.

To really get multiple stations working, I need to get my data display pages to allow the user to select which station they want to see.  I already have a script stations.js which creates a drop-down list that could be added to the top of weather.html, etc.  However, I can't quite figure out how to use that list to change which station is displayed.

This is what I want to happen:
Loading weather.html displays the data for the default weather station, as set by stations.js.
Loading weather.html?stationid=X will load the data for station X.
Changing the selected station in the drop-down list will reload the page with ?stationid=selected

It's the last part I can't quite figure out (reloading the page with a new URL).  Any suggestions are welcome.  I'm opening a branch for working on that in the repository.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 05, 2014, 12:14:30 AM
I've updated the repository with the multiple station select versions of the display pages.  It ain't pretty, so if you have any suggestions for making it look better, fire away.

If you choose not to use the multiple station select version, just hardcode your stationid in wsdata.php.

Update: I have a little more tweaking to do.  I'd like to make the station list box go away if there's only one station in the database.  That way, the (I'm assuming) typical single station user won't have to look at a useless box, while the code will still apply the stationid automatically.

Update 2: That was a little easier than I thought.  I'm just hiding the station select drop down list if there's only one station.

Next task: a configuration page!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dwoolridge on August 13, 2014, 09:37:40 AM
I'm excited to try skydvrz program for the C84612 but have a question...

How large are the data packets sent from the C84612 gateway to the La Crosse Alerts site?

The reason I ask is my model airplane club will have to subscribe to a wireless internet package through one of the cell phone carriers.  I need to determine if the 100Mb per month plan is adequate, or if we need the 1Gb per month plan for just the weather station.  I figured that if I knew the data packet size, I could get a good estimate of total bandwidth knowing the station sends the data every 4 minutes.

Thanks,

--Doug
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 13, 2014, 10:41:10 AM
How large are the data packets sent from the C84612 gateway to the La Crosse Alerts site?

Are you going to use LaCrosse or SkySpy? 

There are many different packet types/sizes, but primarily there are the SDP, RTC and Ping packets that handle most of the daily chatter that the PWS does with the server.  I think you will be in the range of 300 bytes every 4 minutes or so - that works out well with your 100 MB / Month plan.  I also factored in some overhead for packet ACKs.  It is a two-way conversation between PWS and server and you will have to pay for both sides of the data conversation.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 14, 2014, 12:10:00 PM
Hello Everyone and thanks to those who have spent many hours reverse engineering this. I had an opportunities to meet the LaCrosse product engineer last January and asked why a standalone application or protocol info wasn't available. His response was that the weather station technology is licensed from another company and they don't want provide that.

I'm having an issue with SkySpy working on WIN7 Ultimate. The program won't communicate with the gateway if I have the Windows firewall turned on for the local or private network. If I turn it off everything works OK. I have looked the firewall application rules and can see that ssmonitor.exe is listed in the 'incoming' rules, but I don't see an entry in the 'outgoing' rules. Any ideas?

-Skydvrz, I think you have a typo in the user manual when refering to the weather station as model C86412 instead of C84612.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 14, 2014, 05:37:26 PM
I'm having an issue with SkySpy working on WIN7 Ultimate. The program won't communicate with the gateway if I have the Windows firewall turned on for the local or private network. If I turn it off everything works OK. I have looked the firewall application rules and can see that ssmonitor.exe is listed in the 'incoming' rules, but I don't see an entry in the 'outgoing' rules. Any ideas?

ssMonitor does not do any outbound TCP/IP unless you are using an external MySQL database.

If you are using an external MySQL database then you need to open port 3306.  If you are using the embedded database, then you don't need to open this port.

You do need to open up the firewall to SkySpyService.EXE on port 8000.  SkySpyService is the brains of the outfit.  The Service chats with the Monitor over "named pipes" that don't need a network connection, but they do need to be co-located on the same PC.

BTW, you can look at all of your TCP/IP connection bindings by opening a DOS box and running netstat.  The port number used is the number to the right of the colon  ":".  The high port numbers are usually outbound.

Last, I recommend using a hardware firewall instead of software.  I use a router/firewall/WiFi Access Point box at my house.  A hardware firewall/router allows you to run servers and games on your home LAN and access them from the outside world in relative safety.

SkySpy was written and tested on Win7-64 Pro.

Quote
-Skydvrz, I think you have a typo in the user manual when refering to the weather station as model C86412 instead of C84612.

Was that the latest release? Thanks for the heads up.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 14, 2014, 07:07:27 PM
I'm using the latest version of SkySpy configured with the internal DB.

I thought I had added SkySpyService.exe to the firewall program list during troubleshooting, but when I checked again it wasn't there. After adding it, it was listed as 'C86412 Gateway Interface and  Web Server' (C84612). Both the sensor data and web server are now working with Windows 'private networks' firewall turned on.

Is adding this entry to Windows Firewall something that should happen during install or does it need to be done manually?
I do have Avast running, but no other firewall software other than what's built into WIN7.

I do have hardware firewall and load balancing hardware between my LAN and the Internet.

Thanks!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 15, 2014, 11:18:27 AM
I'm using the latest version of SkySpy configured with the internal DB.

I thought I had added SkySpyService.exe to the firewall program list during troubleshooting, but when I checked again it wasn't there. After adding it, it was listed as 'C86412 Gateway Interface and  Web Server' (C84612). Both the sensor data and web server are now working with Windows 'private networks' firewall turned on.

Good!  Apparently my dyslexia is pervasive - or should I say lexdyxia ;-)

Quote
Is adding this entry to Windows Firewall something that should happen during install or does it need to be done manually?

I don't know of a good way to automatically add a firewall exception to Windows.  The end user will probably have to make an exception entry manually.  Doesn't it pop up and warn you the first time an unknown app tries to access the Internet?  Norton does that.

Quote
I do have hardware firewall and load balancing hardware between my LAN and the Internet.

Then you probably don't need Windows Firewall, unless you distrust the other PCs/users on your LAN.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 15, 2014, 01:07:43 PM
Avast (free version) didn't warn of SkySpy trying to make outside connection. I normally run Comodo Firewall which does catch anything trying to make in or out bound connection.

New problem: I can't seem to get SkySpy to work on XP thin-client PC that I use for data logging applications. I closed SkySpy runing on WIN7 PC, changed the gateway to point at the XP PC & reset, opened XP firewall for SkySpy & started SkySpy on XP. After a few minutes here's what shows in the debug window...

Ping Received
Special SDP Len=210
JabberCount=1
Special SDP Len=210
JabberCount=2
Special SDP Len=210
JabberCount=3
repeats to =6 and then starts over.

I switched back over to WIN7 PC and it works fine.

What would you suggest I do next to discover the cause of the problem? ](*,)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 15, 2014, 02:22:18 PM
Update: When I switched back to the WIN7 PC a 2nd time I did some get  'Special SDP= 210' for awhile. Now I'm get an occasional 'Special SDP-30, but the sensor values & WU are updating.

Should I reset the C84612 by removing the batteries?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 15, 2014, 02:37:55 PM
Update: When I switched back to the WIN7 PC a 2nd time I did some get  'Special SDP= 210' for awhile. Now I'm get an occasional 'Special SDP-30, but the sensor values & WU are updating.

Should I reset the C84612 by removing the batteries?

No, just wait.  The Gateway is trying to update the LCX server with missed updates.  Once the buffer is flushed, then they will go away.

Is the "Internet" indicator on the LCD display active?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 15, 2014, 07:11:15 PM

When the gateway is pointed at the XP system the LCD "Internet" icon was flashing or blinking. While connected to the WIN7 system it is a steady on.

I switched back to the XP system and the Special SDP=210 continued for over 4 hours and never did correct. I then switched back to WIN7 and had the 210 message about 20 times before the sensors updated and the =210 stopped.

I powered cycled the gateway & LCD while on XP and it had no effect. I keep thinking it's something simple or oblivious. :?
I don't want to load wireshark and dig in to the nuts & bolts to see what's different if I don't need to.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on August 16, 2014, 02:42:33 AM

When the gateway is pointed at the XP system the LCD "Internet" icon was flashing or blinking. While connected to the WIN7 system it is a steady on.

I switched back to the XP system and the Special SDP=210 continued for over 4 hours and never did correct. I then switched back to WIN7 and had the 210 message about 20 times before the sensors updated and the =210 stopped.

I powered cycled the gateway & LCD while on XP and it had no effect. I keep thinking it's something simple or oblivious. :?
I don't want to load wireshark and dig in to the nuts & bolts to see what's different if I don't need to.
The blinking led means it's not connecting to the sky spy service.  Are you sure you reset to gateway to the correct in?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 16, 2014, 08:49:24 PM
The blinking led means it's not connecting to the sky spy service.  Are you sure you reset to gateway to the correct in?

When the Gateway proxy is pointed to the XP machine, the red LED is on solid and the Green LED is flashing often at the same rate the "*=210" messages are appearing in SkySpy debug window. When looking at the Gateway webpage, the "last contact" timer seems to update when "Ping Received" appears in Debug window.

Everything appears to be the same between the XP (not working) and the WIN7 (working). I'll try restoring XP to a date before I first installed the previous version of SkySpy which I couldn't get to work either and then reinstalling SkySpy.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 17, 2014, 03:24:34 AM
I've posted a rudimentary version of my configuration page in the configuration branch of my repository Weather-ERF-Gateway-1000U (https://github.com/kennkong/Weather-ERF-Gateway-1000U).

It requires the addition of some columns to the stations table, which can be done using the alterstations.sql script.  The weather.sql script has been updated to reflect the additions.

This is "banging the rocks together" quality work, so if anyone wants to offer any improvements, it would be welcome.

Since this function could be obtained from any db admin tool, I think this is a dead end.  I'm teaching myself as I go, so it hasn't been a total waste of time.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 17, 2014, 11:09:56 AM
When the gateway is pointed at the XP system the LCD "Internet" icon was flashing or blinking. While connected to the WIN7 system it is a steady on.

Why are you trying to run it on an XP box?

If the Internet icon continues to flash and you see Special packets for hours on end, then you need to look a the serial number you entered into your settings.  The behavior your describe is usually due to a bad serial number.  You can copy C:\ProgramData\SkySpyData\SkySpy.ini from the working machine to the non-working machine, if you want to duplicate the working settings on the Win7 box.

The non-working box will jabber for a while and then should sync up. 

If that doesn't help, then send me both of the INI files.

You do not have to stop the SkySpy service when you copy/overwrite the INI file.  It gets re-read each time a packet comes in from the gateway.

If you are running ssMonitor, then I recommend shutting it down before overwriting the INI file.  You can restart it afterwords.

You do not have to keep resetting/power-cycling the PWS hardware - in fact I recommend against it.  Whatever you do, don't reset your Gateway serial number!  Do not try to re-register the Gateway with SkySpy!  Step away from the Gateway button and keep your hands where I can see them. :-)

I haven't reset my PWS hardware in about 8 months.  (You are not working with the LaCrosse servers any more, so everything you know is wrong :-) ) 

A Wireshark sniff probably wouldn't reveal much.  Your Gateway does not like the serial number or the Epoch (AKA memory map index) for historical data is not synching properly.  You'd need to capture hours of data to see the problem. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: semoore on August 17, 2014, 02:31:25 PM
Why are you trying to run it on an XP box?
The XP box was a low power Acer R1600 thin-client that is used for solar data logging task and had been running  HeavyWeather along with WUHU for a years.
The Red Internet LED was not flashing, It was the Green RF LED that was blinking at various intervals.

I took the easy way out, not having the time to learn what was wrong, I upgraded the Acer to WIN7 32. SkySpy works fine now. :grin:

Unfortunately I didn't save the .ini file before installing WIN7.

Thanks for your help skydvrz.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on August 17, 2014, 02:42:59 PM
I was talking about the Internet indicator on the wall-mount LCD console.  This should stay lit up continuously if SkySpy is working properly.  The indicator on the Gateway will light up in various ways to indicate the overall health of your PWS.  SkySpy has little to no effect on that.

T/H sensor position will affect the light color on your Gateway, if it is out of range, RF-wise.

I had to reposition my outside T/H sensor and Rain Gauge so that everything would fly in formation at my house.  My rain gauge now sits on a fence post in my pasture and the T/H sensor is under an arbor on the side of my garage.  My Gateway is mounted in an forced-air-cooled box in my attic, co-located with some other network gear. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on August 17, 2014, 03:32:59 PM
This is "banging the rocks together" quality work, so if anyone wants to offer any improvements, it would be welcome.

The only thing I might add to the database is the gateway firmware version number. I suspect this is encoded in some of the packets, although I haven't looked for it. There could be differences in communication for different firmware versions, which could make it important to know. I made a separate table for this, which includes the model number, so as to possibly support similar models as well.

My firmware version is 1.21.00, compiled on 7/4/08 (from the GW's web page). I'm curious what version others have.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on August 17, 2014, 05:08:22 PM
My firmware version is 1.21.00, compiled on 7/4/08 (from the GW's web page). I'm curious what version others have.
Ditto.
Speaking from my vast lack of experience, I doubt the gateway firmware will have much to do with it.  It's pretty much a commodity device that relays what it receives on the RF to the TCP/IP.  Most of the logic must reside in the weather station.  For example, you have to push a button on the weather station to reset the connection to the gateway (i.e. the weather station recognizes the existence of the gateway, not the other way round.)
Title: 2 Problems...
Post by: dwoolridge on September 01, 2014, 10:03:09 AM
I'm really excited to get SkySpy working with my new C84612 weather station, but have run into two separate problems...

1)  I can't get the Gateway talking to the SkySpy service.  Here are the steps I'm following:
  A)  I did a factory reset on the Gateway just to be sure I'm starting from scratch
  B)  I got the gateway working and reporting data to LacrosseAlerts.com (I can see it updating on the website just fine)
  C)  With the SkySpy service running (but with the SkySpy monitor not opened yet...)
      i)  Run Gateway Advanced Setup
         a)   Change the "www.proxyserver.de" to match the PC-IP
         b)  Change the port to 8000
         c)  Check the "Use" box
         d)  Leave the "Use DHCP" box checked
         e)   Changed the "Gateway" and "DNS IP" to my router's IP (192.168.0.1)
NOTE:  I didn't change the IP or Netmask addresses (see attached screen-shot)
         f)  Click Set
         g) Click Reboot

After the reboot, the Green RF light flashes continuously for several seconds then stops.  "Internet" on the weather display turns off at this point.

2) On one of my early runs of SkyMonitor, I accidentally un-checked "Use Embedded MySQL" and clicked OK.  Now, even after uninstalling and re-installing SkySpy, it continues to try to access the MySQL database.  Any action in the Monitor tries to access the database.  This makes it so I can't get into SkySpy and correct the setting.  Any ideas on how I can fix this?

Windows 7, 64-bit OS.


Thanks,

Doug Woolridge
Title: Problem #2 solved...
Post by: dwoolridge on September 01, 2014, 10:16:07 AM
I just found a way to solve problem #2 (described in the previous post). 

I found the C:\ProgramData\SkySpyData\SkySpy.ini file and manually changed the "DBEmbedded" value from 0 to 1.  The monitor no longer tries to connect to database with every menu pick.

--Doug
Title: Re: Problem #2 solved...
Post by: skydvrz on September 10, 2014, 02:46:18 PM
I just found a way to solve problem #2 (described in the previous post). 

I found the C:\ProgramData\SkySpyData\SkySpy.ini file and manually changed the "DBEmbedded" value from 0 to 1.  The monitor no longer tries to connect to database with every menu pick.

Sorry for not getting back to you!  I have been extremely busy of late.

I will fix that in the next release. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: dwoolridge on September 10, 2014, 03:40:57 PM
Thanks for the reply!  More urgent for me would be figuring out question #1 described in my post (3 posts up).

I've tried everything and can't get SkySpy talking to the C84612 weather station.


Thanks,

--Doug
Title: Re: 2 Problems...
Post by: skydvrz on September 11, 2014, 11:20:27 AM
Doug,


1)  I can't get the Gateway talking to the SkySpy service.  Here are the steps I'm following:
  A)  I did a factory reset on the Gateway just to be sure I'm starting from scratch

Don't do factory resets.  That will mess things up.

Quote
  B)  I got the gateway working and reporting data to LacrosseAlerts.com (I can see it updating on the website just fine)

Did you copy your LaCrosse serial number found on their web site?  It changes each time you re-register your Gateway with LaCrosse.

The number on the LaCrosse web site MUST match the one you use in SkySpy.  Otherwise your C84612 "Internet" indicator will go out, the clock will start to drift and your Gateway will send a constant stream of junk to SkySpy.  You enter the serial number into SkySpy using Monitor.

Quote
  C)  With the SkySpy service running (but with the SkySpy monitor not opened yet...)
      i)  Run Gateway Advanced Setup
         a)   Change the "www.proxyserver.de" to match the PC-IP
         b)  Change the port to 8000
         c)  Check the "Use" box
         d)  Leave the "Use DHCP" box checked
         e)   Changed the "Gateway" and "DNS IP" to my router's IP (192.168.0.1)
NOTE:  I didn't change the IP or Netmask addresses (see attached screen-shot)

A netmask of 255.255.255.0 is correct for a 192.168.x.x subnet

Quote
         f)  Click Set
         g) Click Reboot

Your other settings seem normal. 

Are you absolutely sure you got your workstation IP correct?  If you are using DHCP to assign the address of your workstation PC, then it may not be the IP address you think it is, or it may change over time.  Consider using a static IP for your workstation.

Are you sure the Gateway has a unique IP address on your LAN?  If you assign a static IP address to a device on your LAN that matches the GW, then bad things will happen.

Quote
After the reboot, the Green RF light flashes continuously for several seconds then stops.  "Internet" on the weather display turns off at this point.

This is normal, but you should see sensor reading update in SkySpy Monitor and/or Wunderground in 5 minutes or so.  The GW sends new sensor readings every 4-5 minutes.

It may take up to 15-60 minutes for things to stabilize on the C84612 display. 

It is not necessary to run Monitor all the time.  SkySpy Service has all the smarts.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on October 07, 2014, 10:57:12 AM
All:

I have shut down public access to my server that some of you have been using to store your weather samples.  I have posted warnings months ago that this day would come, so I hope it is not a big surprise to anyone.

Please use your own server or switch to Embedded MySQL mode. 
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 370Dancer on October 15, 2014, 02:42:47 PM
Hello,
New to the forum, but a long time user of Lacrosse ERF-100 to TX-60U setups for monitoring pool and spa temps.
I discovered this thread this morning, and read the entire thing. Fabulous info, and progress in providing an alternate way to monitor weather data!
So, my million dollar question is whether SkySpy would have any applicability to a simple source such as the TX60-U, which is 2 temps and humidity only.
I won't be near a station setup to test this for another few days, but I am excited that I might be able to use WUnderground instead of the clunky Weather Direct site, and the spotty Lacrosse pay site.

As a side note for those interested, Lacrosse has changed something, or Weather Direct can't handle their newer sensors, thus forcing you into a Lacrosse online monitor subscription.  I tried to connect a new TX70-U with water sensor, and the registration process on Weather Direct threw an error of device not recognized PEId=7, or something like that.  TX60s still work.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on October 15, 2014, 07:22:22 PM
I discovered this thread this morning, and read the entire thing.

You are a vary patient person.  That is a lot of reading :-)

Quote
So, my million dollar question is whether SkySpy would have any applicability to a simple source such as the TX60-U, which is 2 temps and humidity only.

Probably not, since you didn't mention a Gateway module. Don't forget that SkySpy and the other software mentioned here works off of the Gateway chit-chat, not the sensors or the display.  That sensor->display comms are all radio-frequency transmissions.

SkySpy pretends to be the Lacrosse server, using your home LAN/Internet connection.  It does not listen directly to the sensors RF comm.

Quote
I won't be near a station setup to test this for another few days, but I am excited that I might be able to use WUnderground instead of the clunky Weather Direct site, and the spotty Lacrosse pay site.

I am not sure if you could send water temps to Wunderground.  WU will toss any data that has a big difference between your station and other nearby stations.  I ran across that during SkySpy development - you will receive a nastygram email from them telling you your weather station is screwed up and they start blanking your graphs and tabular data.  If your spa is running at 104F and the outside temp is 60F, guess what is going to happen :-)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 370Dancer on October 15, 2014, 08:45:54 PM
 Thx.  I understand about the gateway.  The TX60 uses the same ERF to post to Weatherdirect.  I will just give it a shot when i get back to florida, and post initial results.  Weatherdirect and Lacrosse appear to be moving in different dierctions, or Lacrosse has just changed their business model.  I am mainly looking for online remote monitoring outside of my local network.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on October 15, 2014, 08:51:44 PM
If the ERF module is the same as the one I am using then you might get it to work.

You could try using the built-in webserver page in SkySpy.  Just point your browser at the SkySpy machine's IP and you can look at your sensor readings in tabular form.  SkySpy shows a hokey looking web page - mostly for debugging or the weather obsessed.  :grin:
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 05, 2014, 07:21:04 PM
skydvrz: Is the version you posted on July 25 the most current? And is it now recommended over 1.1.0.72? Thanks!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on November 05, 2014, 08:56:03 PM
The most recent installer was built on 24 July, 2014

The latest SkySpy Server reports v1.2.0.86 to Wunderground - click the "About this PWS" link up near where your WU Station ID appears on their web page when you get SkySpy up and running.

SkySpy Monitor is up to v1.1.0.177

They are two separate programs, BTW.

You can find the latest installer here, containing both the Server and Monitor:

https://www.dropbox.com/sh/g06wjf0ed36nh4u/AADdx6liYZRuBnR2p2rueXC_a?dl=0 (https://www.dropbox.com/sh/g06wjf0ed36nh4u/AADdx6liYZRuBnR2p2rueXC_a?dl=0)

Yes, please use the most recent version.

Select Embedded mode since I have shut down my public database server.  Embedded mode does not need a full-blown MySQL database server as did previous versions.  It creates a single-session mini-DB on your hard drive.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on November 05, 2014, 09:10:06 PM
skydvrz: Is the version you posted on July 25 the most current? And is it now recommended over 1.1.0.72? Thanks!

Is that you filling up my bug reporting system with error reports?  :grin:

Follow the instruction manual for Embedded mode.  That should fix it.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 05, 2014, 09:26:21 PM
skydvrz: Is the version you posted on July 25 the most current? And is it now recommended over 1.1.0.72? Thanks!

Is that you filling up my bug reporting system with error reports?  :grin:

Follow the instruction manual for Embedded mode.  That should fix it.

Oops. Yup that's me. Problems started today when mySQL told me to set a new password then wouldn't take any. I finally remembered to open the SkySpy .ini file and set the embed value to 1. Still can't figure out why it's not working in embedded mode though.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 06, 2014, 01:09:07 AM
Well, I started from scratch: uninstalled the old version, deleted the SkySpy folder in c:\Program Data and reinstalled the newer version. Before running it the first time, I used Gateway Advanced Setup to reset the gateway's ip to its actual ip and then verified it was reporting to the LaCrosse website (it was). Then I set the ip for the SkySpy pc and fired up SkySpy. I set using embedded mySQL and set the serial number to match the one on the Lacrosse website. Also I set the info for WU and for the mini website.

Unfortunately, no data is going through SkySpy, none is going to WU. I even tried disabling the firewall.

If I reset the gateway, the Lacrosse website starts showing updates.

If anyone can suggest what I'm missing, I'd sure appreciate it!

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 06, 2014, 11:47:15 PM
Thanks for the reply!  More urgent for me would be figuring out question #1 described in my post (3 posts up).

I've tried everything and can't get SkySpy talking to the C84612 weather station.


Thanks,

--Doug

Did you ever get this solved? The same exact thing is happening to me now.

Looking at the DHCP client list for my router, the gateway always keeps its ip address no matter what the Advanced Gateway Setup uses. I suppose that's normal. What is odd that a random device "LaCrosse_Weather_Gateway" shows up, sometimes with its own ip address. When it does that the mac address is always different from the gateway by 1 character at the end. Other times it simply renames another device on the network. (The other device continues to function fine.)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on November 07, 2014, 08:20:37 PM
Looking at the DHCP client list for my router, the gateway always keeps its ip address no matter what the Advanced Gateway Setup uses. I suppose that's normal. What is odd that a random device "LaCrosse_Weather_Gateway" shows up, sometimes with its own ip address. When it does that the mac address is always different from the gateway by 1 character at the end. Other times it simply renames another device on the network. (The other device continues to function fine.)

Your Gateway is simply grabbing new IP addresses from your DHCP server (usually your firewall/router/WiFi Access Point).  It is OK for the Gateway address to shift around, as long as it can find your SkySpy PC on your LAN.  The Gateway initiates all network conversations and the SkySpy PC simply replies back to the sender's address.

The thing that could be messing you up is if your SkySpy PC has a DHCP-assigned address.  Is your current IP address for your SkySpy PC the same as the one you entered into your Gateway? 

Your PC's address may shift around when your "DHCP lease" expires, when you reset your router and for other reasons. Normally this is not a problem, but if another device on your LAN expects your PC to be at a certain address - and it is not - then bad things will happen.  The Gateway won't be able to find your PC and send it packets.

So, to cut down the crazyness factor, you may need to use a static IP address on your PC. 

Most routers allow you to set up a block of static IP addresses that the built in DHCP server will avoid.  Conversely, some routers allow you to set up the range of DHCP addresses; The remaining ones are for use with static assignments. 

You put things like printers, web servers, PCs and other network appliances in the router's static address range with fixed addresses.

I use static IP addresses for all of my computers here at my house.  I only use DHCP for smartphones and my Gateway module.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 09, 2014, 07:19:24 PM
Looking at the DHCP client list for my router, the gateway always keeps its ip address no matter what the Advanced Gateway Setup uses. I suppose that's normal. What is odd that a random device "LaCrosse_Weather_Gateway" shows up, sometimes with its own ip address. When it does that the mac address is always different from the gateway by 1 character at the end. Other times it simply renames another device on the network. (The other device continues to function fine.)

Your Gateway is simply grabbing new IP addresses from your DHCP server (usually your firewall/router/WiFi Access Point).  It is OK for the Gateway address to shift around, as long as it can find your SkySpy PC on your LAN.  The Gateway initiates all network conversations and the SkySpy PC simply replies back to the sender's address.

The thing that could be messing you up is if your SkySpy PC has a DHCP-assigned address.  Is your current IP address for your SkySpy PC the same as the one you entered into your Gateway? 

Your PC's address may shift around when your "DHCP lease" expires, when you reset your router and for other reasons. Normally this is not a problem, but if another device on your LAN expects your PC to be at a certain address - and it is not - then bad things will happen.  The Gateway won't be able to find your PC and send it packets.

So, to cut down the crazyness factor, you may need to use a static IP address on your PC. 

Most routers allow you to set up a block of static IP addresses that the built in DHCP server will avoid.  Conversely, some routers allow you to set up the range of DHCP addresses; The remaining ones are for use with static assignments. 

You put things like printers, web servers, PCs and other network appliances in the router's static address range with fixed addresses.

I use static IP addresses for all of my computers here at my house.  I only use DHCP for smartphones and my Gateway module.

I've always used static ips on my home network as well so there's no problem there.

I'm still not seeing any data going through SkySpy. If the gateway is set to use my pc as the proxy, the red light flashes 3 times over and over again. If the gateway is set to no proxy, then the LaCrosse website updates properly.

I've just moved where the gateway connects to the network and gave it a new ethernet cable.

I wish I could figure this out! :)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: kj5fa on November 09, 2014, 09:43:52 PM
I am new to this and WAY over my head, but I seem to have the same issue as flamand. When I change the IP address to point to my SkySpy computer, the Internet icon goes away and the reporting stops going to the LacrosseAlert page and is never seen by Sky Spy Monitor.
I used the latest version of SkySpyInstaller posted in this thread. I am running Win7.
Do I need to install mySQL if using the embedded database?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 09, 2014, 10:30:35 PM
I am new to this and WAY over my head, but I seem to have the same issue as flamand. When I change the IP address to point to my SkySpy computer, the Internet icon goes away and the reporting stops going to the LacrosseAlert page and is never seen by Sky Spy Monitor.
I used the latest version of SkySpyInstaller posted in this thread. I am running Win7.
Do I need to install mySQL if using the embedded database?

You don't need to install mySQL. Just always leaved the "use embedded mysql" box checked.

Not that that helps our problem any ;)
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 12, 2014, 09:16:05 PM
I'm still trying to figure out why no data is going to skyspy. Could anyone who has it working tell me whether you get data pinging a gateway's original ip address while it's going through a proxy? I mean, if GAS has the gateway (192.168.1.111) using the skyspy pc as proxy (192.168.1.222), what happens when you ping 192.168.1.111?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 12, 2014, 10:25:21 PM
I'm still trying to figure out why no data is going to skyspy. Could anyone who has it working tell me whether you get data pinging a gateway's original ip address while it's going through a proxy? I mean, if GAS has the gateway (192.168.1.111) using the skyspy pc as proxy (192.168.1.222), what happens when you ping 192.168.1.111?

I have my gateway set to use a proxy and can ping it from other machines on my home network. The gateway also has a web server listening on port 80, so you should be able to point a browser to its IP address and see some gateway info. That may not prove much though, since the problem is likely going the other way -- from the GW to the skyspy machine. If you have a firewall on your skyspy machine it's probably only blocking incoming connections. I don't use Windows so I doubt I'll be much help with firewall issues.

If you have another computer on your local network you could try connecting from it to the skyspy machine using telnet. Use whatever the skyspy machine's IP address is and whatever port skyspy is listening on.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 13, 2014, 12:45:16 AM
I have my gateway set to use a proxy and can ping it from other machines on my home network. The gateway also has a web server listening on port 80, so you should be able to point a browser to its IP address and see some gateway info. That may not prove much though, since the problem is likely going the other way -- from the GW to the skyspy machine. If you have a firewall on your skyspy machine it's probably only blocking incoming connections. I don't use Windows so I doubt I'll be much help with firewall issues.

If you have another computer on your local network you could try connecting from it to the skyspy machine using telnet. Use whatever the skyspy machine's IP address is and whatever port skyspy is listening on.

When I point my browser to the gateway's ip and the gateway is configured to use the spysky pc as the proxy, the proxy name shows the spysky pc's ip address and port but the proxy ip is empty (0.0.0.0) and the last contact line says "no contact". Is that correct?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 13, 2014, 09:12:25 AM
If you have another computer on your local network you could try connecting from it to the skyspy machine using telnet. Use whatever the skyspy machine's IP address is and whatever port skyspy is listening on.

When I point my browser to the gateway's ip and the gateway is configured to use the spysky pc as the proxy, the proxy name shows the spysky pc's ip address and port but the proxy ip is empty (0.0.0.0) and the last contact line says "no contact". Is that correct?
[/quote]
I don't think so. Mine shows the following:
Code: [Select]
Use Proxy   Yes
Proxy Server Name   192.168.1.10
Proxy Port   80
Proxy Server IP   192.168.1.10
Last contact      7915959800

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 13, 2014, 09:36:42 PM
My ERF gateway page is different from keckec's in the previous message in that there is no entry in the line for "proxy server ip". Should that be populated when GAS is set to use a proxy? Would that mean that there's a problem with how GAS is working in my network vs a problem in the skyspy service?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on November 13, 2014, 10:11:16 PM
My ERF gateway page is different from keckec's in the previous message in that there is no entry in the line for "proxy server ip". Should that be populated when GAS is set to use a proxy? Would that mean that there's a problem with how GAS is working in my network vs a problem in the skyspy service?
Read page 4 of the users guide (readme2.pdf) It has to be populated.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on November 13, 2014, 11:26:52 PM
It looks like you might have a Proxy Port problem.
Page 4 of the Readme2.pdf (install guide) talks about setting up the gateway with the GAS and that the port needs to be set to 8000.
I pointed my browser to the gateway's IP and here is what mine shows.

Use Proxy   Yes
Proxy Server Name   192.168.0.44
Proxy Port   8000
Proxy Server IP   192.168.0.44
Last contact   77289091000


I have my gateway set to use a proxy and can ping it from other machines on my home network. The gateway also has a web server listening on port 80, so you should be able to point a browser to its IP address and see some gateway info. That may not prove much though, since the problem is likely going the other way -- from the GW to the skyspy machine. If you have a firewall on your skyspy machine it's probably only blocking incoming connections. I don't use Windows so I doubt I'll be much help with firewall issues.

If you have another computer on your local network you could try connecting from it to the skyspy machine using telnet. Use whatever the skyspy machine's IP address is and whatever port skyspy is listening on.



When I point my browser to the gateway's ip and the gateway is configured to use the spysky pc as the proxy, the proxy name shows the spysky pc's ip address and port but the proxy ip is empty (0.0.0.0) and the last contact line says "no contact". Is that correct?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 13, 2014, 11:40:45 PM
My ERF gateway page is different from keckec's in the previous message in that there is no entry in the line for "proxy server ip". Should that be populated when GAS is set to use a proxy? Would that mean that there's a problem with how GAS is working in my network vs a problem in the skyspy service?
Read page 4 of the users guide (readme2.pdf) It has to be populated.

Sorry ...I was unclear.

I ran Gateway Advanced Server. The proxyserver field was set to be the spysky pc's ip, using port 8000. Then I clicked the set and reboot boxes and then exited GAS.

I opened a web browser at pointed it to the spysky pc's ip. In the third group of information, at the end I see:

Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on November 14, 2014, 11:55:22 AM


Sorry ...I was unclear.

I ran Gateway Advanced Server. The proxyserver field was set to be the spysky pc's ip, using port 8000. Then I clicked the set and reboot boxes and then exited GAS.

I opened a web browser at pointed it to the spysky pc's ip. In the third group of information, at the end I see:

Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.

You are sure that after you changed the data with GAS you clicked on SET, waited and then clicked on REBOOT.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 14, 2014, 01:06:43 PM


Sorry ...I was unclear.

I ran Gateway Advanced Server. The proxyserver field was set to be the spysky pc's ip, using port 8000. Then I clicked the set and reboot boxes and then exited GAS.

I opened a web browser at pointed it to the spysky pc's ip. In the third group of information, at the end I see:

Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.

You are sure that after you changed the data with GAS you clicked on SET, waited and then clicked on REBOOT.

Yes. The line before the ones I quoted says Use Proxy - Yes
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 14, 2014, 01:25:08 PM
Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.

I think the GW might not fill in the Data Server IP field until it makes a connection. In my setup the GW connects to a web server with php scripts to save data and keep the GW happy.

I'll try disabling the web server and cycle power on the GW. Then I'll look at its web page to see if/when the Data Server IP and Last Contact fields get filled in.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: MtHighVs on November 15, 2014, 12:37:50 AM
Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.

I think the GW might not fill in the Data Server IP field until it makes a connection. In my setup the GW connects to a web server with php scripts to save data and keep the GW happy.

I'll try disabling the web server and cycle power on the GW. Then I'll look at its web page to see if/when the Data Server IP and Last Contact fields get filled in.

I think you are right that the Data Server IP field is filled in when it makes a connection with the skyspy server.

Have you tried opening port 8000 to all traffic in the firewall on the spysky machine it seems that I had some trouble with that when I first installed and open the port for testing and when it starting working I make a rule for port 8000 in the firewall
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 15, 2014, 12:03:47 PM
Proxy Server Name   192.168.115
Proxy Port                   8000
Data Server IP           0.0.0.0
Last contact           no contact

Somehow the Data Server IP is not being set and I suspect that's consistent with spysky not receiving any data.

As a test I turned off my web server so the GW would have nothing to connect to. Then rebooted the GW. The GW's web page was filled in with all the above fields except Last contact showed "no contact." The Data Server IP was the same as the Proxy Server Name, which in my case is the IP address of my web server. I then restarted the web server, and after a couple of minutes the GW filled in the all the fields.

It's possible the Data Server IP field doesn't get set to 0.0.0.0 until after it's failed to connect a number of times.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: kj5fa on November 15, 2014, 07:56:56 PM
My results look like:

Proxy Server Name   192.168.121
Proxy Port                   8000
Data Server IP         192.168.121
Last contact           no contact

But I haven't ever made a connection that I have seen. Guess it's just failed enough but it came up with that as soon as I re-subscribed after my gateway started wigging out after I changed some values I shouldn't have apparently. I reset everything, git the PWS talking to internet and then used GAS to re-direct. As soon as I logged into ERF-Gateway Settings & States the results above were there.

But I still do not get any data reflected on the SkySpy Monitor.  :sad:

And the Internet icon is not displayed.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 16, 2014, 12:52:35 AM
Proxy Server Name   192.168.121
But I still do not get any data reflected on the SkySpy Monitor.  :sad:

And the Internet icon is not displayed.
Is 192.168.121 the full IP address you entered? This looks incomplete. It should have four numbers separated by three periods.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: kj5fa on November 16, 2014, 09:52:23 AM
Sorry missed a .1.

It should have been:

192.168.1.121
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 16, 2014, 10:05:31 AM
Sorry missed a .1.

It should have been:

192.168.1.121
OK. I was thinking if you only entered three numbers into GAS, it could have been the problem.

You could install Wireshark and capture network traffic to see if anything is getting through. It's a bit complicated to use, but if set monitor only port 8000 it could tell if the GW is sending anything.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: kj5fa on November 16, 2014, 12:30:23 PM
This was quick and dirty so it may be useless info. I am over my head with this, but trying to get this to work.

Thanks again for all the help!  \:D/

Installed Wireshark. Monitoring my wireless connection from this skyspy computer. Found the capture filters and created a new filter by using the TCP or UDP port 80(HTTP) and changing it to be port 8000 instead. Changed it to 8000 in the filter name and filter string.

Nothing came threw initially, because I had stopped re-directing data with GAS. After going into GAS and using the 192.168.1.121 address, data starting showing up in Wireshark!!! What does tell me?

I've attached a screenshot of what I was seeing. Is png the preferred graphic format?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 16, 2014, 12:56:27 PM
Nothing came threw initially, because I had stopped re-directing data with GAS. After going into GAS and using the 192.168.1.121 address, data starting showing up in Wireshark!!! What does tell me?

I've attached a screenshot of what I was seeing. Is png the preferred graphic format?
All the traffic is going one way. It appears skyspy is not replying. I think it still could be a firewall issue, where skyspy is not seeing the packets from the GW, or its reply is being blocked. Does skyspy keep a log that might tell if it sees anything?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 17, 2014, 04:03:45 PM
Nothing came threw initially, because I had stopped re-directing data with GAS. After going into GAS and using the 192.168.1.121 address, data starting showing up in Wireshark!!! What does tell me?

I've attached a screenshot of what I was seeing. Is png the preferred graphic format?
All the traffic is going one way. It appears skyspy is not replying. I think it still could be a firewall issue, where skyspy is not seeing the packets from the GW, or its reply is being blocked. Does skyspy keep a log that might tell if it sees anything?

I've completely disabled my firewall and made sure nothing about port 8000 was set in my router. Still no data is going to skyspy. Can you think of another way to look for a firewall problem?

I've thought about another test using an ethernet crossover cable to connect the gateway directly to the skyspy pc and let the pc get internet connectivity via wireless. i'm sure it's more involved than just plugging in a cable though.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 18, 2014, 12:15:27 AM
Can you think of another way to look for a firewall problem?

I've thought about another test using an ethernet crossover cable to connect the gateway directly to the skyspy pc and let the pc get internet connectivity via wireless. i'm sure it's more involved than just plugging in a cable though.

Could you copy and paste all the info from the GW's web page into a reply to this thread, or post a screen shot. Or if you'd rather, send it in a PM to me. I want to check the rest of the fields.  Thanks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 18, 2014, 12:48:15 AM
These are for with and without using a proxy.

Can you think of another way to look for a firewall problem?

I've thought about another test using an ethernet crossover cable to connect the gateway directly to the skyspy pc and let the pc get internet connectivity via wireless. i'm sure it's more involved than just plugging in a cable though.

Could you copy and paste all the info from the GW's web page into a reply to this thread, or post a screen shot. Or if you'd rather, send it in a PM to me. I want to check the rest of the fields.  Thanks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 18, 2014, 09:07:10 AM
These are for with and without using a proxy.

Can you think of another way to look for a firewall problem?

I've thought about another test using an ethernet crossover cable to connect the gateway directly to the skyspy pc and let the pc get internet connectivity via wireless. i'm sure it's more involved than just plugging in a cable though.

Could you copy and paste all the info from the GW's web page into a reply to this thread, or post a screen shot. Or if you'd rather, send it in a PM to me. I want to check the rest of the fields.  Thanks.
I think the problem is the proxy server name field doesn't have enough numbers in the IP.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: flamand on November 18, 2014, 09:02:22 PM
Quote
I think the problem is the proxy server name field doesn't have enough numbers in the IP.

Good old user error. I have "looked" at that screen so many times over the past couple of weeks and cannot believe I missed it. Weather Underground is getting data now and I couldn't be happier.

Thank you so very much!!!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 18, 2014, 10:33:00 PM
Quote
I think the problem is the proxy server name field doesn't have enough numbers in the IP.

Good old user error. I have "looked" at that screen so many times over the past couple of weeks and cannot believe I missed it. Weather Underground is getting data now and I couldn't be happier.

Thank you so very much!!!
Glad it's working. I can't tell you how many times I've done that sort of thing.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on November 19, 2014, 04:05:40 PM
We Central Floridians got our first "cold snap" of the year last night.  It got down to 39 F, giving me the first real opportunity to examine the wind chill numbers in the sensor data packets (nybble offset 111, length 3).  Although winds were quite light (< 3 mph, gusts to 7), there should have been some difference between the current outside temp (offset 75, length 3) but there wasn't.  Can anyone confirm that their station is outputting a wind chill less than the outside temp?

Update: My research into how wind chill is calculated shows that calculations aren't valid for wind speeds less than 3 mph (NOAA Wind Chill Chart (http://www.crh.noaa.gov/ddc/?n=windchill)).  However, I checked my data and found a few data points above 3.  Of course, I have no idea what calculations LaCrosse is performing.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on November 19, 2014, 07:27:10 PM
We Central Floridians got our first "cold snap" of the year last night.  It got down to 39 F, giving me the first real opportunity to examine the wind chill numbers in the sensor data packets (nybble offset 111, length 3).  Although winds were quite light (< 3 mph, gusts to 7), there should have been some difference between the current outside temp (offset 75, length 3) but there wasn't.  Can anyone confirm that their station is outputting a wind chill less than the outside temp?

Update: My research into how wind chill is calculated shows that calculations aren't valid for wind speeds less than 3 mph (NOAA Wind Chill Chart (http://www.crh.noaa.gov/ddc/?n=windchill)).  However, I checked my data and found a few data points above 3.  Of course, I have no idea what calculations LaCrosse is performing.

I originally found this field by doing a check through all my collected station packets, looking at the difference between current outside temp at offset 75 and the (then unknown) value at offset 111. I found the value at 111 was lower only when it was both windy and cold, and never higher than the outside temp. I didn't do any calculations on the values, and it could be something else.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on November 20, 2014, 04:40:12 PM
I originally found this field by doing a check through all my collected station packets, looking at the difference between current outside temp at offset 75 and the (then unknown) value at offset 111. I found the value at 111 was lower only when it was both windy and cold, and never higher than the outside temp. I didn't do any calculations on the values, and it could be something else.
Your climate is much more likely to experience wind chill than mine, but I'll keep an eye on it this winter.  Last winter was extremely mild even by Central Florida standards.  As I recall, it only dropped below 40 F for a few days. I haven't tried to find the long range forecast for this winter, but I certainly don't want one like we had in 1982-3 just to check the functionality of my weather station.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: KennKong on November 26, 2014, 05:45:07 PM
I finally got a wind chill value less than the outside temperature, but it doesn't agree with either the old or new formulas from the NOAA page.  At 6:41 a.m., the outside temp was 56.7 F, with a wind of 9 mph, and the station reported a wind chill of 53.2 F.  The NOAA formulas aren't even valid above 50 F.  Nonetheless, the old formula yielded 50.1, while the new formula yielded 54.6.

I need to convert the formulas to deg C and kph, and test them with the raw data packet values.

The good news is the station does report a wind chill under some circumstances.  How those values compare with accepted formulas remains to be investigated.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 01:15:30 AM
Howdy all.  I'm new here and I hope it's appropriate to ask for help on this existing thread.

I have a LaCrosse C84612 and I just got it setup and talking to the LaCrosse cloud.  But I'd really like for it to be talking to WX-Underground.  So I downloaded SkySpy and installed it (version 1.1.0.177).    I have both the SkySpy Monitor and SkySpy service running.  Per the SkySpy User's Manual I also ran the Gateway advanced setup.  As far as I know it ran correctly.  It came up with a bunch of numbers in all the boxes.  It seems to have found my LaCrosse gateway.  So I copied the PC-IP address into the "Use" field, made sure it was set for port 8000, and clicked the "Use" box.  I then clicked the "Set" button, waited a few seconds, clicked "Reboot", and closed the GAS.  I did not see any sort of activity when I clicked "Set" or "Reboot".  Not sure if this is normal.

But... I don't see any activity on the SkySpy monitor.  If I understand correctly I should be seeing Service Debug Info records every 30 seconds or so - right?  Unfortunately I can't find any info on what to do if that data is not updating.  Any tips would be greatly appreciated. 

I'll warn you in advance - I'm smart enough to hit the buttons you tell me to - and report the results - but I'm no expert when it comes to IT network kind of stuff.

Thanks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 02:40:03 AM
I  tried some things based on what I found in this thread.  Here's the update...

- I made sure my windows firewall has SkySpy as an exception.
- I checked that my PC's IP address is in fact the one I entered into the "Use" box on GAS.
- I tried disabling "Microsoft Security Essentials"...

It does not seem to be updating the LaCrosse website any more.  Is this because SkySpy intercepts the data?

I am running Windows 7

The PC-IP I get is 192.168.1.64  This seems to be the IP of the laptop I'm running SkySpy on.
   When I point a browser at that IP address I get my "WiRNS" page (a utility for talking to my ReplayTV).
   The "Actual IP" shown in GAS is 192.168.1.71   When I point a browser at that IP address I get the data shown in the attachment below.

"Have you tried opening port 8000 to all traffic in the firewall on the spysky machine?"  Not yet.  Without
   a spotter that's above my pay-grade.  I have tried looking at the settings for my wireless router and it
   does indicate that its firewall is active.  My LaCrosse gateway device is currently plugged directly into my
   wireless router.

As above - any help will be greatly appreciated.  Thanks.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2015, 01:07:24 PM
I  tried some things based on what I found in this thread.  Here's the update...

Spork,

You may have SkySpy set to use a non-existent external MySQL database.  Please make sure you select "Use Embedded MySQL" in the SkySpy Monitor setup form.

Once you have SkySpy up and running, your weather data will no longer appear on the LaCrosse web site.  It will appear on Weather Underground if you have the correct WU password and Station ID set up in SkySpy Monitor.

Quote
It does not seem to be updating the LaCrosse website any more.  Is this because SkySpy intercepts the data?

Actually, it doesn't update because you have reconfigured your weather station to report to SkySpy instead of Lacrosse, using GAS.

Quote
The PC-IP I get is 192.168.1.64  This seems to be the IP of the laptop I'm running SkySpy on.
When I point a browser at that IP address I get my "WiRNS" page (a utility for talking to my ReplayTV).

Don't forget to use the correct port 8000 number in the URL.  Something like (lurkers: don't click the links in the URLs below - they will NOT work):

http://192.168.1.64:8000

That should work from any PC on your home LAN, as long as SkySpy Service is running on 192.168.1.64.  If you are sitting at the PC running SkySpy Service, then you can use:

http://localhost:8000


to view your tabular weather data.  Localhost means "the IP address of the machine I am sitting at"

If you do not add the 8000 part to the URL, your browser will default to port 80.  This is not what you want and will probably show some other web site.

Quote
   The "Actual IP" shown in GAS is 192.168.1.71   When I point a browser at that IP address I get the data shown in the attachment below.

Ignore the "Actual IP".  It is not useful.

So, make sure you are using Embedded mode and write me back if you are still having problems.

Best regards,

Kevin

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 01:45:18 PM
Thanks very much for the response.  When I added the ":8000" that solved that problem.  I now see the attached when I point my browser to that address.

And each time I do point my browser to that address I get one record of activity in the SkySpy monitor - as in the 2nd attachment.

I feel certain I selected "embedded" but now I can't even find where that setting would be.  Take a peek at the remaining attachments if you would to see what my UI looks like with each of the "home", "tools", "settings", and "help" tabs selected.  It seems like I'm missing stuff under "tools" and "settings".
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 01:46:13 PM
I hit the limit on attachments.  Here are the other two tabs...

Thanks again for your help.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 02:04:29 PM
So I pretty clearly messed up my ribbon somehow.  Last night I did see all the normal icons - today I don't.  I uninstalled SkySpy monitor and re-installed just now.  Still no luck.  Then I tried customizing the ribbon.  I found some of the settings that were missing but they now appear under a new tab "RibbonTab1".  I do have "embedded" checked.  If you can tell me how to do a "factory reset" on SkySpy I'll do that.

Still no activity reports though.

Thanks for your patience.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 02:30:30 PM
I think I've managed to reconfigure my ribbon properly.  I do have "embedded" checked - but still not activity except when I open a browser window to http://localhost:8000 (in which case I get one activity record).
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2015, 05:24:22 PM
Thanks very much for the response.  When I added the ":8000" that solved that problem.  I now see the attached when I point my browser to that address.

Good!  We are making progress.  This shows that your SkySpy Service is running properly in the background, but is not recording weather samples.

Quote
And each time I do point my browser to that address I get one record of activity in the SkySpy monitor - as in the 2nd attachment.

Yes, Monitor should show a GET request each time you open your browser on the "mini web page" or refresh it.

I will answer your other questions on their respective posts.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2015, 05:28:17 PM
So I pretty clearly messed up my ribbon somehow.  Last night I did see all the normal icons - today I don't.  I uninstalled SkySpy monitor and re-installed just now.  Still no luck.  Then I tried customizing the ribbon.  I found some of the settings that were missing but they now appear under a new tab "RibbonTab1".  I do have "embedded" checked.  If you can tell me how to do a "factory reset" on SkySpy I'll do that.

You somehow figured out a way to trash your Monitor registry entry.   :grin:

The ribbon configuration (or lack thereof) is stored in the Windows Registry.  It sounds like you have unsnarled it if you can now see your ribbon buttons.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2015, 05:34:43 PM
I think I've managed to reconfigure my ribbon properly.  I do have "embedded" checked - but still not activity except when I open a browser window to http://localhost:8000 (in which case I get one activity record).

Are you sure that you have GAS set up to send data to your IP (proxy settings)?  In an earlier post you mentioned that you were playing with GAS again.

Hint:  Once you have GAS set up to use your PC as a proxy, then you should never touch it again.  Do not reset your Internet Gateway module either by pushing its button.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 05:57:34 PM
Quote from: skydvrz
You somehow figured out a way to trash your Monitor registry entry.

Yes.  I had been messing with the "Configure ribbon" button on SkySpy monitor.  On the one hand I don't want to mess things up beyond repair, but on the other hand I hate to be the guy asking for help without having tried everything I can first.  I also hate being the guy that says "it doesn't work - what should I do?".  So I'm trying to provide as much info as I can.

For what it's worth - I had not messed with the registry to mess it up.  I'm pretty sure I did all that damage with the "configure ribbon" button.  I did look at the registry this morning because I assumed the settings must be stored there if a re-install didn't fix it.  But I still didn't mess with the registry directly.  I'm pretty sure I've got it straightened out with the "configure ribbon" button.

Are you sure that you have GAS set up to send data to your IP (proxy settings)?

Unfortunately when you use words like "proxy" I start getting out of my element.  I'm not *sure* but I think I have it set correctly.  I'm not at home at the moment so I can't do a screen grab from GAS - but taking an image out of your doc I can tell you what I have.

In the box marked PC-IP I have 192.168.1.64.  If I drop down the box I have some other address, but when I select it most of the other boxes go blank.  I also like the 192.168.1.64 address because it shows the proper serial number for my LaCrosse gateway.

So I copied that address into the "use" box at the very bottom.  I have both "Use DHCP" and "Use" checked.  I have 8000 in the "Port" box.

Generally speaking I have it looking pretty much like it does in your doc.  I hit "Set", waited a few seconds, and hit "Reboot".

Quote
In an earlier post you mentioned that you were playing with GAS again.

Yes.  I understand that it's supposed to be "set it and forget it", but I've been messing with it a fair bit since I haven't been able to see data coming across.  Do I need to start fresh somehow?

Quote
Do not reset your Internet Gateway module either by pushing its button.

I would not swear to it - but I don't think I've reset the Gateway since I got it talking to the LaCrosse site last night.  That being said - that claim is somewhat suspect since it has not updated the Lacrosse site since 9:43p last night.  I understand it won't update LaCrosse when SkySpy is running - but I left the system up and my laptop running SkySpy is 20 miles away at the moment.  So I would guess it would go back to updating LaCrosse(?)

Thanks again.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 09:01:48 PM
Now that I'm home I'm able to capture a screen grab from the Gateway Advanced Setup.  The first image shows the exact configuration I see when I hit Set and then Reboot.

The second grab shows the alternate PC-IP in the drop-down box.  But you can see that everything else is either blank or zero when I select that - so I don't.

I hope this is of some help.  I'm kind of tempted to re-register to the LaCrosse site since I may have messed that up - but I think I'll wait to hear from you before making a bigger mess.

Thanks again.



Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 07, 2015, 09:58:05 PM
Now that I'm home I'm able to capture a screen grab from the Gateway Advanced Setup.  The first image shows the exact configuration I see when I hit Set and then Reboot.

That looks good.  Don't change anything on the gateway module or GAS.

Quote
The second grab shows the alternate PC-IP in the drop-down box.  But you can see that everything else is either blank or zero when I select that - so I don't.

You can ignore that one too.  You may have a second network port on your PC that you are not using - this is probably it.

Do you have another PC on your LAN that you can try a test on?

Try to enter the full IP address URL (not the localhost one) into a browser from another PC on your LAN.  If your network is not blocking port 8000, then you should see the mini-web site in the browser.   This test will not work if you are not on-site, unless you do some heroic stuff with your router.

Quote
I hope this is of some help.  I'm kind of tempted to re-register to the LaCrosse site since I may have messed that up - but I think I'll wait to hear from you before making a bigger mess.


Don't do that - you'll just waste trouble shooting time.  You will remain "registered" with them forever.  Nothing SkySpy does changes that. 

SkySpy is a "clone" of the LaCrosse web server.  GAS simply redirects your weather station Gateway module to send its data to the SkySpy PC and the SkySpy Service instead of LaCrosse.  SkySpy then fakes out the Gateway module into thinking it is talking to LaCrosse by mimicking all the chit-chat that LaCrosse uses to retrieve your weather data.

So in a nutshell, "registration" has no meaning for SkySpy, since there are no revenue protection issues like LaCrosse has.  They want to verify that you have a valid account with them.  SkySpy is free and doesn't care about any of that stuff.

You do have to configure SkySpy with your current registration number issued by LaCrosse.  This number will change each time you "re-register", so don't re-register!  If you use the wrong registration number, SkySpy and the Gateway will jabber to one another continuously, your weather station wall display clock will start to drift, and the "Internet" indicator next to the time of day numbers will shut off.

If you have already re-registered, make sure you use the 7FFFxxxxxxxx number that was last issued by LaCrosse.  This number is stored/refreshed in your Gateway module each time you re-register.

I explain in the manual about how you go about retrieving your 7FFFxxxxxxxxxxxx number from LaCrosse.

I hope this makes sense!

One other thing... Can you email me your SkySpy.ini file, found in C:\ProgramData\SkySpyData

You can send it to me at

skyspyATidk-inc.com

Replace AT with the appropriate character for email addresses.

If you are paranoid, you can replace your passwords with xxxxxx in the INI file before you send it to me but don't touch anything else!  Use Notepad or some other simple text editor.  Do not use a word processor!

Best regards,

Kevin

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 10:16:58 PM
Thanks very much.  I'm going to do that test right now from my wife's laptop.

I just checked the 7fff number on the LaCrosse site.  It still matches the one I have in SkySpy monitor.

I'll send you the .ini file right after I try the test.

Rick C.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 07, 2015, 10:30:29 PM
OK - I just sent you the .ini file and ran the suggested test from my wife's laptop.  She is on the same wireless router as me - so I assume that means she's on the same LAN.  She got "website not found" when we entered: http://192.168.1.64:8000

I'm betting the problem is becoming obvious now to everyone but me.

Thanks again.


Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rockhopper on January 08, 2015, 02:09:46 PM
Hi all! Was curious about doing the same thing, capturing the LaCrosse data without using their server, when I stumbled upon page 1 of this thread a couple days ago. I've read through the whole thing with great interest, and I've looked at some of the software you all have developed.

I've got pretty extensive background in programming, so first thing I've got to say is I'm quite impressed with all you've done, especially all the time skydvrz must have spent about a year ago puzzling out the messages sent back and forth between the gateway and the server!  =D&gt; Well done.

So... I'm curious if anyone has solved the very first question on the thread: getting this all to work with the TX60U-IT temperature/humidity sensor. I've got a couple of those, not the weather station.

The package with the TX60U-IT includes the same GW1000U gateway that you've been using with the weather station, so I imagine the conversation between it and the server must be very similar. Tried skyspy (and some error log messages from yesterday were coming from me, skydvrz, so safe to ignore those), but realized quickly that it won't work with the TX60U-IT. The "7FFF" registration numbers for the weather station are "0001" numbers for the TX60U units. So unless someone else has already explored that difference, looks like that's where I'd have to start investigating. But I assume that there will be many more similarities than differences.

I'm also very interested in hearing how far the Linux variations have gotten. It looks like three different versions have gotten off the ground? In the long run, I'd like to get a PHP or Python version running on a cloud server, but I also have no qualms about working via the Windows version while puzzling out the protocols.

Much thanks to you all for all the work that's already been put into this!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 08, 2015, 07:17:03 PM
Rockhopper,

There is nothing special about the 7FFF number assigned by LaCrosse.  Whatever you type in to SkySpy is used to handshake with the Gateway.  Try it and find out.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rockhopper on January 08, 2015, 07:25:15 PM
Hmm. I seem to recall that when I tried the "0001" value yesterday, skyspy came back with an error message stating that it required 16 digits (ok) and it had to start with "7FFF".
Will double-check that.

I'll also check the INI file to see if I can bypass the error message that way.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rockhopper on January 08, 2015, 09:24:15 PM
Okay, confirmed that "0001" as the first 4 digits triggers an error message in SkySpy. It wants "7FFF". But I was indeed able to bypass that by entering the "0001" value for the TX60U sensor directly into the INI file.

SkySpy's mini browser is accessible on the network, and if I flip the GW back and forth between talking to the LaCrosse server and SkySpy, I do get a ping message when flipping it to SkySpy. Then, about every minute, it shows an RTC packet request, a ping, an RTC packet request, and a ping. However none of the sensor readings change to a non-zero value.

So they're "talking", but I'm guessing that the data types sent by the TX60U are not matching those expected by SkySpy for the C84612 weather station.

Back to the original question: has anyone yet got this working with the TX60U-IT sensors, as in the original thread post? Or are we on new territory? Very encouraging so far.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: rockhopper on January 08, 2015, 09:40:02 PM
Spork:

Try turning off your Windows firewall entirely, just to see if communications start coming through. I was getting nothing on mine, even though SkySpy ("Weather Info Monitor") was showing up in the firewall exceptions. Then I tried just turning off the firewall entirely and bingo, I was able to get to the mini website from another computer, and my gateway was now pinging SkySpy. Neither worked until I turned off the firewall.

Will need to investigate a solution better than this all or nothing approach, but if that allows communications through then at least we've zeroed in on the issue.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 08, 2015, 10:35:08 PM
Thanks very much for the tip rockhopper.  Skydvrz actually just got it sorted out for me.  The firewall was in fact half the problem.  I had tried turning it off yesterday - and that didn't fix it.  Skydvrz figured out that the computer was protecting the .ini file from me, SkySpy, etc.  Once that was sorted out turning off the firewall did fix it.

You're right - I'm hoping to configure my firewall to allow messages on port 8000.  Hopefully without crashing the internet.

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 08, 2015, 10:40:47 PM
OK - I just turned my firewall back on - but I made a rule to allow incoming TCP messages on port 8000.  Hopefully that will work.  But to be honest I'm not sure if I need the rule for incoming or outgoing and TCP or UDP.  I'll try things until it works.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: spork on January 08, 2015, 11:00:14 PM
Alrighty - the firewall is back on with a rule to allow incoming TCP messages on port 8000 - and SkySpy monitor is still updating.  I really appreciate all the help!

Now I'll set it up to talk to WX-Underground.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on January 09, 2015, 07:13:34 PM
All:

I am going to start a new thread called SkySpy here in the LaCrosse section.  This thread is getting pretty long and has drifted quite a bit since I started SkySpy development.

See you over there!
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: gpiveta on September 16, 2015, 02:17:47 PM
Hello dear friends
I am new in this forum just acquire two stations this model c84612 in america and live in the Pantanal of Brazil
Excuse me for my English but ...
Would I like to help using the SkySpy and this ERF-100 gateway accompanying, ie how do the pc configuration and network and gateway, you need to be on the same network this server or not? It should set the router's ip address?
Ie like the step-by-step instructions on using some manual or about
Please if you can send email thank
gpiveta@ig.com.br

grateful
Gustavo
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: 10ACTony on September 16, 2015, 02:26:07 PM
Hello dear friends
I am new in this forum just acquire two stations this model c84612 in america and live in the Pantanal of Brazil
Excuse me for my English but ...
Would I like to help using the SkySpy and this ERF-100 gateway accompanying, ie how do the pc configuration and network and gateway, you need to be on the same network this server or not? It should set the router's ip address?
Ie like the step-by-step instructions on using some manual or about
Please if you can send email thank
gpiveta@ig.com.br

grateful
Gustavo

Everything you need is in the SkySpy forum.  Latest version and setup guide.

Here's the link to the forum and post #100 has the latest version.


http://www.wxforum.net/index.php?topic=24882.0

Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: gpiveta on September 16, 2015, 02:39:00 PM
Hello dear friends
I am new in this forum just acquire two stations this model c84612 in america and live in the Pantanal of Brazil
Excuse me for my English but ...
Would I like to help using the SkySpy and this ERF-100 gateway accompanying, ie how do the pc configuration and network and gateway, you need to be on the same network this server or not? It should set the router's ip address?
Ie like the step-by-step instructions on using some manual or about
Please if you can send email thank
gpiveta@ig.com.br



grateful
Gustavo

Everything you need is in the SkySpy forum.  Latest version and setup guide.

Here's the link to the forum and post #100 has the latest version.


http://www.wxforum.net/index.php?topic=24882.0

Thank you 10ACTony
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: PDM1964 on November 24, 2016, 07:32:50 PM
For anyone who is still using this weather station and may be looking for alternative software.  I recently switched over to weewx with the Interceptor driver.  Spent some time working with mwall and got the clock to set etc. \:D/ As much as I have loved using SkySpy I am looking forward to running in Linux now.

https://github.com/matthewwall/weewx-interceptor

https://www.wunderground.com/personal-weather-station/dashboard?ID=KUTPLAIN2
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: ddurdle on October 13, 2017, 10:35:10 AM
Is there any way to leverage this to send different forecasts through the gateway?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on October 19, 2017, 07:18:53 PM
Why would you want to?  Are you trying to convince people that Alaska is warm in the winter or that Florida is cool in the summer? :lol:

The answer is yes, you could easily do that, but your weather station would be marked as low-accuracy and may disappear from public weather maps.

My own weather station is picking up roof heat and direct sun here in Florida, so it reads several degrees hotter than actual in the noon-day sun.  I get marked down for that by Weather Underground.  I have my (non-LaCrosse) weather sensors on an 8' mast, but it is still not high enough off the roof when it is >100F outside.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: ddurdle on October 19, 2017, 07:28:07 PM
Not sure if that was directed at me.

GW1000U ERF-100 is also meant for pulling weather data to populate the forecast on La Crosse displays, but La Crosse has decided to arbitrarily kill the service, which make thousands of the devices they sold for displaying weather forecasts to be completely useless before their natural-end-of-life.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: skydvrz on October 20, 2017, 11:16:01 AM
Sorry, you confused me with the word "forecast" and your desire to send different ones.  A forecast is a prediction of future weather. 

Your weather station records and reports current sensor readings.  The LaCrosse unit can do rudimentary forecasts but these are not reported anywhere except on the LCD display.  The raw sensor readings can be sent to various weather sites in near real-time. 

The LaCrosse "forecast" is generated by firmware in the display itself.  There is no forecast information sent from the LaCrosse web site back to your display.

There are alternatives to the LaCrosse web site.  Read backwards through the posts here and you will see links to various software packages that can post real-time weather data to Wunderground and possibly other crowd-sourced weather sites.  Wunderground uses your weather data to make forecasts, but these are only visible on their web site.  The service is free.

SkySpy connects the LaCrosse gateway module to your PC.  From there the PC software sends your weather station sensor samples to Wunderground and/or IKiteSurf.  SkySpy completely replaces the LaCrosse web site.

Unfortunately SkySpy is no longer supported (I wrote it).  There are some other software packages that may work for you.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: Red Dragon on November 08, 2017, 08:31:13 PM
What he is asking is the same thing I'm asking. I have a display that gets the forecast from the gateway. Weather direct gets the forecast from accuweather and sends it to my gateway to the display. Now that service is gone how can we used the gateway to send a forecast to the display? Like form accuweather or somewhere?
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: keckec on March 08, 2018, 03:20:22 PM
My GW-1000U stopped working a few weeks ago. I tried the factory reset several times, but didn't help. On http://support.lacrossealertsmobile.com/articles/gateway-troubleshoot-replacement-parts (http://support.lacrossealertsmobile.com/articles/gateway-troubleshoot-replacement-parts) Lacrosse says the gateway may stop working after several years. Yikes, that's planned obsolescence for you!

On a hunch I opened the gateway up and found a MX25L2005 flash chip inside. These have a 100,000 write-cycle endurance, and I have my gateway configured to save history every 5 minutes. This would use up the write cycles in a little over a year, and mine is about four years old. I replaced the flash chip with a Macronix MX25L2006E, did a factory reset and re-registration, and it's all working again. The part was less than $5.00 including shipping from Digikey Electronics. At that price, I'll order a couple of spares.

BY the way, the PHP code referened way earlier in this thread worked just fine for re-registration, and I have packet captures of it all if anyone is interested.
Title: Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
Post by: timmerk on June 17, 2022, 06:26:34 PM
I replaced the flash chip with a Macronix MX25L2006E, did a factory reset and re-registration, and it's all working again.

keckec, thank you! I just got some replacements. Do you know which of the three SOIC8 ICs it is, along with which pin is #1? The factory burned off the numbers on mine, and I can only tell pin #1 on the bottom one: