Author Topic: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100  (Read 139254 times)

0 Members and 2 Guests are viewing this topic.

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #375 on: July 29, 2014, 02:25:21 PM »
I recommend using the embedded database for now, if you want to try out the program.

The full-up MySQL install is fairly complex, and not well tested.  That is why I don't cover it in the manual.

I could send you MySQL scripts to create the two schemas that are required.  I saw from your error messages that you had table and DB hostname issues.  You may not have given yourself sufficient privileges on MySQL to create and modify new tables.

Before you overwrite them, could you send me a backup of the two schemas (if any) on your MySQL server box:

weather2
utility2

I'd like to see what got created, so I can debug/update the internal MySQL creation scripts that are supposed to fire on initial installation.

Offline dld121

  • Member
  • *
  • Posts: 9
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #376 on: July 29, 2014, 02:32:31 PM »
I am happy to send you what it built, but I am not a MYSQL expert.  Can you tell me what to do to back it up?

I was using MYSQL on  port 3308 and set your app to do the same.

The DB settings are:
My INI settings are:
Code: [Select]
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you
# *** upgrade to a newer version of MySQL.
[client]
port=3308

[mysqld]

# Remove leading # and set to the amount of RAM for the most important data
# cache in MySQL. Start at 70% of total RAM for dedicated server, else 10%.
innodb_buffer_pool_size = 128M

# Remove leading # to turn on a very important data integrity option: logging
# changes to the binary log between backups.
# log_bin

# These are commonly set, remove the # and set as required.
basedir = "C:\\mysql56\\"
datadir = "C:\\mysql56\\data\\"
port = 3308
#server_id = ....


# Remove leading # to set options mainly useful for reporting servers.
# The server defaults are faster for transactions and fast SELECTs.
# Adjust sizes as needed, experiment to find the optimal values.
join_buffer_size = 128M
sort_buffer_size = 2M
read_rnd_buffer_size = 2M

sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

Offline 10ACTony

  • Senior Member
  • **
  • Posts: 71
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #377 on: July 29, 2014, 02:48:58 PM »
I need some brave souls to download, install and test the next version of SkySpy.  Feedback is welcome!

Make sure you back up any existing installation and data before installing the new version.

Here is a link to the users manual.  I recommend reading it prior to starting the installation.  As usual, the installer will put another copy in your installation folder.

https://www.dropbox.com/s/6htjoyuxkgezia9/ReadMe2.pdf

Here is a link to the installer:

https://www.dropbox.com/s/4qpv7destypx726/SkySpyInstall.exe


After thinking about it for a bit, I'm guessing I should have Uninstalled the old version first.  I guess I'm just too used to just over writing the old version.  Maybe it would be a good idea to put a note in the quick setup reminding us to uninstall first.  Will try uninstalling old and do a clean install of the new version. When I get back home.

After recovering from my brain fart the installation went smoothly without any issues.
« Last Edit: July 29, 2014, 06:54:07 PM by 10ACTony »
//
No trees were destroyed in the sending of this message though a significant number of electrons were terribly inconvenienced.
//

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #378 on: July 29, 2014, 07:21:22 PM »
After recovering from my brain fart the installation went smoothly without any issues.

Great!  Which mode are you using?  Embedded or Full MySQL?  I need to shut my public MySQL server off soon, before I get in Dutch with my ISP.  I am up to a 319 gigabytes of Internet usage this month.  Eeek!

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #379 on: July 29, 2014, 08:09:31 PM »
I am happy to send you what it built, but I am not a MYSQL expert. 

You are in for a spot of trouble then.  If you are a intermediate or better programmer and a network person, then you can dive right in and muddle your way through.  Otherwise, bad things will happen.  Setting up any sort of database engine is non-trivial.

Quote
Can you tell me what to do to back it up?

Download, configure and run MySQL Workbench. The Community version is free.

Use the Workbench Data Export feature to back up your schemas and data.  Zip or RAR the resulting SQL files and send them to me.

Quote
I was using MYSQL on  port 3308 and set your app to do the same.

If you are going to run your own server, then you should probably leave the port set to the default 3306.  I used 3308 for my public server since I run more than one MySQL server on my LAN and need to keep them separate and also allow for port forwarding through my router-firewall.

Quote
The DB settings are:

??? Nothing came through.

Quote
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
# *** DO NOT EDIT THIS FILE. It's a template which will be copied to the
# *** default location during install, and will be replaced if you

As the file header says, it is just a template for initial installation. You need to use the Workbench to actually configure MySQL.  There are hundreds of settings.  Most of the defaults will work, but there are a few that probably need tweaking, depending on your computer hardware and needs.  This is where the programming and network skills come in.

You must run the Workbench utility on the same PC as the database engine if you want to adjust settings.  Once you have MySQL configured, you can monitor it from any workstation on your LAN using the same utility.

You need to restart MySQL to make any settings changes "take".


Here is a typical MySQL settings file (mysql.ini) for a v5.6.19 installation on Windows Server 2008, 64-bit.  Do NOT use the settings here as they are specific to the server hardware used.  These are not the settings I use on my server, since I can't access them just now.

Code: [Select]
# Other default tuning values
# MySQL Server Instance Configuration File
# ----------------------------------------------------------------------
# Generated by the MySQL Server Instance Configuration Wizard
#
#
# Installation Instructions
# ----------------------------------------------------------------------
#
# On Linux you can copy this file to /etc/my.cnf to set global options,
# mysql-data-dir/my.cnf to set server-specific options
# (@localstatedir@ for this installation) or to
# ~/.my.cnf to set user-specific options.
#
# On Windows you should keep this file in the installation directory
# of your server (e.g. C:\Program Files\MySQL\MySQL Server X.Y). To
# make sure the server reads the config file use the startup option
# "--defaults-file".
#
# To run run the server from the command line, execute this in a
# command line shell, e.g.
# mysqld --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# To install the server as a Windows service manually, execute this in a
# command line shell, e.g.
# mysqld --install MySQLXY --defaults-file="C:\Program Files\MySQL\MySQL Server X.Y\my.ini"
#
# And then execute this in a command line shell to start the server, e.g.
# net start MySQLXY
#
#
# Guildlines for editing this file
# ----------------------------------------------------------------------
#
# In this file, you can use all long options that the program supports.
# If you want to know the options a program supports, start the program
# with the "--help" option.
#
# More detailed information about the individual options can also be
# found in the manual.
#
# For advice on how to change settings please see
# http://dev.mysql.com/doc/refman/5.6/en/server-configuration-defaults.html
#
#
# CLIENT SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by MySQL client applications.
# Note that only client applications shipped by MySQL are guaranteed
# to read this section. If you want your own MySQL client program to
# honor these values, you need to specify it as an option during the
# MySQL client library initialization.
#
[client]
no-beep

# pipe
# socket=mysql
port=3306

[mysql]

default-character-set=utf8


# SERVER SECTION
# ----------------------------------------------------------------------
#
# The following options will be read by the MySQL Server. Make sure that
# you have installed the server correctly (see above) so it reads this
# file.
#
# server_type=3
[mysqld]

# The next three options are mutually exclusive to SERVER_PORT below.
# skip-networking
# enable-named-pipe
# The Pipe the MySQL Server will use
# socket=mysql

# The TCP/IP Port the MySQL Server will listen on
port=3306

# Path to installation directory. All paths are usually resolved relative to this.
# basedir="C:/Program Files/MySQL/MySQL Server 5.6/"

# Path to the database root
datadir="C:/ProgramData/MySQL/MySQL Server 5.6/data\"

# The default character set that will be used when a new schema or table is
# created and no character set is defined


# The default storage engine that will be used when create new tables when
default-storage-engine=INNODB

# Set the SQL mode to strict
sql-mode="STRICT_TRANS_TABLES,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION"

# Enable Windows Authentication
# plugin-load=authentication_windows.dll

# General and Slow logging.
log-output=NONE
general-log=0
general_log_file="WOLVERINE.log"
slow-query-log=0



# Binary Logging.
# log-bin

# Error Logging.
log-error="WOLVERINE.err"

# The maximum amount of concurrent sessions the MySQL server will
# allow. One of these connections will be reserved for a user with
# SUPER privileges to allow the administrator to login even if the
# connection limit has been reached.
max_connections=151

# Query cache is used to cache SELECT results and later return them
# without actual executing the same query once again. Having the query
# cache enabled may result in significant speed improvements, if your
# have a lot of identical queries and rarely changing tables. See the
# "Qcache_lowmem_prunes" status variable to check if the current value
# is high enough for your load.
# Note: In case your tables change very often or if your queries are
# textually different every time, the query cache may result in a
# slowdown instead of a performance improvement.
query_cache_size=0

# The number of open tables for all threads. Increasing this value
# increases the number of file descriptors that mysqld requires.
# Therefore you have to make sure to set the amount of open files
# allowed to at least 4096 in the variable "open-files-limit" in
# section [mysqld_safe]
table_open_cache=2000

# Maximum size for internal (in-memory) temporary tables. If a table
# grows larger than this value, it is automatically converted to disk
# based table This limitation is for a single table. There can be many
# of them.
tmp_table_size=118M

# How many threads we should keep in a cache for reuse. When a client
# disconnects, the client's threads are put in the cache if there aren't
# more than thread_cache_size threads from before.  This greatly reduces
# the amount of thread creations needed if you have a lot of new
# connections. (Normally this doesn't give a notable performance
# improvement if you have a good thread implementation.)
thread_cache_size=10

#*** MyISAM Specific options
# The maximum size of the temporary file MySQL is allowed to use while
# recreating the index (during REPAIR, ALTER TABLE or LOAD DATA INFILE.
# If the file-size would be bigger than this, the index will be created
# through the key cache (which is slower).
myisam_max_sort_file_size=100G

# If the temporary file used for fast index creation would be bigger
# than using the key cache by the amount specified here, then prefer the
# key cache method.  This is mainly used to force long character keys in
# large tables to use the slower key cache method to create the index.
myisam_sort_buffer_size=226M

# Size of the Key Buffer, used to cache index blocks for MyISAM tables.
# Do not set it larger than 30% of your available memory, as some memory
# is also required by the OS to cache rows. Even if you're not using
# MyISAM tables, you should still set it to 8-64M as it will also be
# used for internal temporary disk tables.
key_buffer_size = 25M

# Size of the buffer used for doing full table scans of MyISAM tables.
# Allocated per thread, if a full scan is needed.
read_buffer_size=64K
read_rnd_buffer_size=256K

# This buffer is allocated when MySQL needs to rebuild the index in
# REPAIR, OPTIMZE, ALTER table statements as well as in LOAD DATA INFILE
# into an empty table. It is allocated per thread so be careful with
# large settings.
sort_buffer_size=256K

#*** INNODB Specific options ***
# innodb_data_home_dir=0.0

# Use this option if you have a MySQL server with InnoDB support enabled
# but you do not plan to use it. This will save memory and disk space
# and speed up some things.
# skip-innodb

# Additional memory pool that is used by InnoDB to store metadata
# information.  If InnoDB requires more memory for this purpose it will
# start to allocate it from the OS.  As this is fast enough on most
# recent operating systems, you normally do not need to change this
# value. SHOW INNODB STATUS will display the current amount used.
innodb_additional_mem_pool_size = 47M

# If set to 1, InnoDB will flush (fsync) the transaction logs to the
# disk at each commit, which offers full ACID behavior. If you are
# willing to compromise this safety, and you are running small
# transactions, you may set this to 0 or 2 to reduce disk I/O to the
# logs. Value 0 means that the log is only written to the log file and
# the log file flushed to disk approximately once per second. Value 2
# means the log is written to the log file at each commit, but the log
# file is only flushed to disk approximately once per second.
innodb_flush_log_at_trx_commit=1

# The size of the buffer InnoDB uses for buffering log data. As soon as
# it is full, InnoDB will have to flush it to disk. As it is flushed
# once per second anyway, it does not make sense to have it very large
# (even with long transactions).
innodb_log_buffer_size=9M

# InnoDB, unlike MyISAM, uses a buffer pool to cache both indexes and
# row data. The bigger you set this the less disk I/O is needed to
# access data in tables. On a dedicated database server you may set this
# parameter up to 80% of the machine physical memory size. Do not set it
# too large, though, because competition of the physical memory may
# cause paging in the operating system.  Note that on 32bit systems you
# might be limited to 2-3.5G of user level memory per process, so do not
# set it too high.
innodb_buffer_pool_size=755M

# Size of each log file in a log group. You should set the combined size
# of log files to about 25%-100% of your buffer pool size to avoid
# unneeded buffer pool flush activity on log file overwrite. However,
# note that a larger logfile size will increase the time needed for the
# recovery process.
innodb_log_file_size=48M

# Number of threads allowed inside the InnoDB kernel. The optimal value
# depends highly on the application, hardware as well as the OS
# scheduler properties. A too high value may lead to thread thrashing.
innodb_thread_concurrency = 10

# The increment size (in MB) for extending the size of an auto-extend InnoDB system tablespace file when it becomes full.
innodb_autoextend_increment=64

# The number of regions that the InnoDB buffer pool is divided into.
# For systems with buffer pools in the multi-gigabyte range, dividing the buffer pool into separate instances can improve concurrency,
# by reducing contention as different threads read and write to cached pages.
innodb_buffer_pool_instances=8

# Determines the number of threads that can enter InnoDB concurrently.


# Specifies how long in milliseconds (ms) a block inserted into the old sublist must stay there after its first access before
# it can be moved to the new sublist.
innodb_old_blocks_time=1000

# It specifies the maximum number of .ibd files that MySQL can keep open at one time. The minimum value is 10.
innodb_open_files=300

# When this variable is enabled, InnoDB updates statistics during metadata statements.
innodb_stats_on_metadata

# When innodb_file_per_table is enabled (the default in 5.6.6 and higher), InnoDB stores the data and indexes for each newly created table
# in a separate .ibd file, rather than in the system tablespace.
innodb_file_per_table=1

# Use the following list of values: 0 for crc32, 1 for strict_crc32, 2 for innodb, 3 for strict_innodb, 4 for none, 5 for strict_none.
innodb_checksum_algorithm=0

# The number of outstanding connection requests MySQL can have.
# This option is useful when the main MySQL thread gets many connection requests in a very short time.
# It then takes some time (although very little) for the main thread to check the connection and start a new thread.
# The back_log value indicates how many requests can be stacked during this short time before MySQL momentarily
# stops answering new requests.
# You need to increase this only if you expect a large number of connections in a short period of time.


# If this is set to a nonzero value, all tables are closed every flush_time seconds to free up resources and
# synchronize unflushed data to disk.
# This option is best used only on systems with minimal resources.


# The minimum size of the buffer that is used for plain index scans, range index scans, and joins that do not use
# indexes and thus perform full table scans.
join_buffer_size=256K

# The maximum size of one packet or any generated or intermediate string, or any parameter sent by the
# mysql_stmt_send_long_data() C API function.
max_allowed_packet = 160M

# If more than this many successive connection requests from a host are interrupted without a successful connection,
# the server blocks that host from performing further connections.


# Changes the number of file descriptors available to mysqld.
# You should try increasing the value of this option if mysqld gives you the error "Too many open files".
open_files_limit=4161

# Set the query cache type. 0 for OFF, 1 for ON and 2 for DEMAND.
query_cache_type=0

# If you see many sort_merge_passes per second in SHOW GLOBAL STATUS output, you can consider increasing the
# sort_buffer_size value to speed up ORDER BY or GROUP BY operations that cannot be improved with query optimization
# or improved indexing.
sort_buffer_size=256K

# The number of table definitions (from .frm files) that can be stored in the definition cache.
# If you use a large number of tables, you can create a large table definition cache to speed up opening of tables.
# The table definition cache takes less space and does not use file descriptors, unlike the normal table cache.
# The minimum and default values are both 400.
table_definition_cache=1400

# Specify the maximum size of a row-based binary log event, in bytes.
# Rows are grouped into events smaller than this size if possible. The value should be a multiple of 256.
binlog_row_event_max_size=8K

# If the value of this variable is greater than 0, a replication slave synchronizes its master.info file to disk.
# (using fdatasync()) after every sync_master_info events.


# If the value of this variable is greater than 0, the MySQL server synchronizes its relay log to disk.
# (using fdatasync()) after every sync_relay_log writes to the relay log.


# If the value of this variable is greater than 0, a replication slave synchronizes its relay-log.info file to disk.
# (using fdatasync()) after every sync_relay_log_info transactions.


secure-auth = off
net_buffer_length = 2M
net_write_timeout = 1800
wait_timeout = 1800
basedir = "C:/Program Files/MySQL/MySQL Server 5.6"
interactive_timeout = 28800
innodb_lock_wait_timeout = 10
net_read_timeout = 1800

performance_schema = off


After you have a functional MySQL installation, you need to set up some user accounts.  You should probably create an account for SkySpy that only has access to the two necessary schemas and limit it to logging in from a specific IP address or subnet on your LAN if you use DHCP. This will keep the bad guys out of your server. You use the Workbench to create MySQL accounts and give them privileges.

So...  Do you see why I suggest using Embedded mode?  It is a heckuva lot simpler than full up MySQL.   :grin:



Offline 10ACTony

  • Senior Member
  • **
  • Posts: 71
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #380 on: July 30, 2014, 01:10:32 AM »
skydvrz

Anyway to get the data from the old database (weather) into the new (weather2) ??
//
No trees were destroyed in the sending of this message though a significant number of electrons were terribly inconvenienced.
//

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #381 on: July 30, 2014, 10:19:23 AM »
Anyway to get the data from the old database (weather) into the new (weather2) ??

If you are using a full-up MySQL database engine, then yes.  Back it up and send it to me and I will translate it to the new format.  Are you still on my server?

Otherwise you can view/download your historical data from Weatherunderground.

Offline 10ACTony

  • Senior Member
  • **
  • Posts: 71
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #382 on: July 30, 2014, 12:21:26 PM »
Anyway to get the data from the old database (weather) into the new (weather2) ??

If you are using a full-up MySQL database engine, then yes.  Back it up and send it to me and I will translate it to the new format.  Are you still on my server?

Otherwise you can view/download your historical data from Weatherunderground.
I'm on my own server, I'll send you the backup later today. Everything seems to be working great since I remembered to uninstall first.

Thanks.
//
No trees were destroyed in the sending of this message though a significant number of electrons were terribly inconvenienced.
//

Offline benmargolin

  • Member
  • *
  • Posts: 3
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #383 on: July 31, 2014, 07:57:12 AM »
I finally got around to trying some of this code, and used what I think was the latest snapshot of keckec's as well as kennkong. I couldn't get either one to work out of the box :( but after some cajoling I got the keckec stuff to mostly work -- I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!

I also ended up using this for determining my serial number: running mitmdump (http://mitmproxy.org/doc/mitmdump.html), which made it really easy.

I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(

I really appreciate all the work everyone's put into this effort! I wish I had more time to contribute.

Offline keckec

  • Senior Member
  • **
  • Posts: 62
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #384 on: August 01, 2014, 11:01:20 AM »
I finally got around to trying some of this code, and used what I think was the latest snapshot of keckec's as well as kennkong. I couldn't get either one to work out of the box :( but after some cajoling I got the keckec stuff to mostly work -- I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!
Sorry my php/schema isn't fully working. I'm sort-of feeling my way along, as this is the first time I've worked with PHP or SQL. The db schema has changed substantially several times, as I learn more.

I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(
I don't mind using github, and I currently use git with my own repo. So far my code is rough, and as you found out a lot of it doesn't work very well yet. I don't plan to send to WU, but store data for myself and a few friends, and provide web pages to view graphs, etc. I've been saving captures of all the packets to and from the station, and so far have about 500K packets. Right now I'm working on php code to import it into the db, which I doubt would be of much interest to others.

I'm still not sure the best way to gather rain data from the station. There are quite a few fields transmitted from the station that have to do with rain, but none that I see directly indicates bucket tips except the history packets. Even with history there is a compromise between storage depth and bucket-tip time granularity. And the history record may lose bucket tips when the history interval is changed. I'm still working on it.

Offline KennKong

  • Member
  • *
  • Posts: 30
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #385 on: August 01, 2014, 04:48:26 PM »
I was missing the schema for the rain history table, and it doesn't seem to ever record any of the historical min/maxes (code missing for the most part), but at least there's sensorvalues getting into the DB!
I gather from context that you're talking about keckec's version of historical values, but it does bring up a point about my version that may not be clear.  My `records` table must be prepopulated with the records you want to keep, the request script only updates existing records.  It doesn't create missing ones.  This is because certain records like minimum wind and rainfall are useless, so why keep them?
Quote
I do think it'd be really great if we could all agree to use a versionable repo like github to share stuff through, it seems as though there's at least 3 php forks going now, all with slightly different featuresets :( I like the features that kennkong seems to have (WUG posting, passthrough to Lacrosse, simple charting), but the schema seems to have diverged somewhere, for one thing. :(

I really appreciate all the work everyone's put into this effort! I wish I had more time to contribute.
I've never used github or any other repository before, but I'm not opposed to using any particular one.  In the meantime, I've updated my code weather.tgz  I've reconfigured everything to reside in one directory, which on my system I've set up as a virtual host.  These should work in the root directory of a web server, too.  As I've said before, I'm a total newb at web services, so these are hardly shining examples of best practices.  Any suggestions for improvement are welcome.

Update: I've created a github repository for this project Weather Gateway
« Last Edit: August 01, 2014, 07:09:36 PM by KennKong »

Offline benmargolin

  • Member
  • *
  • Posts: 3
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #386 on: August 02, 2014, 06:41:57 AM »
Kenn, thanks a lot for this, I'm making some progress getting your stuff to work for me.
However, PLEASE include the data dump for the 'sensors' table, and any others that have "generic" data! Pretty tricky to figure out what is supposed to populate those...

Edit: fixuprain() still not working properly for me; commented out the call to it, for now. I also didn't have an admin@localhost user on my mysql install, so the views didn't work; I adjusted them to be the right user but maybe setting SQL SECURITY INVOKER would be a better solution. Also had to change the HTTP-IDENTIFY to be HTTP_IDENTIFY (underscore vs hyphen). Not sure why on your GW it (apparently!) sends a hyphen...?

You can see tweaks I've made so far in my fork at https://github.com/benmargolin/Weather-ERF-Gateway-1000U/compare/kennkong:master...master (really not much, mostly rearranging some stuff; no SQL changes documented, sadly.)

Anyhow, thanks again!
« Last Edit: August 02, 2014, 07:57:16 AM by benmargolin »

Offline KennKong

  • Member
  • *
  • Posts: 30
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #387 on: August 02, 2014, 05:28:42 PM »
Kenn, thanks a lot for this, I'm making some progress getting your stuff to work for me.
However, PLEASE include the data dump for the 'sensors' table, and any others that have "generic" data! Pretty tricky to figure out what is supposed to populate those...
I updated weather.sql to pre-populate the sensors table.  I added a file records.sql which has a script to populate the records table after you have at least one sensor reading
Quote
Edit: fixuprain() still not working properly for me; commented out the call to it, for now.
Looking more carefully at the code, fixuprain() might need an existing record for the last rain reset date in the sensordatevalues table to work.
Quote
I also didn't have an admin@localhost user on my mysql install, so the views didn't work; I adjusted them to be the right user but maybe setting SQL SECURITY INVOKER would be a better solution.
I didn't check the generated schema script closely enough.
Quote
Also had to change the HTTP-IDENTIFY to be HTTP_IDENTIFY (underscore vs hyphen). Not sure why on your GW it (apparently!) sends a hyphen...?
The problem is not with my gateway, it's with my webserver.  It doesn't have apache_request_headers() built-in, so I borrowed some code to create one.  That code changes the underscores to hyphens for some reason.  If your server has apache_request_headers() that leaves the underscores unchanged, well, how am I supposed to know that?  Remember, total webserver newb here.

Anyway, I pushed a few changes to the repository.  Thanks for your comments, I have a lot to learn.

Offline KennKong

  • Member
  • *
  • Posts: 30
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #388 on: August 04, 2014, 12:24:13 AM »
Thanks to keckec, I've tested my code with another station.  This helped me debug the fixuprain issue, which has been fixed and pushed to the repository.

To really get multiple stations working, I need to get my data display pages to allow the user to select which station they want to see.  I already have a script stations.js which creates a drop-down list that could be added to the top of weather.html, etc.  However, I can't quite figure out how to use that list to change which station is displayed.

This is what I want to happen:
Loading weather.html displays the data for the default weather station, as set by stations.js.
Loading weather.html?stationid=X will load the data for station X.
Changing the selected station in the drop-down list will reload the page with ?stationid=selected

It's the last part I can't quite figure out (reloading the page with a new URL).  Any suggestions are welcome.  I'm opening a branch for working on that in the repository.

Offline KennKong

  • Member
  • *
  • Posts: 30
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #389 on: August 05, 2014, 12:14:30 AM »
I've updated the repository with the multiple station select versions of the display pages.  It ain't pretty, so if you have any suggestions for making it look better, fire away.

If you choose not to use the multiple station select version, just hardcode your stationid in wsdata.php.

Update: I have a little more tweaking to do.  I'd like to make the station list box go away if there's only one station in the database.  That way, the (I'm assuming) typical single station user won't have to look at a useless box, while the code will still apply the stationid automatically.

Update 2: That was a little easier than I thought.  I'm just hiding the station select drop down list if there's only one station.

Next task: a configuration page!
« Last Edit: August 05, 2014, 05:24:52 PM by KennKong »

Offline dwoolridge

  • Member
  • *
  • Posts: 4
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #390 on: August 13, 2014, 09:37:40 AM »
I'm excited to try skydvrz program for the C84612 but have a question...

How large are the data packets sent from the C84612 gateway to the La Crosse Alerts site?

The reason I ask is my model airplane club will have to subscribe to a wireless internet package through one of the cell phone carriers.  I need to determine if the 100Mb per month plan is adequate, or if we need the 1Gb per month plan for just the weather station.  I figured that if I knew the data packet size, I could get a good estimate of total bandwidth knowing the station sends the data every 4 minutes.

Thanks,

--Doug

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #391 on: August 13, 2014, 10:41:10 AM »
How large are the data packets sent from the C84612 gateway to the La Crosse Alerts site?

Are you going to use LaCrosse or SkySpy? 

There are many different packet types/sizes, but primarily there are the SDP, RTC and Ping packets that handle most of the daily chatter that the PWS does with the server.  I think you will be in the range of 300 bytes every 4 minutes or so - that works out well with your 100 MB / Month plan.  I also factored in some overhead for packet ACKs.  It is a two-way conversation between PWS and server and you will have to pay for both sides of the data conversation.


Offline semoore

  • Member
  • *
  • Posts: 7
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #392 on: August 14, 2014, 12:10:00 PM »
Hello Everyone and thanks to those who have spent many hours reverse engineering this. I had an opportunities to meet the LaCrosse product engineer last January and asked why a standalone application or protocol info wasn't available. His response was that the weather station technology is licensed from another company and they don't want provide that.

I'm having an issue with SkySpy working on WIN7 Ultimate. The program won't communicate with the gateway if I have the Windows firewall turned on for the local or private network. If I turn it off everything works OK. I have looked the firewall application rules and can see that ssmonitor.exe is listed in the 'incoming' rules, but I don't see an entry in the 'outgoing' rules. Any ideas?

-Skydvrz, I think you have a typo in the user manual when refering to the weather station as model C86412 instead of C84612.
« Last Edit: August 14, 2014, 02:21:20 PM by semoore »

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #393 on: August 14, 2014, 05:37:26 PM »
I'm having an issue with SkySpy working on WIN7 Ultimate. The program won't communicate with the gateway if I have the Windows firewall turned on for the local or private network. If I turn it off everything works OK. I have looked the firewall application rules and can see that ssmonitor.exe is listed in the 'incoming' rules, but I don't see an entry in the 'outgoing' rules. Any ideas?

ssMonitor does not do any outbound TCP/IP unless you are using an external MySQL database.

If you are using an external MySQL database then you need to open port 3306.  If you are using the embedded database, then you don't need to open this port.

You do need to open up the firewall to SkySpyService.EXE on port 8000.  SkySpyService is the brains of the outfit.  The Service chats with the Monitor over "named pipes" that don't need a network connection, but they do need to be co-located on the same PC.

BTW, you can look at all of your TCP/IP connection bindings by opening a DOS box and running netstat.  The port number used is the number to the right of the colon  ":".  The high port numbers are usually outbound.

Last, I recommend using a hardware firewall instead of software.  I use a router/firewall/WiFi Access Point box at my house.  A hardware firewall/router allows you to run servers and games on your home LAN and access them from the outside world in relative safety.

SkySpy was written and tested on Win7-64 Pro.

Quote
-Skydvrz, I think you have a typo in the user manual when refering to the weather station as model C86412 instead of C84612.

Was that the latest release? Thanks for the heads up.

Offline semoore

  • Member
  • *
  • Posts: 7
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #394 on: August 14, 2014, 07:07:27 PM »
I'm using the latest version of SkySpy configured with the internal DB.

I thought I had added SkySpyService.exe to the firewall program list during troubleshooting, but when I checked again it wasn't there. After adding it, it was listed as 'C86412 Gateway Interface and  Web Server' (C84612). Both the sensor data and web server are now working with Windows 'private networks' firewall turned on.

Is adding this entry to Windows Firewall something that should happen during install or does it need to be done manually?
I do have Avast running, but no other firewall software other than what's built into WIN7.

I do have hardware firewall and load balancing hardware between my LAN and the Internet.

Thanks!

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #395 on: August 15, 2014, 11:18:27 AM »
I'm using the latest version of SkySpy configured with the internal DB.

I thought I had added SkySpyService.exe to the firewall program list during troubleshooting, but when I checked again it wasn't there. After adding it, it was listed as 'C86412 Gateway Interface and  Web Server' (C84612). Both the sensor data and web server are now working with Windows 'private networks' firewall turned on.

Good!  Apparently my dyslexia is pervasive - or should I say lexdyxia ;-)

Quote
Is adding this entry to Windows Firewall something that should happen during install or does it need to be done manually?

I don't know of a good way to automatically add a firewall exception to Windows.  The end user will probably have to make an exception entry manually.  Doesn't it pop up and warn you the first time an unknown app tries to access the Internet?  Norton does that.

Quote
I do have hardware firewall and load balancing hardware between my LAN and the Internet.

Then you probably don't need Windows Firewall, unless you distrust the other PCs/users on your LAN.


Offline semoore

  • Member
  • *
  • Posts: 7
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #396 on: August 15, 2014, 01:07:43 PM »
Avast (free version) didn't warn of SkySpy trying to make outside connection. I normally run Comodo Firewall which does catch anything trying to make in or out bound connection.

New problem: I can't seem to get SkySpy to work on XP thin-client PC that I use for data logging applications. I closed SkySpy runing on WIN7 PC, changed the gateway to point at the XP PC & reset, opened XP firewall for SkySpy & started SkySpy on XP. After a few minutes here's what shows in the debug window...

Ping Received
Special SDP Len=210
JabberCount=1
Special SDP Len=210
JabberCount=2
Special SDP Len=210
JabberCount=3
repeats to =6 and then starts over.

I switched back over to WIN7 PC and it works fine.

What would you suggest I do next to discover the cause of the problem? ](*,)

Offline semoore

  • Member
  • *
  • Posts: 7
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #397 on: August 15, 2014, 02:22:18 PM »
Update: When I switched back to the WIN7 PC a 2nd time I did some get  'Special SDP= 210' for awhile. Now I'm get an occasional 'Special SDP-30, but the sensor values & WU are updating.

Should I reset the C84612 by removing the batteries?

Offline skydvrz

  • Senior Contributor
  • ****
  • Posts: 224
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #398 on: August 15, 2014, 02:37:55 PM »
Update: When I switched back to the WIN7 PC a 2nd time I did some get  'Special SDP= 210' for awhile. Now I'm get an occasional 'Special SDP-30, but the sensor values & WU are updating.

Should I reset the C84612 by removing the batteries?

No, just wait.  The Gateway is trying to update the LCX server with missed updates.  Once the buffer is flushed, then they will go away.

Is the "Internet" indicator on the LCD display active?

Offline semoore

  • Member
  • *
  • Posts: 7
Re: LaCrosse Wireless Internet Gateway Model GW1000U ERF-100
« Reply #399 on: August 15, 2014, 07:11:15 PM »

When the gateway is pointed at the XP system the LCD "Internet" icon was flashing or blinking. While connected to the WIN7 system it is a steady on.

I switched back to the XP system and the Special SDP=210 continued for over 4 hours and never did correct. I then switched back to WIN7 and had the 210 message about 20 times before the sensors updated and the =210 stopped.

I powered cycled the gateway & LCD while on XP and it had no effect. I keep thinking it's something simple or oblivious. :?
I don't want to load wireshark and dig in to the nuts & bolts to see what's different if I don't need to.

 

anything