WXforum.net

Weather Station Hardware => WiFiLogger/Meteo-Pi => Topic started by: WiFiLogger on December 14, 2020, 12:51:30 PM

Title: New things for Davis console
Post by: WiFiLogger on December 14, 2020, 12:51:30 PM
Will I find here some users (3 people) to test and help improve newest prototypes of connectivity devices for Davis consoles?
Title: Re: New things for Davis console
Post by: Herbaldew on December 14, 2020, 01:07:01 PM
I would be happy to.
Title: Re: New things for Davis console
Post by: PaulMy on December 14, 2020, 01:22:58 PM
I can if needed.


Enjoy,
Paul
Title: Re: New things for Davis console
Post by: fkapp on December 14, 2020, 02:23:56 PM
I can help if needed.
I am running WLL so not sure if this impacts what you are asking for help testing.

Can you share a bit more information on what testing help you need?

Thanks
Frank
Title: Re: New things for Davis console
Post by: WiFiLogger on December 14, 2020, 03:29:23 PM
I can help if needed.
I am running WLL so not sure if this impacts what you are asking for help testing.

Can you share a bit more information on what testing help you need?

Thanks
Frank

1 prototype is a set of boards to take out logger from console.
Logger is outside to dont generate extra heat, but the main feature is that batteries inside console do not power WL IP, or WiFiLogger, but just logger memory.
Batteries should work for 3 months with WiFiLogger, or WL IP.
I am looking someone with WL IP to check this design.

2 prototype is Ethernet device very close to WL IP. It is sending data to WL. com and WU and share localy on port 22222. Uploads are with 1min resolution, so no RapidFire. It should be robust, not filled with multiple options.
Here I need people with Weewx, or other problematis software, to help me make it robust.
Title: Re: New things for Davis console
Post by: davidmc36 on December 15, 2020, 04:04:20 AM
I have a spare console or two receiving wireless ISS data and a spare RPi that could run WeeWx or something.
Title: Re: New things for Davis console
Post by: WiFiLogger on December 15, 2020, 06:57:23 AM
I have a spare console or two receiving wireless ISS data and a spare RPi that could run WeeWx or something.

Perfect. I will prepare prototype unit for you.
It should be ready on the end of this week.

In general Weewx is not prepared for any error in connection. It is because RS232 is very robust. WL IP was very good. WiFiLogger because of WiFi reconnection from time to time it is not working perfectly with WeeWx.
New Ethernet logger has to work perfectly with WeeWx, or there is no place for it on market.
Title: Re: New things for Davis console
Post by: davidmc36 on December 15, 2020, 07:30:51 AM
Cool. I set up WeeWx before and also have CumulusMX running now with MeteoPi.
Title: Re: New things for Davis console
Post by: WiFiLogger on December 15, 2020, 07:41:20 AM
Cool. I set up WeeWx before and also have CumulusMX running now with MeteoPi.

No rush. Shipping is about 14 days. So you will have it in 17-20 days :)
Title: Re: New things for Davis console
Post by: TomKeffer on December 15, 2020, 09:56:29 AM
In general Weewx is not prepared for any error in connection. It is because RS232 is very robust. WL IP was very good. WiFiLogger because of WiFi reconnection from time to time it is not working perfectly with WeeWx.

Not true at all. WeeWX sets a timeout parameter for all serial operations. It the connection times out, it retries 3 times. If that fails, it releases the connection and retries from the top.

Not sure I understand the rest of your comment.

Regards,

-Tom Keffer (author of WeeWX).
Title: Re: New things for Davis console
Post by: davidmc36 on December 16, 2020, 07:00:57 PM
I believe I have been successful so far. Web server is running but has no data fed into it yet. I would have to take down one of the other stations to use hardware to try it out.

Once connected to console I expect the worst I have now is to correct one file path maybe.

 [ You are not allowed to view attachments ]
Title: Re: New things for Davis console
Post by: davidmc36 on December 17, 2020, 09:08:36 AM
I am successful with setting the program to Simulator. Stumbled at first as I forgot about setting the file permission to allow weeconfig to work.

 [ You are not allowed to view attachments ]
Title: Re: New things for Davis console
Post by: WiFiLogger on December 18, 2020, 08:51:07 AM
In general Weewx is not prepared for any error in connection. It is because RS232 is very robust. WL IP was very good. WiFiLogger because of WiFi reconnection from time to time it is not working perfectly with WeeWx.

Not true at all. WeeWX sets a timeout parameter for all serial operations. It the connection times out, it retries 3 times. If that fails, it releases the connection and retries from the top.

Not sure I understand the rest of your comment.

Regards,

-Tom Keffer (author of WeeWX).
Tom it is perfect that you are here and you are active.
Cilents who bought WiFiLogger claim that it is not stable with WeeWX. WeeWX is a great software. There is multiple people wanted to use it.
With WiFiLogger WeeWX crash from time to time. Some users wrote me that crash is every 4 days.
WiFiLogger has some features to reboot it when something is not right. This mechanism include not only internal WatchDog, but also external hardware monitoring chip.

Some clientes to solve thairs problems, they are writing scripts to reboot WiFiLogger, but also to check if WeeWX demon is still working and turn it on again if crashed.

Maybe there is a way, to prevent WeeWX from crashes?
Title: Re: New things for Davis console
Post by: TomKeffer on December 18, 2020, 10:09:22 AM
Tom it is perfect that you are here and you are active.
Cilents who bought WiFiLogger claim that it is not stable with WeeWX. WeeWX is a great software. There is multiple people wanted to use it.
With WiFiLogger WeeWX crash from time to time. Some users wrote me that crash is every 4 days.
WiFiLogger has some features to reboot it when something is not right. This mechanism include not only internal WatchDog, but also external hardware monitoring chip.

Some clientes to solve thairs problems, they are writing scripts to reboot WiFiLogger, but also to check if WeeWX demon is still working and turn it on again if crashed.

Maybe there is a way, to prevent WeeWX from crashes?

LOL! I'd say it's the other way around. If your goal is to emulate the original Davis logger, then it should not crash the driver, which has been stable for 10+ years.

What, exactly, is the problem?
Title: Re: New things for Davis console
Post by: WiFiLogger on December 21, 2020, 12:16:38 PM
Tom it is perfect that you are here and you are active.
Cilents who bought WiFiLogger claim that it is not stable with WeeWX. WeeWX is a great software. There is multiple people wanted to use it.
With WiFiLogger WeeWX crash from time to time. Some users wrote me that crash is every 4 days.
WiFiLogger has some features to reboot it when something is not right. This mechanism include not only internal WatchDog, but also external hardware monitoring chip.

Some clientes to solve thairs problems, they are writing scripts to reboot WiFiLogger, but also to check if WeeWX demon is still working and turn it on again if crashed.

Maybe there is a way, to prevent WeeWX from crashes?

LOL! I'd say it's the other way around. If your goal is to emulate the original Davis logger, then it should not crash the driver, which has been stable for 10+ years.

What, exactly, is the problem?

I can't say. I don't use WeeWX. I have some info from my clients, that they are not satisfied with connecting WeeWX to WiFiLogger.
I am attaching LOG from Jerry. I hope I do not publish Jerry's sensitive data.

When WiFiLogger loose WiFi connection, then it reconnecting and test connection. It can take 3 minutes, then it reboot itself if there is no internet.
During this situation WeeWX crash, or I believe it is during this operation. Frequency: 2-4 days.
Title: Re: New things for Davis console
Post by: TomKeffer on December 21, 2020, 01:48:13 PM
Thanks for that. Looks like the user is running on a Mac.

That is a very short snippet of a log, and it does not have timestamps, so it's hard to piece together exactly what happened.

It looks like weewxd (the WeeWX daemon) had a connection to the logger, downloaded some data, then the connection timed out, then it was reset by peer (the logger, in this case). This is followed by numerous time outs, and more resets. Correct?

Does your IP logger have any keep alive strategy? Or, if there is no activity for a while, does it drop the connection?

-tk
Title: Re: New things for Davis console
Post by: TomKeffer on January 02, 2021, 09:12:40 PM
Thanks for that. Looks like the user is running on a Mac.

That is a very short snippet of a log, and it does not have timestamps, so it's hard to piece together exactly what happened.

It looks like weewxd (the WeeWX daemon) had a connection to the logger, downloaded some data, then the connection timed out, then it was reset by peer (the logger, in this case). This is followed by numerous time outs, and more resets. Correct?

Does your IP logger have any keep alive strategy? Or, if there is no activity for a while, does it drop the connection?

-tk

@WiFiLogger?
Title: Re: New things for Davis console
Post by: Intheswamp on January 02, 2021, 10:50:47 PM
I think the demon must have got'im...
Title: Re: New things for Davis console
Post by: johnd on January 03, 2021, 03:47:56 AM
@WiFiLogger?

On vacation at the moment I believe. Should be back as from 11th Jan.
Title: Re: New things for Davis console
Post by: k8tek on January 11, 2021, 12:15:48 PM
I cannot get weeex to work with the WiFi logger either. It keeps dropping the connection. Cumulus MX works fine. I think the problem has something to do with weewx hogging the connection.
Title: Re: New things for Davis console
Post by: TomKeffer on January 11, 2021, 01:44:45 PM
I cannot get weeex to work with the WiFi logger either. It keeps dropping the connection. Cumulus MX works fine. I think the problem has something to do with weewx hogging the connection.

WeeWX works fine with the standard Davis logger. It does depend on being able to control the state of the logger. But, until @WiFiLogger gets back to us, we'll never know what the issue is.

Meanwhile, if you're having trouble, post to the weewx-user (https://groups.google.com/forum/#!forum/weewx-user) forum.
Title: Re: New things for Davis console
Post by: k8tek on January 11, 2021, 02:24:32 PM
I cannot get weeex to work with the WiFi logger either. It keeps dropping the connection. Cumulus MX works fine. I think the problem has something to do with weewx hogging the connection.

WeeWX works fine with the standard Davis logger. It does depend on being able to control the state of the logger. But, until @WiFiLogger gets back to us, we'll never know what the issue is.

Meanwhile, if you're having trouble, post to the weewx-user (https://groups.google.com/forum/#!forum/weewx-user) forum.

Your belcherstown skin is what made me attempt to get away from cumulus.
Title: Re: New things for Davis console
Post by: TomKeffer on January 11, 2021, 02:31:58 PM
Credit goes to Pat O'Brien for that.
Title: Re: New things for Davis console
Post by: mcrossley on January 12, 2021, 07:15:34 AM
I cannot get weeex to work with the WiFi logger either. It keeps dropping the connection. Cumulus MX works fine. I think the problem has something to do with weewx hogging the connection.

I had to put quite a bit of extra code into Cumulus MX to make it work "fairly reliably" with the WiFi logger compared to the Davis device. The WiFi logger connectivity is not very stable.
Title: Re: New things for Davis console
Post by: davidmc36 on January 12, 2021, 07:37:02 AM
So the instability is related to those programs but no issue it seems in my case with WFL running almost one year just uploading to CWOP and getting picked up by my MeteoPi.

Will be interesting to see how it can be solved.
Title: Re: New things for Davis console
Post by: WiFiLogger on January 12, 2021, 08:48:53 AM
Does your IP logger have any keep alive strategy? Or, if there is no activity for a while, does it drop the connection?

ut, until @WiFiLogger gets back to us, we'll never know what the issue is.

Tom
I am very sorry for the delay. I had a lot of urgent work before vacations all ASAP.

I would like to describe connection in WiFiLogger now.
There is 3 options, but "Collision" I have not tested and maybe it is not finished. Collision is when WiFiLogger internal functions are able to read station even if TCP port 22222 is opened.
Other more civilized options are:
- "PAUSE with Disconnection"
- "PAUSE"

In general WL IP and WiFiLogger need to make access to console to read data for export. So disconnection option in Cumulus and WeeWX is very needed for proper working.
This is part on PC software like Cumulus, WeatherLink and WeeWX.
On WiFiLogger there are also such options:
WiFiLogger can't disconnect port 22222 every 1 minute, because reading of archive records 513 pages takes 2 minutes 15 sec. And uploading firmware to console takes about 5-7 minutes. But anyway it needs to take care of own export options e.g. to WL.COM, or WU if those are enabled.

So "PAUSE with Disconnection" will close TCP port if there is no command send to console via this port for 60 sec.
Cumulus is sending one command to console to receive continue readings of LOOP. So here is little problem, because one command is sent on the beginning then there is no TX data to console, so WiFiLogger will close such port after 60 sec.
No problem if Cumulus has set discon period. It is closing connection itself every 1 minute.

option "PAUSE" is blocking WiFiLogger against reading console when TCP port is OPEN.
Here, please note that TCP port 22222 will be closed and opened for new software connection.

e.g. WeatherLink PC is reading data via TCP port. Then we opening Cumulus and Cumulus is starting to read console on port 22222. So Cumulus is closing TCP port for WeatherLink PC and new socket is opened for Cumulus. Ofcourse WL PC can make reconnection and close socket for Cumulus.
On the begining it was different. TCP port was assigned to first software. New TCP socket was not opened, before first one was closed.
But Cumulus was making reconnections without closing old port. So Cumulus was trying to open sockets 10 times to access console, but was not releasing first socket which WiFiLogger was reserving for it.
Now new connection drop old.

Also WiFiLogger is working in main loop. TCP port is maintain in same loop where exports to weather services (WL.com, Wu etc....) are made. If connection to WU will be invalid then TIMEOUT for socket is 15 sec. So WeeWX will not have access to console.
On the other hand users knows about that and they are closing all exports and other activities of WiFiLogger, leaving just TCP port access.

In general connection to WeeWX is working fine, but WiFiLogger has a few routines to reboot itself if needed.
I had info from clients that WeeWX with WiFiLogger crash every 4 days. So 4 days it is not issue with TCP port access which can be reset every 1 minute. It is more like WiFiLogger is rebooting. After reboot It takes minimum 8 sec. to works again, sometimes more when archive records have problem to read on the beginning.
I do believe that here will be main problem.

On the other hand why WeeWX is crashing?

Title: Re: New things for Davis console
Post by: TomKeffer on January 12, 2021, 09:10:10 AM
Perhaps it's a language issue, but I'm not understanding much of what you are saying.

It sounds like port 22222 is being used to both communicate with the console, and to communicate with the computer, as well as perform updates, etc. So you are having to multiplex between its many uses. Is that correct? Keep in mind, that while I'm very familiar with the Davis logger, I know nothing about your logger.

On the other hand why WeeWX is crashing?

Why indeed? Perhaps it's because your logger has a very different state management strategy than the Davis logger it is trying to emulate. If I'm understanding you correctly, and I may not be, you cannot have a logger which simply closes a connection to go off and do something else. You'll have to find a way to teach it to maintain the connection and open up a new one.

Or, maybe I'm completely misunderstanding you.
Title: Re: New things for Davis console
Post by: mcrossley on January 12, 2021, 09:24:44 AM
So "PAUSE with Disconnection" will close TCP port if there is no command send to console via this port for 60 sec.
Cumulus is sending one command to console to receive continue readings of LOOP. So here is little problem, because one command is sent on the beginning then there is no TX data to console, so WiFiLogger will close such port after 60 sec.

Which is not how the Davis logger works, it does not arbitrarily close the connection.

Quote
But Cumulus was making reconnections without closing old port. So Cumulus was trying to open sockets 10 times to access console, but was not releasing first socket which WiFiLogger was reserving for it.
Now new connection drop old.

It keeps opening new sockets because the old ones repeatedly get connection closed by peer or connection aborted type error - I cannot close a socket that is already closed or in an error state. Reconnection often successfully opens a socket then fails to get any response to Davis commands, and Cumulus has close and reconnect several times before the logger finally makes a good connection that it responds over.  The same code reconnecting to the Davis logger always works first time.
Title: Re: New things for Davis console
Post by: WiFiLogger on January 13, 2021, 06:01:05 AM
It sounds like port 22222 is being used to both communicate with the console, and to communicate with the computer, as well as perform updates, etc. So you are having to multiplex between its many uses. Is that correct? Keep in mind, that while I'm very familiar with the Davis logger, I know nothing about your logger.

While socket with port 22222 is opened, then it makes direct access to console.
About firmware update. I have mentioned about console firmware update.
While firmware is being uploaded to console, then there can't be disruption for about 6 minutes.
Title: Re: New things for Davis console
Post by: GHammer on January 18, 2021, 06:02:35 PM
I have had no problems with the WiFiLogger 2 since I use WeeWX to handle services I wish to send my data to.
In that case, there is no competing use of the port/wifi chip and all is well.
So, if used for a single connection to a single program, there should be no error until you have a firmware update. Then, there will be no connection and if the program you use with the WiFiLogger can't handle its absence, it will likely crash. I'd like to have a 'Dont update automatically' option in WiFiLogger for that reason.

An alternative for WeeWX would be to use the WeeWX-MQTTSubscribe driver to get data into WeeWX and the MQTT export to send from WiFiLogger. At the time, there is an issue with the data from the WiFiLogger and it will sometime produce values that are a string instead of the numeric that WeeWX-MQTTSubscribe/WeeWX wants.

If Wojtek can clean the MQTT export up, that would solve the connection problem as far as WeeWX is concerned as there would be no direct connection between WeeWX and the WiFiLogger.

Mind you, I do not run multiple programs accessing the logger, nor do I report to any service as I use WeeWX to do that.

Title: Re: New things for Davis console
Post by: davidmc36 on March 30, 2021, 07:48:04 AM
So I have been running this new Ethernet Logger for a while now. It works very well with an up-time now of almost 19 days.

Using Weather Display or Weatherlink it pauses for WL.com uploads every minute.

I decided to try Weewx on the Pi and when first starting up I got nothing. Then after trying WD and WL again, they would not connect. The Ethernet Logger was still running fine and sending to WL.com

Then I changed the IP to static on the Ethernet Logger and Weewx started working fine. The other programs would still connect and allow WL.com uploads.

The thing with Weewx running now is that it DOES NOT pause for WL.com

I wonder what the method is to make it do that.
Title: Re: New things for Davis console
Post by: GHammer on March 31, 2021, 10:13:42 AM
So I have been running this new Ethernet Logger for a while now. It works very well with an up-time now of almost 19 days.

Using Weather Display or Weatherlink it pauses for WL.com uploads every minute.

I decided to try Weewx on the Pi and when first starting up I got nothing. Then after trying WD and WL again, they would not connect. The Ethernet Logger was still running fine and sending to WL.com

Then I changed the IP to static on the Ethernet Logger and Weewx started working fine. The other programs would still connect and allow WL.com uploads.

The thing with Weewx running now is that it DOES NOT pause for WL.com

I wonder what the method is to make it do that.

There is no method for WeeWX to pause data collection.
Curious as to what service wl.com is. If I put that in a browser I get a strange website that has nothing to do with weather.
Title: Re: New things for Davis console
Post by: mcrossley on March 31, 2021, 10:43:02 AM
Curious as to what service wl.com is. If I put that in a browser I get a strange website that has nothing to do with weather.

It's an abbreviation for weatherlink.com
Title: Re: New things for Davis console
Post by: davidmc36 on March 31, 2021, 11:36:58 AM
......

There is no method for WeeWX to pause data collection.......

OK Good to know.

At least I got WeeWx rolling again. Had it a while ago and this week drew a total blank ](*,) for a while.

My options are open.