Author Topic: WiFiLogger - Connect your Davis console directly to the Internet via WiFi  (Read 109349 times)

0 Members and 1 Guest are viewing this topic.

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #525 on: September 18, 2018, 09:49:23 AM »
I have found problem in 1.22. I am repairing it. Please do not update with 1.22.

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #526 on: September 18, 2018, 10:19:23 AM »
(Hint for exportfile. Don't use the same separator inside bracket-values as separator used to separate different items.)

I am not sure if I am understanding.
Could you paste example?

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #527 on: September 18, 2018, 10:20:56 AM »
Too late for me, just finished the update 10 min ago :(
Should I go back or just wait it out?

Enjoy,Paul

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #528 on: September 18, 2018, 10:27:14 AM »
Too late for me, just finished the update 10 min ago :(
Should I go back or just wait it out?
Enjoy,Paul

It cause reboot while NTP update. I didn't see it, because I have long interval in my unit.
With version 1.22 every time when NTP will want to renew time WFL will be rebooted.
Quit big issue. I have repaired that. In 10 min I will publish it.

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #529 on: September 18, 2018, 10:37:01 AM »
Newest firmware: http://wifilogger.net/files/WiFiLogger123.zip
BIN file has to be updated. +
From html folder files: "ftpexport.html", "setup.html", "export.html" has to uploaded.
New things:
- MQTT real time data (wflexp.json) export
- weathercloud.net
- FTP address can be IP number now.

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #530 on: September 18, 2018, 10:50:16 AM »
Thanks Wojtek,
I am now on v.1.23
Should I be watching for something in particular?

I had NTP Server: pool.ntp.org
Update: 15 min

Enjoy,
Paul

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #531 on: September 18, 2018, 11:08:16 AM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #532 on: September 18, 2018, 12:04:34 PM »
Thanks Wojtek,
I have extended the NTP server time, as I think the 15 min was the default or from past setting that I hadn't paid any further attention to.
I have enabled Weathercloud.

I've tried to understand MQTT and see how or for what I could use that, but not there yet  :oops:

Enjoy,
Paul


Offline johnd

  • Forecaster
  • *****
  • Posts: 4827
    • www.weatherstations.co.uk
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #533 on: September 18, 2018, 02:02:56 PM »
I've tried to understand MQTT and see how or for what I could use that, but not there yet  :oops:

You can point the finger at me for suggesting that (and pestering Wojtek for it).  :twisted:

But why I'm interested in it is that it should make possible very frequent updates to web page dashboards without loading the sending device (ie WFL in this case) with significant FTP uploads every few seconds. It would be nice to think that WFL could manage one MQQT upload per loop but not sure whether this is feasible or not. The overall idea is that you do a light-weight simple upload to an MQTT broker (eg effectively a server somewhere in the cloud) which can then act as a relay post for the data, potentially passing the data to multiple clients and maybe different dashboards too. So, much of the load for data distribution is taken off the sending device.

There's a bit more of a hurdle in understanding MQTT because you have to understand the role of the broker, but once you've grasped that then the overall concept is straightforward. You do also need a subscription plan to use a broker, but there are free low-usage plans available (eg at cloudmqtt.com and having a higher throughput plan doesn't cost megabucks.

Of course, having the MQTT source is only half of the picture - you also need some MQTT-using dashboard code to consume it. But hopefully having at least one device available in WFL that can generate an MQTT feed will now stimulate some software writers to fill in the other parts of the equation.

Prodata Weather Systems
Prodata's FAQ/support site for Davis stations
Includes many details on 6313 Weatherlink console.
UK Davis Premier Dealer - All Davis stations, accessories and spares
Cambridge UK

Sorry, but I don't usually have time to help with individual issues by email unless you are a Prodata customer. Please post your issue in the relevant forum section here & I will comment there if I have anything useful to add.

Offline blacklistedcard

  • Member
  • *
  • Posts: 44
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #534 on: September 18, 2018, 10:42:11 PM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)

With version 1.23 can I re-enable the NTP?

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #535 on: September 18, 2018, 10:57:32 PM »
ALL the uploads ;) seem to be working fine with the new version, but MQTT will have to wait for me.  And thanks johnd for the MQTT information, I will not lose any sleep over it.

Enjoy,
Paul

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #536 on: September 19, 2018, 03:01:07 AM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)
With version 1.23 can I re-enable the NTP?

Yes I have make changes in NTP function. Now WU should work ok. Please let me know if there are still problems.

Offline blacklistedcard

  • Member
  • *
  • Posts: 44
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #537 on: September 19, 2018, 12:35:30 PM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)
With version 1.23 can I re-enable the NTP?

Yes I have make changes in NTP function. Now WU should work ok. Please let me know if there are still problems.

I re-enabled it and so far it looks good....

Offline StudleyWeather

  • Member
  • *
  • Posts: 1
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #538 on: September 19, 2018, 02:41:14 PM »
I have to say this is a great piece of kit - installed fairly easily into my Vantage Vue and the real time data webpage is great.  I do however have one query - I am running WeatherLink 6.0.4 locally on a PC and have set up the TCP/IP connections for it (TCP port is 22222)  It successfully found the WiFi Logger but now when I try to poll data into WeatherLink I often get Communication error - it has polled successfulyl 3 times today but in the comms log it keeps saying communication error.  I am running v1.20 firmware on the logger.  On the logger I have the web interface enabled and TCP port set to Auto Pause with pause duration of 10 mins.  On Weatherlink it is set to poll hourly at 2 mins past the hour.  Is there anything else I need to set up/look at to enable both the real time page and WeatherLink to populate.  Thanks

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #539 on: September 20, 2018, 08:48:51 AM »
I have to say this is a great piece of kit - installed fairly easily into my Vantage Vue and the real time data webpage is great.  I do however have one query - I am running WeatherLink 6.0.4 locally on a PC and have set up the TCP/IP connections for it (TCP port is 22222)  It successfully found the WiFi Logger but now when I try to poll data into WeatherLink I often get Communication error - it has polled successfulyl 3 times today but in the comms log it keeps saying communication error.  I am running v1.20 firmware on the logger.  On the logger I have the web interface enabled and TCP port set to Auto Pause with pause duration of 10 mins.  On Weatherlink it is set to poll hourly at 2 mins past the hour.  Is there anything else I need to set up/look at to enable both the real time page and WeatherLink to populate.  Thanks
Pause duration works with manual pause. On first page Real Time Data you trigger manual pause by clicking time. This is old function probably I will erase that function.
Auto pause is triggered automatically when connection on port 22222 is appeared. Its stay paused as long as connection is on, but there is a safety feature here. When there is no data transfer from PC to WFL then after 60s WFL is closing connection. PC software if it's working can renew this connection.

I will check once more WL 6.0.4. Testing is time consuming. I will let you know.

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #540 on: September 20, 2018, 08:49:16 AM »
(Hint for exportfile. Don't use the same separator inside bracket-values as separator used to separate different items.)

I am not sure if I am understanding.
Could you paste example?

To separate items, when extracting file to insert to database, one usually uses a separator between items that isn't used anywhere else in the file. This to simplify extraction of each item. If the items can't be separated from each other, much more work has to be done to be able to easily extract.

Example, (from wflexp.json): "stnmod":17,"ver":4.33 .............. "xt":["---","---","---","---","---","---","---"], ..... "sunrt":"6:42"
In this case it isn't possible to separate items from each other with the separator "," due to that the comma is used inside the xt-items array. It isn't possible to use the separator ":" due to it is used inside the value for item sunrt, i.e 6:42.

To use separators for easy access to make operations on the items:
Example:     "stnmod":17;"ver":4.33 .............. "xt":["---","---","---","---","---","---","---"];"sunrt":"6.42"
In this example it's easy to get each item by splitting them on the separator ";" also after that it is easy to split the name from the value by splitting on the ":". All values has to have a meaning, thereby giving the 6.42 the same meaning as 6:42 but makes the handling of the file sooo much easier.




Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #541 on: September 20, 2018, 09:06:37 AM »
Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Try to upload once more "system.html" and "spark-md5.js". Please use system.html from newest firmware.
Which web browser do you use?
This button will be enabled when proper information from WFL will comes, but it's 2s delay. Do you know how to use debug console in Chrome? CTRL + SHIFT + J -> then Network tab

File is in JSON format. You don't need to parse it. It's very common file. You need to find library which will parce it for you.
http://php.net/manual/en/function.json-decode.php
https://www.w3schools.com/js/js_json_parse.asp

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #542 on: September 20, 2018, 09:29:23 AM »
Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Try to upload once more "system.html" and "spark-md5.js". Please use system.html from newest firmware.
Which web browser do you use?
This button will be enabled when proper information from WFL will comes, but it's 2s delay. Do you know how to use debug console in
Chrome? CTRL + SHIFT + J -> then Network tab

File is in JSON format. You don't need to parse it. It's very common file. You need to find library which will parce it for you.
http://php.net/manual/en/function.json-decode.php
https://www.w3schools.com/js/js_json_parse.asp
No change. Using Firefox. Tried to do it on my phone and it worked.

OK, thanks. Used to parse data to suit my needs....

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #543 on: September 20, 2018, 11:04:09 AM »
The Realtime.txt, where is this file supposed to end up?

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #544 on: September 20, 2018, 12:27:09 PM »
Quote
The Realtime.txt, where is this file supposed to end up?
In my FTP Archive Records Export Setup tab I have Folder path:  /public_html/wifi-console

to get http://www.komokaweather.com/wifi-console/realtime.txt

Enjoy,
Paul

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #545 on: September 20, 2018, 03:45:35 PM »
Quote
The Realtime.txt, where is this file supposed to end up?
In my FTP Archive Records Export Setup tab I have Folder path:  /public_html/wifi-console

to get http://www.komokaweather.com/wifi-console/realtime.txt

Enjoy,
Paul
The only file I get is the wflexp.json.

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 5509
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #546 on: September 20, 2018, 05:34:50 PM »
Quote
The only file I get is the wflexp.json.
Wojtek may have an answer. 


From that one setting in FTP Archive Records Export setup I get both realtime.txt and wflexp.json

http://www.komokaweather.com/wifi-console/wflexp.json
http://www.komokaweather.com/wifi-console/realtime.txt

I am on v1.23

Enjoy,
Paul

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #547 on: September 20, 2018, 06:49:45 PM »
Quote
The only file I get is the wflexp.json.
Wojtek may have an answer. 


From that one setting in FTP Archive Records Export setup I get both realtime.txt and wflexp.json

http://www.komokaweather.com/wifi-console/wflexp.json
http://www.komokaweather.com/wifi-console/realtime.txt

I am on v1.23

Enjoy,
Paul
I'm on v1.23, but I only get .json..... (guess I'm too picky, but it would be great to be able to see that the whole system is updated ok to match 1.23.)
Edit: Reinstalled 1.23, still no file.
« Last Edit: September 20, 2018, 08:45:24 PM by Lars_Olsson »

Offline WiFiLogger

  • Forecaster
  • *****
  • Posts: 733
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #548 on: September 21, 2018, 08:06:31 AM »
I'm on v1.23, but I only get .json..... (guess I'm too picky, but it would be great to be able to see that the whole system is updated ok to match 1.23.)
Edit: Reinstalled 1.23, still no file.

Probably your FTP server is using some other commands. Please open http://IP_WFL/debug wait for interaval time or click on TEST text in FTP windows.
You will have information like below. Could you send it to me on priv or share with us here?
Command connected
220 ProFTPD 1.3.5a Server (Debian) [185.135.88.161]
331 Password required for username
230 User username logged in
215 UNIX Type: L8
200 Type set to I
227 Entering Passive Mode (185,135,88,161,152,121).
FTPPasv: Data port: 39033
FTPPasv: Data connected
150 Opening BINARY mode data connection for /public_html/station/wflexp.json
FTP Writing
226 Transfer complete
Data disconnected
227 Entering Passive Mode (185,135,88,161,170,131).
FTPPasv: Data port: 43651
FTPPasv: Data connected
150 Opening BINARY mode data connection for /public_html/station/realtime.txt
FTP Writing
226 Transfer complete
Data disconnected
200 Type set to A
221 Goodbye.
Command disconnected

Offline Lars_Olsson

  • Member
  • *
  • Posts: 47
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #549 on: September 21, 2018, 10:22:23 AM »
I'm on v1.23, but I only get .json..... (guess I'm too picky, but it would be great to be able to see that the whole system is updated ok to match 1.23.)
Edit: Reinstalled 1.23, still no file.

Probably your FTP server is using some other commands. Please open http://IP_WFL/debug wait for interaval time or click on TEST text in FTP windows.
You will have information like below. Could you send it to me on priv or share with us here?
Command connected
220 ProFTPD 1.3.5a Server (Debian) [185.135.88.161]
331 Password required for username
230 User username logged in
215 UNIX Type: L8
200 Type set to I
227 Entering Passive Mode (185,135,88,161,152,121).
FTPPasv: Data port: 39033
FTPPasv: Data connected
150 Opening BINARY mode data connection for /public_html/station/wflexp.json
FTP Writing
226 Transfer complete
Data disconnected
227 Entering Passive Mode (185,135,88,161,170,131).
FTPPasv: Data port: 43651
FTPPasv: Data connected
150 Opening BINARY mode data connection for /public_html/station/realtime.txt
FTP Writing
226 Transfer complete
Data disconnected
200 Type set to A
221 Goodbye.
Command disconnected
Started Debug on Firefox and got

new TCP Client
new TCP Client
new TCP Client

Then tried the Test-button and got

Command connection failed
ARCHIWUM EXPORT. Int: 5
02:00:27 1-1-1970
02:00:27 1-1-1970
DavisGetLPSFull(): CRC BAD

and then

Renew RTD
DavisGetLPSFull(): CRC BAD
Custom FTP 3
Command connection failed
new TCP Client
Renew RTD
Renew RTD


During this time the wflexp.json-file stopped updating.

Closed debug on Firefox and opened it on my phone and got

NTP Time: 16:14:00 21-09-2018
Weather station time: 16:14:01 21-09-2018
DavisCheckTime(): Weather Station time OK
Undergroud 5
Renew RTD
DavisGetLPSFull(): CRC BAD
Client response time: 400
success
Custom FTP 3
Command connected
220 Welcome to "site" FTP.
331 User "user" OK. Password required
230 OK. Current restricted directory is /
215 UNIX Type: L8
200 TYPE is now 8-bit binary
227 Entering Passive Mode (46,30,211,100,229,117).
FTPPasv: Data port: 58741
FTPPasv: Data connected
150 Accepted data connection
FTP Writing
226-File successfully transferred
Data disconnected

wlexp.json updated.


Edit.
I think I've seen somewhere in the thread about all the parameters in the file, but can't find it. Someone who can help me? (Meaning of them.)
« Last Edit: September 21, 2018, 12:56:06 PM by Lars_Olsson »