Author Topic: WiFiLogger - Connect your Davis console directly to the Internet via WiFi  (Read 18100 times)

0 Members and 1 Guest are viewing this topic.

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #525 on: September 13, 2018, 05:28:53 PM »
Ok...  Right now I have it going to my raspberry pi that is a ntp gps pps time server.   

I think problem is when NTP is renew, it's updating UTC time, then is recalculated to console local time.
Spikes are data shifted in time. UTC +/- Time zone +/- summer time.
When you disable NTP in WFL, then it will stop updating internal WFL timers UTC and Local.
Please check it. That will help to solve this problem.

Offline redlynr

  • Member
  • *
  • Posts: 45
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #526 on: September 13, 2018, 05:30:47 PM »
so Iím not sure I understand - currently thereís no way to update WU (for year to date, etc. charting) if say 5 inches of rain was missed due to internet outage?

Now WFL is transmitting current data when it can.
WU takes two rain values: day rain and hour rain. If WFL will be lack of the Internet for 24h WU will miss that rain.

Ok - so as long as I have power, and the outage is less than a day, there will be no loss of rainfall?  Thatís nice!

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #527 on: September 13, 2018, 05:40:05 PM »
so Iím not sure I understand - currently thereís no way to update WU (for year to date, etc. charting) if say 5 inches of rain was missed due to internet outage?

Now WFL is transmitting current data when it can.
WU takes two rain values: day rain and hour rain. If WFL will be lack of the Internet for 24h WU will miss that rain.

Ok - so as long as I have power, and the outage is less than a day, there will be no loss of rainfall?  Thatís nice!

Console is always gives rain as daily or 24h. It is cleared at midnight or at 9 am. So when there was a rain at 10 am this value will be send with every transmission to WU. It will change when it will rise, or be cleared.
Arch. records have rain in intervals. We will check what is better for WU.
- daily rain
- rain from arch. rec. + catch up

Offline blacklistedcard

  • Member
  • *
  • Posts: 38
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #528 on: September 15, 2018, 03:50:33 PM »
Ok...  Right now I have it going to my raspberry pi that is a ntp gps pps time server.   

I think problem is when NTP is renew, it's updating UTC time, then is recalculated to console local time.
Spikes are data shifted in time. UTC +/- Time zone +/- summer time.
When you disable NTP in WFL, then it will stop updating internal WFL timers UTC and Local.
Please check it. That will help to solve this problem.

Confirming that setting the value of zero fixes the problem....


Offline ocala

  • Forecaster
  • *****
  • Posts: 3437
    • http://www.wunderground.com/weatherstation/WXDailyHistory.asp?ID=KFLBELLE2
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #529 on: September 16, 2018, 08:36:30 AM »
so Iím not sure I understand - currently thereís no way to update WU (for year to date, etc. charting) if say 5 inches of rain was missed due to internet outage?

Now WFL is transmitting current data when it can.
WU takes two rain values: day rain and hour rain. If WFL will be lack of the Internet for 24h WU will miss that rain.

Ok - so as long as I have power, and the outage is less than a day, there will be no loss of rainfall?  Thatís nice!
Let me also add that with battery back power it will still keep logging data. I'm not sure of the days but the shorter archive interval you have the quicker the data builds up. Example a 1 minute interval will fill up the memory faster then say a 10 minute interval. The exact days are somewhere in this thread somewhere.

Offline Lars_Olsson

  • Member
  • *
  • Posts: 26
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #530 on: September 18, 2018, 09:43:24 AM »
Is the .bin supposed to be used as well? My setup menu doesn't allow me to pick a file for Firmware Update anymore.

(Hint for exportfile. Don't use the same separator inside bracket-values as separator used to separate different items.)

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #531 on: September 18, 2018, 09:49:23 AM »
I have found problem in 1.22. I am repairing it. Please do not update with 1.22.

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #532 on: September 18, 2018, 10:19:23 AM »
(Hint for exportfile. Don't use the same separator inside bracket-values as separator used to separate different items.)

I am not sure if I am understanding.
Could you paste example?

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 4044
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #533 on: September 18, 2018, 10:20:56 AM »
Too late for me, just finished the update 10 min ago :(
Should I go back or just wait it out?

Enjoy,Paul

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #534 on: September 18, 2018, 10:27:14 AM »
Too late for me, just finished the update 10 min ago :(
Should I go back or just wait it out?
Enjoy,Paul

It cause reboot while NTP update. I didn't see it, because I have long interval in my unit.
With version 1.22 every time when NTP will want to renew time WFL will be rebooted.
Quit big issue. I have repaired that. In 10 min I will publish it.

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #535 on: September 18, 2018, 10:37:01 AM »
Newest firmware: http://wifilogger.net/files/WiFiLogger123.zip
BIN file has to be updated. +
From html folder files: "ftpexport.html", "setup.html", "export.html" has to uploaded.
New things:
- MQTT real time data (wflexp.json) export
- weathercloud.net
- FTP address can be IP number now.

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 4044
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #536 on: September 18, 2018, 10:50:16 AM »
Thanks Wojtek,
I am now on v.1.23
Should I be watching for something in particular?

I had NTP Server: pool.ntp.org
Update: 15 min

Enjoy,
Paul

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #537 on: September 18, 2018, 11:08:16 AM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 4044
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #538 on: September 18, 2018, 12:04:34 PM »
Thanks Wojtek,
I have extended the NTP server time, as I think the 15 min was the default or from past setting that I hadn't paid any further attention to.
I have enabled Weathercloud.

I've tried to understand MQTT and see how or for what I could use that, but not there yet  :oops:

Enjoy,
Paul


Offline johnd

  • Forecaster
  • *****
  • Posts: 2779
    • www.weatherstations.co.uk
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #539 on: September 18, 2018, 02:02:56 PM »
I've tried to understand MQTT and see how or for what I could use that, but not there yet  :oops:

You can point the finger at me for suggesting that (and pestering Wojtek for it).  :twisted:

But why I'm interested in it is that it should make possible very frequent updates to web page dashboards without loading the sending device (ie WFL in this case) with significant FTP uploads every few seconds. It would be nice to think that WFL could manage one MQQT upload per loop but not sure whether this is feasible or not. The overall idea is that you do a light-weight simple upload to an MQTT broker (eg effectively a server somewhere in the cloud) which can then act as a relay post for the data, potentially passing the data to multiple clients and maybe different dashboards too. So, much of the load for data distribution is taken off the sending device.

There's a bit more of a hurdle in understanding MQTT because you have to understand the role of the broker, but once you've grasped that then the overall concept is straightforward. You do also need a subscription plan to use a broker, but there are free low-usage plans available (eg at cloudmqtt.com and having a higher throughput plan doesn't cost megabucks.

Of course, having the MQTT source is only half of the picture - you also need some MQTT-using dashboard code to consume it. But hopefully having at least one device available in WFL that can generate an MQTT feed will now stimulate some software writers to fill in the other parts of the equation.

Prodata Weather Systems
Prodata's dedicated Davis EnviroMonitor website
UK Davis Premier Dealer - All Davis stations, accessories and spares
Littleport, Ely, Cambs UK

Offline blacklistedcard

  • Member
  • *
  • Posts: 38
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #540 on: September 18, 2018, 10:42:11 PM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)

With version 1.23 can I re-enable the NTP?

Offline PaulMy

  • Forecaster
  • *****
  • Posts: 4044
    • KomokaWeather
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #541 on: September 18, 2018, 10:57:32 PM »
ALL the uploads ;) seem to be working fine with the new version, but MQTT will have to wait for me.  And thanks johnd for the MQTT information, I will not lose any sleep over it.

Enjoy,
Paul

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #542 on: September 19, 2018, 03:01:07 AM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)
With version 1.23 can I re-enable the NTP?

Yes I have make changes in NTP function. Now WU should work ok. Please let me know if there are still problems.

Offline blacklistedcard

  • Member
  • *
  • Posts: 38
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #543 on: September 19, 2018, 12:35:30 PM »
Should I be watching for something in particular?
Enjoy,
Paul
No Paul. If you are interested in MQTT this option is now avaible. Also I have add weathercloud.net service.
+ now you can use IP address for NTP/FTP, before was only server name allowed.
I hope that this problem with time shifting on WU was repaired now.

15 min for NTP is to much in my opinion. I don't think that time synchronization is needed so frequently.
It always checking time on startup. When NTP time was reach from the Internet then console time is checked and updated.
1 min can be diffrence between console and NTP. With bigger difference console time will be updated.
In my opionion this mechanizm can be triggered 2-4 times a day (interval 360-720)
With version 1.23 can I re-enable the NTP?

Yes I have make changes in NTP function. Now WU should work ok. Please let me know if there are still problems.

I re-enabled it and so far it looks good....

Offline StudleyWeather

  • Member
  • *
  • Posts: 1
Re: WiFiLogger - Connect your Davis console directly to the Internet via WiFi
« Reply #544 on: September 19, 2018, 02:41:14 PM »
I have to say this is a great piece of kit - installed fairly easily into my Vantage Vue and the real time data webpage is great.  I do however have one query - I am running WeatherLink 6.0.4 locally on a PC and have set up the TCP/IP connections for it (TCP port is 22222)  It successfully found the WiFi Logger but now when I try to poll data into WeatherLink I often get Communication error - it has polled successfulyl 3 times today but in the comms log it keeps saying communication error.  I am running v1.20 firmware on the logger.  On the logger I have the web interface enabled and TCP port set to Auto Pause with pause duration of 10 mins.  On Weatherlink it is set to poll hourly at 2 mins past the hour.  Is there anything else I need to set up/look at to enable both the real time page and WeatherLink to populate.  Thanks

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
I have to say this is a great piece of kit - installed fairly easily into my Vantage Vue and the real time data webpage is great.  I do however have one query - I am running WeatherLink 6.0.4 locally on a PC and have set up the TCP/IP connections for it (TCP port is 22222)  It successfully found the WiFi Logger but now when I try to poll data into WeatherLink I often get Communication error - it has polled successfulyl 3 times today but in the comms log it keeps saying communication error.  I am running v1.20 firmware on the logger.  On the logger I have the web interface enabled and TCP port set to Auto Pause with pause duration of 10 mins.  On Weatherlink it is set to poll hourly at 2 mins past the hour.  Is there anything else I need to set up/look at to enable both the real time page and WeatherLink to populate.  Thanks
Pause duration works with manual pause. On first page Real Time Data you trigger manual pause by clicking time. This is old function probably I will erase that function.
Auto pause is triggered automatically when connection on port 22222 is appeared. Its stay paused as long as connection is on, but there is a safety feature here. When there is no data transfer from PC to WFL then after 60s WFL is closing connection. PC software if it's working can renew this connection.

I will check once more WL 6.0.4. Testing is time consuming. I will let you know.

Offline Lars_Olsson

  • Member
  • *
  • Posts: 26
(Hint for exportfile. Don't use the same separator inside bracket-values as separator used to separate different items.)

I am not sure if I am understanding.
Could you paste example?

To separate items, when extracting file to insert to database, one usually uses a separator between items that isn't used anywhere else in the file. This to simplify extraction of each item. If the items can't be separated from each other, much more work has to be done to be able to easily extract.

Example, (from wflexp.json): "stnmod":17,"ver":4.33 .............. "xt":["---","---","---","---","---","---","---"], ..... "sunrt":"6:42"
In this case it isn't possible to separate items from each other with the separator "," due to that the comma is used inside the xt-items array. It isn't possible to use the separator ":" due to it is used inside the value for item sunrt, i.e 6:42.

To use separators for easy access to make operations on the items:
Example:     "stnmod":17;"ver":4.33 .............. "xt":["---","---","---","---","---","---","---"];"sunrt":"6.42"
In this example it's easy to get each item by splitting them on the separator ";" also after that it is easy to split the name from the value by splitting on the ":". All values has to have a meaning, thereby giving the 6.42 the same meaning as 6:42 but makes the handling of the file sooo much easier.




Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Offline WiFiLogger

  • Senior Contributor
  • ****
  • Posts: 171
Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Try to upload once more "system.html" and "spark-md5.js". Please use system.html from newest firmware.
Which web browser do you use?
This button will be enabled when proper information from WFL will comes, but it's 2s delay. Do you know how to use debug console in Chrome? CTRL + SHIFT + J -> then Network tab

File is in JSON format. You don't need to parse it. It's very common file. You need to find library which will parce it for you.
http://php.net/manual/en/function.json-decode.php
https://www.w3schools.com/js/js_json_parse.asp

Offline Lars_Olsson

  • Member
  • *
  • Posts: 26
Also. When I access the System menu, the button on updating html-files is clickable, but the button to make system-upgrade isn't. it's disabled.

Cheers.
PS. Thanks for a fine product.

Try to upload once more "system.html" and "spark-md5.js". Please use system.html from newest firmware.
Which web browser do you use?
This button will be enabled when proper information from WFL will comes, but it's 2s delay. Do you know how to use debug console in
Chrome? CTRL + SHIFT + J -> then Network tab

File is in JSON format. You don't need to parse it. It's very common file. You need to find library which will parce it for you.
http://php.net/manual/en/function.json-decode.php
https://www.w3schools.com/js/js_json_parse.asp
No change. Using Firefox. Tried to do it on my phone and it worked.

OK, thanks. Used to parse data to suit my needs....

Offline Lars_Olsson

  • Member
  • *
  • Posts: 26
The Realtime.txt, where is this file supposed to end up?

 

anything