Author Topic: FOSHKplugin - brief introduction  (Read 25458 times)

0 Members and 1 Guest are viewing this topic.

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #300 on: November 09, 2022, 04:18:38 PM »
Hi!

In the next FOSHKplugin version, there will be a new function that might appeal to some people.

I at least use FOSHKplugin here for dozens of forwarding destinations. It works without having to worry about it. It simply runs.
The problem is more with the remote stations that are supposed to accept and process the weather data. If they are (temporarily) unavailable, it is difficult to notice.
FOSHKplugin then simply tries to send the valid data at the next interval.
Any errors are logged in the log file, but who looks at it every day without a reason?
I certainly don't!

In order to still receive information relatively promptly if a forward destination is not reachable for a longer period of time, as of v0.10 there is the possibility to be informed via push notification (Pushover).
After a configurable number of successive unsuccessful attempts, a push message is sent:

Code: [Select]
FOSHKplugin
From FOSHKplugin on 2022-11-09 at 19:10
forward FWD-17 (EW - PWT @Leo Tablet) was unsuccessful 10 times since FOSHKplugin start (last result: TIMEOUT)
* 192.168.15.236 for ws@192.168.15.215; 11-09-2022 19:10:32 *

or

Code: [Select]
FOSHKplugin
From FOSHKplugin on 2022-11-09 at 17:21
forward FWD-53 (INFLUX2MET - InfluxDB 2.x-Forward of metric values to LoxBerry) was unsuccessful 10 times since 09.11.2022 17:15:35 (last result: : Failed to establish a new connection: [Errno 113] No route to host)
* 192.168.15.237 for ws@192.168.15.221; 09.11.2022 17:21:13 *

Included are the forward designation (Forward-nn), the forward type (FWD_TYPE) and any forward comment (FWD_CMT) - together with the error message and the time of the last successful transmission.

This message is sent exactly ONCE after the number of failed attempts specified in the config file under Warning\FWD_WARNINT. The default value is 10.
With each successful attempt, the internal counter is reset - so you really only get a warning if the specified number of failed attempts occur after each other.
After a push message, another message is only generated for this forward if at least one forward was successful before and thus the internal counter was reset.
This function can be deactivated via Warning\FWD_WARNING = False in the config file.
As usual, however, this can also be deactivated at runtime via http (http://ipaddress:port/FOSHKplugin/fwdwarning=disable) or via UDP (Plugin.fwdwarning=disable) or activated via enable.

In addition, there is also a simple html page at http://ipaddress:port/FOSHKplugin/fwdstat that shows the status of all activated forwards. There you can see at a glance whether and since when a forward has been hanging and, if so, with what error message.

The dimensioning of FWD_WARNINT is not yet completely clear. The default is currently 10 - i.e. 10 unsuccessful attempts.
With a transmission interval to the target of 30 seconds, a warning would come after 5 minutes (10*30 seconds) - if the target could not be reached continuously.
If, however, the interval is 1 minute, the warning will only come after 10 minutes of uninterrupted inaccessibility.
And if the transmission interval is 10 minutes, a warning is only given after 10*10 minutes = 100 minutes.

Perhaps I should not specify FWD_WARNINT globally for all forwards, but allow a setting option per forward?
Or would it perhaps make more sense to specify a fixed time (such as 15 or 30 or 60 minutes) as a global setting?
What do you think? What do you think makes sense? What would you wish for?

And would you use this feature at all?
I don't know how many users actually use the push function of FOSHKplugin - I like the notification that a sensor's battery needs to be changed or a thunderstorm is coming or that a sensor no longer reports (e.g. because the cap of a WH51 has come off again).

Regards, Oliver

Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #301 on: November 09, 2022, 04:23:20 PM »
I definitely like this! I'd be interested in beta testing the new release when available.
Ecowitt Wittboy| FOSHKplugin on Pi 4


Offline thunder.ky

  • Weathering all the things!
  • Member
  • *
  • Posts: 44
  • GW2000B, WS90, WeeWX, FOSHKplugin
    • thunder.ky
Re: FOSHKplugin - brief introduction
« Reply #302 on: November 09, 2022, 04:34:08 PM »
Hi Oliver,

And would you use this feature at all?

I'm definitely interested in using this feature. Looking forward to the new version! Just able to give a quick response now, but will give it some thought and offer some additional feedback to your questions.

Also, happy to beta test if you're looking for it.
Ecowitt GW2000B v2.2.2
Ecowitt WS90 v1.2.0
WeeWX v4.9.1
FOSHKplugin v0.10
https://wx.thunder.ky
  |  
  |  
  |  
  |  
  |  

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #303 on: November 11, 2022, 01:28:41 AM »
Hi!

I have revised the forward warning a bit more this night.
There is now a global variable Warning\FWD_WARNINT that specifies the number of failed forwards before a warning is issued via push notification.
In addition, an individual value can be configured in each forward with FWD_WARNINT. This setting overrides the global default for that specific forward and allows the warning to be disabled individually by setting FWD_WARNING = 0.
This way, the warning behaviour can optionally be configured separately for each forward.

On the forward status page, different font colours show at a glance the status of the forwards: black: everything is ok, orange: the last forward was unsuccessful and red: the number of successive unsuccessful attempts exceeds the defined warning limit:

 [ You are not allowed to view attachments ]

In addition, the formatting of date and time in all outputs - such as CSV, LOG or even in the push notification - can now be adapted via another optional setting Config\DT_FORMAT. This can then be used to determine whether this should be output as 10.11.2022 17:34:20 or 10/11/2022 17:34:20 or 11/10/2022 05:34:20 pm or whatever.
Here, the internal possibilities of the time library in Python are simply used - for dd.mm.yyyy hh:mm:ss (as usual for Germany), one simply enters in the config file under [Config]:
Code: [Select]
DT_FORMAT = %d.%m.%Y %H:%M:%S
However, this is also the default that is used with the existing configuration. For UK/US users, these variables must be swapped somewhat.

I'm still thinking about what should be a sensible value for the number of failed forward attempts as a default before a warning is given. Weather services such as WOW sometimes have temporary problems accepting data. I don't want to trigger warnings too often.

Regards, Oliver

Offline thunder.ky

  • Weathering all the things!
  • Member
  • *
  • Posts: 44
  • GW2000B, WS90, WeeWX, FOSHKplugin
    • thunder.ky
Re: FOSHKplugin - brief introduction
« Reply #304 on: November 11, 2022, 09:12:04 AM »
Perhaps I should not specify FWD_WARNINT globally for all forwards, but allow a setting option per forward?
Or would it perhaps make more sense to specify a fixed time (such as 15 or 30 or 60 minutes) as a global setting?
What do you think? What do you think makes sense? What would you wish for?

Our Network Management System does something similar with alerts occurring after a certain number of missed polls. There's the option to send repeating alerts, but I find it a bit too much to have repeating alerts. So it won't send a follow up alert unless things are reset by whatever caused the trigger being restored.

Having both a global setting, and an individual setting that overrides the global seems like it would be good.
Ecowitt GW2000B v2.2.2
Ecowitt WS90 v1.2.0
WeeWX v4.9.1
FOSHKplugin v0.10
https://wx.thunder.ky
  |  
  |  
  |  
  |  
  |  

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #305 on: November 13, 2022, 04:00:10 AM »
Hi!

In the new FOSHKplugin version, it will be possible to temporarily store missed forwards to InfluxDB/InfluxDB2 and transfer them when the forward destination is available again.
Technically, all data that could not be delivered to the target (for example, because the server is not available) will be stored locally as files.
If the server is then accessible again at some point, all this data is subsequently sent.

The interesting question is for which other forward types this procedure would still make sense. Because only a few weather services allow the subsequent upload of outdated data.
Do you know which weather services do support this? Does anyone already have their own experience?

Is there anyone here who would like to test the extended InfluxDB/InfluxDB2 support as a pre-beta? Please PM me. Thanks!

Regards, Oliver

Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #306 on: December 10, 2022, 04:04:02 PM »
Would you happen to have an example of forwarding Ecowitt protocol to Home Assistant?
Ecowitt Wittboy| FOSHKplugin on Pi 4


Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #307 on: December 10, 2022, 04:16:10 PM »
Hi!

HA is not directly supported by FOSHKplugin.
Either you use a MQTT forward or an EW forward and let the HA integration do the rest.

Oliver

Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #308 on: December 11, 2022, 10:40:48 AM »
Hi!

HA is not directly supported by FOSHKplugin.
Either you use a MQTT forward or an EW forward and let the HA integration do the rest.

Oliver

I should have been more clear. Here is the EW forward that I'm trying to setup, and it's not working. Do you see anything wrong with it?

[Forward-4]
FWD-ENABLE = True
FWD-TYPE = EW
FWD_CMT = Home Assistant
FWD_URL = http://192.168.0.80:8123/api/webhook/xxxxxxxxxxxxxxxxxxxxxxxxx
Ecowitt Wittboy| FOSHKplugin on Pi 4


Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #309 on: December 11, 2022, 10:51:55 AM »
Hi!

You should use the underscore in FWD_TYPE and FWD_ENABLE not FWD-TYPE etc.

Oliver

Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #310 on: December 11, 2022, 11:33:40 AM »
Hi!

You should use the underscore in FWD_TYPE and FQD_ENABLE not FWD-TYPE etc.

Oliver

Thank you! I knew that this had to be a ME problem! Rather embarrassing since the correct syntax was above and below the mistakes!
Ecowitt Wittboy| FOSHKplugin on Pi 4


Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #311 on: January 13, 2023, 02:12:24 AM »
Hi!

Another teaser ...

For all those who use several weather stations locally, the next version of FOSHKplugin offers a new feature for web-based searches.
With the URL http://ipaddress:port/FOSHKplugin/scanWS, all consoles accessible in the network segment are searched for via usual UDP broadcast and displayed as an overview page:

 [ You are not allowed to view attachments ]

A click on the displayed ipaddress then leads directly to the WebUI of the console in question in a new browser tab - provided this console has a WebUI.

If I am unsure whether all consoles in the network are accessible or whether one has hung up, I use this option. Or if I need the MAC address for any API attempts.

Perhaps this is also interesting for other users.

If you would like to participate in the internal beta test to be able to use all the new functions a little earlier, please contact me by PM.
Most of the changes and enhancements are already described in the changelog (Version History). If there are any questions about the changelog: please ask! So I can see the need for further documentation.

This version has already been running here for months - the internal beta test is only to ensure that there are no major errors - for example during installation or update - and serves to maintain the documentation. This is because questions usually arise that I can then also answer immediately as documentation.
So I would be very happy to have a few testers.

Oliver

Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #312 on: January 13, 2023, 09:00:19 AM »
I'll be happy to test!
Ecowitt Wittboy| FOSHKplugin on Pi 4


Offline KC5JIM

  • Senior Contributor
  • ****
  • Posts: 221
    • KC5JIM Weather
Re: FOSHKplugin - brief introduction
« Reply #313 on: January 13, 2023, 03:45:20 PM »
I've got .10 installed and it's working perfectly.
Ecowitt Wittboy| FOSHKplugin on Pi 4


Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #314 on: January 13, 2023, 06:16:00 PM »
Hi!

Thank you very much for the feedback.
Would 2 or 3 more users like to test FOSHKplugin v0.10? Please get in touch. Thank you!

Oliver

Online broadstairs

  • Forecaster
  • *****
  • Posts: 651
Re: FOSHKplugin - brief introduction
« Reply #315 on: January 21, 2023, 07:22:26 AM »
Oliver I'd be more than happy to test for you. I have just installed it on my openSUSE system and got it working. Just two comments on an openSUSE system sudo is not configured by default and it's normal to run with su, also openSUSE  does not use apt but 'zypper install' to install things. Fortunately it runs OK using su and all the required s/w was already installed.

Stuart
Ecowitt GW1003 with ultrasonic wind gauge, lightning sensor and PM2.5 sensor with Personal Weather Tablet as a console.

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #316 on: January 21, 2023, 10:45:53 AM »
Hi!

Quote
I'd be more than happy to test for you.
Thanks for wanting to test FOSHKplugin!
I'm a bit dependent on the users - if there are no or hardly any volunteers, the release drags on forever because I have to test it all by myself.
Actually, the public beta test was supposed to start at Christmas - but I didn't have enough feedback from other users.

For the installation, I could query whether apt-get exists and if not, I could then install the required packages with zypper.
I could proceed analogously with sudo: if sudo exists, it is used, if not, it is not used.
This should at least allow the installation to go through without any problems.
I'll prepare this tonight and send you the link to the current pre-beta.

Oliver

Online broadstairs

  • Forecaster
  • *****
  • Posts: 651
Re: FOSHKplugin - brief introduction
« Reply #317 on: January 21, 2023, 11:16:27 AM »
Thanks Oliver. The issue with sudo is that it exists as a command but is not set up to work as default. Not sure if you can tell easily if it is setup.

Stuart
Ecowitt GW1003 with ultrasonic wind gauge, lightning sensor and PM2.5 sensor with Personal Weather Tablet as a console.

Online Rover1822

  • Forecaster
  • *****
  • Posts: 1564
    • Mini Wind and Solar Data project
Re: FOSHKplugin - brief introduction
« Reply #318 on: January 21, 2023, 11:18:01 AM »
I'm waiting for the JAVA port. Don't hit me Oliver :)
Ambient:
  WS-2000
  PM 2.5(2)
  WH31B(2)
  WH40E
  WH31P
EcoWitt:
  GW1100
  GW1000(4)
  WH31(2)
  WH57
  WH51(12),
  WH40
  WH5360B
  WN34S
  WittBoy WS90 + GW2000
  WS90 (other one) + GW1100
Personal Sites: Weather Cam

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #319 on: January 29, 2023, 04:25:30 AM »
Hi!

By now, the installation of FOSHKplugin v0.10 should also work on systems other than Debian-based. I have now successfully tested this with OpenSuse (thanks to @broadstairs) and Fedora.
Additionally, the package manager of Synology is supported - but I don't have a test possibility.
Is there anyone who would like to try FOSHKplugin on a Synology NAS? Please PM!
Otherwise, I am still looking for a user who would like to try out the new InfluxDB v2 support. It works here. But as usual, the devil is in the details - so I would like another test opportunity.

[offtopic]
Quote
I'm waiting for the JAVA port.
Incidentally, I already warned about Java as a "saviour" when it was introduced. A programming language for every device, harhar.
This extremely lame programming language, dependent on a commercial and proprietary big (!) runtime environment, is responsible for a multitude of "joys" in the IT world today (actually always have been).
Now that the corresponding support has also been removed from the last browser, I have to resort to an even more difficult crutch to operate numerous IP KVMs.
In my view, Java is not even a language that should be taught at the universities of this world. It would have been better to stick to Pascal or teach Python.
Java is dead (hopefully!).
[/offtopic]

Oliver

Offline thunder.ky

  • Weathering all the things!
  • Member
  • *
  • Posts: 44
  • GW2000B, WS90, WeeWX, FOSHKplugin
    • thunder.ky
Re: FOSHKplugin - brief introduction
« Reply #320 on: January 31, 2023, 12:36:33 PM »
Hi Oliver,

So far my testing of v0.10 has gone well. I've just recently enabled Pushover notifications and they have been working flawlessly.

Speaking of Pushover, is it possible to alert on additional parameters? I'd be interested in alerts such as configurable temperature, rain rate & rain amount.

Thanks,
Arthur
Ecowitt GW2000B v2.2.2
Ecowitt WS90 v1.2.0
WeeWX v4.9.1
FOSHKplugin v0.10
https://wx.thunder.ky
  |  
  |  
  |  
  |  
  |  

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #321 on: January 31, 2023, 06:35:29 PM »
Hi!

Thank you for the feedback. I don't hear any problems from the other testers either. So the public beta test can start soon.
However, I still have to take care of the installation on the Synology NAS (I have received offers for testing - thank you - I'll get back to you).

Quote
is it possible to alert on additional parameters?
So far, this is not yet possible.
This feature is actually already on my internal wish list (things that could still be done). So far, I have simply been missing the trigger - my own need.
In terms of content, however, this can quickly become very complex - because one may not want to receive a message based on a singular value, but on chained events. And you probably want to be able to define the text to be sent yourself.
I'm still tinkering with a definition language.

I'm also still thinking about how to prevent messages that are too frequent.
Let's say you want to receive a push message when the temperature drops below 0C. But then you might receive this push message again and again - with every incoming data record (with temperature below 0C).
So I have to buffer the status "already reported". And cancel the status again (temperature > 0C) to be able to warn again later.
For each individual definition.

So I have it on my desk. But I can't promise that it will come in v0.10 ...

Oliver

Offline thunder.ky

  • Weathering all the things!
  • Member
  • *
  • Posts: 44
  • GW2000B, WS90, WeeWX, FOSHKplugin
    • thunder.ky
Re: FOSHKplugin - brief introduction
« Reply #322 on: January 31, 2023, 06:52:15 PM »
So far, this is not yet possible.
This feature is actually already on my internal wish list (things that could still be done). So far, I have simply been missing the trigger - my own need.
In terms of content, however, this can quickly become very complex - because one may not want to receive a message based on a singular value, but on chained events. And you probably want to be able to define the text to be sent yourself.
I'm still tinkering with a definition language.

Thanks Oliver. Totally get how alerts can get out of hand. Wouldn't want it to become to "noisy". Happy to know that it's being thought of.
Ecowitt GW2000B v2.2.2
Ecowitt WS90 v1.2.0
WeeWX v4.9.1
FOSHKplugin v0.10
https://wx.thunder.ky
  |  
  |  
  |  
  |  
  |  

Online olicat

  • Forecaster
  • *****
  • Posts: 1179
  • GWxx00, HPx5x1C, WN1900C, WN1980C & FOSHKplugin
    • FOSHKplugin
Re: FOSHKplugin - brief introduction
« Reply #323 on: February 07, 2023, 06:20:46 PM »
Hi!

Quote
I'd be interested in alerts such as configurable temperature, rain rate & rain amount.
Do you have a few examples under which conditions you would like to receive a push notification?
Is this more complex than temperature < x or rainrate > y or  uvi > 10?

Anybody else needs this feature?

Oliver

Offline thunder.ky

  • Weathering all the things!
  • Member
  • *
  • Posts: 44
  • GW2000B, WS90, WeeWX, FOSHKplugin
    • thunder.ky
Re: FOSHKplugin - brief introduction
« Reply #324 on: February 08, 2023, 09:44:38 AM »
Hi Oli,

Quote
I'd be interested in alerts such as configurable temperature, rain rate & rain amount.
Do you have a few examples under which conditions you would like to receive a push notification?
Is this more complex than temperature < x or rainrate > y or  uvi > 10?

What's I've done for now is set up e-mails from Ecowitt to Pushover.

Some of the conditions that I have set in the Ecowitt alerts are:
temp > 90F
rainrate > 0.25in/hr
lightning < 10miles
indoor temp > 85F

They are currently have an interval of 1 hour. However it could potentially get noisy if one of these conditions remains after the hour has elapsed.

I like the way our Network Management System works. When an alert is triggered it does not repeat unless it first has been reset by the alert condition not meeting the threshold. There is an option for "escalation" which then allows you to send an alert every X time periods.
Ecowitt GW2000B v2.2.2
Ecowitt WS90 v1.2.0
WeeWX v4.9.1
FOSHKplugin v0.10
https://wx.thunder.ky
  |  
  |  
  |  
  |  
  |