Topic: FOSHKplugin - brief introduction

[KC5JIM]

If we don't find a solution, we will post in the Meteobridge board (in wxforum or/and in meteohub.com).

I suggest to post the solution even in the case you find a solution.

I will definitely post any solution. Asking for help publicly and then hoarding the answer is ridiculous.

[Gyvate]

I didn't mean to keep to solution secret - I only meant to say that when/if we are not successful with our assessments and tests, we will present the case to the Meteobridge fora.
If/when we find the solution, we will surely present it in the wxforum.
Now, for the next steps, @kd7eir will need to have a thunderstorm with a good number of strikes nearby ...

[Gyvate]

The culprit was not Meteobridge but a special block in Meteotemplate for depicting the lightning situation on a Meteotemplate (home)page. Meteobridge sends proper data, provided you use the proper API string with the proper selectors.

This block (lightningew) uses all three lightning values L (lightning count), LD (lightning distance) and LT ([last] lightning time stamp). In order to use it properly, you have to extend the database schema by two new columns for LD and LT (three new scripts to replace the old ones will do this job - they will also extend the MT database schema from 4 to 8 extra temperature, humidity, soil moisture and gas sensors (O₃, SO₂, NO₂, CO₂, CO)).

The way the update script of this block processes the data and also depicts a day-related history makes this saving of lightning count data happen. What is saved by the block is not the growing total count but the count per time slot (actual, 5 minutes, hour and day).
So what @kd7eir observed is just the result of how this Meteotemplate block works. No Meteobridge to blame.

By the way - the proper files for the database schema extension can be found at http://www.meteotemplate.com/forum/viewtopic.php?f=50&t=2064

[KC5JIM]

The culprit was not Meteobridge but a special block in Meteotemplate for depicting the lightning situation on a Meteotemplate (home)page. Meteobridge sends proper data, provided you use the proper API string with the proper selectors.

This block (lightningew) uses all three lightning values L (lightning count), LD (lightning distance) and LT ([last] lightning time stamp). In order to use it properly, you have to extend the database schema by two new columns for LD and LT (three new scripts to replace the old ones will do this job - they will also extend the MT database schema from 4 to 8 extra temperature, humidity, soil moisture and gas sensors (O₃, SO₂, NO₂, CO₂, CO)).

The way the update script of this block processes the data and also depicts a day-related history makes this saving of lightning count data happen. What is saved by the block is not the growing total count but the count per time slot (actual, 5 minutes, hour and day).
So what @kd7eir observed is just the result of how this Meteotemplate block works. No Meteobridge to blame.

By the way - the proper files for the database schema extension can be found at http://www.meteotemplate.com/forum/viewtopic.php?f=50&t=2064

What is the api string that you are now using in MeteoBridge for Meteotemplate?

[Gyvate]

I guess you are mainly interested in the lightning portion:
....&L=[lgt0total-dmax.0:]&LD=[lgt0dist-lastval.0:]&LT=[lgt0dist-lasttime=epoch]&.....
the rest depends largely on what and how many sensors of which type you use.

[KC5JIM]

I guess you are mainly interested in the lightning portion:
....&L=[lgt0total-dmax.0:]&LD=[lgt0dist-lastval.0:]&LT=[lgt0dist-lasttime=epoch]&.....
the rest depends largely on what and how many sensors of which type you use.

Thank you, that is exactly what I needed!

[KC5JIM]

I guess you are mainly interested in the lightning portion:
....&L=[lgt0total-dmax.0:]&LD=[lgt0dist-lastval.0:]&LT=[lgt0dist-lasttime=epoch]&.....
the rest depends largely on what and how many sensors of which type you use.

Lightning is working fine now, thank you again for all your help!

[KC5JIM]

I spoke too soon. I have FOSHK feeding one website, MB feeding another, and Ecowitt feeding Ecowitt.net. All of this comes from one GW1000, so the data is the same.

FOSHK rain fall = .48", Ecowitt rain fall = .48", MB rain fall = 0.00"

The rain rate is 2.91"/hr (seriously!) MB does show that correctly, but it still shows 0.00" for rain fall.

I learned my lesson. FOSHK is the absolute champion and FREE. MB is the absolute gutter and COSTS $76!

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker. At least now I have an extra Pi4 available for use by something that works.

[WA4OPQ]

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker. At least now I have an extra Pi4 available for use by something that works.

Put WeeWx on the Pi4, it does APRS/CWOP

[olicat]

Hi!

The only thing FOSHK does not do for me that MB does is APRS/CWOP

I have already looked at APRS as another forward target (feature) for FOSHKplugin. Sounds doable.
Are there any other interested parties?

Oliver

[Gyvate]

I spoke too soon. I have FOSHK feeding one website, MB feeding another, and Ecowitt feeding Ecowitt.net. All of this comes from one GW1000, so the data is the same.

FOSHK rain fall = .48", Ecowitt rain fall = .48", MB rain fall = 0.00"

The rain rate is 2.91"/hr (seriously!) MB does show that correctly, but it still shows 0.00" for rain fall.

I learned my lesson. FOSHK is the absolute champion and FREE. MB is the absolute gutter and COSTS $76!

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker. At least now I have an extra Pi4 available for use by something that works.

As already said in the Meteobridge Forum https://forum.meteohub.de/viewtopic.php?f=61&t=15866&sid=ccfa5e699fc5905e2088aa47fa5ec696, where you have posted the same text
it's most likely not Meteobridge not working, what's needed is the proper use of the selectors to make Meteobridge send the data one wants to send
here it should be [rain0total-daysum:--] and [rain0rate-lastval:1] which will provide the desired result(s) (for Meteotemplate that's ...&R=[rain0total-daysum:--]&RR=[rain0rate-lastval:1]...
in a http request). Sorry to say that, MB does what you told it to do  - you tell it the right thing the right way, it will do what's expected.

It's rewarding studying the MB WiKi section on selectors (https://www.meteobridge.com/wiki/index.php/Templates) - or asking more experienced users for help / solutions

[KC5JIM]

I spoke too soon. I have FOSHK feeding one website, MB feeding another, and Ecowitt feeding Ecowitt.net. All of this comes from one GW1000, so the data is the same.

FOSHK rain fall = .48", Ecowitt rain fall = .48", MB rain fall = 0.00"

The rain rate is 2.91"/hr (seriously!) MB does show that correctly, but it still shows 0.00" for rain fall.

I learned my lesson. FOSHK is the absolute champion and FREE. MB is the absolute gutter and COSTS $76!

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker. At least now I have an extra Pi4 available for use by something that works.

As already said in the Meteobridge Forum https://forum.meteohub.de/viewtopic.php?f=61&t=15866&sid=ccfa5e699fc5905e2088aa47fa5ec696, where you have posted the same text
it's most likely not Meteobridge not working, what's needed is the proper use of the selectors to make Meteobridge send the data one wants to send
here it should be [rain0total-daysum:--] and [rain0rate-lastval:1] which will provide the desired result(s) (for Meteotemplate that's ...&R=[rain0total-daysum:--]&RR=[rain0rate-lastval:1]...
in a http request). Sorry to say that, MB does what you told it to do  - you tell it the right thing the right way, it will do what's expected.

It's rewarding studying the MB WiKi section on selectors (https://www.meteobridge.com/wiki/index.php/Templates) - or asking more experienced users for help / solutions

The site that MB is feeding is it's own Weather Dashboard, so the problem is 100% with MB. Even if it were not feeding it's own weather dashboard, those values you listed are already in my string, so that would not be the cause.

[Gyvate]

"it's own" Weather dashboard is a variant of the Weather34 dashboard, live data without history compared to the original W34 .

In my case the template works with MB without issues, shows all the proper values all the time , also during rain events.

Maybe something went wrong in the archiving process ...

Did you check the database history for that respective date for which no rain data is shown in the W34 dashboard ?
The MB system log would give more info (unless you rebooted and it started anew).
There could have been a conflict between a backup and the data archiving and the database was locked, or temporarily locked for some other reason.
Would be interesting to know.

What would be helpful is if you documented the situation with screen copies (from raw live data, min/ax, system log, hourly and (if activated) minute based history and the final issue - with time stamps).

Is this a one time observation or does it happen repeatedly (documentation !!!) ?

Complex software (and MB is a complex piece of software, much more complex than the FOSHKplugin or the weather34 template which are "broker" middleware or a front-end (both a nice piece too )) is difficult to get 100% bug free. That's general knowledge.

The issue could also be with the sqlite database used by MB.

If you were with the weewx or CumulusMX communities, by the way also a very nice piece of software, both, but not bug free either, the first thing everybody asks for when you complain about something not working the way you expect is: system log, system log, system log ....: adequate documentation of the event. And that's professional issue management.

So let's do the same here. Just writing a value is such and such - or not such and such - is not helpful.
Product bashing neither.

So far, many of the issues you reported here have turned out to be user or handling issues and were related to other software and not MB issues (whereby MB is not 100% error/bug free !). They may have helped others not to hit certain obstacles - that's fine. The purpose of this community is meant to help each other. Also those who are not experts in handling with the weather hardware or software they purchased. And, it's also good to call a spade a spade - but this should be backed by documentation others can understand, verify and reconstruct the real or only pseudo bug.

I think that I can claim that I'm not biased here and do report MB bugs when I run into them - but always documented for the developer and the community to understand, verify and correct or avoid.

[lantana]

This may be relevant. I logged this issue on the Meteobridge forum on June 9:

[rain0total-daysum] error


I have a an event that is setup to run "before every full hour" and FTP a file to my hosted web server. The timestamps in the file show the minute as '59', so it is still within the hour.

The problem I have is that the [rain0total-daysum] seems to be reset to zero before the actual end of day. For example, yesterday in the 10:59pm (hour 23) file, the daily rain total was correctly shown as 12.3 mm, but the 11:59pm (hour 24) file shows the daily rain total as 0.0 mm. I would expect it to still show as 12.3 mm.

https://forum.meteohub.de/viewtopic.php?f=56&t=15772

[olicat]

Gentlemen,

would you please continue the MB-specific questions in a different thread.
Users interested in FOSHKplugin will certainly find hardly any useful information when solving a problem with MeteoBridge.

This thread should actually (primarily) be about FOSHKplugin.
Thank you!
(and sorry!)

Oliver

[CHPlay]

Hello,
 I think this is possible but I'm missing a step and hopefully someone can help me out. 

What I understand is FOSHKplugin can translate AmbientWeather to Ecowitt.  Ecowitt can then be picked up by the GW1000 driver in WeeWX. 
I have a WS-5000 with a custom output URL configured, and I can go to my FOSHKplugin/Weewx system (http://192.168.1.13:8080) and I can see the output from the FOSHKplugin. 

How do I bridge this final gap and tell FOSHKplugin to translate or do I just point the GW1000 driver to a certain URL?
I have a GW1100 on order if this won't work, but it seems like I'm only missing 1 step. 


Thanks!

[Gyvate]

Hello,
 I think this is possible but I'm missing a step and hopefully someone can help me out. 

What I understand is FOSHKplugin can translate AmbientWeather to Ecowitt.  Ecowitt can then be picked up by the GW1000 driver in WeeWX. 
I have a WS-5000 with a custom output URL configured, and I can go to my FOSHKplugin/Weewx system (http://192.168.1.13:8080) and I can see the output from the FOSHKplugin. 

How do I bridge this final gap and tell FOSHKplugin to translate or do I just point the GW1000 driver to a certain URL?
I have a GW1100 on order if this won't work, but it seems like I'm only missing 1 step. 

Thanks!

In a nutshell: it won't work and you need your GW1100 to have weewx directly communicate with it through the GW1000 API. FOSHKplugin can convert formats but as far as I am aware it cannot emulate the GW1000 API which weewx works with together with its very stable GW1000 driver.

If you are only behind the basic sensors which the WH65 outdoor array provides, I could imagine that you could make FOSHKplug make emulate the custom server and weewx can receive its data via the Interceptor driver. But the interceptor driver is limited to the basic sensors, no extra sensors. By the way, this you could have already now without FOSHKplugin using in the WS-2000 console the WU protocol and set the interceptor driver to WU-client.

So again, if it's only about the basic sensors (indoor temp/hum, pressure; outdoor temp/hum; rain; wind; solar; UV), then one can also in a WS-2000 use the WU protocol and the interceptor driver with the WU-client settings (still path, interval, IP of the weewx server need to be configured in either the WS-2000 or via WSView app). You can also use FOSHKplugin sending Ambient protocol from your console, have it converted by FOSHKplugin to Ecowitt format and then use the weewx interceptor driver with ecowitt-client (but there's no added value in this unless you want to use FOSHKplugin for posting to more recipients which need Ecowitt protocol).

You want to get extra sensors (WH31-type, CO2, PM2.5, lightning etc.), the interceptor driver doesn't process them. Then you better use a GW1000/GW1100 and the weewx GW1000 API driver.

[olicat]

Hi!

How do I bridge this final gap and tell FOSHKplugin to translate or do I just point the GW1000 driver to a certain URL?

There's no way.
FOSHKplugin can actually convert data in the Ambient Weather protocol to Ecowitt and make it available to weewx, among other things.
This way, however, the GW1000 driver cannot be used on the part of weewx, as it explicitly uses the API functions of the GW1000 and independently queries compatible devices.

Instead, you have to use the interceptor driver, which receives and processes the Ecowitt messages coming from the weather station.

In FOSHKplugin you have to create a forward that converts the incoming Ambient messages to Ecowitt and forwards them to the configured port of weewx.
The default port for the weewx interceptor driver is 8000:

Code: [Select]

[Forward-9]
FWD_TYPE = RAWEW
FWD_URL = http://ipaddressOfWeeWX-host:8000

Regards, Oliver

[olicat]

Sorry Gyvate,

I have to correct you in this regard:

You want to get extra sensors (WH31-type, CO2, PM2.5, lightning etc.), the interceptor driver doesn't process them. Then you better use a GW1000/GW1100 and the weewx GW1000 API driver.

the WH31 and WH51 sensors are supported through weewx/Interceptor already. Same for WH41/43.
With some modifications some more sensors are possible like WH57 or WH55.

I always expected the Interceptor driver to get a major update. But this is still not the case.
So I also would - if the possibility exists - rather use the GW1000 API driver.

Regards, Oliver

[Gyvate]

There are some modifications to the driver available on GitHub, but none by the original author.
Some of the mods have other unwanted side effects. Found out myself the "hard" way.

Of course, if someone is familiar with Python coding, understands the logic of the program and the structure of the data sent/received, they can modify the Interceptor driver to also process the extra sensor data. "Just" extend the sensor map and add the corresponding coding ... It's no witchcraft. But you better thoroughly understand the whole program not to create other unwanted side effects.

With the GW1000 API driver in place, there is no big need for extending the interceptor driver to the full GW1000 sensor range. And the WS-5000 user number (or WS-2000 with extra sensors) doesn't seem to big large enough for creating "pressure".

At the end of the day a 30 US$ investment for the GW1100/GW1000 is in my opinion the cleaner approach (and comes with a hardware backup for your WS-2000/HP2551 console, just in case).

[CHPlay]

This way, however, the GW1000 driver cannot be used on the part of weewx, as it explicitly uses the API functions of the GW1000 and independently queries compatible devices.

Instead, you have to use the interceptor driver, which receives and processes the Ecowitt messages coming from the weather station.

In FOSHKplugin you have to create a forward that converts the incoming Ambient messages to Ecowitt and forwards them to the configured port of weewx.
The default port for the weewx interceptor driver is 8000:

Code: [Select]

[Forward-9]
FWD_TYPE = RAWEW
FWD_URL = http://ipaddressOfWeeWX-host:8000

Regards, Oliver

Thank you Oliver.  I misunderstood how the GW1000 driver worked.  I'll give the interceptor driver a try.  Thank you for making such an awesome plugin and providing so much support for it too!

[olicat]

Hi!

Since some services only support a selection of sensors from the Fine Offset universe, I have long wanted to be able to specify more precisely which fields are used for a forward.
I now think I have found a way that makes this possible in an understandable way. Here is the link to the explanation of the remap function implemented in FOSHKplugin with the upcoming version v0.09.
Perhaps your questions or suggestions will result in improvements to the function and/or the documentation.

recipe Remapping keys/values (from v0.09)

I would be happy to receive suggestions for improvement or general comments. Or just take it as a teaser.

Regards, Oliver

[olicat]

Hi!

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker.

With next version FOSHKplugin will also support APRS/CWOP.

It was implemented yesterday - now I'm testing this.
Notes, advice and suggestions for improvement are welcome!

Have a look to the recipe Sending weather station data via APRS/CWOP (from v0.09).

Oliver

[KC5JIM]

Hi!

The only thing FOSHK does not do for me that MB does is APRS/CWOP, and that is 100% NOT a deal breaker.

With next version FOSHKplugin will also support APRS/CWOP.

It was implemented yesterday - now I'm testing this.
Notes, advice and suggestions for improvement are welcome!

Have a look to the recipe Sending weather station data via APRS/CWOP (from v0.09).

Oliver

Wonderful news!

[olicat]

Hi!

If anyone would like to test FOSHKplugin's APRS/CWOP support (or the other new features) please PM me.
I have just received the first success report regarding APRS from a radio amateur. Thanks, @wxwol!


"killing" new features of upcoming FOSHKplugin v0.09:

• Leaf moisture level for Meteotemplate, WC, Awekas, Weather365 and WSWin - instead of 0..99 it is now sent as Davis-like level 0..15 (float)
• new, improved sun hours calculation sunhours (according to https://github.com/Jterrettaz/sunduration) with dynamic, location-dependent threshold value (thank you Werner!), requires Coordinates\LAT and Coordinates\LON
• new forward type EWUDP (UDPEW) - converts incoming EW, WU and AMB messages to Ecowitt/UDP (e.g. for Personal Weather Tablet/UDP broadcast listener)
• Remap function FWD_REMAP implemented - Output keys can now be defined with values of all known internal keys
• New forward type APRS enables data to be sent to CWOP

These internal beta tests give me clear indications of any problems with the program itself, but also of gaps in the documentation.
Unfortunately, however, it cannot be completely ruled out that well-known functions may suddenly no longer work as before. The operation of beta software is therefore by no means without risk.
Even if I use this version in productive use without problems ...
I would be glad!
Thanks.

Oliver