Topic: Forwarding and publishing of weather data

[Julius]

Been experimenting with the FTP output/forwarding options, in order to get the data of 3 different devices (or 2 in my case, still waiting for the MeteoRain IoT LoRa unit) into one display via WeeWX. This ends up being harder than expected with the Barani devices, since the FTP output lacks a lot of options. We can't bind 3 device outputs into 1 ftp upload file, let alone properly pre-format the way the output units look. The way allmeteo allows setting this up now is not very useful. It's lacking in how it uploads the data. It either sends a file every upload (and overwrites the existing one), or it adds to an existing file. It misses precisely those options I hoped would be there, and so within seconds I would have to script it all on a server with each new ftp upload, causing even more delay before it can be accurately merged and viewed. I tried doing this server-side, it's one huge conversion mess, where each value needs scripting and binding with timestamps.

Similar issues exist with forwarding to sites like WU; It cannot easily bind MeteoHelix, MeteoWind and MeteoRain output into one ID, if WU even allows receiving them separately. allmeteo can provide parts, but WU expects miles/hour output, which MeteoWind does not provide, to name one. So, again, you first have to convert it from an FTP inflow, then upload it separately to WU, causing delays and time-mismatches.
This is just WU. It would be better if barani could, for example, work in full cohesion with weewx or openweather formatting, and stack or concatenate data somewhere before doing the upload(s).
Anyone that has parsed all outputs into one dashboard already? If so, how did you go about it?

It's disappointing to find out that the weather.allmeteo.com visualizations are entirely locked behind auth, and lack theming. I'd much rather embed parts of it publicly for some other website. For now, the only viable option seems to be using raw output from The Things Network for whatever display view allows for that as a source. I don't understand why they've chosen to re-invent the wheel here, limiting their output this way. Not one api that understands all devices and allows for embedding or viewing on a self-hosted or *public* site? And why should each device visually reside in separate 'stations'. As long as they all are located within meters apart, the dashboard should show it all on 1 page, at the very least.

[Julius]

Can someone please tell us where to get the embed code for the widgets used here? http://bmcb.club/index.php?page=widget.php
Why is this all so hard to find for allmeteo data?

[Tln7559]

Julius,

The widgets used by BMCB are their own production at their own server.
.
Reading your previous message, IMHO it boils down to:
- if you have multiple datastreams and less 'end-applications', in some way you must merge the datastreams before you feed data to the 'end-applications'.
- even more important is that organisations often have a limitiation on calls/day, and/or they limit the intervaltime between calls
[at least for the 'free' subscriptions. For paid services different rules apply]

Data-merging needs an 'intermediary' script of some kind.
In my own system configuration I apply 'Domoticz & Auxiliaries' for that function:
- script types are lua/dzVents or Python (dependent on best availability&features vs required functions, and due to 'matter of taste')
- data from various sources is received 'at own speed & time, in own layout' and feeds devices or virtual devices in Domoticz
- result is that sensor-data is 'harmonized' at those devices
- from the collection of (virtual) devices a new data collection is compiled (in appropriate format & units as required by the 'end-application') and the related uploadfile is built, and transferred in the appropriate layout & timing.
.
In that way I realised working solutions for uploads to WU, MetOffice/WOW, PWSW and AWEKAS, as well as for my own website.

[Julius]

The widgets used by BMCB are their own production at their own server.

Yes, I can see that, everybody sees that. It's not their own production, since it pulls data from barani devices' output. The point is: ALL barani users would like to be able to show these widgets. They look exactly like the widgets barani provides on its website, but more elaborate, even showing the allmeteo logo. This is really weird, comes across as very strange for all (new) owners of barani devices. They will all wonder where to get the code to run widgets that way, only to find it's nowhere to be found. BMCB apparently uses some kind of vendor lock-in, if you want to show your data, you need to use their server(s) and software. This is not what the meteorological community is looking for.

[Tln7559]

Julius,

IMHO this is more or less the characteristic of a 'closed' network:
for the Barani-systems you need to go through the SigFox-netork or through a LoraWAN-network:
usually the apllication at the 'exit-point' then determines how you get the data.
They are rather clear for the description of this exit at data-level.
Widgets is another aspect.

[Julius]

My point is; People are no longer buying weather stations and sensor hardware when they can't quickly and easily view and/or show its data output online. Of course, you can write entire python constructs, use numerous micro-servers to bind all your sensors into one overview for a website, but still, with the amount of different sources out there, this gets to be a very tedious task. You want it to work, without having to build your own API or ftp storage site etc. I think way too few brands realize this.

[Mattk]

Well depends on how much you want to pay for what are no doubt propriety services that someone has spent countless hours writing entire python scripts, binding multiple servers, yes this is a very tedious task and probably not something anybody is going to hand over for nothing. Not sure what is really being expected but if it's a free lunch then not going to happen.

[johnd]

Well depends on how much you want to pay for what are no doubt propriety services that someone has spent countless hours writing entire python scripts, binding multiple servers, yes this is a very tedious task and probably not something anybody is going to hand over for nothing. Not sure what is really being expected but if it's a free lunch then not going to happen.

There's also the issue that users will never agree on exactly which presentational features they need and how they should best be presented. In some ways it's a good solution to have a feed of the basic data available in some well-documented format and allow the user to commission the final step of designing the presentation according to their own preferences. This is obviously one of the approaches that Davis are providing with the v2 API of weatherlink.com. (Yes, they also have the presentations available on the wl.com platform but I'm sure they know well that whatever they do is not going to please everyone.)

I guess the ideal solution is for there to be enough paying customers worldwide for the final presentation layer that it might pay software houses to provide 2 or 3 different designs to suit different sets of user requirements. but I can't see that happening.

[Julius]

There's also the issue that users will never agree on exactly which presentational features they need and how they should best be presented. In some ways it's a good solution to have a feed of the basic data available in some well-documented format and allow the user to commission the final step of designing the presentation according to their own preferences.

Well, this is precisely what - for example - the Weather Station plugin by Pierre Lanoy for wordpress tries to achieve, and what WeeWX and the MeteoBridge Pro with weather34 come very close to achieving, giving the user full control of that, within the available parameters to use. I've literally been messing with about every package out there, relating to sensor-data presentation, but those 3 I just mentioned have it down to the tee, if only someone could have those 3 join forces and perhaps even standardize what they're willing to work with, especially where the input value syntaxes are concerned, so that most big brands tailor to that standard. Those vendor lock-in features are the death of usability, it's so frustrating to have to encounter that time and again.
In the network-/server- and computer/datacenter monitoring you can clearly see things slowly culminate towards open standards, which then are used by packages like netdata and grafana, and in the end everybody more or less wants the same flexibility in presentation, which is why both their user bases have exploded the last 10 years.