Author Topic: Feeding MeteoTemplate from weewx?  (Read 5922 times)

0 Members and 1 Guest are viewing this topic.

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Feeding MeteoTemplate from weewx?
« on: December 20, 2016, 07:57:25 AM »
What is the preferred/official method to feed MeteoTemplate from weewx?  A quick search through Google reveals that there are several options that others are using, the most are based on transferring CSV files.  Someone in the weewx Google group shared a PHP script the copies between weewx's database and MT's database, but that sort of implied they're running on the same system and MySQL instance.  (Yes, I know that MySQL can be accessed remotely, but my web host's firewall rules will not let me do that from my weewx server at home.)

I am currently feeding MT from MeteoBridge, and have been spoiled by the officially-supported simple URL posting procedure.  I do not see a similar URL report option in weewx, which is why I'm asking what the preferred/official method is and what the pitfalls are of the various CSV file formats.....
WU Gold Stars for everyone! :lol:

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #1 on: December 20, 2016, 12:39:16 PM »
What is the preferred/official method to feed MeteoTemplate from weewx?

i cannot tell you an official method, but these are all feasible:

option 1: simulate cumulus.  use the weewx-crt extension to emit a realtime.txt file every archive interval.  use ftp or rsync (built in to weewx) to transfer the realtime.txt file to meteotemplate.

option 2: simulate weather-display.  use the weewx-wd extension to emit clientraw.txt.  use ftp or rsync to transfer the clientraw.txt to meteotemplate.

option 3: write a restful extension to weewx that uses the meteotemplate http interface (same api that meteohub/bridge/plug uses).

option 4: use the 'custom' input to meteotemplate.  create a meteotemplate.txt.tmpl file your weewx skin, and fill it with whatever values you want to feed to meteotemplate (see the meteotemplate 'custom text file' instructions).  use ftp or rsync to transfer the file to meteotemplate.

option 5: use a database copy such as the php example you cited.

imho, option 3 should be the most robust.  in fact, maybe i'll write that restful uploader now...

m

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #2 on: December 20, 2016, 12:58:50 PM »
You can use the Meteobridge script, which is basically an HTTP/GET request, just make sure you use the same units as are the standardized ones for MB

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #3 on: December 20, 2016, 01:28:06 PM »
You can use the Meteobridge script, which is basically an HTTP/GET request, just make sure you use the same units as are the standardized ones for MB

i do not have meteotemplate.  i do not have a meteohub/plug/bridge.  i have no intention of buying/installing either.

i would like to write a weewx plugin that will push data into meteotemplate by invoking an http GET request using some meteotemplate URL.  (this is the same mechanism used by WU, wetter.com, windguru, etc)

in order to do that, i need to know the following:

what is that URL?

what are the units of each field in that URL?

what is the format for each field in that URL?  (e.g., are the times in UTC or local time?  what format are the time strings?  how many decimal places for numeric values?)

what is NULL for each field?

if there is no value for a field, should i send NULL or should i omit the field?

assuming all of that works, does meteotemplate aggregate those data into its database, or must one also install a meteotemplate cron job to suck in data from realtime.txt (or equivalent)?

m

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #4 on: December 20, 2016, 01:32:33 PM »
That is all technically possible to do, but not currently because of the way how this currently works in the backend. I will look into it once Im finished with what Im working on right now with regards to the template, no-one has ever asked for an API because they usually could use one of the options already available. So its not like I dont want to create it, there just was no reason for it. I think it should not be a problem and I will try most likely at the end of this year next week. Right now there is no documentation for such thing and there are a few minor things that I will also have to change slightly in some of the core files.

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Re: Feeding MeteoTemplate from weewx?
« Reply #5 on: December 20, 2016, 01:57:37 PM »
i would like to write a weewx plugin that will push data into meteotemplate by invoking an http GET request using some meteotemplate URL.  (this is the same mechanism used by WU, wetter.com, windguru, etc)

in order to do that, i need to know the following:

what is that URL?

Does this help?  This is the posting URL that I have in my MeteoBridge, with domain name and posting password munged to protect the innocent.

http://www.example.com/weather/update/meteobridge.php?t=[th0temp-avg5]&h=[th0hum-avg5]&p=0&d=[th0dew-avg5]&w=[wind0wind-avg5]&g=[wind0wind-max5]&b=[wind0dir-avg5]&rr=[rain0rate-act]&r=[rain0total-sumday]&tmax=[th0temp-max5]&tmin=[th0temp-min5]&year=[YYYY]&month=[MM]&day=[DD]&hour=[hh]&minute=[mm]&seconds=[ss]&pass=posting-password

Quote
what are the units of each field in that URL?

what is the format for each field in that URL?  (e.g., are the times in UTC or local time?  what format are the time strings?  how many decimal places for numeric values?)

what is NULL for each field?

if there is no value for a field, should i send NULL or should i omit the field?

I will defer these answers to Jáchym, although I suspect that the answers are that MeteoTemplate only needs numeric values and assumes that they are whatever the template has configured as default units.  In terms of no value for a field, I static-coded a zero for the barometer entry in the above URL since my station's sensor is trashed.  (I don't remember what the necessary MeteoBridge variable was named off the top of my head, whether it was sea pressure or not.  [thb0press] versus [thb0seapress])

Quote
assuming all of that works, does meteotemplate aggregate those data into its database, or must one also install a meteotemplate cron job to suck in data from realtime.txt (or equivalent)?

This is the beauty of the MeteoBridge input versus the other text file options -- the posting URL also adds the data into MT's database as one transaction.....
WU Gold Stars for everyone! :lol:

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Re: Feeding MeteoTemplate from weewx?
« Reply #6 on: December 20, 2016, 02:10:10 PM »
That is all technically possible to do, but not currently because of the way how this currently works in the backend. I will look into it once Im finished with what Im working on right now with regards to the template, no-one has ever asked for an API because they usually could use one of the options already available.

Unless there is something more desirable from one of the other input options, I would think that the existing ~update/meteobridge.php file would be sufficient as the incoming API:

Code: [Select]
<?php

############################################################################

# Meteotemplate
http://www.meteotemplate.com
Free website template for weather enthusiasts
Author: Jachym
#           Brno, Czech Republic
First release: 2015
#
############################################################################
#
# Database update from Meteobridge
#
A script which updates database from data provided by Meteobridge.
#
############################################################################
# Version (change log - http://meteotemplate.com/blog/?page_id=42)
#
v8.0 Pineapple 2016-07-25
#
############################################################################

// get configuration
include("../config.php");

$updateInterval 60// minimum update interval in seconds (corrensponds to your setting in meteobridge - how often do 
you run this script)

#############################################################################################
################## DO NOT EDIT BELOW ########################################################
#############################################################################################

$password $_GET['pass'];
// if user updated this file, but has not yet set the update password, use the admin one
if(isset($updatePassword)){
if($password!=$updatePassword){
echo "Incorrect upload password!";
die;
}
}
else{
if($password!=$adminPassword){
echo "Incorrect upload password!";
die;
}
}

$updateInterval $updateInterval 30// allow some time for server or MB to send the data

// get data
$t $_GET['t'];
$tC $t;
$tmax $_GET['tmax'];
$tmin $_GET['tmin'];
$h $_GET["h"];
$d $_GET["d"];
$w $_GET["w"];
$wMS $w;
$g $_GET["g"];
$b $_GET["b"];
$p $_GET["p"];
$r $_GET["r"];
$rr $_GET["rr"];
if(isset($_GET["s"])){
$s $_GET["s"];
}
else{
$s "";
}

$year $_GET["year"];
$month $_GET["month"];
$day $_GET["day"];
$hours $_GET["hour"];
$minutes $_GET["minute"];
$seconds $_GET["seconds"];

if($dataTempUnits=="F"){
$t = ($t 9/5) + 32;
$tmax = ($tmax 9/5) + 32;
$tmin = ($tmin 9/5) + 32;
$d = ($d 9/5) + 32
}
if($dataWindUnits=="kmh"){
$w $w 3.6;
$g $g 3.6;
}
if($dataWindUnits=="mph"){
$w $w 2.236936;
$g $g 2.236936;
}
if($dataWindUnits=="kt"){
$w $w 1.943844;
$g $g 1.943844;
}
if($dataPressUnits=="mmhg"){
$p $p 0.75006375541921;
}
if($dataPressUnits=="inhg"){
$p $p 0.029529987508;
}
if($dataRainUnits=="in"){
$r $r 0.0393701;
$rr $rr 0.0393701;
}

$tRange = array($limitTempMin,$limitTempMax); // temperature range
$hRange = array($limitHumidityMin,$limitHumidityMax); // humidity range
$dRange = array($limitTempMin,$limitTempMax); // dew point range
$pRange = array($limitPressureMin,$limitPressureMax); // pressure range
$wRange = array($limitWindMin,$limitWindMax); // wind speed and wind gust range
$bRange = array($limitBearingMin,$limitBearingMax); // wind bearing range
$rRange = array($limitRainMin,$limitRainMax); // daily total precipitation range
$rrRange = array($limitRainRateMin,$limitRainRateMax); // rain rate range (units of rain per hour)
$sRange = array($limitSolarMin,$limitSolarMax); // solar radiation range

// check parameters fall within specified range
if($t >= $tRange[0] && $t <= $tRange[1]){
$tValid true;
}
else{
$tValid false;
}
if($tmax >= $tRange[0] && $tmax <= $tRange[1]){
$tmaxValid true;
}
else{
$tmaxValid false;
}
if($tmin >= $tRange[0] && $tmin <= $tRange[1]){
$tminValid true;
}
else{
$tminValid false;
}
if($h >= $hRange[0] && $h <= $hRange[1]){
$hValid true;
}
else{
$hValid false;
}
if($d >= $dRange[0] && $d <= $dRange[1]){
$dValid true;
}
else{
$dValid false;
}
if($p >= $pRange[0] && $p <= $pRange[1]){
$pValid true;
}
else{
$pValid false;
}
if($w >= $wRange[0] && $w <= $wRange[1]){
$wValid true;
}
else{
$wValid false;
}
if($g >= $wRange[0] && $g <= $wRange[1]){
$gValid true;
}
else{
$gValid false;
}
if($r >= $rRange[0] && $r <= $rRange[1]){
$rValid true;
}
else{
$rValid false;
}
if($rr >= $rrRange[0] && $rr <= $rrRange[1]){
$rrValid true;
}
else{
$rrValid false;
}
if($s >= $sRange[0] && $s <= $sRange[1] && $s!=""){
$sValid true;
}
else{
$sValid false;
}
if($b >= $bRange[0] && $b <= $bRange[1]){
$bValid true;
}
else{
$bValid false;
}

// calculate apparent temperature - this formula works for C and m/s

$e = ($h/100)*6.105*pow(2.71828, ((17.27*$tC)/(237.7+$tC)));
$a round(($tC 0.33*$e-0.7*$wMS-4),1);

if($dataTempUnits=="F"){
$a = ($a 9/5) + 32;
}

if($tValid==true && $hValid==true && $wValid==true){
$aValid true;
}
else{
$aValid false;
}

// create string for data upload to MySQL
$parameters = array();
$values = array();

// check last update in database
$result mysqli_query($con,"SELECT DateTime FROM alldata ORDER BY DateTime DESC LIMIT 1");
while($row mysqli_fetch_array($result)){
$lastUpdate date(strtotime($row['DateTime']));
}
if($lastUpdate == ""){ // this is a fix if the database is completely empty
$lastUpdate 0;
}

$currentTime strtotime($year."-".$month."-".$day." ".$hours.":".$minutes.":".$seconds);
if((date($currentTime) - $lastUpdate)>($updateInterval-1)){ // last update must be at least 5 mins ago
$timeValid true;
$current date('Y-m-d H:i:s',$currentTime);
array_push($parameters,"DateTime");
array_push($values,"'".$current."'");
}
else{
$timeValid false;
}

if($tValid){
array_push($parameters,"T");
array_push($values,$t);
}
if($tmaxValid){
array_push($parameters,"Tmax");
array_push($values,$tmax);
}
if($tminValid){
array_push($parameters,"Tmin");
array_push($values,$tmin);
}
if($dValid){
array_push($parameters,"D");
array_push($values,$d);
}
if($aValid){
array_push($parameters,"A");
array_push($values,$a);
}
if($hValid){
array_push($parameters,"H");
array_push($values,$h);
}
if($pValid){
array_push($parameters,"P");
array_push($values,$p);
}
if($wValid){
array_push($parameters,"W");
array_push($values,$w);
}
if($gValid){
array_push($parameters,"G");
array_push($values,$g);
}
if($bValid){
array_push($parameters,"B");
array_push($values,$b);
}
if($rValid){
array_push($parameters,"R");
array_push($values,$r);
}
if($rrValid){
array_push($parameters,"RR");
array_push($values,$rr);
}
if($sValid){
array_push($parameters,"S");
array_push($values,$s);
}

if((count($parameters)>0) && $timeValid == true){ // upload to MySQL if at least one parameter is valid and time from last u
pdate at least 5 mins
$query "INSERT INTO alldata (".implode(',',$parameters).") values (".implode(',',$values).")";
mysqli_query($con,$query);
}

if((date($currentTime) - $lastUpdate)>$minimumAlertInterval){ // check if station is reporting and send alert if activated
if($alertActive){
// check if alert was sent already
if(file_exists("alertSent.txt")){
// alert was sent already, do nothing
}
else{
// station not reporting, alerts active and alert not yet sent -> sent alert
$subject "Weather Station Offline";
$lastReport date("Y-m-d H:i",$lastUpdate);
$message "Weather station has not reported in the last ".round($minimumAlertInterval/60,0)." minut
es. Last db record found is from "
.$lastReport.".";
mail($alertEmail,$subject,$message);
$alertFile fopen("alertSent.txt""w") or die("Unable to open file!");
$txt "Alert sent";
fwrite($alertFile$txt);
fclose($alertFile);

// now make a note in the error log that mail was sent
if(file_exists("../admin/errorLog.txt")){
$errorContent file_get_contents("../admin/errorLog.txt");
$errorContent $errorContent."\n\n".date("Ymd H:i",time())."\n".$message."\nSending email t
o"
.$alertEmail.".";
file_put_contents("../admin/errorLog.txt"$errorContent);
}
else{
$errorFile fopen("../admin/errorLog.txt""w") or die("Unable to open file!");
$errorText date("Ymd H:i",time())."\n".$message."\nSending email to ".$alertEmail.".";
fwrite($errorFile$errorText);
fclose($errorFile);
}
}
}
}
?>

Since weewx has become a popular platform in the PWS arena, I would argue that a symlink for weewx.php pointing to meteobridge.php be included in future template versions.  That way, it will appear that weewx is supported natively by MeteoTemplate, and nobody needs to know the secret. ;)

Of course, I have to throw in my customary disclaimer that I am not a programmer...and have never even looked at Python code until Sunday afternoon when I installed weewx.....
WU Gold Stars for everyone! :lol:

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #7 on: December 20, 2016, 02:40:59 PM »
As I said, I do need to make a few minor changes and I can then create some kind of an API for this. Right now it will not be that easy and there is no documentation, just give me some time :)

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #8 on: December 20, 2016, 02:44:23 PM »
That is all technically possible to do, but not currently because of the way how this currently works in the backend. I will look into it once Im finished with what Im working on right now with regards to the template, no-one has ever asked for an API because they usually could use one of the options already available.

jachym,

please have some discussion before you publish an http(s) interface for pushing data into meteotemplate.  propose an api syntax on wxforum - i guarantee you'll learn a lot.

many, many people have done these already.  some of them are really good.  some of them are complete rubbish.

as soon as you publish the interface, you have to support it, and deal with backward compatibility.  this is not something that you want to change.  nor is there any reason to iterate on an interface - we already have many years of experience and existence proofs of both good and bad.

you *could* use the WU interface; make meteotemplate accept data in WU format.  that has the advantage that you could simply point any device that 'speaks' WU directly to meteotemplate, and it will just work (e.g., the fine offset observers, the acurite gateways, etc).  the only question is whether the WU api provides all of the data you need for meteotemplate.

i'm not saying the WU api is an example of a good api - in fact, it is not.  but it is well known, supported by many devices/software, and well understood, even if the behavior of the servers that sit behind it are not

meanwhile, if you could clarify the behavior of the http/get interface that meteobridge uses, i can make weewx work with that, then transparently (to the user) transition to the new api when you figure it out.

m

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #9 on: December 20, 2016, 02:45:07 PM »
That is all technically possible to do, but not currently because of the way how this currently works in the backend. I will look into it once Im finished with what Im working on right now with regards to the template, no-one has ever asked for an API because they usually could use one of the options already available.

Unless there is something more desirable from one of the other input options, I would think that the existing ~update/meteobridge.php file would be sufficient as the incoming API:

Code: [Select]
<?php

############################################################################

# Meteotemplate
http://www.meteotemplate.com
Free website template for weather enthusiasts
Author: Jachym
#           Brno, Czech Republic
First release: 2015
#
############################################################################
#
# Database update from Meteobridge
#
A script which updates database from data provided by Meteobridge.
#
############################################################################
# Version (change log - http://meteotemplate.com/blog/?page_id=42)
#
v8.0 Pineapple 2016-07-25
#
############################################################################

// get configuration
include("../config.php");

$updateInterval 60// minimum update interval in seconds (corrensponds to your setting in meteobridge - how often do 
you run this script)

#############################################################################################
################## DO NOT EDIT BELOW ########################################################
#############################################################################################

$password $_GET['pass'];
// if user updated this file, but has not yet set the update password, use the admin one
if(isset($updatePassword)){
if($password!=$updatePassword){
echo "Incorrect upload password!";
die;
}
}
else{
if($password!=$adminPassword){
echo "Incorrect upload password!";
die;
}
}

$updateInterval $updateInterval 30// allow some time for server or MB to send the data

// get data
$t $_GET['t'];
$tC $t;
$tmax $_GET['tmax'];
$tmin $_GET['tmin'];
$h $_GET["h"];
$d $_GET["d"];
$w $_GET["w"];
$wMS $w;
$g $_GET["g"];
$b $_GET["b"];
$p $_GET["p"];
$r $_GET["r"];
$rr $_GET["rr"];
if(isset($_GET["s"])){
$s $_GET["s"];
}
else{
$s "";
}

$year $_GET["year"];
$month $_GET["month"];
$day $_GET["day"];
$hours $_GET["hour"];
$minutes $_GET["minute"];
$seconds $_GET["seconds"];

if($dataTempUnits=="F"){
$t = ($t 9/5) + 32;
$tmax = ($tmax 9/5) + 32;
$tmin = ($tmin 9/5) + 32;
$d = ($d 9/5) + 32
}
if($dataWindUnits=="kmh"){
$w $w 3.6;
$g $g 3.6;
}
if($dataWindUnits=="mph"){
$w $w 2.236936;
$g $g 2.236936;
}
if($dataWindUnits=="kt"){
$w $w 1.943844;
$g $g 1.943844;
}
if($dataPressUnits=="mmhg"){
$p $p 0.75006375541921;
}
if($dataPressUnits=="inhg"){
$p $p 0.029529987508;
}
if($dataRainUnits=="in"){
$r $r 0.0393701;
$rr $rr 0.0393701;
}

$tRange = array($limitTempMin,$limitTempMax); // temperature range
$hRange = array($limitHumidityMin,$limitHumidityMax); // humidity range
$dRange = array($limitTempMin,$limitTempMax); // dew point range
$pRange = array($limitPressureMin,$limitPressureMax); // pressure range
$wRange = array($limitWindMin,$limitWindMax); // wind speed and wind gust range
$bRange = array($limitBearingMin,$limitBearingMax); // wind bearing range
$rRange = array($limitRainMin,$limitRainMax); // daily total precipitation range
$rrRange = array($limitRainRateMin,$limitRainRateMax); // rain rate range (units of rain per hour)
$sRange = array($limitSolarMin,$limitSolarMax); // solar radiation range

// check parameters fall within specified range
if($t >= $tRange[0] && $t <= $tRange[1]){
$tValid true;
}
else{
$tValid false;
}
if($tmax >= $tRange[0] && $tmax <= $tRange[1]){
$tmaxValid true;
}
else{
$tmaxValid false;
}
if($tmin >= $tRange[0] && $tmin <= $tRange[1]){
$tminValid true;
}
else{
$tminValid false;
}
if($h >= $hRange[0] && $h <= $hRange[1]){
$hValid true;
}
else{
$hValid false;
}
if($d >= $dRange[0] && $d <= $dRange[1]){
$dValid true;
}
else{
$dValid false;
}
if($p >= $pRange[0] && $p <= $pRange[1]){
$pValid true;
}
else{
$pValid false;
}
if($w >= $wRange[0] && $w <= $wRange[1]){
$wValid true;
}
else{
$wValid false;
}
if($g >= $wRange[0] && $g <= $wRange[1]){
$gValid true;
}
else{
$gValid false;
}
if($r >= $rRange[0] && $r <= $rRange[1]){
$rValid true;
}
else{
$rValid false;
}
if($rr >= $rrRange[0] && $rr <= $rrRange[1]){
$rrValid true;
}
else{
$rrValid false;
}
if($s >= $sRange[0] && $s <= $sRange[1] && $s!=""){
$sValid true;
}
else{
$sValid false;
}
if($b >= $bRange[0] && $b <= $bRange[1]){
$bValid true;
}
else{
$bValid false;
}

// calculate apparent temperature - this formula works for C and m/s

$e = ($h/100)*6.105*pow(2.71828, ((17.27*$tC)/(237.7+$tC)));
$a round(($tC 0.33*$e-0.7*$wMS-4),1);

if($dataTempUnits=="F"){
$a = ($a 9/5) + 32;
}

if($tValid==true && $hValid==true && $wValid==true){
$aValid true;
}
else{
$aValid false;
}

// create string for data upload to MySQL
$parameters = array();
$values = array();

// check last update in database
$result mysqli_query($con,"SELECT DateTime FROM alldata ORDER BY DateTime DESC LIMIT 1");
while($row mysqli_fetch_array($result)){
$lastUpdate date(strtotime($row['DateTime']));
}
if($lastUpdate == ""){ // this is a fix if the database is completely empty
$lastUpdate 0;
}

$currentTime strtotime($year."-".$month."-".$day." ".$hours.":".$minutes.":".$seconds);
if((date($currentTime) - $lastUpdate)>($updateInterval-1)){ // last update must be at least 5 mins ago
$timeValid true;
$current date('Y-m-d H:i:s',$currentTime);
array_push($parameters,"DateTime");
array_push($values,"'".$current."'");
}
else{
$timeValid false;
}

if($tValid){
array_push($parameters,"T");
array_push($values,$t);
}
if($tmaxValid){
array_push($parameters,"Tmax");
array_push($values,$tmax);
}
if($tminValid){
array_push($parameters,"Tmin");
array_push($values,$tmin);
}
if($dValid){
array_push($parameters,"D");
array_push($values,$d);
}
if($aValid){
array_push($parameters,"A");
array_push($values,$a);
}
if($hValid){
array_push($parameters,"H");
array_push($values,$h);
}
if($pValid){
array_push($parameters,"P");
array_push($values,$p);
}
if($wValid){
array_push($parameters,"W");
array_push($values,$w);
}
if($gValid){
array_push($parameters,"G");
array_push($values,$g);
}
if($bValid){
array_push($parameters,"B");
array_push($values,$b);
}
if($rValid){
array_push($parameters,"R");
array_push($values,$r);
}
if($rrValid){
array_push($parameters,"RR");
array_push($values,$rr);
}
if($sValid){
array_push($parameters,"S");
array_push($values,$s);
}

if((count($parameters)>0) && $timeValid == true){ // upload to MySQL if at least one parameter is valid and time from last u
pdate at least 5 mins
$query "INSERT INTO alldata (".implode(',',$parameters).") values (".implode(',',$values).")";
mysqli_query($con,$query);
}

if((date($currentTime) - $lastUpdate)>$minimumAlertInterval){ // check if station is reporting and send alert if activated
if($alertActive){
// check if alert was sent already
if(file_exists("alertSent.txt")){
// alert was sent already, do nothing
}
else{
// station not reporting, alerts active and alert not yet sent -> sent alert
$subject "Weather Station Offline";
$lastReport date("Y-m-d H:i",$lastUpdate);
$message "Weather station has not reported in the last ".round($minimumAlertInterval/60,0)." minut
es. Last db record found is from "
.$lastReport.".";
mail($alertEmail,$subject,$message);
$alertFile fopen("alertSent.txt""w") or die("Unable to open file!");
$txt "Alert sent";
fwrite($alertFile$txt);
fclose($alertFile);

// now make a note in the error log that mail was sent
if(file_exists("../admin/errorLog.txt")){
$errorContent file_get_contents("../admin/errorLog.txt");
$errorContent $errorContent."\n\n".date("Ymd H:i",time())."\n".$message."\nSending email t
o"
.$alertEmail.".";
file_put_contents("../admin/errorLog.txt"$errorContent);
}
else{
$errorFile fopen("../admin/errorLog.txt""w") or die("Unable to open file!");
$errorText date("Ymd H:i",time())."\n".$message."\nSending email to ".$alertEmail.".";
fwrite($errorFile$errorText);
fclose($errorFile);
}
}
}
}
?>

Since weewx has become a popular platform in the PWS arena, I would argue that a symlink for weewx.php pointing to meteobridge.php be included in future template versions.  That way, it will appear that weewx is supported natively by MeteoTemplate, and nobody needs to know the secret. ;)

Of course, I have to throw in my customary disclaimer that I am not a programmer...and have never even looked at Python code until Sunday afternoon when I installed weewx.....

That MB script you posted above is not from version 10 and should only be used by those who installed the template prior to release of v10. I will make something more useful, its just that it is a bit hectic at this time of the year so I will try when I get to it, hopefully before the end of the year

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #10 on: December 20, 2016, 02:46:11 PM »
That is all technically possible to do, but not currently because of the way how this currently works in the backend. I will look into it once Im finished with what Im working on right now with regards to the template, no-one has ever asked for an API because they usually could use one of the options already available.

jachym,

please have some discussion before you publish an http(s) interface for pushing data into meteotemplate.  propose an api syntax on wxforum - i guarantee you'll learn a lot.

many, many people have done these already.  some of them are really good.  some of them are complete rubbish.

as soon as you publish the interface, you have to support it, and deal with backward compatibility.  this is not something that you want to change.  nor is there any reason to iterate on an interface - we already have many years of experience and existence proofs of both good and bad.

you *could* use the WU interface; make meteotemplate accept data in WU format.  that has the advantage that you could simply point any device that 'speaks' WU directly to meteotemplate, and it will just work (e.g., the fine offset observers, the acurite gateways, etc).  the only question is whether the WU api provides all of the data you need for meteotemplate.

i'm not saying the WU api is an example of a good api - in fact, it is not.  but it is well known, supported by many devices/software, and well understood, even if the behavior of the servers that sit behind it are not

meanwhile, if you could clarify the behavior of the http/get interface that meteobridge uses, i can make weewx work with that, then transparently (to the user) transition to the new api when you figure it out.

m

Updates via WU have been supported for quite a while already, it is built-in

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #11 on: December 20, 2016, 02:50:47 PM »
Since weewx has become a popular platform in the PWS arena, I would argue that a symlink for weewx.php pointing to meteobridge.php be included in future template versions.  That way, it will appear that weewx is supported natively by MeteoTemplate, and nobody needs to know the secret. ;)

thanks for posting the meteobridge script.  that answers my questions about how meteotemplate behaves and what units it expects.

a symlink would work, or you could do it with your apache (or nginx or lighttpd) configuration using aliases.

or you could simply call it 'update.php' and use that in the http GET.  it is, afterall, just a script that updates the data, and is not specific to any particular client.

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #12 on: December 20, 2016, 02:56:20 PM »
Please do NOT use that script, it is outdated and as I mentioned above, should only be used by those who have been using meteotemplate prior to release of version 10.

I will create a standard API with full documentation, but it just wont be today

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #13 on: December 20, 2016, 02:58:34 PM »
Updates via WU have been supported for quite a while already, it is built-in

i think you mean "meteotemplate can pull data from wu".

i mean "make meteotemplate accept data using the same api that software/hardware uses to send data to wu"

m

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Re: Feeding MeteoTemplate from weewx?
« Reply #14 on: December 20, 2016, 03:08:35 PM »
Updates via WU have been supported for quite a while already, it is built-in

I believe that he is referring to using the protocol that several stations/devices use to push data *to* WU, not pulling data from WU and into Meteotemplate.  (AFAIK, the WU push protocol is very similar to what you have already built for MeteoBridge input.)  Besides, with WU having serious reliability issues and losing data lately, is it wise to put all your eggs in one basket and feed Meteotemplate from WU?

I was unaware that anything changed with the meteobridge.php file recently.  I am still running Meteotemplate 8.0 Pineapple, due to spending much of the months of September and October in the hospital…and November medically/professionally recovering from September and October.  Looks like I will be spending the Christmas weekend updating Meteotemplate...and trying to figure out how weewx works.  (FWIW, I already have weewx capturing Acu-Rite sensor data off of the air into MySQL using a USB SDR dongle, and would like to push that data into Meteotemplate and also send it via packet radio on APRS.....)
« Last Edit: December 20, 2016, 03:11:48 PM by vreihen »
WU Gold Stars for everyone! :lol:

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #15 on: December 20, 2016, 03:17:40 PM »
There are some changes which help especially me as a developer with debugging etc. So I really do not want new users to use it.

I see your point with WU, the only problem is that WU has defined units, whereas if I made standardized api I could inlcude the units as parameters in the API so it would accept more. If you stick with just WU, rain has to be in in, wind speed in mph etc

Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #16 on: December 20, 2016, 03:37:05 PM »
I see your point with WU, the only problem is that WU has defined units, whereas if I made standardized api I could inlcude the units as parameters in the API so it would accept more. If you stick with just WU, rain has to be in in, wind speed in mph etc

this is exactly why you should start a thread on wxforum with your proposed api for pushing data into meteotemplate.

why should a server have to accept any combination of units, instead of simply specifying a single set or unit system?  based on the code i have written to interface with many, many different types of systems, that flexibility, at the data intake level, does not add value.

imho, you would benefit far more from simply accepting the wu protocol - it makes meteotemplate usable out-of-the-box for many, many more devices than if you create yet another protocol.  wu, weatherlink, and rainwise probably account for 99% of the out-of-the-box uploading protocols.  (netatmo and other obfuscated and/or encrypted protocols do not count)

not only that, your code would be easier to test and maintain.

but propose the api and see what real users have to say about it.  *before* you deploy it.

just trying to be helpful,

m

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #17 on: December 20, 2016, 03:40:21 PM »
Yes I agree with that. It would be a plugin anyway so the advantage of that is that it can be updated/changed anytime regardless of the whole template version, which I do not want to update too often. But some changes in the core files also have to be made so before I can start working on the API I will make the necessary modifications in the core files.

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Re: Feeding MeteoTemplate from weewx?
« Reply #18 on: December 20, 2016, 05:52:45 PM »
It would be a plugin anyway so the advantage of that is that it can be updated/changed anytime regardless of the whole template version...

OK, this is starting to make sense to me now!  The update folder scripts concept was getting a little long in the tooth, and using the plugin architecture to also handle data uploads/imports makes perfect sense from a supportability view.

I will be patient for the time being, so that be perfect interface can be developed and supported on both ends with proper documentation.  Honestly, I have at least another week's worth of work before I would feel comfortable sending weewx data to a "production" web server anyway...with one of my remaining tasks being how to configure an interface with an external barometer module on the Raspberry Pi I'm using to host weewx.....
WU Gold Stars for everyone! :lol:

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #19 on: December 20, 2016, 05:59:12 PM »
Yes that is why I try to make more problematic scripts as plugins, because I cannot update the template as a whole on a daily basis if necessary. I also sort of know how this will work. What I will do is a compromise - it will use the standardized API which WILL support more units, which can be passed as parameters, however - it will use by default the WU units, which means that if no units are provided, it will work with the WU units. This way it will work with the standard WU protocol and if units are different, it will be possible to set that.

No parameters for units -> use WU
Parameteres specified for units -> use those


Offline mwall

  • Contributor
  • ***
  • Posts: 126
Re: Feeding MeteoTemplate from weewx?
« Reply #20 on: December 24, 2016, 12:43:54 AM »
hey vreihen,

here is a candidate release 0.1rc1 for a weewx uploader that will inject data directly into meteotemplate using the meteobridge.php plugin.

https://github.com/matthewwall/weewx-meteotemplate

i don't have a meteotemplate instance, so this is untested; it might take a couple of iterations to shake out the bugs.

it should be trivial to update the uploader once jachym defines a standard api for meteotemplate.  (if he uses the wu protocol, you can simply use the wu uploader that comes with weewx and point it at your meteotemplate instance)

m

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #21 on: December 24, 2016, 08:30:14 AM »
I plan on work on the Api now that v11 is out,  please keep in mind what I said about the meteobridge script,  it is not even part of the template since version 9, the file was deprecated

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #22 on: December 25, 2016, 01:50:29 PM »
OK, so for the API, I was thinking about something like this:

...default API URL....php?

and then the compulsory parameters passed as GET variables:
- password (the update password specified in MT control panel) for authorization
- time -> UNIX timestamp
- T -> current outdoor temperature
- H -> current outdoor humidity
- P -> current pressure
- W -> average wind speed
- G -> wind gust
- R -> cumulative daily rain total
- RR -> current rain rate (per hour)
- B -> wind bearing (degrees)

- unitsT -> C/F
- unitsP -> inhg/hpa/mmhg
- unitsW -> kmh/ms/mph/kt
- unitsR -> mm/in

Then optional:
- S -> solar radiation

If some parameter is N/A, then it would not be included in the URL or set to nothing ("")

Timestamp, password and at least one parameter must always be included.

Dewpoint and apparent (feels like) are calculated automatically by the script using T, H, W if available

Offline vreihen

  • El Nińo chaser
  • Forecaster
  • *****
  • Posts: 880
  • K2BIG
Re: Feeding MeteoTemplate from weewx?
« Reply #23 on: December 25, 2016, 07:04:30 PM »
My only suggested change is to not call things passwords, to avoid confusion with the template's maintenance password.  Can it be referred to as auth-token or pre-shared-key on both sides?????
WU Gold Stars for everyone! :lol:

Offline Jáchym

  • Meteotemplate Developer
  • Forecaster
  • *****
  • Posts: 8572
    • Meteotemplate
Re: Feeding MeteoTemplate from weewx?
« Reply #24 on: December 25, 2016, 08:23:56 PM »
Hi,
hmmm the thing is - it is referred to as "update password" all over the template - control panel, wiki etc. (renaming it would probably cause even greater confusion). I dont really like using the word "token" - tokens are usually temporary dynamic codes that authorize you, this is static. I tried to explain thoroughly in the wiki the difference between the "admin password" and "update password". Admin password is never passed in a URL, I had to make two because devices such as Meteobridge, or in fact this too, cannot use the POST command and I did not like the main password being passed in URL... it is secured by B-crypt now :D
And.... just to make sure, since the update password is considered as something that in case someone hacked your site, is likely to know, unlike the admin, if you actually provide the admin password to the update script, it will also accept it, just that it will give you a notice that you should not be using this password. But it will work, since my assumption is, if you know the admin password, you must know the update password as well (since even if you did not, with the admin you could login to the control panel of Meteotemplate and change the update password to whatever you want if you see what I mean).

Even the WU upload protocol refers to this sort of variable as "password"

 

anything