Author Topic: Software development forum  (Read 12191 times)

0 Members and 1 Guest are viewing this topic.

Offline iisfaq

  • Christchurch Weather Live
  • Senior Contributor
  • ****
  • Posts: 216
    • Christchurch Weather Live
Software development forum
« on: June 10, 2011, 10:57:24 PM »
I would like to see a forum for software development around weather. Not so much for Web Site displaying of data but more writing your own code for weather stations.

Like commuicating with a station over a serial port, the Davis LOOP format, calculating % chance of it snowing, THW, DewPoint, HeatIndex and other "code" related weather data, current weather conditions from the data we have available and heaps more.

I am writing my own Weather Software for my Davis Vantage Pro II at the moment and it would be nice to have a forum where people who are interested in code and weather related algorithms could communicate.

I guess that most people are not in this league but everyone could gain from this in the end.

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: Software development forum
« Reply #1 on: June 11, 2011, 12:29:31 AM »
Sounds like a good idea to me.

Software Development Forum is now available http://www.wxforum.net/index.php?board=104.0

Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline iisfaq

  • Christchurch Weather Live
  • Senior Contributor
  • ****
  • Posts: 216
    • Christchurch Weather Live
Re: Software development forum
« Reply #2 on: June 11, 2011, 01:04:43 AM »
Sounds like a good idea to me.

Software Development Forum is now available http://www.wxforum.net/index.php?board=104.0

Best regards,
Ken

Great news - hopefully it will be a popular forum....

cheers

chris

Offline engamor

  • Member
  • *
  • Posts: 1
Hello! Hints for interfacing wanted
« Reply #3 on: June 23, 2011, 04:15:06 AM »
Dear friends, we are a company operating in the design and manufacture of meteorological instruments. So far we have made several separate instruments, but now we should design a complete datalogger and weather station. So we are also developing our software. The first atsk is to understand what is rewuired by the customers. We would welcome any suggestion. Thanks to all who want to establish a contact and discuss the requirements.

Offline DFELD336

  • Member
  • *
  • Posts: 2
Re: Software development forum
« Reply #4 on: June 25, 2011, 08:53:45 AM »
   I would like a software package that would drop my data right into Microsoft Acess 2007. I use access to keep my weather data in after I take the *.csv file from VWS, and place it an Excel Spreadsheet then into access. This would cut my work considerably.

Offline hswaters

  • Senior Member
  • **
  • Posts: 52
  • hswaters
Re: Software development forum
« Reply #5 on: July 12, 2011, 08:49:05 PM »
I would like to see a forum for software development around weather. Not so much for Web Site displaying of data but more writing your own code for weather stations.

Like commuicating with a station over a serial port, the Davis LOOP format, calculating % chance of it snowing, THW, DewPoint, HeatIndex and other "code" related weather data, current weather conditions from the data we have available and heaps more.

I am writing my own Weather Software for my Davis Vantage Pro II at the moment and it would be nice to have a forum where people who are interested in code and weather related algorithms could communicate.

I guess that most people are not in this league but everyone could gain from this in the end.


I could volunteer to work out a couple of subroutines for you, depending on the language, I am best with VB.

Offline iisfaq

  • Christchurch Weather Live
  • Senior Contributor
  • ****
  • Posts: 216
    • Christchurch Weather Live
Re: Software development forum
« Reply #6 on: July 13, 2011, 07:51:36 AM »
I could volunteer to work out a couple of subroutines for you, depending on the language, I am best with VB.

Hi that could be great.

Do you want to PM me off this forum post so we can discuss options?

cheers

chris

Offline adnadeau

  • Member
  • *
  • Posts: 10
Re: Software development forum
« Reply #7 on: July 17, 2011, 09:05:29 AM »
Hi Chris,

I don't know if you've seen this thread:   http://www.wxforum.net/index.php?topic=10721.0  about serial communications with the Vue and Pro II.  The Xbee setup works really well.  I'm not interested in collecting data and I have no Davis software, so I wrote a script in Autoit to display some of the data from the Pro II console in a GUI on my laptop in real time.  It queries the console with the Loop command and breaks out some of the current readings.  Autoit is free and is a language similar to DOS Basic, but has lots more functions and can work with Windows functions.  Autoit scripts can also be compiled into .exe files.

Cheers,
Alan

Offline hswaters

  • Senior Member
  • **
  • Posts: 52
  • hswaters
Re: Software development forum
« Reply #8 on: July 17, 2011, 09:19:06 AM »
Hi Chris,

I don't know if you've seen this thread:   http://www.wxforum.net/index.php?topic=10721.0  about serial communications with the Vue and Pro II.  The Xbee setup works really well.  I'm not interested in collecting data and I have no Davis software, so I wrote a script in Autoit to display some of the data from the Pro II console in a GUI on my laptop in real time.  It queries the console with the Loop command and breaks out some of the current readings.  Autoit is free and is a language similar to DOS Basic, but has lots more functions and can work with Windows functions.  Autoit scripts can also be compiled into .exe files.

Cheers,
Alan

One thing that many people may not know is that you can get the personal version of VB for free from Microsoft.  Just log on and download it.  At this time I have VB 2008.  I think that VB 2010 has been released as well.

Offline Ray Proudfoot

  • Forecaster
  • *****
  • Posts: 446
    • Cheadle Hulme Weather
Re: Software development forum
« Reply #9 on: August 11, 2011, 02:21:56 PM »
Being a keen user of Microsoft Access I have designed a database that imports the contents of the Dayfile.txt generated by Cumulus.

Once this data is in a table the world is your oyster. Attached is a screenshot of a form which contains the results of 111 queries and 13 VBA subroutines.

Dayfile.txt contains one record created just after midnight containing the previous day's stats. Each time the database is opened it deletes the contents of the table holding the Dayfile.txt and imports it again. It then runs the 13 VBA subroutines which write the information seen in the Longest Runs part of the form to another table.

Finally, the form you see below uses the DLookup command to retrieve the information from the relevant query. An example for the maximum temperature in the last 7 days is =DLookUp("[Last7MaxTemp]","qryLast7_MaxTemp")

I started small and have developed what you see now.

« Last Edit: August 11, 2011, 02:28:47 PM by Ray Proudfoot »

Offline hswaters

  • Senior Member
  • **
  • Posts: 52
  • hswaters
Re: Software development forum
« Reply #10 on: August 11, 2011, 02:58:45 PM »
Being a keen user of Microsoft Access I have designed a database that imports the contents of the Dayfile.txt generated by Cumulus.

Once this data is in a table the world is your oyster. Attached is a screenshot of a form which contains the results of 111 queries and 13 VBA subroutines.

Dayfile.txt contains one record created just after midnight containing the previous day's stats. Each time the database is opened it deletes the contents of the table holding the Dayfile.txt and imports it again. It then runs the 13 VBA subroutines which write the information seen in the Longest Runs part of the form to another table.

Finally, the form you see below uses the DLookup command to retrieve the information from the relevant query. An example for the maximum temperature in the last 7 days is =DLookUp("[Last7MaxTemp]","qryLast7_MaxTemp")

I started small and have developed what you see now.



That is spectacular.  I used to work with a fellow who was an Access guru.  Unfortunately the most of my co-workers were chemical engineers and despite my insistence that Access was the best tool for data bases they never used anything but Excel.  I think they even tried to send email with it.

Offline Cynjut

  • Senior Member
  • **
  • Posts: 59
    • Bellevue, NE local weather
Re: Software development forum
« Reply #11 on: August 11, 2011, 03:12:44 PM »
I'm relatively new here, but I've been a professional programmer for FAR longer than I've been doing weather systems.

I've marveled at the way that people tightly couple their systems - VWS, for example, insists on doing the data logging and storage, and creating displays, and managing websites all at the same time.  Maybe I'm just old-fashioned, but there must be a simpler way to design this stuff.  The CarterLake stuff, seems to insist on

For example - a weather station front-end that generates a data stream (with or without tags, XML or not, I'm not sure what it should look like yet) and sends it out to a data sink, or better yet, a series of data sinks.

Maybe something like XMPP?  Each sender would log into the XMPP server and communicate with listeners that are also logged into the service.  This way, you can have the data captured on one server, stored on another, and displayed on a third.

The only reason I'm bringing it up is the problem I'm having getting to where I need to be with my system.  I need to build a server that can listen to up to four different weather stations and collect data from each.  The data needs to be normalized and stored in a database, and then excerpts of that data will be used to create graphics for various needs.  We're also thinking "other uses" for the weather data - things that are simple enough, but because of the proprietary nature of the stations we're using, becomes unnecessarily difficult.

One example is a simple "X10" interface.  Right now, the program we're developing to open and close the greenhouse windows has to interpret data from a JPG file on the webserver.  If we had a "data network" interface, we could simply have the window manager listen in on the conversation and when the conditions warrant, change the settings of the windows.

By splitting the data reporting up from the data use, it simplifies a lot of what we're trying to do.  I also noticed that a lot of people are using Visual Basic for their programs, which implies MS-Access as the database of choice.  Anyone that's used MS-Access knows that it's a sub-optimal solution for anything more complicated than toys.  My adding this data interface to the scheme, the choice of database becomes system independent.  The data storage subsystem could be written to use Access, SQL Server, MySQL, or Postgresql.  Since the interface front end is the same for all systems (XMPP again), the backends become independent, allowing for layers of data security.

Just thinking out loud....

Offline Ray Proudfoot

  • Forecaster
  • *****
  • Posts: 446
    • Cheadle Hulme Weather
Re: Software development forum
« Reply #12 on: August 11, 2011, 03:53:41 PM »
That is spectacular.  I used to work with a fellow who was an Access guru.  Unfortunately the most of my co-workers were chemical engineers and despite my insistence that Access was the best tool for data bases they never used anything but Excel.  I think they even tried to send email with it.

Thanks. I was self-taught from a £20 book back in the mid-90s. It took me ages to understand referential integrity! But once the penny dropped it came totally naturally. My employer has been using my Access skills for the past 12-13 years so they must think I know what I'm doing.  :-)

The beauty of Access, Excel, Outlook and Word is the common programming language they use making it easy to import and export data and even send an email from any of them.

A lot of people denigrate Access but it all depends on how much data you need to hold and how well the structure is designed. My Dayfile table contains just 824 records but even if it were to run for 10 years it would only contain 3650. A tiny number. SQL databases can hold far more data of course. If you stored a record every minute then over 10 years you would end up with 60*24*365*10 = 5,265,000. Probably too big for Access but then again, who is storing records every minute?

At work we often find people using Excel to store records rather than a database probably because they don't know how to design one. Excel is perfect for complex calculations. We often joke that people probably want a spreadbase!  :lol:

Offline SLOweather

  • Global Moderator
  • Forecaster
  • *****
  • Posts: 3456
    • Weatherelement Moline IL
Re: Software development forum
« Reply #13 on: August 11, 2011, 03:54:37 PM »
We're working on some of these issues with WeatherElement, like multiple stations reporting to one server, data graphing, and even the value-added services like remote control. Dunno if I'd use X-10 though. That's kinda the "toy" status of Access. ;)

The "tightly coupled" problem that you mention is a factor of the climate in which those programs were developed, and perhaps the lack of foresight to allow for future expandability. This is the same problem with the weather stations themselves. No one knew at the time, for instance, that so many people would want to put their weather data on the internet.

You can see that also in the continuous threads about stations and the lack of specialized sensors (pool sensors, for example).

There is a crossover that hasn't happened yet, and the current state of the art is kludgy at best. I could do what you are talking about with a Davis station, Homeseer, a dedicated computer, and an X-10 interface. Better would be all that and some Adicon hardware in place of the X-10. But, it's still a kludge, and made out of weather station equipment, a Home Automation (HA) program, and some HA hardware.

Y'know, I saw a post here the other day that linked to a Davis ModBus interface that Ambient sells. That's getting where you might need it to be, getting the weather data into an industrial platform.

Offline hswaters

  • Senior Member
  • **
  • Posts: 52
  • hswaters
Re: Software development forum
« Reply #14 on: August 11, 2011, 03:55:54 PM »
I sent you a personal message, not related to Access or Excel.

Offline iisfaq

  • Christchurch Weather Live
  • Senior Contributor
  • ****
  • Posts: 216
    • Christchurch Weather Live
Re: Software development forum
« Reply #15 on: August 11, 2011, 04:14:35 PM »
The system I am developing is based on a seperation of data collection and presentation.

Currently the system works for Davis Vantage Pro II as that is all I have to test with.

But it is moduler and uses interfaces to dynamically load a WeatherStation DLL and call methods on that.

The system runs as a Windows Service which polls the station for data on an interval and stores it in a SQL Server database. It uses about 0.2% cpu and runs in about 10MB +/- of ram. Probably more as windows always lies about ram usage.

There is no UI for the windows service as that is a different part of the system. Web UI/Windows UI to come later.

The Windows Service also hosts a WCF web service so you can talk to the Windows Service and get data from the station via a web page (running on the same or differnet server) or a windows app or what ever. So although I only store data once per minute in the SQL database some of the parameters of the station are updated every 3 seconds and can be read via the service.

It was all starting to work well but my serial data logger died and so has development until I get the station converted to NZ format (I bought it from the US - different wireless frequencies) and Davis would not support me in the US and the agent in NZ would not support me with an illegal station!

So it has been sent away and I guess I will get it back in a couple of weeks - $400 NZD

And hopefully they will then repair/replace the faulty data logger under warranty.

There is a lot of parameters that the station software must generate itself from data available. Hence the development of this forum for asking how to do things.

Chris

Offline nolversteeg

  • Member
  • *
  • Posts: 38
    • meteoeersel
Re: Software development forum
« Reply #16 on: September 04, 2011, 04:08:45 PM »
Hello Weatherstation software Thinkers,

I read this topic just today, and it compares with he problem I'm have,
Hardware in use, Davis VP2+ and a Envoy 8x ,at first the Envoy8x give the problem because it's a data logger with no time structure for export of data to the outside ,I have solved it by using a keystroke generator ,to download the logger data to a MySql data base. works great. But for the Davis VP2 console i need the same I have made a MySql for the Davis VP2 with the data as HTM text, so i can export this file by hand, works also great . Now i want to make a php script to do this export, but i can't find the file because its a Say's "$sql = "LOAD DATA INFILE \'C:\\\\xampp\\\\tmp\\\\phpF8C1.tmp\' INTO TABLE `davisdata` FIELDS TERMINATED BY \';\' ENCLOSED BY \'\"\' ESCAPED BY \'\\\\\' LINES TERMINATED BY \'\\r\\n\'# 1 rij(en) bijgewerkt.\n"
     . "";"  .
I hope you understand my question, if not try me.
greetings Nol Versteeg

Offline dean.martin

  • Senior Member
  • **
  • Posts: 77
Re: Software development forum
« Reply #17 on: July 07, 2016, 10:35:11 PM »
Being a keen user of Microsoft Access I have designed a database that imports the contents of the Dayfile.txt generated by Cumulus.

Once this data is in a table the world is your oyster. Attached is a screenshot of a form which contains the results of 111 queries and 13 VBA subroutines.

Dayfile.txt contains one record created just after midnight containing the previous day's stats. Each time the database is opened it deletes the contents of the table holding the Dayfile.txt and imports it again. It then runs the 13 VBA subroutines which write the information seen in the Longest Runs part of the form to another table.

Finally, the form you see below uses the DLookup command to retrieve the information from the relevant query. An example for the maximum temperature in the last 7 days is =DLookUp("[Last7MaxTemp]","qryLast7_MaxTemp")

I started small and have developed what you see now.

Amazing! How did you do this? This is exactly what I need but I am planning to import the download.txt from weatherlink instead of that Dayfile.txt from Cumulus. I hope you can share with me your file if you dont mind. Thanks!

 

anything