Weather Related Organizations > CWOP Forum

Another Shakeup at CWOP

<< < (2/5) > >>

Anthony:
I hardly ever actually go to the CWOP site and check my data. I always rely on the daily error reprting e-mails that I get. Have not seen anything unusual there. Except that I have been getting a few barometric pressure errors. But when I compare with the local Mear. Readings are only of by .01 or .02.

jwyman:
Guys,
 I must have been asleep for a decade or too much Codiene in the cough syrup but whats going on here? I am missing the gist of the subject matter here about CWOp.. Whats going on? Something we should know about?  Thanks from Mr. Slowness ( I have had the flu for the last 4 days)... :(((((((((((((((


Jim

tinplate:

--- Quote from: WeatherHost on February 04, 2008, 08:19:01 AM ---These types of issues are exactly why I don't get involved in those sites.  I send to our own site and to WU.

--- End quote ---

I'd like to respond in defense of CWOP. The purpose of CWOP is not to display web pages of weather info like WU. CWOP is a cooperative program that makes weather observations from personal weather stations available to meteorologists and the systems they use. CWOP data adds to the observation density to improve computer forecasting models. NWS local stations often use CWOP data in monitoring developing weather situations. CWOP data is even important in community safety. If there were to be airborne dispersal of hazardous material, there is software that emergency responders use to predict the movement of the plume so that people can be evacuated. The ability of this software to make accurate plume models depends on information such as that provided by CWOP. This data is not just for the benefit of folks looking up the weather on the internet. CWOP data is being used for serious scientific purposes. Once in the MADIS system, it's used by numerous projects. People contributing their data through CWOP are providing a public service, as are those who make it possible for the data to get from the personal weather stations to the MADIS servers. This is a good program, and I hope it does not die.

This is where the CWOP data ends up: http://www-sdd.fsl.noaa.gov/MADIS_Overview/MADIS_Overview.html
Steve
SoftWx

SLOweather:
Here's the background (long) as posted to another forum to which I belong.

---


From: Dave Anderson KG4YZY
>   To: 'Discussion of weather data quality issues'
>   Sent: Sunday, February 03, 2008 8:36 PM
>   Subject: [wxqc] Unexpected loss of 1/2 the servers that support CWOP.
>
>
>   
>
>   Hello all,
>
>   
>
>   I am here to regretfully announce that the third.aprs.net and
> fourth.aprs.net servers that I personally own and operate will no
> longer be supporting the CWOP program.
>
>   
>
>   Allow me to give a brief history on events, so you understand my decision.
>
>   
>
>   I am a ham radio operator.  The APRS-IS network was built by hams, for hams.   
> CWOP, non ham users, are guests on our network to get to MADIS.   CWOP users
> account for 1/8 of the total station count on APRS, the other 7/8 are
> non weather ham radio users.  The traffic is disproportionate though,
> as CWOP users take up a full < of the bandwidth passing thru the servers.
>
>   
>
>   Back January of last year, the previous group of server sysops that
> had ran the servers for CWOP since day one abruptly pulled support for
> the CWOP program one Friday.  No excuse was given beyond that it was
> in the best interest of both parties.  This was a devastating blow to CWOP.
>
>   
>
>   The core group of server sysops were contacted about this, and we
> spung into action to help get CWOP taken care of.  It required extensive work to get this
> to happen, as we had only three servers at the time.    In less than 48 hours we
> managed to get a 4th server online, and had started moving users over.
>
>   
>
>   Shortly after this, I observed that the method that the stations
> were using to submit the data was horribly wrong for the APRS-IS
> network (does not follow the spec), knowing that as the load
> increased, we would have problems.  I started writing a white paper,
> which I have sent to several of you on request, that outlined
> everything a developer needed to know to write a program to use the APRS-IS network without causing undue network load.
>
>   
>
>   During the development of this white paper there were 8 if I recall
> folks that were involved with the "fine tuning" of it.  At the time,
> we wanted to set the host name for CWOP users to use as cwop.aprs.net,
> so later we could move you guys to your own dedicated network of
> servers (thinking ahead).  There were also a few other items that we wanted addressed, such as the polling time, and
> polling interval.     Dave Helms, one of the CWOP administrators refused to
> accept some of the items in the draft.   After several attempts to talk reason
> into him, I just said the heck with it, and let it sit in my temp
> directory, never finished.  No one else picked up the ropes to keep
> going, either, and everyone had the last version of the draft!
>
>   
>
>   Here's the problems. The core sysops have been begging Dave to get
> the developers to fix their software.  Dave, fearing that developers
> would pull out of the CWOP program with the changes the core was
> wanting to be made, has had very little done about this.  When most of
> you have problems with data loss, or difficulty in connecting to a
> server, it's because the policies that the core put in place were
> never implemented by most of the software developers because Dave never passed it on to them.  We can only operate a network so long as the
> software -you- use is set to work with it correctly.    So a year later, after
> suggestions have been made, most of you are still using software that
> is broken, or have incorrect settings in place to work 100% correctly with the servers.
> And we've been asking over, and over, and over for these changes to be
> implemented, as even as far back as October we started noticing a
> detrimental impact on the network, but nothing has been done.
>
>   
>
>   Jump to today, here's the problems that still exist that are causing
> server sysops problems and are causing you problems due to the fact
> that most of the CWOP programs don't follow the rules:
>
>   
>
>   The lion share of you have software that uses your PC clock for
> polling time, meaning our servers have to accept thousands of stations
> every 5 minutes then sit relatively idle for 4 more minutes.  This
> with the addition of new Christmas gift stations this year has been
> causing countless crashes, data loss, and other issues that  the
> server sysops have been working on.  Each server can only accept 50
> backlogged connections at a shot, and frequently port 14580 way exceeds that.  So it's caused instabilities in the network, as networking queue
> times go very high when this happens affecting HAMS as well as CWOP.   We've
> asked this to be fixed, and heard silence.
>
>   
>
>   Polling interval is another issue.  MADIS takes the data from the
> APRS-IS network via Findu only every 15 minutes, but we still have
> stations that were sending data in every minute.  Keep in mind, you -don't stay- connected to the
> server, you connect, drop off, and disconnect.    To Russ Chadwick's credit
> action was finally taken just here recently about these stations, and
> we've banned them at the server level, but that still doesn't prevent
> them from going to connect, and tying up a server slot every minute.
> 5 minutes is the standard that amateur radio operators have used on
> APRS for probably 10 years now.  Why that wasn't just hard coded in
> programs, I don't know, but again, this is another issue we've been asking over and over to get fixed, and it yet is.
>
>   
>
>   Round Robin DNS still doesn't work for the most of you.  Over = of
> the CWOP users use third alone.  Many of the programs simply only look
> up the IP address of the rotate.aprs.net when you first load the
> program or connect the first time, and then it -never- goes back to
> ask if the IP has changed, or check if there is another one for load
> balancing.  Again, asked over and over for this to be addresses with the developers, still hasn't been.
>
>   
>
>   I am by no means saying that every piece of CWOP software is broken!!!
> Numerous apps -do it right- as the developers of those programs have
> been in this forum, seen some of our technical posts about problems
> and corrected their code.  But the amount of developers who seem to
> care about doing it right is very few, and the largest CWOP
> applications that the bulk of the users use, simply do not follow the rules.
>
>   
>
>   So who suffers?  We all do.  As the server sysops, we have to sit,
> being sucker punched every 5 minutes right now.  You do as not all of
> your weather data makes it thru.
>
>   
>
>   APRS-IS is the internet side of a VHF radio system on 144.390mhz
> that does packet radio.  Weather is only a small fraction of APRS's
> overall functionality, so as system operators we have to be sensitive
> to this, not to allow CWOP to cause harm to the amateur radio
> operators that rely on this for emergency communications and safety of life traffic.
>
>   
>
>   The servers are ran by a 100% volunteer group of system operators
> that in the case of the core servers, are all ran in datacenters.
> Fourth alone is a 8 core Xeon box running VMWare ESX server with
> Windows 2003 running the fourth virtual machine on that box.  I spent
> $4500.00 for that server to -SUPPORT- cwop.  I also donate about 6 meg
> of tier1 quality backbone bandwidth every day to this program.  Unlike
> Findu, where Steve gets support from his users of that website, the
> server sysops, who without there would be no network to use, we get
> zero support from the user community.  That' after thousands of dollars in hardware and hundreds of hours of time.
>
>   
>
>   So how was I repaid for my generosity?Huh?
>
>   
>
>   7         Requests to make changes to the client software went unanswered,
> allowing things to spiral to where they are now.
>
>   7        Every time anything happens to the network, the server sysops are
> automatically crucified for doing something wrong.
>
>   7        I can't think of the last time anyone stopped to tell me how much
> they appreciate what I do.
>
>   
>
>   So it all boils down to this morning, when I posed the message at
> 10:24 am to Seth about the standing of the software problems.
>
>   
>
>   Shortly afterward, I go this from Dave Helms:
>
>   
>
>   
>
>   Dave,
>
>   
>
>   I, too, have absolutely no interest in revisiting the events of
> January-March
> 2007 (publicly or otherwise).  I am willing to support the notion of
> segmenting CW traffic to improve stability of APRS-IS, to include developing a plan for
> implementing the cwop.aprs.net round robin server name.   In last year's crisis
> mode, I agreed to contact virtually all CW and Ham weather station
> operators to facilitate the switch from Tier2 servers to Core servers
> included on the rotate.aprs.net RR server domain.  In the future, I
> will rely on encouraging individuals to read the CWOP News Page to
> advertise the RR server name change, and through routine client
> application upgrades rather than making individual email contacts.
>
>   
>
>   As we move forward, I request one thing... that we not make the "CW'
> (non-Ham) volunteers second class users.  Their contributions are equally valuable to NOAA
> as validated Ham Radio users who send weather reports to APRS-IS and NOAA.   If
> weather traffic must be (temporarily) throttled back to preserve
> APRS-IS stability, then all weather contributions should be equally
> reduced, Ham and non-Hams alike.
>
>   
>
>   Going forward, we need to find a way to add capacity to the APRS-IS
> to take the load of the expected growing number of users.  Hence, we
> need to develop an architecture that will allow exploitation of both
> Ham Radio APRS-IS server resources and non-Ham servers which may be
> available. These non-Ham servers and bandwidth may be donated from
> government, universities, and private sector groups.
>
>   
>
>   Regards,
>
>   
>
>   Dave
>
>   
>
>   
>
>   
>
>   
>
>   Finally after a year, Dave concedes to allowing us to start using
> CWOP.APRS.NET.  Well the problem is that it's a day late, and a dollar short.
> He further more says it'll only be a news post on a website, and for
> when folks update software.  Well if the last change to rotate is any
> indication of how well that will go, there still are 800 of you using the old servers to this DAY.
> So if he's not going to help promote it, we'll have a network that is
> in shambles here in the next 90 days due to load.  I just don't think
> Dave understands the gravity of the problem.
>
>   
>
>   Then the straw that broke the camel's back was in the second paragraph.
> Insisting that ham stations also be reduced to that of non hams.  Well first
> off, ham stations are generally sharing a very limited amount of bandwidth on
> 144.390mhz (1200bps).  From peer pressure alone, no ham will beacon weather more
> than every 5 minutes without hearing about it from other local hams.  We have a
> system in place that works well, using nothing but peer pressure.
>
>   
>
>   The fact that he's insisting hams back off, on their own network, that was
> built by  hams, really offended me, and so far most of the hams' I've shared
> this with seem to agree.  Dave and the NWS pay NOTHING for what we do on the
> network.  How dare he tell us what to do on our own network.  CWOP are GUESTS on
> our network.  Dave has in the past shown callous towards the hams, never this
> blatant, but I've seen it in other messages.  This disgusts me.  This would be
> no different than me coming into your house as a guest and telling you can only
> spend 5 hours a day watching TV.
>
>   
>
>   So I write Dave back a -very- long message, and at this point I'm very upset
> about this.  I lay blame for the problems we have right now where it needs to be
> placed, squarely on his shoulders.  He was the reason the white paper was given
> up on, as if it wasn't Dave's way, it was no good.  Yeah, I'll admit my message
> to him was long, and rather blunt.  I was direct with him, and factually telling
> him it's come to a point where change has to occur, or the network will change
> around him.
>
>   
>
>   Well, here was his response to that:
>
>   
>
>   
>
>   
>
>   
>
>   
>
>   Dave,
>
>   
>
>   In the time your spent wiring this email, you would could have instead been
> making progress on the a white paper your said you would complete last year.   
> Don't blame the client developers for something you knew that was needed over a
> year ago.
>
>   
>
>   Dave Helms
>
>   
>
>   
>
>   
>
>   
>
>   Accusing me of not finishing a document that he opposed?!!?!?!?!?!!  Excuse
> me?  Draft 3 was what would have gotten published had he not objected to the
> hostname.  The rest -is- done right now.
>
>   
>
>   Well if the NWS employee that is supposed to be our liaison to the developers
> is taking this attitude towards me, with the thousands I have spent to support
> CWOP, I hereby pull my official support from the program.
>
>   
>
>   I realize this will be a heavy hit for CWOP, as I take a full one half of the
> server capacity with me.   I am greatly sorry for that.  The last people I
> wanted to hurt were the ones I was out to help, but with Dave Helms at the
> "Helm", it's over for me.  I cannot continue to be badgered, beaten, ignored,
> then finally blamed for it to begin with over a volunteer operation.
>
>   
>
>   If any of you would like to talk about this more, I encourage you to send me a
> private e-mail at dave@aprsfl.net.  I'm sure this message will get me banned on
> this forum, so I'll not be checking back.
>
>   
>
>   I'm glad I've been able to help up till now, and I really am truly sorry for
> going out this way, but I just can't go on with this type of pervasive
> attitude.. Let alone one that is so callous towards the folks that this network
> was built for, ham radio operators.
>
>   
>
>   Regards,
>
>   Dave Anderson
>
>   KG4YZY

weatherforyou:
Wow!   :shock:

I e-mailed Russ Chadwick offering assistance from weatherforyou.com, whether it be servers or whatever.

I'm wondering more about how their system is designed.  They mentioned round-robin DNS, which I found a long time ago doesn't work well.  Although we still use round-robin to a small extent, mostly for redundancy, there are much more sophisticated load balancing systems that are free for Linux servers.  HAProxy is one we've used for quite some time.

Navigation

[0] Message Index

[#] Next page

[*] Previous page