Author Topic: NOAA Weather Radio Org Quality / Reliability Tools  (Read 8065 times)

0 Members and 1 Guest are viewing this topic.

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
NOAA Weather Radio Org Quality / Reliability Tools
« on: December 15, 2017, 01:45:19 AM »
ANNOUNCING


Available now,
for NWROrg Stream Providers ONLY...
A suite of tools to enable all providers to judge performance, reliability and quality, and receive notification of extended 'non-availability or possible quality issues' for example.

This initial QRA suite release consists of four divisions:
Email Notifications, Online Reports and Tools, and an Administration section.
The fourth division is the NWROrg Star Stream icons for consistently reliable and good quality streams.

Email Notifications, and Online Reports /Tools are available for all verified providers of NWR streams listed on NOAA All Hazards Weather Radio (NOAA Weather Radio Org)


Important Note: The ONLINE suite of reports and tools MUST be accessed with a WebAudio API compatible browser.  This includes most newer browsers, BUT EXCLUDES ALL VERSIONS OF Internet Explorer.

The following posts illustrate a quick overview of what is currently available with the QRA suite.

How Do I get it? Subscribe / Sign UP Read the form thoroughly. Non validated streams / providers may experience a delay.
Web access is unique to YOU. If you lose your link code, you'll need to resubscribe. If You provide multiple streams, you MUST submit a separate form for each stream.

Can I get More Info? Certainly! Click Here!

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
EMAIL Alerts or Notifications
« Reply #1 on: December 15, 2017, 01:45:55 AM »

Email Notifications
Available to providers by subscription


And





Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Your Home Page and Quality History
« Reply #2 on: December 15, 2017, 01:46:28 AM »

ONLINE Reports and Tools
Available ONLINE Only, by Unique Private Link Access, these will NOT work with Internet Explorer, any version. A WebAudio API compatible browser must be used... Firefox, Chrome, e.g.

When you access your private 'home' page, among some other info, you will see this:


ONLINE Quality Log HISTORY
Select any date to review your log for that date as in the example





Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Stream Availability History and Live Real Time Quality Analysis
« Reply #3 on: December 15, 2017, 01:46:54 AM »

ONLINE Availability HISTORY


ONLINE LIVE REAL-TIME Quality Analysis
(click image to view a DEMO example)




Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
And Finally... NWROrg Stream Stars!
« Reply #4 on: December 15, 2017, 01:47:29 AM »




The QRA suite automatically Awards or Demotes within a 7 date analysis.
This is from the 'administration' monitor.. not available otherwise


CREDIT:
QRA developed by Doug Chappelle, Chappelle Weather
who spent much time, energy, and hair pulling to develop this...
even though plagued by numerous gremlins, frustrations,
and the constant nagging from a certain team member, still managed to pull this off!
Doug earns 5 NWRarrrrgh stars for survival.



Special thanks to Ken True who keeps everything tied together,
who developed and releases the NWROrg PWS templates to
valid provider websites upon request. Without him and the
other NWRorg team members, this would never be possible.


Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #5 on: December 15, 2017, 10:38:58 AM »
 ;)
As providers begin to sign up, please note that the information
you place in the form must match the data we have on file...
...critically, the URL for your stream access,...
that must match exactly the data already on file.

The request process is automated, and will work smoothly if your request 'self-validates'

Also note that this process requires the use of the 'request' form, and we cannot generally correct, modify, or add other than through that tool. Use that form specifically for any communication related to the QRA suite, simply fill out appropriate information, leave any of the 'subscription' boxes un-ticked, and enter your message.

For any changes to your STREAM Info,  to Add a stream, ANY query Un-related to QRA, etc,
use the "Submit Stream" or "Contact NWROrg" buttons on the website


Also, ensure your email provider/client does not block email from us...
"Save A Life"
« Last Edit: December 15, 2017, 10:42:46 AM by NWROrg Team »

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #6 on: December 16, 2017, 10:56:43 AM »
As providers begin to utilize the QRA, you might post observations and notes here...
remember this is still the initial release, and parameters may change as we observe
overall response, and performance, especially in refinement...

There will be some 'strange anomalies' at times... for example, a stream
or reception can be good at the moment, yet be poor the next.
Daily reports information are run once daily, and may not reflect actual
parameters if checked a few minutes later...or the next day...

That can happen to anybody, for many reasons...

So before questioning a 'star' award, for example, a 'current' bad quality
when listened to, or a missing stream with a star, ... the star is generated
with parameter averages over, currently, a 7 day period... with the most
emphasis on the reliability status... >95% or >98% uptime ...  even failure
"today" on one 'quality' parameter won't necessarily 'remove' or 'prevent' a star,
but 'consistent' failure in 2 or more, will.

A New Stream, or one that has been 'adjusted' would take up to 7 days to 'qualify'.

We are watching overall but it will take time to refine the 'Reality'
especially since:
One of the most difficult 'parameters' to isolate is line frequency hum... and it's multiples.
We're aware of that 'weak point'... so on some content,
even an extreme hum level can escape the analysis, at least presently....
that's a toughie. especially since NWROrg isn't the Broadcaster, where
(sometimes) very expensive equipment and monitoring can detect and deal with it.

Currently,for example, there are some 'starred' streams with bad HUM, or Noise, or even
Clipping, etc when heard... but they're 'passing' the other parameters, on average...

So your comments and suggestions are needed and desired, but please be kind.  #-o

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
URL Incorrect
« Reply #7 on: December 16, 2017, 12:02:42 PM »
OK, Here's a GOTCHA

BROADWAVE STREAMERS:
The actual URL used on NWROrg to access your direct Broadwave strem,
ends with MP3
not M3U or any of the other tags Broadwave wants
you to add on.
simply (e.g.) http://ipaddress.you:port#/broadwave.mp3
That's the address config that should validate when you
submit your QRA request...

Sorry for any confusion....

ALSO:  OTHER SERVERS
You may be offering a stream from some server other
than WxRadioDNS,... note that the stream url you
submit may not validate against our platform.., there
are special issues with Third Parties, especially Commercial
providers, some we can solve, some we can't... as we state
in the How To Page on the site... you may receive a 'failed' notice...
Please be patient with that, as we research it...
If we can get the stream, we should be able to get QRA for you,
but it may be delayed because that must be checked manually.




« Last Edit: December 16, 2017, 12:25:21 PM by NWROrg Team »

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
My Stream Access URL
« Reply #8 on: December 18, 2017, 06:13:19 PM »
Let's expand on that url issue that may occur as you attempt to register...

The 'HOW TO' page at NWrOrg has been revised in an attempt to explain what may happen...
There are generally 3 ways to get a stream online at NOAA Weather Radio ORG, and we have about 250+'stream urls, and approximately 167 are currently 'displayed' , from approximately 140 providers.
These stream source URLs fall into basically 3 confusing categories.


Please go to the 'HOW TO PAGE" Getting it Live on the Web

Having read the 'orange' notes under each type, you might understand better why there could be an
invalid URL on your QRA submission...

...and it is true we have run into a bit more confusion than we actually expected as we developed this...
We have located some issues with our notifications and confirmations and are working on them.

Again, a brief explanation of why:
Today, the MP3 codec is supported by most browsers and devices,.
Today, HTML5 Audio using MP3 is supported by most newer browsers and devices.
Today, NWrOrg scripts are designed  to play a selected stream directly in the site, in such browsers / devices,
    WITHOUT the need for plugins, external players, or an awkward "choose your type" menu on the website.
The streams simply Play When Clicked, on the NOAA Weather Radio Site with no popup players.  Hopefully...  :-P
(Do Note that the main page player does offer a 'pop-out' player option, however.)

Invariably, no matter how well we try to explain that some stream access urls in the three categories may be modified from what you expect, {to encompass maximum cross-platform capability), there can be an issue with the addition of the QRA tool request validation.

Once we match the urls, then your QRA reports/notifications come to you exactly as we are 'hearing' them at the time of report. And we cannot generate a report other than on the URL we forward to any user accessing NOAA Wether Radio...
« Last Edit: December 18, 2017, 06:32:34 PM by NWROrg Team »

Offline UpstateWeather.com

  • Signature
  • Contributor
  • ***
  • Posts: 146
    • Upstate Weather
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #9 on: December 20, 2017, 01:47:28 PM »
On behalf of all of us - thanks so much for building these tools! Great stuff!

Offline USAF_Pride

  • Member
  • *
  • Posts: 13
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #10 on: December 25, 2017, 12:18:02 PM »
I don't get it.  I've tweaked everything I can and my stream (KIG63) sounds better than a lot of streams with stars.  Would someone please provide some guidance?  ](*,)

Offline ValentineWeather

  • Forecaster
  • *****
  • Posts: 6362
    • Valentine Nebraska's Real-Time Weather
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #11 on: December 25, 2017, 01:16:34 PM »
I don't get it.  I've tweaked everything I can and my stream (KIG63) sounds better than a lot of streams with stars.  Would someone please provide some guidance?  ](*,)

Agree my stream is very clean so I think the stars means (attention) something is wrong from what I can tell.
Randy

Offline USAF_Pride

  • Member
  • *
  • Posts: 13
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #12 on: December 25, 2017, 01:56:25 PM »
Agree my stream is very clean so I think the stars means (attention) something is wrong from what I can tell.

It means they are either "Good" or "Great"

Code: [Select]
QRA streams: ☆Good Quality: 22, ★Great Quality: 30

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #13 on: December 25, 2017, 03:27:00 PM »
Gold Star parameters explained in the 'tool tip' in the Legend on Main Page Player
Measured over 7 Days Inclusive / rolling ...  Averages
Please keep in mind... this rates the STREAM... not necessarily the stream provider... stuff happens.

Good Quality (Hollow Star)
Stream uses both channels or Full Mono
Stream bit rate of 32Kbps or more
Sampling of 22K/sec or more
Audio Quality score  5 or 6
Availability >=95%

Great Quality (Solid Star)
Stream uses both channels or Full Mono
Stream bit rate of 32Kbps or more
Sampling of 22K/sec or more
Audio Quality score 6
Availability >=98%

The two streams referenced above:

WXN82 is left channel only, and the Stream Sampling rate is 16K, (Stream Bit Rate is good: 32 K)

KIG63 Dec 22- the server was LIVE, but there was No Audio throughout the day.
Additionally the Stream Sampling Rate is 8000 Hz,  Stream bit rate is 21.1 K.

Offline USAF_Pride

  • Member
  • *
  • Posts: 13
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #14 on: December 25, 2017, 08:54:39 PM »
KIG63 was off due to an error on my part trying to get a better signal (reducing gain), but that was corrected.  If you manually listen to the Petoskey or Hagerstown streams (hollow star) and compare it to mine, which is better?  I'm not sure bit rate or sampling makes that much of a difference for this type of broadcast.

Offline Andy G

  • Forecaster
  • *****
  • Posts: 306
    • SunPrairieWeather.com
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #15 on: December 25, 2017, 08:59:39 PM »
KIG63 was off due to an error on my part trying to get a better signal (reducing gain), but that was corrected.  If you manually listen to the Petoskey or Hagerstown streams (hollow star) and compare it to mine, which is better?  I'm not sure bit rate or sampling makes that much of a difference for this type of broadcast.

I had a outage as well. When that outage fell of of the current 7 day period I received a solid star. Availability >=98% required for a solid star.
Once you can accept the universe as being something expanding into an infinite nothing which is something, wearing stripes with plaid is easy.
~Einstein


SunPrairieWeather.com
Davis VP2 24 FARS

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #16 on: December 25, 2017, 10:34:07 PM »
KIG63 was off due to an error on my part trying to get a better signal (reducing gain), but that was corrected.  If you manually listen to the Petoskey or Hagerstown streams (hollow star) and compare it to mine, which is better?  I'm not sure bit rate or sampling makes that much of a difference for this type of broadcast.
Why KIG63... obviously... no contest, no argument. It sounds better than many, in fact. It's been "good" since it was added...

...and the referenced feeds have also been 'good', or at least much better.  Get a good signal to them, then what? 

What some may question are the 'pre-qualifier stream parameters' --- what long hours, experimentation, and testing have determined presents the product best to the listener, overall, and in various 'situations' ...  those are "Is the Audio In Both Channels (Or Full Mono)?" Are both the digital audio stream's BitRate and Sampling frequency sufficient for best quality under a wide range of conditions?

Also keep in mind that the analysis generating 'stars' reflect ONLY what was sensed at the time of any analysis, and could change in moments, or the next morning... Static / noise by itself will NOT kill a Hollow Star....but can 'demote' a 'Solid' to 'Hollow'...  The analysis is automatic, not manual... and under refinement, even as this is typed...

We also felt originally as mentioned, regarding the 'rates'... We didn't expect the need for higher 'bit rate' and 'sampling rate' when we began this project about a year ago. And in fact, we need to update some of the information on the website.

We first believed a bitrate of 8Kb/s and a sample rate (frequency) of 8KHz, (and then later - 16KHz) should be adequate.  In fact, may work well on pure audio BW of 3750 Hz, in most cases...
But,    #-o    4KHz is NOT the max freq being streamed on virtually all sources....! At the very beginning it may be received by an FM radio not operating as 'narrow band', and processed through a 15KHz BW FM circuit, and then sent to a device with an encoder installed, which converts to digital MP3, with some sampling frequency and bit rate, sent through various connections to a device which may be operating at some different "personal settings".... including volume level setting.
Most streams, then, include noise, hiss, hum,at varying levels, etc to 15KHz or higher>.. Many have some clipping, excessive input volume, etc... None of which existed in the original NWR audio before it left the transmitter. We were wrong, overall. 

Long hours were spent. This established guidelines across many 'situations' and 'encoders' and 'reception' variables. Today those parameters have been locked in at 32Kb/s (Bit Rate) for smoothest audio levels and 22,050HZ Sampling Frequency minimum for best frequency reproduction with minimum aliasing. In fact, we could suggest the CD standard 44.1KHz sampling, again because of the extended audio bandwidth of most audio amplifiers connected to the encoders....and don't ignore the wider bandwidth on the 'listener's' end, as the 'digital mp3' is recoded to Analog.

One might note, especially to the 'louder streams'.. at times the audio may 'pop' ('thump') or 'blank' for a moment...even 'appear' to 'lock' or 'skip' ... that can be due to a low bitrate, especially when coupled with a low sample frequency and higher levels.

We've been evaluating this for months.  Throw in noise, low or excessive levels, a single channel sent, especially when encoded as 'stereo', weak reception, then the Bit Rate and Sampling frequency baselines became apparent.  These were not set arbitrarily.

So besides Overall Average Quality, and Stream availability, which qualify for a stream star,
the 'pre-qualifiers' are

Stream uses both channels if Stereo Encoded,  or Full Mono Mode ( with both channels)
Stream bit rate of 32Kb/s or more
Sampling rate (frequency) of 22.050 KHz/sec or more

Those three directly affect the Overalls in many cases, are easily reconciled, and present the stream better to the listener... regardless of any failures in the 'graded' 6 point analysis.
« Last Edit: December 26, 2017, 08:58:53 AM by NWROrg Team »

Offline Cutty Sark Sailor

  • Moderator
  • Forecaster
  • *****
  • Posts: 3393
    • Frankfort Weather - TwinHollies WeatherCenter
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #17 on: December 26, 2017, 09:50:10 AM »
   #-o    4KHz is NOT the max freq being streamed on virtually all sources....! At the very beginning it may be received by an FM radio not operating as 'narrow band', and processed through a 15KHz BW FM circuit, and then sent to a device with an encoder installed, which converts to digital MP3, with some sampling frequency and bit rate, sent through various connections to a device which may be operating at some different "personal settings".... including volume level setting.
Most streams, then, include noise, hiss, hum,at varying levels, etc to 15KHz or higher>.. Many have some clipping, excessive input volume, etc... None of which existed in the original NWR audio before it left the transmitter. We were wrong, overall. 
```
So besides Overall Average Quality, and Stream availability, which qualify for a stream star,
the 'pre-qualifiers' are

Stream uses both channels if Stereo Encoded,  or Full Mono Mode ( with both channels)
Stream bit rate of 32Kb/s or more
Sampling rate (frequency) of 22.050 KHz/sec or more

Those three directly affect the Overalls in many cases, are easily reconciled, and present the stream better to the listener... regardless of any failures in the 'graded' 6 point analysis.
Let me illustrate what team may be speaking of:
« Last Edit: December 26, 2017, 09:58:12 AM by Cutty Sark Sailor »
 


Offline 92merc

  • BismarckWeather.net
  • Moderator
  • Forecaster
  • *****
  • Posts: 1307
  • BismarckWeather.net
    • BismarckWeather.net
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #18 on: December 26, 2017, 04:04:17 PM »
That is one bonus of using the RPI method Tim had pointed out.  He figured out how to clip sound below 150hz to get rid of the 60 cycle hum I could hear in my stream.  I've also capped my stream at 3k to get rid of the upper frequency noise.

I do still have some noise.  I'm waiting a new power supply to arrive I'm hoping will fix the issues.  I've been bouncing between 5 points and 6 points.  So I get a hollow star.  I'm satisfied with that for the time being.
https://www.BismarckWeather.net
Davis VP2, Cumulus, WeatherDisplay, Blitzortung, Saratoga Scripts, NOAA Stream via PI

Offline USAF_Pride

  • Member
  • *
  • Posts: 13
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #19 on: December 26, 2017, 07:42:19 PM »
Let me illustrate what team may be speaking of:


Is this my actual stream or an example?

Offline Cutty Sark Sailor

  • Moderator
  • Forecaster
  • *****
  • Posts: 3393
    • Frankfort Weather - TwinHollies WeatherCenter
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #20 on: December 26, 2017, 09:22:20 PM »
No, that's one of my streams, examined back in August... the Red traces are the 'peaks' green is 'current' instant.  The attached image in the post has more 'looks'
 


Offline USAF_Pride

  • Member
  • *
  • Posts: 13
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #21 on: December 27, 2017, 10:05:29 AM »
Ok.  I'm 6 miles LOS from the antenna (with no obstructions).  I'd be curious if your graph would look the same as my stream.

Offline Maumelle Weather

  • Forecaster
  • *****
  • Posts: 1824
    • Maumelle Weather
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #22 on: December 27, 2017, 04:51:12 PM »
Hi Folks,

Got a couple of emails regarding the availability of my streams being below 84%. System has not been touched in 66 days, since it was last restarted. The only adjustment I have made within the last month was to lower the main computer volume down about 10% (sitting at 85% volume). Attached is the image from Broadcastify on stream availability.

System has been at 100% the whole time. Daily graphs for the last week show the same.
GR2AE, GR3, Cumulus

Offline NWROrg Team

  • Save A Life
  • Moderator
  • Contributor
  • *****
  • Posts: 131
    • NOAA All Hazards Weather Radio
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #23 on: December 27, 2017, 05:04:17 PM »
John,...
A router DNS glitch  caused a failure to connect on an houly check (10 an) to Ken's Streams, yours, and one other... ... we don't think any others were affected, and it was quickly fixed...
...sigh... --2017-12-27 10:05:34--  http://xxxxxxxxxxxXXXXXXXXXXXXXX
Resolving xxxxxxxxxxxx (xxxxxxxxx)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘xxxxxxxxxxxxxxx'
--2017-12-27 10:05:56--  http://xxxxxxxxxxxxxxxxxxxx
Resolving xxxxxxxxxxxxxxx (xxxxxxxxxxxxxxx)... failed: Temporary failure in name resolution.
wget: unable to resolve host address ‘xxxxxxxxxxxx

Additionally there were Other DNS failures for about 41  sources on the previous hourly (9am) run
Sorry for the issue folks...
« Last Edit: December 27, 2017, 05:20:30 PM by NWROrg Team »

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9244
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: NOAA Weather Radio Org Quality / Reliability Tools
« Reply #24 on: December 27, 2017, 05:17:06 PM »
Yes, I saw the messages and checked the issue and saw that *.dyndns.org addresses were temporarily unresolvable at your census server.  No worries... "Internet happens"  :)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

 

anything