WXforum.net

Weather Software => Station Software Development => Topic started by: m77 on August 11, 2011, 05:13:48 PM

Title: predicting imminent rainfall
Post by: m77 on August 11, 2011, 05:13:48 PM
I am interested in whether or not you can predict rainfall using data available from a pws, not hours ahead but maybe 5 mins in advance.

not sure what use it may be but its more of a challenge.

what sort of variables would be needed from a pws to give a better than 50.50 call of rain?
Title: Re: predicting imminent rainfall
Post by: SLOweather on August 11, 2011, 06:12:04 PM
Divide the current 10 minute average wind speed by 12. That gives the approximate wind run in miles for 5 minutes.

Estimate the current average wind direction.

Take the weather station that direction and that distance and set it down.

When the station gets wet, it'll rain at the original location in about 5 minutes.

 :lol:

Seriously, I don't know of anything a PWS can do with the granularity required for that kind of prediction. Around here (central California coast), I've noticed that in classic winter storms, (our rainy season), the barometer drops and drops, and it's windy. The rain generally doesn't start until the baro bottoms out and the wind dies down.

But trying to predict rain from those 2 observances is going to be pretty hard. I can get a better idea from looking at radar, or other close by stations if the rainfall is hard enough.

If it's really important, I'd consider a more elegant solution to what I described above. Figure out the prevailing directions that the storm comes from, and the average speed at which they travel.

Put one of those Hydreon RG-11 Optical Rain Gauges at the calculated location, set to the most sensitive "It's raining" setting, and use the relay contacts to signal you some how.
Title: Re: predicting imminent rainfall
Post by: SLOweather on August 11, 2011, 06:36:34 PM
I'm curious, what's your definition of high ground level dew point?

Two things are needed: (1) falling barometric pressure, and (2) high ground level dew point temperature.

I'm not sure exactly *how* but cloud level (meaning elevated atmospheric dew point temperature) also comes into play, because of the lapse rate of temperature with increasing altitude.

It's possible for "rain" to occur below a cloud, but no water to actually reach the ground, ie: virga. That occurs when the surface air is much drier (low dew point temp) than the upper air (high dew point temp).
Title: Re: predicting imminent rainfall
Post by: TNETWeather on August 11, 2011, 06:50:06 PM
Around here normally you are only assured of getting rain when it hits your head.  Thunderstorm cells are very fickle and even watching radar can give you a false idea if it is actually going to rain in a location.

Can't tell how many times I've watched GRLevel3 radar and seen a cell approaching only to have it be done before it gets there, or skip over and regenerate past our location.

We have a location just north of us by about 3-4 miles that due to a little ridge in front of it, gets a lot of the moisture that passed over us and dumps on them on a very regular basis.  Great lightning shows from that area as well.

Its also no uncommon for a cell that ends up generating rain doesn't have any measurable rain from it leading up to when it actually starts raining.

I know a lot of places in the country when you see certain cloud types, its going to rain and rain over a wide area, but we get a lot smaller cells normally in the desert unless it is a fairly large storm.
Title: Re: predicting imminent rainfall
Post by: m77 on August 12, 2011, 01:39:42 AM
thanks, I think I may study

pressure
pressure tendency
air temp minus dew point
the above tendency
relative humidity
rh tendency

any others?
Title: Re: predicting imminent rainfall
Post by: d_l on August 12, 2011, 11:26:21 AM
Rapidly rising barometric pressures, rapidly dropping temperatures, and suddenly gusty winds are indicative of a microburst from a cell and may be accompanied or soon followed by rainfall.
Title: Re: predicting imminent rainfall
Post by: m77 on August 12, 2011, 01:04:02 PM
wow thanks guys.

I am still looking at a few things but at the moment I have a 10% ish strike rate of rain within 15 mins.

any advice is welcome.

I suppose one use could be to use your data to generate a "rain likely" warning on a website.

it interests me anyhow.

I will look at dl's advice on gusts, pressure rise and temp fall and see of the strike goes up.
Title: Re: predicting imminent rainfall
Post by: wuhu_software on August 13, 2011, 09:56:10 PM

I have always wondering if using fuzzy logic or a neural network system, developed using historical data, would produce good results. I looked around for weather prediction based on either of these concepts but did not find anything.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 03:41:46 AM
thanks all.

I looked at lots of variables individually and in combination.

I only looked at scenarios where it wasn't already raining, to make it fair.

the strongest variables were pressure and (air temp much minus dew point).

wind speed, direction, humidity and temp didn't really have as much of an effect.

when pressure was equal to or below 1010mb and t-dp was equal to or below 0.6 I got the following

rain within 15 mins 1 in 12
within 30 ...1 in 7
within 45...1 in 5
within 60...1 in 5
within 120...1 in 3

each is four times more likely than choosing blindly.

when temp = d point and pressure <= 1010 the odds were
1 in 3, 1 in 2, 2 in 3, 8 in 10 and 100%.

pretty pleased with that study.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 07:40:04 AM
ok, extended the study a little to look further ahead.

when pressure <= 1010 and (T-DP) <= 0.6

the chance of rain (>=0.2mm) within...

15m   8%
30m   14%
45m   18%
1h   22%
2h   33%
6h   55%
12h   69%
24h   77%
36h   82%
48h   86%


obviously i'd like to expand my analysis onto more than 3 months worth of data to see how it stacks up all year.

Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 07:44:10 AM
... and when pressure <= 1010 and T-DP =0

15m   35% - which is 16 x more accurate than picking at random!
30m   50%
45m   65%
1h   80%
2h   100%
6h   100%
12h   100%
24h   100%
36h   100%
48h   100%


(although this scenario occurred only 20x, whereas the previous one happened 2600x)
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 02:15:54 PM
my mistake - that data is based on 466 days worth of 15 min logging, so a year and 3 months or so.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 04:15:39 PM
thanks Tele man.

you are correct, this rule of thumb that I have suggested will not work at another location easily. there will be many factors influencing local conditions and thus rainfall.

our altitude is about 90m and average pressure is 1014.

when the pressure <=1010 and dpd is <=0.6 I think there is a good chance (70%) of rain within 12 hours. when dpd is zero rain is likely within an hour (80%).

I will follow this rule of thumb with interest.

what does tdp stand for exactly?



Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 04:19:27 PM
i'd also be interested to hear if anyone else can test this using their own data and seeing how results differ, using pressure and dpd.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 04:21:43 PM
I think it is also important to say HOW MUCH MORE the conditions improve your prediction by.

my first was four times more likely than average and the stricter condition was 16x

Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 05:03:19 PM
dry bulb!

got it.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 05:37:42 PM
yes thanks I use the cloud base calculation.

0.6 tdp is equivalent to a cloud base of around 94m agl for me with anything lower increasing the chances of rain
 when pressure is under 1010.

so extending the rule of thumb... cloudbase under 100m (clouds this low probably indicate pressure <1010) brings rain within 12h. sounds quite like a no brainer when you put it like this!
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 05:54:28 PM
do you mean the curves for forecasted times ahead or as pressure and tdp alter?

hours ahead chances are posted above?
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 06:04:28 PM
ie posts 13, 14.
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 06:09:00 PM
good question if understand it correctly.

so you are asking what variables would be present for the probability to be x in y hours?

what do you think?

I have an excel sheet with altering probabilities for time as you tweak the factors.

?
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 06:10:02 PM
...OK, it looks like only a "two-line" curve, correct?

AZ CWOP - DW6988 (inactive)


?
Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 06:20:54 PM
ah ok.

I would have to alter it 0.1 at a time and record probability.

not on pc so could do it tomorrow.

Title: Re: predicting imminent rainfall
Post by: m77 on August 14, 2011, 06:27:42 PM
what do you predict or no idea?

will post in about 10 hours when on pc.

i will even go from higher than 0.6 and will draw graphs for 15mins, 30 mins etc
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 01:34:35 AM
this site uses 4F. my calculation was 0.6C

0.6C is about 1F
4F is about 2.2C

and that looks like they can detect mist whereas I am going for rain. interesting setup though.
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 02:19:43 AM
   15m   30m   45m   1h   2h   6h   12h   24h   36h   48h
0.6   8.05%   13.84%   18.32%   22.05%   32.62%   55.34%   68.74%   76.64%   81.79%   86.49%
0.5   8.16%   14.07%   18.42%   22.04%   32.54%   55.10%   67.13%   75.67%   81.06%   86.07%
0.4   8.17%   14.19%   18.37%   21.88%   32.20%   54.98%   66.49%   75.49%   80.92%   85.99%
0.3   8.03%   13.26%   17.72%   21.30%   32.43%   53.24%   64.96%   73.77%   79.28%   85.19%
0.2   7.78%   13.17%   18.36%   22.36%   32.73%   52.89%   62.87%   72.85%   79.44%   82.83%
0.1   10.68%   16.50%   22.82%   28.16%   37.38%   58.25%   68.45%   78.16%   84.95%   86.41%
0   35.00%   50.00%   65.00%   80.00%   100.00%   100.00%   100.00%   100.00%   100.00%   100.00%
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 02:25:19 AM
or from dpd <= 2

   15m   30m   45m   1h   2h   6h   12h   24h   36h   48h
2   5.88%   10.09%   13.47%   16.26%   24.59%   43.30%   56.57%   68.65%   75.89%   80.66%
1.7   6.24%   10.76%   14.36%   17.28%   26.03%   45.15%   58.35%   69.54%   76.53%   81.20%
1.5   6.61%   11.28%   14.99%   18.02%   27.04%   47.04%   59.95%   70.49%   77.15%   81.85%
1.2   6.97%   11.91%   15.81%   18.97%   28.64%   49.76%   62.52%   72.15%   78.10%   82.89%
1   7.21%   12.20%   16.30%   19.56%   29.46%   51.09%   63.94%   73.38%   79.15%   84.10%
0.6   8.05%   13.84%   18.32%   22.05%   32.62%   55.34%   68.74%   76.64%   81.79%   86.49%
0.5   8.16%   14.07%   18.42%   22.04%   32.54%   55.10%   67.13%   75.67%   81.06%   86.07%
0.4   8.17%   14.19%   18.37%   21.88%   32.20%   54.98%   66.49%   75.49%   80.92%   85.99%
0.3   8.03%   13.26%   17.72%   21.30%   32.43%   53.24%   64.96%   73.77%   79.28%   85.19%
0.2   7.78%   13.17%   18.36%   22.36%   32.73%   52.89%   62.87%   72.85%   79.44%   82.83%
0.1   10.68%   16.50%   22.82%   28.16%   37.38%   58.25%   68.45%   78.16%   84.95%   86.41%
0   35.00%   50.00%   65.00%   80.00%   100.00%   100.00%   100.00%   100.00%   100.00%   100.00%
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 02:34:43 AM
...leading to these rules of thumb when press <= 1010

dpd
2   50-50 within 12h   likely within 36h
1.7   50-50 within 12h   likely within 36h
1.5   50-50 within 6h   likely within 36h
1.2   50-50 within 6h   likely within 36h
1   50-50 within 6h   likely within 36h
0.6   50-50 within 6h   likely within 24h
0.5   50-50 within 6h   likely within 24h
0.4   50-50 within 6h   likely within 24h
0.3   50-50 within 6h   likely within 24h
0.2   50-50 within 6h   likely within 24h
0.1   50-50 within 6h   likely within 12h
0   50-50 within 30m   likely within 1h
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 03:05:30 AM
i think the obvious step is to look at what happens to the %s as i vary pressure and DPD, to give a full look up chart.

i will probably lump pressures together eg 1005-1010, 1010-1015, 1015-1020
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 07:08:26 AM
just realised those chances are for less than or equal to the dpd, as opposed to equal to it.

will correct and add in the pressure factor in the next hour or so.
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 11:14:51 AM
yes our 2010 pressure average was 1014.5!  ?

what do you mean about 100% threshold. are you referring to the 0.6?
Title: Re: predicting imminent rainfall
Post by: m77 on August 15, 2011, 11:33:09 AM
this suggests it should be 1014-1016?

www.metoffice.gov.uk/climate/uk/averages/7100_1km/MSLP_Average_1971-2000_17.gif
Title: Re: predicting imminent rainfall
Post by: m77 on August 16, 2011, 08:33:51 AM
i am developing some expressions rather than using the % chances for a piece of php script.

any comments on these particular phrases and % thresholds?

or should i just quote the % chance category? eg "33-49%"

it is aimed at a local comunity and school rather than experts in meteorology.

40-59   even chance
60-79   likely
80+   extremely likely

are these too many?
Title: Re: predicting imminent rainfall
Post by: m77 on August 16, 2011, 08:44:55 AM
using these cutoffs i get a nice summary table:

where E = even chance, L= likely and X = extemely likely

and 15m = next 15 minutes, 1h = next 1 hour etc etc

so E24h means evens chance of rain in the next 24 hours

Title: Re: predicting imminent rainfall
Post by: m77 on August 16, 2011, 08:55:57 AM
so at the moment we have pressure = 1015.9 and DPD = 3.1 so that translates to

Even chance of rain in the next 48 hours


we will see!

Title: Re: predicting imminent rainfall
Post by: m77 on August 16, 2011, 03:15:47 PM
second thoughts...

help - is this a simple way to notify locals of rain chances?

for example if a sentence read

"The chance of rain is 50:50 in the next 6 hours, LIKELY in the next 12 hours and EXTREMELY LIKELY in the next 24 hours"

does that mean anything or is it too much?

or should it be reduced to

"The chance of rain is 50:50 in the next 6 hours"

or even just

"The chance of rain is EXTREMELY LIKELY in the next 24 hours"
Title: Re: predicting imminent rainfall
Post by: Cynjut on August 17, 2011, 09:18:21 AM
The only problematic thing I can see here is the granularity of your report.

Your rain chances are 50% because the conditions at your station say that half the time these conditions exists it rains at your station.  From what I remember about the weather report (when I was a radio announcer), the likelihood at each locality also figures into that percentage.

For a larger forecast reporting center (TV, NWS, Radio), the area is much larger.  For them, the percentage chance of rain isn't just the likelihood that the rain will fall at a specific location, it's that "within the sound of my voice", the rain will fail as a percentage.

So, if 20% of the listening area is "100%" going to get rain, the percentage was 20%.  If it's 50% in 50% of the listening area, it was "30%" (we always rounded up to 10s instead of using the things like "20-30%").  The trick, then, is for your prediction aggregater to take all of the "50%"s and bring them into a model that takes your data and the data from other stations like yours pull it into a forecast that works for them.  The larger the reporting area, the more stations would need to be contacted.

Other than that, this has been a fascinating process to watch.
Title: Re: predicting imminent rainfall
Post by: m77 on August 17, 2011, 10:43:34 AM
interesting idea, many thanks.

however i have no idea how big the actual audience is (and more importantly who cares!) and os i will assume calculations to be done on the school roof / rain gauge rather than any sort of catchment area as you suggest. rain fall distribution is patchy and hit and miss anyhow.

i have smoothened the data (as there was noise and inconsistencies - sometimes a decrease in DPD or pressure led to a slight decrease in the rain chance and not increase a sexpected)

by using Excel's data analysis tool i performed a bit of multiple regression to get best fit equations for the probabilities for each time period.

so there is a formula for the rain chance when given DPD and pressure for 1h, 2h, 6h, 12h etc etc

for example i found the 12h formula to be

12hchance = 11.129 + (DPD * -0.105 ) + ( pressure * -0.010 )

i used the 20% "description" cutoffs (Low, small, even, high, very high) and have now used php to lookup the pressure and DPD and put them through the formulae. This produces the chances shown in red on my own personal data page here:

http://www.maidenerleghweather.com/lastrecord.php

i do not broadcast this page as it's too much data but it has everything for me at a quick glance!!!

it's really just to get the php and formulae up and running!
Title: Re: predicting imminent rainfall
Post by: m77 on August 18, 2011, 01:22:56 AM
I'm pretty sure that the 12h formula I posted won't work for you as its based on my data.

perhaps look at your own data and derive formulae and compare.
Title: Re: predicting imminent rainfall
Post by: m77 on August 18, 2011, 09:46:53 AM
just struck me that the %s by hour don't mean much unless you relate them to the normal average chance of rain in that given period!

hence i have worked out the amount of times rain is more likely than normal

eg http://www.maidenerleghweather.com/lastrecord.php

1h%: 12

means there is a 12% chance of rain within the hour

but if you were picking blindly the rain would come 5.97% of the time within the hour.

so this is 2.0 x more likely than average

hence the addition of 1hx: 2

does this make more sense and is it more useful??
;-)


at the moment it is raining  :-P and the lines read:

1h%: 12 2h%: 18 6h%: 35 12h%: 48 24h%: 63 36h%: 74 48h%: 80
1hx: 2 2hx: 1.9 6hx: 1.8 12hx: 1.6 24hx: 1.4 36hx: 1.4 48hx: 1.3

so in retrospect, it is twice as likely - and it actually is!!!!

bit of backward thinking but we shall see from hereonin.
Title: Re: predicting imminent rainfall
Post by: m77 on August 22, 2011, 03:50:40 PM
yes its a pretty good indicator I reckon.

the hourly predictor has a larger variation compared to the longer term 48h which fluctuates very little.

one addition that I will make is the historical change so that I can track how the percentage has changed periodically, perhaps every 15mins for the last 2h to are a tendency - a rise or fall in the percent.
Title: Re: predicting imminent rainfall
Post by: m77 on August 23, 2011, 03:29:21 AM
ok my data page now looks like this:


chance1h%: 9 2h%: 14 6h%: 26 12h%: 38 24h%: 53 36h%: 63 48h%: 70
compare to average1hx: 1.5 2hx: 1.4 6hx: 1.3 12hx: 1.3 24hx: 1.2 36hx: 1.2 48hx: 1.1
historical 1h%9 9 8 8 8 8 8 8 9 9 8 7
historical 1h compare to av1.5 1.5 1.3 1.3 1.3 1.3 1.3 1.3 1.5 1.5 1.3 1.2

each value represents - now, 5m, 10m, 15, 20, 30m, 45m, 60m, 75m, 90m, 105m, 120m ago in time

from this data i can see that rain is becoming more and more likely - the chance has increased from 7/8% to a 9% and the av compare has risen from a 1.2x to 1.5x more likely than average

this way i can see what the trend / tendency is and can probably make more of a call on what's going to come.

check it out live at http://www.maidenerleghweather.com/lastrecord.php
Title: Re: predicting imminent rainfall
Post by: m77 on August 24, 2011, 02:00:27 PM
humidty rise?
dpd fall?

why not? Im interested.
Title: Re: predicting imminent rainfall
Post by: m77 on August 24, 2011, 02:13:24 PM
pressure drop?

or is it mostly convectional?

ours is frontal.
Title: Re: predicting imminent rainfall
Post by: xykotik on August 25, 2011, 11:11:25 PM
what sort of variables would be needed from a pws to give a better than 50.50 call of rain?

You don't need any fancy-schmancy variable whatchamacallits or historical data set thingamajiggers to know if it is going to rain.  The statistical probability for anything to happen is always 50/50.  It either will, or it won't.  Accuracy beyond that and you may be a labeled a witch, so be careful.