Author Topic: New Acurite Lightning Detector  (Read 18144 times)

0 Members and 1 Guest are viewing this topic.

Offline Storm017

  • Senior Contributor
  • ****
  • Posts: 172
Re: New Acurite Lightning Detector
« Reply #75 on: September 06, 2018, 07:47:24 PM »
A T-Storm passed through the area...captured some data. Hopefully this will help.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #76 on: September 08, 2018, 12:40:04 PM »
So I found some C++ code that takes what's in the register in the AS3935 chip for distance of a lightning strike.  It's all in kilometers so you'll have to do your own calculations to get miles.  I'm guessing the 28 that is returned in the data stream is the HEX value in the register.  If so, the values break down as follows:

case 0x28:
         km = 40;
         break;
      case 0x25:
         km = 37;
         break;
      case 0x22:
         km = 34;
         break;
      case 0x1F:
         km = 31;
         break;


Hopefully that helps.  I haven't got around to getting this setup in weewx yet.  I've missed my storm season as I have a bunch of projects on the go that never seem to get done in time.

Here's the Spec sheet from Mouser:

https://www.mouser.com/ds/2/588/ams_AS3935_Datasheet_EN_v5-1214568.pdf

Look at the chapter on Statistical Distance Estimation

I'm not sure how the acurite lightning detector is using this data to send out to it's console

Thanks for that, but unless I'm missing some context that code is pointless.  The case statements are just turning the hex literal into a decimal literal, but assuming those are in some int data type means no conversion is necessary.  Hex 28 is 40 decimal.  So I have no idea why someone thought that was needed.   Maybe the understanding changed over time and the maintainers didn't notice that "lookup" table didn't actually do anything any longer.

However, it's not relevant to the acurite unless I'm missing some bits.  There only appear to be 5 bits available, so you've got 0-31, (0x1f).   I think 0x1f (31) is an invalid value.  That's what it reset to at power up.  Though I think the status bit indicates whether the distance value is valid or not.

I haven't looked at the data sheet in a year or two.  I don't remember there being any clues that seemed relevant for how the acurite sensor is encoding it.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #77 on: September 08, 2018, 12:51:27 PM »
A T-Storm passed through the area...captured some data. Hopefully this will help.

So taking a quick look, you've got three distance correlations in that file.  (There are actually 5 but 3 have the same values.)

rtl_433, console
20, 15
10,  7   (repeated 2 more times)
12,  9

Did I interpret that correctly?


Offline Storm017

  • Senior Contributor
  • ****
  • Posts: 172
Re: New Acurite Lightning Detector
« Reply #78 on: September 08, 2018, 04:40:15 PM »
Yes that is correct.  I'm going to set the console to display KM and capture data during the next time storms pass through.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #79 on: September 08, 2018, 04:45:24 PM »
3 data points isn't a lot to have confidence, but just looking at those, it looks like the relationship is the console value in miles is 3/4 of what the raw value in the message is.  That doesn't quite seem like any magic number that I recognize.  If that it the actualy conversion, it means the range 31 - 0 raw is 23.25 to 0 miles.  IIRC the spec is up to 25 miles.

Kilometers would be useful at some point, but I'd be interested in more Miles correlations.  The larger the range and the more data points, the easier it will be to see if we've got the right conversion.

Offline Storm017

  • Senior Contributor
  • ****
  • Posts: 172
Re: New Acurite Lightning Detector
« Reply #80 on: September 08, 2018, 05:22:19 PM »
Hopefully I get some storms that stick around and will try to get more data points.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #81 on: September 08, 2018, 06:17:29 PM »
Actually if you get some brief / far off storms that only register a strike or two that would also be helpful.  Large values for distance that are stable for long enough to be sure of the correlation would be best.

« Last Edit: September 09, 2018, 03:12:08 PM by rct »

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #82 on: September 09, 2018, 10:03:24 AM »
Did you look at my data? I found no obvious correlation.

I also had the 10 and 12 as 7 and 9, however I took a bunch of KM readings too.

The closest I got to was sensor value / 1.4 rounded, however I then got a longer distance reading that did not correspond.

I also had a few readings from rtl433 that were not whole numbers.  Not sure how that happened.
« Last Edit: September 09, 2018, 11:23:42 AM by wcndave »

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #83 on: September 09, 2018, 03:30:09 PM »
Did you look at my data? I found no obvious correlation.

I also had the 10 and 12 as 7 and 9, however I took a bunch of KM readings too.

The closest I got to was sensor value / 1.4 rounded, however I then got a longer distance reading that did not correspond.

I also had a few readings from rtl433 that were not whole numbers.  Not sure how that happened.

The problem with the data you provided is that it has been modified by your weewx config/processing, so it is approximate and probably misleading.  I think you are storing 5 minute buckets in the database and were doing averaging.  The 6045 transmits every 8 seconds when it is in active mode.  So that's a long time.

As you said you are seeing some floating point numbers for distances. The numbers coming out of rtl_433 can only be integers.

Those floating point numbers are caused by the aggregation (averaging) you are (or were?) doing in your weewx config.  Your SQL statements show you are rounding which shouldn't be necessary.  @craigthom suggested changes to your config.  You should discard the data from when you were averaging.

The most reliable data for doing correlations is the raw messages from rtl_433.  I think there is a way to capture those when weewx is running.

That being said -- Dividing by 1.4 is pretty close to multiplying by 3/4.  The reciprocal of 3/4 is 1.3333...   That is for a conversion to miles.  Given the 5 minute buckets and possibly averaging, I'm not surprised that you would see numbers that don't fit.

Note: if you remove the batteries/power cycle the sensor, it resets the storm distance to 31 (0x1f).

31 is either an invalid value or some of the other bits indicate that it shouldn't be used for a distance calculation.



« Last Edit: September 09, 2018, 03:36:21 PM by rct »

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #84 on: September 09, 2018, 03:40:20 PM »
I've cleaned up and posted a Jupyter notebook that I've used to look at the lightning detector data.  The notebook uses data from Summer 2018.  I've extracted only messages where there are changes in strike count, storm distance, and USSB1.

Some patterns can be seen in the data.

See https://github.com/rct/device-decoding-notes/blob/master/acurite-6045-lightning/rct-data/lightning-breakdown-1.ipynb

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #85 on: September 09, 2018, 04:53:33 PM »
A T-Storm passed through the area...captured some data. Hopefully this will help.

For what it's worth I also posted a Jupyter notebook where I was examining the data @Storm017 provided.

https://github.com/rct/device-decoding-notes/blob/master/acurite-6045-lightning/storm017/storm017-data.ipynb

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #86 on: September 09, 2018, 05:04:36 PM »
Did you look at my data? I found no obvious correlation.

I also had the 10 and 12 as 7 and 9, however I took a bunch of KM readings too.

The closest I got to was sensor value / 1.4 rounded, however I then got a longer distance reading that did not correspond.

I also had a few readings from rtl433 that were not whole numbers.  Not sure how that happened.

The problem with the data you provided is that it has been modified by your weewx config/processing, so it is approximate and probably misleading.  I think you are storing 5 minute buckets in the database and were doing averaging.  The 6045 transmits every 8 seconds when it is in active mode.  So that's a long time.

As you said you are seeing some floating point numbers for distances. The numbers coming out of rtl_433 can only be integers.

Those floating point numbers are caused by the aggregation (averaging) you are (or were?) doing in your weewx config.  Your SQL statements show you are rounding which shouldn't be necessary.  @craigthom suggested changes to your config.  You should discard the data from when you were averaging.


Hi, I have config for doing deltas for number of strikes, however the distance has never been modified by weewx config, as there's no "module" for it.

That's the reason I was trying to figure out how to get it to work ;-)

So distance is raw data.  The floating numbers are odd, and could be discarded, the other values match with other posts, so I think you can use them as reliable data.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #87 on: September 09, 2018, 07:01:25 PM »
@wcndave - take a look at this post from @craigthom again. From what he says you need to use the min aggregator.  The default for things that aren’t defined is to average them, which explains what you are seeing

Did you get anything going on this?  I have added 2 fields to record the data, however I can't make sense of them, nor be sure how to get weewx to use them.

in my sensor map I have ...

I think this is more of a WeeWX issue, but here's what I discovered a year or so ago when I was messing with this.

The value for lightning strikes is, as noted, a total count in seven bits, so it wraps from 127 to 0.  That's from the sensor.  If the value is the same in consecutive records there were no strikes during that loop interval.

If you put an entry in the [[deltas]] section of the [SDR] driver config that's something like this

Code: [Select]
[sdr]
...
    [[deltas]]
        lightningStrikes = lightningStrikesTotal

you'll get just the number of strikes, not the internal running total.

But that just gives you the lightning strikes per packet received.  Each packet will have a new value, and there are dozens of packets during the loop time before weewx writes to the database.

By default, for things it doesn't already know about, weewx averages the values received by packets, so you won't get the total lightning strikes for the reporting interval (usually five minutes) but an average of the strikes per packet.

To fix this you'll need to add or edit the Accumulator top level section in weewx.conf.  I added it at the very end.

Code: [Select]
[Accumulator]
    [[lightningStrikes]]
        extractor = sum
    [[lightningDistance]]
       extractor = min

The distance extractor gives you the closest lightning strike during the interval.  Without it it averages the values, so if you get two with a distance of 1 and eight with a distance of 20, you'd end up with a value of 18 or so.  I want to record the smallest distance during the interval, not the average.  If you want the average you can leave that out.

The "min" only works for weewx 3.7.2 or later.  I got Tom to add it for this very reason.

I haven't looked at this in a year, but I got those lines from my weewx.conf.

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #88 on: September 10, 2018, 04:58:42 AM »
OK, makes sense, thanks.  Although my non-floating distance data is still correct, with 500 strikes in one 4 hour period I had quite a lot of data!

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #89 on: September 11, 2018, 09:31:22 AM »
Out of interest, what does happen to the delta when the values wrap around from 127 back to 0?  Do you simply lose one reading?  If they go from 125 to 8, you'd lose 10 strikes...

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #90 on: September 11, 2018, 10:54:07 AM »
OK, makes sense, thanks.  Although my non-floating distance data is still correct, with 500 strikes in one 4 hour period I had quite a lot of data!

If you don't get any changes during the interval, the average computed will be the same as the (one) input number.

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #91 on: September 11, 2018, 11:00:27 AM »
Out of interest, what does happen to the delta when the values wrap around from 127 back to 0?  Do you simply lose one reading?  If they go from 125 to 8, you'd lose 10 strikes...

I believe you are asking how weewx handles counters. Hopefully one of the people with direct weewx knowledge can answer that.

Note: counters are a reasonably common data representation.  Things like the rain gauges count tips.  There aren't infinite bits available so at some point the counter wraps around just like a 4 digit meter (electrical usage, odometer, etc.) that rolls over from 9999 -> 0 or whatever depending on the number of digits it has.

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #92 on: September 11, 2018, 04:08:04 PM »
Out of interest, what does happen to the delta when the values wrap around from 127 back to 0?  Do you simply lose one reading?  If they go from 125 to 8, you'd lose 10 strikes...

I believe you are asking how weewx handles counters. Hopefully one of the people with direct weewx knowledge can answer that.

Note: counters are a reasonably common data representation.  Things like the rain gauges count tips.  There aren't infinite bits available so at some point the counter wraps around just like a 4 digit meter (electrical usage, odometer, etc.) that rolls over from 9999 -> 0 or whatever depending on the number of digits it has.

Sure, and the rain counter would appear to go very high, it counts yearly rainfall for example.

However, if the lightning strikes count up to 127, then it can easily go three times round in one night for me, meaning data could be lost.

eg, if the readings were 108, 121 008, 012
there there have been 13, 14, 4 strikes in each reporting period respectively.

However weewx might count that as 13, 0, 4

Just wondering if there was a way to tell weewx what the wraparound point was, so that it could calculate based on that, however probably better to ask in the weewx group.

Just seemed people here knew about the lightning counter in quite some detail  :-)

Thanks!

Offline rct

  • Senior Member
  • **
  • Posts: 98
  • W2RCT
Re: New Acurite Lightning Detector
« Reply #93 on: September 11, 2018, 04:40:29 PM »
Remember the lightning sensor is sending data every 8 seconds when it is in active mode.   So unless there is an RF problem, it will be a small interval when it wraps around.

Offline wcndave

  • Member
  • *
  • Posts: 46
Re: New Acurite Lightning Detector
« Reply #94 on: September 11, 2018, 06:09:28 PM »
So the delta is done on sensor data received, rather than on db writes?  Ie if it writes to db every 5 minutes, doesn't do the delta calc then...