Topic: WS80 Solar Data Reporting Time Intervals with GW1000+Ecowitt.net vs HP2553BC

[solartempest]

It took a couple weeks, but I finally found the right dataset to show how Ecowitt.net is logging my WS80 solar data versus HP2553BC microSD backups over the different time intervals. How the Ecowitt servers are saving the data has been discussed in other threads for wind data/gusts (with interesting oddities), but I hadn't seen anything for solar data.

Summary: My GW1000 uploads to Ecowitt.net every 1 minute. If you export a single day's readings, the server is providing the data in 5 minute intervals. Only the peak (highest) solar measurement value is saved/provided from the 0:00 to 4:59 of each 5 minute interval. If you export a month's worth of data, the server provides it in 4 hour intervals. In a similar fashion, the peak reading from 00:00:00 to 03:59:59 of each 4 hour interval are reported.

It's a little frustrating for me as a plant enthusiast/collector because the peak solar value is not very useful as compared to the average value over each reporting interval. When using light readings to optimize plant growth and care, the area under the curve is more important than the instantaneous values. That is, the total light collected in a day is analogous to how much water a plant receives in a day being more important than highest instantaneous the rainfall rate.

How off is the reporting with this peak-value-only method? With a couple weeks of data, at 1 minute intervals with my HP2553BC they are off by -5% to +10%. At 5 minute intervals it is +5% to 30%. At 4 hour intervals it's +110% to +275%. I highly doubt there is anyway to change this behaviour, but some sort of user setting would be really beneficial. Maybe for the other sensors using the peak value makes sense but I don't think it does for light measurements.

How did I go about figuring this out? I compared the readings between my Apogee SQ-520 PAR DLI quantum sensor (±5% calibrated with good linearity) and my WS80 lux sensor (±15% and no other info). My GW1000 is set to transmit data at 1 minute intervals to Ecowitt.net. The daily data report is always provided back by the servers with 5 minute time intervals. The monthly data report is provided with 4 hour intervals.

My SQ-520 was set to average light readings over 1 minute and log that data every 1 minute. This was my "true" baseline as it is a scientific calibrated sensor with field settings that I have full control over. Then I set my HP2553BC to log my WS80 data every 1 minute and backed that up to microSD manually. Comparing those two sets of data, they were fairly close.

The % difference for area under the curve was different for all sensors on different days/lighting conditions so that drove me to figure out what was happening. Both the SQ-520 and WS80 are mounted within 3ft from each other and ~12m from the ground. It took many weeks to set up the sensors properly: physical location for consistent data each days, data logging settings, aggregating data, plotting results, and then collecting more data for different weather conditions.

Other WS80 lux sensor topics I'm still investigating are non-linearity and response rate under changing conditions. So far, I have not been looking into water droplet magnification because I coated the top of my WS80 with a superhydrophic silica coating. That's proven to be really helpful in keeping water droplets away and keeping my data consistent.

The graphs below show my solar data and how the Ecowitt.net is saving the peak values for each reporting period only. I have a lot of data, these particular days just are good examples to show how it is done.

 [ You are not allowed to view attachments ]
 [ You are not allowed to view attachments ]
 [ You are not allowed to view attachments ]

[Old Salt]

How did I go about figuring this out? I compared the readings between my Apogee SQ-520 PAR DLI quantum sensor (±5% calibrated with good linearity) and my WS80 lux sensor (±15% and no other info).

This could be comparing apples to oranges.  We know the SQ-520 is designed to accurately measure the aggregate intensity of the wavelengths from 380 to 700nm, but don't have this information for the Ecowitt sensor.  Different wavelengths are absorbed at different rates as they pass through our atmosphere.  This absorption also varies with the season and time of day as the distance the solar radiation travels through the atmosphere changes.  I would expect the differences between the sensors.  You would need to account for this if you wish to calibrate the Ecowitt.

Those that are using the Ecowitt to determine the economic viability of solar power will also have to compensate for the angle of the solar panels.  The angle of incidence of the solar radiation greatly affects the power output.

[solartempest]

This could be comparing apples to oranges.  We know the SQ-520 is designed to accurately measure the aggregate intensity of the wavelengths from 380 to 700nm, but don't have this information for the Ecowitt sensor.

A keen observation, however the exact same argument applies for WS80 sensor itself. It is a lux sensor and not a pyranometer or UV meter. This is identified in the product manual which states "The default conversion factor based on the wavelength for bright sunlight is 126.7 lux / w/m2. This variable can be adjusted by photovoltaic experts based on the light wavelength of interest, but for most weather station owners, is accurate for typical applications, such as calculating evapotransporation and solar panel efficiency.". This is further discussed in another thread: https://www.wxforum.net/index.php?topic=39119.0.

Overall, there are some scientific discussions that have placed general conversion factors for lux to PAR. I'm hoping to achieve a conversion factor well within the stated ±15% accuracy of the WS80 light sensor (honestly that's a huge range). The main benefit of the WS80 sensor in conjunction with the GW1000/HP2553BC is that the data is regularly updated live and easy to download/manage remotely even though it is less accurate. To do the same with calibrated Apogee sensors would probably involve $1000 worth of new equipment and still be a less convenient solution.

Different wavelengths are absorbed at different rates as they pass through our atmosphere.  This absorption also varies with the season and time of day as the distance the solar radiation travels through the atmosphere changes.  I would expect the differences between the sensors.  You would need to account for this if you wish to calibrate the Ecowitt.

Yes, this is why I am comparing the minute-by-minute data between the two sensors over entire days under different conditions. I plan to determine my own conversion factor across the seasons at my specific location. The overall goal is to have a conversion factor for my Ecowitt data, to a reasonable degree if possible, with the scientific calibrated tool that I own. At this time, purchasing a dedicated calibrated pyranometer 360 to 1120nm (rather than 380-700nm) is not planned due to cost reasons as stated above.

Unfortunately due to COVID-19 and other supporting installation issues, I wasn't able to get the Ecowitt heater cord for my WS80/outdoor power supply for the sensor at its current location but I plan to move the unit to a difference place, still around 30ft off the ground, for the winter. It will have almost 100% unobscured view of the sky but the clear protective layer above the sensor (to prevent icing) may reduce light readings by around 15% (as measured by my handheld lux meter). That's the compromise of safety in cold places I guess!

[Mandrake]

This is a fascinating topic and has been touched on lightly before and it has long been recognised that the Lux sensor is less than ideal and only provides a rough estimation of UV values.
This has been recognised by Ecowitt and they will be looking at a enhanced product at some point for the community that uses a real UV sensor.

However like all these things, no promises....although their track record of converting requests into real products to buy is exceptionally good so hang in there!

[solartempest]

Upon further investigation of the WS80 and lux sensor, it seems quite a bit different than what I had expected. Purely looking at the spectral response, comparison between the SQ-520 seemed impossible but might not be so bad actually. The WS80 sensor is likely optimized for accurate lux measurements and not W/m2 at all (depending on sensor chip settings). So if you are looking for the "real" value measured by the sensor, then the most accurate setting is lux (native unit).

The W/m2 conversion is a bit glossed-over in the manual, but a simple explanation is provided on the ambient weather website: https://help.ambientweather.net/help/why-is-the-lux-to-w-m-2-conversion-factor-126-7/ . In brief, the conversion factor is chosen for the spectra of bright sunlight only (as I mentioned previously).

Regarding Old Salt's comment for "apples and oranges" comparison, my understanding is that the WS80 sensor should have infrared compensation built in, so the actually changes in various lighting conditions/seasons/light angle may actually be minimized. This aligns with my data observations so far over various weather and lighting conditions.

The main exception is extremely bright, rapidly changing light which I am still investigating. This is where the biggest difference in the two sensors lies - the UV spectrum. The WS80 visible light sensor has some sensitivity there and my SQ-820 does not. I did some testing with a 395nm UV LED source and the WS80 sensor did pick up the light (reported as upwards of 77w/m2 for 3W source at 1" distance).

As I pointed out earlier, if the main sensor of the WS80 is optimized and integrated purely for lux, then the impact of the UV spectrum should be minimized by the manufacturer for those readings as it is invisible to humans. Being able to pick up UV light, even with such a low spectral response, really seems counterproductive in a lux sensor so that might just be a filtering limitation? It's curious. I need to dig through my data to try and see if I can learn more on why this range is not compensated for by the sensor chip (at least in my limited test so far).

This is a fascinating topic and has been touched on lightly before and it has long been recognised that the Lux sensor is less than ideal and only provides a rough estimation of UV values.
This has been recognised by Ecowitt and they will be looking at a enhanced product at some point for the community that uses a real UV sensor.

I think we might not be giving Ecowitt enough credit here on the UV readings from the WS80, even though the statement is true. My understanding is that the sensor is actually designed to provide these values, although the reported UV index is most likely based on a lookup table or calibration curve of sorts which correlates sunlight to typical UV readings (ideal or measured by other means). In other words, the sensor in the WS80 was chosen and designed specifically for this purpose and the UV index was not an afterthought.

One of the challenges is likely higher package cost of the UV sensor, along with the technical challenges associated with integrating it. To obtain the correct response, the engineers at Ecowitt would need to be very careful with the housing material, installation tolerance, and diffuser design.

For these reasons, I have a better understanding why the WS80/(my HP2553BC) manual states UV calibration should be completed frequently and that intense sunlight may alter the results. If the current clear housing is PMMA or similar material then it will exhibit photo-oxidation over time and affect the sensor accuracy. UV calibration will compensate for that but appears to be a challenging exercise.

It's not so simple as putting in a different sensor in the same or similar housing. Many physical and software design choices are not so easy to describe or explain in just a few words. Coming up with a calibration procedure is also quite difficult... even though it's mentioned so simply in the manual. (Honestly I was hoping for a simple answer, but in my work in commissioning industrial systems even the simple tests take a lot of care and thought. Just figures this is the same.)

[solartempest]

Regarding Old Salt's comment for "apples and oranges" comparison, my understanding is that the WS80 sensor should have infrared compensation built in, so the actually changes in various lighting conditions/seasons/light angle may actually be minimized. This aligns with my data observations so far over various weather and lighting conditions.

Small update: I performed a test using a 750nm IR filter and have verified that the WS80 lux sensor does have infrared compensation enabled.

Today's test condition weather is heavy cloud with some rain which should block our the vast majority of UV light. The 750nm IR filter has a very sharp cutoff, allowing only near-infrared and longer wavelengths to pass through to the WS80 lux sensor. With the 750nm filter covering the WS80 sensor, sunlight light readings went from 18klux immediately to 0lux.

I used my camera with near-infrared capability to verify that there was plenty of ambient IR light available going through the clouds. If I get lucky with the weather, maybe I can get some clear skies this week to repeat the test at peak sun times (to be comprehensive although I do not expect different results).

Today I also found some good technical documentation regarding UV intensity changes due to seasons as well as throughout a day. Still digesting that bit though.

 [ You are not allowed to view attachments ]

[solartempest]

Next update:While I know the WS80 lux sensor does not have a dedicated UV-only sensor, I wanted to check and the sensitivity of it to UV light. This is to gain a little insight on the potential impact of variations in UV light over throughout day/seasons.

I performed two UV tests, one at night and one during the day to assess how the WS80 lux sensor responds. For both tests, I used a 3W 395nm UV LED light approximately 1" away from the sensor. The light was held in place for 2 minutes to ensure a full period of data was recorded (although from my previous posts I know only the peak value is reported by the WS80/GW1000).

Results: Since the WS80 lux sensor is designed to match the photopic curve (human vision / CIE 1931), we expect the impact of ultraviolet light to be minimized in its readings. The sensor's spectral response does include some UV light, as shown in the Night Test. However, in the Day Test just holding the LED light over the sensor blocked more ambient sunlight (despite being a cloudy day) than it contributed in the UV spectrum.

Overall, what this demonstrates is that UV spectral response is not heavily weighted by the WS80 lux sensor. So while there will be some daytime/seasonal shifts in accuracy due to different amounts of UV light, it should not skew the lux readings heavily.

Side Note: I didn't do a w/m2 graph because that would need me to convert/redownload and match up the data all over again versus using DLI or fc like I've already been doing. Not that it isn't impossible, just a lot of work for pretty much the same conclusion.

Further Reference: I've included two excerpts detailing the variations in UV light throughout the day and seasons below. The readings were taken by Q-Lab in Cleveland, which does accelerated weather and corrosion testing. I don't know how good this lab is, but I have done accelerated aging studies and test evaluations for my work (which is a very complex and interesting specialty subject). Full reading here: https://www.q-lab.com/documents/public/cd131122-c252-4142-86ce-5ba366a12759.pdf

 [ You are not allowed to view attachments ]
 [ You are not allowed to view attachments ]