Author Topic: SD card flash life on Wx programs using Raspberry Pi  (Read 4355 times)

0 Members and 1 Guest are viewing this topic.

Offline Nekota

  • Member
  • *
  • Posts: 4
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #25 on: April 04, 2021, 03:04:37 PM »
I have had a dozen or more RPi's and a fair number of SD card corruptions.  What I found is the card is not worn out but corrupted probably due to power glitch or dropout.  The solutions I have found are to use the low level diskpart on windows pc to recover the SD card and a battery backup power supply for the RPi.  Note that diskpart can erase any of your drives so be very careful to select the SD flash card. (For details Google diskpart usb flash recovery).  For the backup power source I use a 2.5A DC power supply and the Adafruit PowerBoost 1000 charger.  I have one RPi recording temperatures from a heat pump for the past 7 years with a 16G SD card.  It has been corrupted several times but since putting the powerboost backup on the RPi I have not had any more corruptions.  I also have powerboost on two other RPi computers (one is for power level monitoring recording two readings every 6 seconds and a PiAware for Flight Aware monitoring SDR radio) and both have had no SD card problems since I started using the powerboost 1000 backups.  Rather than spend money for SLC I put the $ on getting a larger SD flash card (currently 64G is my preference) and let the SD card space through wear leveling provide the longer life and use battery backup for the RPi.   

Offline davidmc36

  • He who dies with the most toys wins!
  • Forecaster
  • *****
  • Posts: 570
  • FN25ie61jw
    • MorewoodW34
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #26 on: April 04, 2021, 03:12:38 PM »
How is wear leveling taken care of?

Offline Nekota

  • Member
  • *
  • Posts: 4
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #27 on: April 04, 2021, 09:21:22 PM »
How is wear leveling taken care of?

Flash memory devices have an embedded controller which performs the mapping of logical blocks of data to the physical blocks on the chip. This is transparent to the operating system and the controller keeps counts of how many times each physical block has been written to. When data is added or changed requiring a write to the flash memory the controller updates the data and writes to a new location having fewer writes and erases the old location. The old location is now available for a new write and the leveling table is updated to account for the write/erase cycles.  There are several algorithms to implement wear leveling with performance and reliability tradeoffs. Search for dynamic vs static wear leveling for more details. An example link is https://www.atpinc.com/blog/how-SSD-wear-leveling-works

Offline WexfordWx

  • Member
  • *
  • Posts: 20
    • Wexford Harbour Weather Info
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #28 on: April 05, 2021, 10:43:39 AM »
I found this thread to late in the day to save my hair from been torn out bunch by bunch. I have run a meteohub controlled station for about 10 years. Until it packed up I used a Sheeva plug computer; in this one, the SD cards lasted an average of 2-3 years. I had to clear the memory several times with a Linux computer as the memory filled up with tables. I kept an RPi as a backup and when the Meteohub died I started using the RPi3. After about 6 weeks it stopped calculating averages then after 6 or 7 months it stopped working altogether. So I got new SD cards and configured them. Three times I had this experience. Boris was little help, he said I must be using cheap SD cards, which I was not.

I gave up. I bought two WiFi Loggers for my two Davis consoles and now I can rest easy knowing I will not have to contend with formatting and configuring Meteohub SD cards ever again.

I have two Meteohub licenses and two Davis Weatherlinks available for charity or educational purposes if someone wishes to contact me. I will repeat the offer under a new Subject.
« Last Edit: April 05, 2021, 10:45:28 AM by WexfordWx »

Offline DaleReid

  • Forecaster
  • *****
  • Posts: 1716
    • Weather at Eau Claire, WI
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #29 on: April 05, 2021, 02:01:49 PM »
How are the memory areas on the meteohub or dataloggers different than the SD cards?

Do the SDs write often and even if there is no activity to log, whereas the dataloggers/hubs only write when a new data point comes in, every few minutes rather than millions of times per second?
ECWx.info
&
ECWx.info/t/index.php

Offline WexfordWx

  • Member
  • *
  • Posts: 20
    • Wexford Harbour Weather Info
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #30 on: April 05, 2021, 05:44:07 PM »
Dataloggers such as WiFiLogger do not generally use SD cards/flash memory for live data, only for holding the software and parameters needed for its operation. They use R/W memory (RAM) for holding the met data, this is transient but this does not matter as data is only computed and stored for a few minutes, i.e. until the next transmission. R/W memory can be used, erased and reused endlessly without deteriorating.

Offline galfert

  • Global Moderator
  • Forecaster
  • *****
  • Posts: 6605
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #31 on: April 05, 2021, 06:06:47 PM »
Boris was little help, he said I must be using cheap SD cards, which I was not.
If you were not using SLC type SD cards then you were using cheap SD cards. You say you weren't using cheap cards but you are not specifying what you were using. Even if it says industrial or long life or even if they say that they are designed for dashcams...not good enough unless it is SLC. Boris may have said you were using cheap SD cards but what he really meant by cheap is non-SLC type. The latest Meteobridge has a requirement of using SLC type cards and there is a link to acceptable SLC type cards.

Here is that list of SLC supported card. Yes this is for the Meteobrige but this is still a good list to go by to know that you are using SLC.
https://www.meteobridge.com/wiki/index.php/Hardware#Raspberry_Pi_Model_3B.2C_3B.2B.2C_4B

Quote
I gave up. I bought two WiFi Loggers for my two Davis consoles and now I can rest easy knowing I will not have to contend with formatting and configuring Meteohub SD cards ever again.
WiFiLogger has no archive storage....it only uploads data to other places.  You aren't comparing apples to apples in terms of capabilities. I'm sure WiFiLogger is great for many users, just know that different devices have different uses. Boris recommends those using Meteohub to upgrade to Meteobridge. He even ran a special upgrade pricing a few months back.
 
Ecowitt GW1000 | Meteobridge on Raspberry Pi
WU: KFLWINTE111  |  PWSweather: KFLWINTE111
CWOP: FW3708  |  AWEKAS: 14814
Windy: pws-f075acbe
Weather Underground Issue Tracking
Tele-Pole

Offline WexfordWx

  • Member
  • *
  • Posts: 20
    • Wexford Harbour Weather Info
Re: SD card flash life on Wx programs using Raspberry Pi
« Reply #32 on: April 05, 2021, 06:46:18 PM »
I stand corrected! Boris did recommend using a swissbit card in my RPi Meteohub alright and I spent hours trying to find an affordable 16GB microSD. Unfortunately the Meteohub advice is to use a much larger card than is apparently needed in the Meteobridge. I could not find one for under 100 and even then I would have to have waited 6 weeks for delivery. I was not aware of the importance of using SLC type, I just looked for industrial grade. Perhaps I was using fakes, I even downloaded software to try and avoid these.

Your point regarding the WiFiLogger is well made. Horses for courses! Regarding the Meteohub versus the WiFiLogger, the latter is suitable for my purposes as I do not use or need the tables generated by the Meteohub, I take raw data and calculate my own.
« Last Edit: April 05, 2021, 06:48:09 PM by WexfordWx »