Author Topic: How do i edit data?  (Read 3180 times)

0 Members and 1 Guest are viewing this topic.

Offline K6SGH

  • Member
  • *
  • Posts: 40
    • K6SGH
How do i edit data?
« on: February 19, 2010, 09:05:32 PM »
Is there a simple way to change the total rainfall to fix an error?  Is the only way to do it is to go to settings/database settings and bring up each 15 minute record to fix the bad datapoint?

example: my rainfall for the 18th of feb shows .25 at 3pm. there was no rainfall then.

so if i bring up the database settings, i can find the 15 minute period when the error occured, how do i change that record so it goes forward? there are over 156 individual records since that point in time and a new one is created every 15 minutes continuing the error. 

can't i just bring up the summary data for the day and change it?  yes, i tried that using settings/recall daily, monthly... and edited that day and each successive day but the summary data on VMS still shows the wrong amount.

it would seem being able to quickly correct a simple error like this should take a few seconds...

help..

steve

Offline sam2004gp

  • Mount Crawford, Virginia
  • Forecaster
  • *****
  • Posts: 2865
  • Weeeeeeeee!!!!
    • Mount Crawford Weather, VA
Re: How do i edit data?
« Reply #1 on: February 19, 2010, 09:09:48 PM »
I am at work, and can not get to my VWS computer at the moment.

But if memory serves me right........

Go to settings>calibration>At Total Rainfall, subtract or enter in a negative number in the amount that you want to remove.  Then go to settings Rainfall> and adjust and remove the same amount for the month and the day on which the error occurred. 

.........If I am wrong guys chime in here and set me right, so this fellow can get through this pain in the butt process.  I won't get a chance to get back on my VWS computer until tomorrow mid day.
« Last Edit: February 19, 2010, 09:11:36 PM by sam2004gp »
SAM --->>> http://www.mountcrawfordweather.org
OS WMR-968 with a Dedicated PWS Weather Computer running VWS v13.01 p09


Offline skysummit

  • Forecaster
  • *****
  • Posts: 806
    • StormCast
Re: How do i edit data?
« Reply #2 on: February 19, 2010, 10:07:12 PM »
I am at work, and can not get to my VWS computer at the moment.

But if memory serves me right........

Go to settings>calibration>At Total Rainfall, subtract or enter in a negative number in the amount that you want to remove.  Then go to settings Rainfall> and adjust and remove the same amount for the month and the day on which the error occurred. 

.........If I am wrong guys chime in here and set me right, so this fellow can get through this pain in the butt process.  I won't get a chance to get back on my VWS computer until tomorrow mid day.

Would that bold part be under Gain or Offset?
Hardware:  Acurite Atlas with Lightning Detector
CoCoRaHs Station: LA-TG-11

NWS's Ponchatoula/Hammond Area's COOP Member
President | Director of Operations - StormCast

Offline K6SGH

  • Member
  • *
  • Posts: 40
    • K6SGH
Re: How do i edit data?
« Reply #3 on: February 20, 2010, 09:34:28 AM »
I attempted to edit the data in the dbase.csv file as there doesn't seem to be a SIMPLE way to either edit or remove data that is incorrect.

I found the columns that had the bad data, removed it by setting those values to zero and adjusting the total rainfall numbers sequentially down to correct the error.  However, when i saved the file it somehow changed the byte size and apparently some formatting because the edited dbase.csv file would not load into VMS.  I tried several more attempts including importing the data into a paradox (dos based) database (of which i am an expert user) but couldn't duplicate the actual format vms needs.

In the Davis Weatherlink software, its EASY to fix a bad data point by simply finding it in the database and changing it as all reports in weatherlink are generated from a SINGLE database.  Apparently, VMS has several non-linked databases and this is the source of the problem i think.

I think it is virtually impossible to take out a bad datapoint that other calculations are dependent on.  In the rain problem i am having, the bad data STARTS at a single 15 minute period.  One should be able to remove that single datapoint, or edit it to the proper value, then ALL OTHER values that are dependent on that data would be automatically recalculated and fixed.  But i don't think vms is structured this way and hence once you get bad data one must use all sorts of convoluted ways to fix it while not actually going to the source of the problem.

Offline sam2004gp

  • Mount Crawford, Virginia
  • Forecaster
  • *****
  • Posts: 2865
  • Weeeeeeeee!!!!
    • Mount Crawford Weather, VA
Re: How do i edit data?
« Reply #4 on: February 20, 2010, 10:39:29 AM »
I am at work, and can not get to my VWS computer at the moment.

But if memory serves me right........

Go to settings>calibration>At Total Rainfall, subtract or enter in a negative number in the amount that you want to remove.  Then go to settings Rainfall> and adjust and remove the same amount for the month and the day on which the error occurred. 

.........If I am wrong guys chime in here and set me right, so this fellow can get through this pain in the butt process.  I won't get a chance to get back on my VWS computer until tomorrow mid day.

Would that bold part be under Gain or Offset?

That Would be Gain.
SAM --->>> http://www.mountcrawfordweather.org
OS WMR-968 with a Dedicated PWS Weather Computer running VWS v13.01 p09


Offline sam2004gp

  • Mount Crawford, Virginia
  • Forecaster
  • *****
  • Posts: 2865
  • Weeeeeeeee!!!!
    • Mount Crawford Weather, VA
Re: How do i edit data?
« Reply #5 on: February 20, 2010, 10:54:10 AM »
I attempted to edit the data in the dbase.csv file as there doesn't seem to be a SIMPLE way to either edit or remove data that is incorrect.

I found the columns that had the bad data, removed it by setting those values to zero and adjusting the total rainfall numbers sequentially down to correct the error.  However, when i saved the file it somehow changed the byte size and apparently some formatting because the edited dbase.csv file would not load into VMS.  I tried several more attempts including importing the data into a paradox (dos based) database (of which i am an expert user) but couldn't duplicate the actual format vms needs.

In the Davis Weatherlink software, its EASY to fix a bad data point by simply finding it in the database and changing it as all reports in weatherlink are generated from a SINGLE database.  Apparently, VMS has several non-linked databases and this is the source of the problem i think.

I think it is virtually impossible to take out a bad datapoint that other calculations are dependent on.  In the rain problem i am having, the bad data STARTS at a single 15 minute period.  One should be able to remove that single datapoint, or edit it to the proper value, then ALL OTHER values that are dependent on that data would be automatically recalculated and fixed.  But i don't think vms is structured this way and hence once you get bad data one must use all sorts of convoluted ways to fix it while not actually going to the source of the problem.

Yes it is difficult to edit dbase.csv file.  ](*,) There is a program that will allow you to do this, but I have not tried it.  But basically it is best not to mess with that file, and use the "convoluted" ways to edit out the data.

........In addition to what I posted last night.  You also need to go to "recall daily, monthy and yearly archive" and edit out the bad rain values.  This will keep them from appearing in any future reports you may output. 

There is a individual file for the Year, a individual file for the Month,  and a individual file for the Day (that the error occurred).   They are .bin files that you will be looking for.  I also recommend that you backup those files and/or VWS as a whole, before you start tinkering with them.   
IE: If your rain error occurred on Feb 14, 2010, then you will need to edit  the 2010.Bin, 2010_02.bin, 2010_02_14.bin files. 
Once you start to edit them you need to reduce the "total rain max" value to the correct amount (current amount minus your correction) for those three files.

Here is another older thread on the subject that I authored.
http://www.wxforum.net/index.php?topic=2726.0
SAM --->>> http://www.mountcrawfordweather.org
OS WMR-968 with a Dedicated PWS Weather Computer running VWS v13.01 p09


Offline K6SGH

  • Member
  • *
  • Posts: 40
    • K6SGH
Re: How do i edit data?
« Reply #6 on: February 21, 2010, 09:35:17 AM »
Thanks Sam for your comments and help.  I did see your prior posting about the.bin files and that was my first attempt at fixing the bad data.  This fixed the data that shows up in the climate reports but the totals that show on my screen for total rain still have the bad data and it appears i'm just going to have to live with it.

Many thanks.

steve

Offline tomcj2

  • Forecaster
  • *****
  • Posts: 848
    • CanbyWeather
Re: How do i edit data?
« Reply #7 on: February 21, 2010, 11:44:38 AM »
Steve,

You mentioned WeatherLink in a previous post so I assume you have  a Davis.  If so, enter the correct yearly rain in WL, setup>set yearly rain, I just tried it with mine, and it adjusted only the yearly rain in WL, and VWS, it did not change the monthly rain amount.  ( Cumulus did show the higher amount in montly, and daily rain amounts)

Davis VP2 (6163), WL 5.9.0..  VWS 14.01 p25, Panasonic HM371A camera. WU & W4U KORCANBY3, CoCoRaHS OR-CC-27