Topic: Blake Larsen Sun Recorder to CMx

[davidmc36]

So I just got my BL Sun Recorder running again and want to get the data into CMx. Or even additionally post the graph images it produces to my webpage.

I have CMx running on a RPi Zero with a Meteo HAT.

The App for the Sun Recorder is currently on a W10 laptop.

I wonder if sending to an FTP server on the RPi, the .dat file from the recorder app, would be the easiest way to go. I was thinking of setting up CMx 4 on the laptop, but converting the CMx 3 files from the RPi does not seem like a very straightforward process.

[PaulMy]

Hi David,
It has been years since I last had the B-L running and I think the process is still the same.  At that time there was a setting in SR to create a SRsunshine.dat file and save it in Cumulus\data folder at a 1-minute interval.

In CumulusMX settings > Extra Sensors > enable

Code: [Select]

Blake-Larsen Sunshine recorder
Use a Blake-Larsen sunshine recorder for sunshine hours
 Use an attached Blake-Larsen sun recorder to determine hours of sunshine - reads the SRsunshine.dat fileand CMX will read the file every 1-minute for the ongoing sunshine hours and logging.

The Charts tab > Daily Charts will then display the sunshine hours, and also uploaded to the standard CumulusMX website if you have that.

The SR graph files on your computer can be uploaded to your website and displayed how and where you want them to display.

I was thinking of setting up CMx 4 on the laptop, but converting the CMx 3 files from the RPi does not seem like a very straightforward process.

Should be relative easy, follow steps 1 through 9 https://cumulus.hosiene.co.uk/viewtopic.php?t=22051

Code: [Select]

A basic workflow to run the utility:
1. Start with a clean install v4
2. Copy your v3 Cumulus.ini and strings.ini files to root of v4
3. Copy your v3 /data and /Reports folders to the v4 install
4. Rename the copied /data folder to /datav3
5. From the command prompt, change the path to your v4 /CumulusMX folder
6. Run MigrateData3to4
Note: If you have any Custom Daily log files, you must supply these file names to the migration utility as parameters as they can have any free-format file name. Note: you do not type the square brackets, they just indicate optional parameters.
Windows:
Code: Select all

MigrateData3to4.exe
Linux:
Code: Select all

dotnet MigrateData3to4.dll
7. Follow the prompts and look for any errors - the utility also outputs a log file to the /MXdiags folder
8. If you use PHP upload, check the upload.php script on your web server is up to date with the v4 zip file versions
9. Done!
Enjoy,
Paul

[davidmc36]

I read a thread where it stated you now just put the .dat file in main CumulusMX folder. I think that does seem easy if both apps are on the PC.

But if my CMx is on the Pi, I only need those two files copied to PC install to transfer all records etc.????

[PaulMy]

Hi David,
In my case in the past the SRsunshine.dat was in the \Cumulus\data folder but that is easy to determine and establish if there was a way from one PC to another.   And maybe there is....

Could you use CMX Dashboard >  Settings > HTTP Files settings?  I had used it just for testing when this was added to CMX by Mark a few years ago and that worked great to get a file from another user's website download to my PC.

Code: [Select]

HTTP File Settings
HTTP Files
Enabled URL Destination Filename Download Interval Upload Enabled Timed Downloads Time of first download of the day Actions

https://some.site.com/someimage.png web/someimage.png 1 min 00:00 Save Settings
URL: This the url used to access your source file. This can be either a URL or <ecowittcameraurl>, in which case the latest Ecowitt web cam image will be downloaded (if available). URLs can contain web tags

Destination Filename: This the path and file name where you want your downloaded file to be saved.
This is either a path on your web server, or a local path where Cumulus is running - the Upload Enabled option controls which.

Download Interval: This a multiple of your web interval, or at the End of Day rollover
For example, if your web interval is 5 minutes and you select "x12", then the download will occur once an hour.

Upload Enabled: This controls if the file is just downloaded to the local file system, or it is then uploaded to your web site.

Timed Downloads: If this is disabled, then the file will be downloaded at the set interval, starting whenever Cumulus is started.
If you enable this option, then the downloads will start at First Download and repeat each interval

First Download: If timed uploads are enabled, then the first download of the day will occur at this time and repeat at each interval until the end of the day
The difficulty I see is, what is the remote URL for the source file?

Enjoy,
Paul

[davidmc36]

......
The difficulty I see is, what is the remote URL for the source file?

Enjoy,
Paul

I wonder if it could be as simple as IP address of the PC "slash" directory of the file? Have CMx fetch it there??

[davidmc36]

Well it works just fine with CMx 4 and Sunshine recorder hosted on the PC.

I put: http://192.168.5.88:8998/C:\Users\david\Desktop\Sun Recorder\SRsunshine.dat  for the URL to fetch.

I put: C:\Users\david\Downloads\CumulusMXDist4088\CumulusMX\SRsunshine.dat   for the destination.

CMx fetches the .dat file.

I don't see all the options to do this in the CMx 3 on RPi.

[davidmc36]

Well Holy Howdy Doody!

It seemed that the experiment with having CMx4 fetch the .dat file within the PC ended up with a blank file. Showed updated time-stamp always, but always empty.

So I started hunting around for some automation that may FTP the file to the CMx3 on RPi. Windows Task Scheduler running a script appeared on the radar.

So. WinSCP has a feature to generate a script to transfer files. Good 'ol Copy/Paste comes in to play. I fire up PowerShell ISE in Administrator mode and run the .ps1 file.

POOF! File is in the CMX folder on the Pi.

I know it is using the file because when I ticked the BL Sun option in CMx, the hours today changed from the 1.5 calculated to 1.1 of the Sun Recorder.

The last hurdle I suppose will be to have it run every 5 minutes initiated by Task Scheduler. The permission thing may be tricky yet for an automated task?????

[mcrossley]

Well it works just fine with CMx 4 and Sunshine recorder hosted on the PC.

I put: http://192.168.5.88:8998/C:\Users\david\Desktop\Sun Recorder\SRsunshine.dat  for the URL to fetch.

I put: C:\Users\david\Downloads\CumulusMXDist4088\CumulusMX\SRsunshine.dat   for the destination.

CMx fetches the .dat file.

It would be simpler to use the Extra Web Files feature. Just specify...
Local file = C:\Users\david\Desktop\Sun Recorder\SRsunshine.dat
Destination Filename = C:\Users\david\Downloads\CumulusMXDist4088\CumulusMX\SRsunshine.dat

Do NOT tick "upload" so MX performs a local copy.
Optionally tick "Realtime" for very frequent copies at your realtime interval

@davidmc36 can you map a drive (mount in Linux parlence) to your PC from the rPi? If so you could use the same mechanism to perform the copy.

Example of mounting a Windows shared drive on a rPi...
sudo mount.cifs //192.168.0.3/d /home/pi/winshare/d -o user=myusername,password=mypassword

[davidmc36]

IC, Web Files. If I need it on PC I will set that up instead of URL.

My struggle now is automation. I can run script with PS or PS ISE in admin mode and all is well. I need it to run auto from Task Scheduler or whatever can initiate it every few minutes.

[davidmc36]

... automation.  ...

This is my script.

Code: [Select]

# Needed for PowerShell 7.3 and newer
$PSNativeCommandArgumentPassing = "Legacy"

& "C:\Program Files (x86)\WinSCP\WinSCP.com" `
  /log="C:\Program Files (x86)\WinSCP\WinSCP.log" /ini=nul `
  /command `
    "open sftp://pi:5292@192.168.5.249/ -hostkey=`"`"ssh-ed25519 255 Tu1usm0aAdVpVLAhtV/Nlg1Xo7OXQkQxVhxOxiiN08Q`"`"" `
    "lcd C:\Users\david\Desktop\SunRecorder" `
    "cd /home/pi/CumulusMX" `
    "put SRsunshine.dat" `
    "exit"

$winscpResult = $LastExitCode
if ($winscpResult -eq 0)
{
  Write-Host "Success"
}
else
{
  Write-Host "Error"
}

exit $winscpResult
So instead of exit can it wait a couple minutes and start over?   Loop?

[davidmc36]

I made a shortcut that will start in Administrator Mode but cannot get Task Scheduler to start it.

[mcrossley]

If you run the script as system so you do not have to specify a username/password...

Code: [Select]

$TaskName="Copy SR File"
$Trigger = New-ScheduledTaskTrigger -AtStartup
$Trigger.Repetition = (New-ScheduledTaskTrigger -once -at "12am" -RepetitionInterval (New-TimeSpan -Minutes 10) -RepetitionDuration (New-TimeSpan -Minutes 10)).repetition
$User= "NT AUTHORITY\SYSTEM"
$Action= New-ScheduledTaskAction -Execute "PowerShell.exe" -Argument "-ExecutionPolicy Bypass -NonInteractive -WindowStyle Hidden -File C:\<PATH_TO_MY_SCRIPT>\<MY_SCRIPT_NAME>.ps1"
Register-ScheduledTask -TaskName $TaskName -Trigger $Trigger -User $User -Action $Action -RunLevel Highest -Force

[davidmc36]

Right on. I stumbled upon some info leading me to -ExecutionPolicy Bypass

What about putting my code into a while loop that is always true, including say a 5 minute wait before starting at the top again?

[92merc]

When I did it, it exported my processes to a plain text file.  I then used Task Scheduler to run:

"C:\Program Files (x86)\WinSCP\WinSCP.exe" /log="c:\zip\dayfile.log" /ini=nul /script="C:\ZIP\dayfile_upload.txt"

This is what I did to generate a txt file.
https://winscp.net/eng/docs/guide_automation

That should bypass all the PS bullscript.

[mcrossley]

But that also requires you to run an FTP server on the destination host.

[davidmc36]

But that also requires you to run an FTP server on the destination host.

As far as FTP, the Pi accepts SFTP connections natively when SSH is enabled. That's how I have been doing it so far. I am using WinSCP. I used it to generate the code I posted above.

[PaulMy]

Hi David,
Sorry if I am requesting something that has already been posted... are the SRsunshine.dat and the 3 graph files created and saved on the PC that is running the Sunrecorder?  Can these files be FTP to your website from the PC?

If so and those files are on a website then you can use the CMX Dashboard HTTP Files settings to download the SRsunshine.dat file to your RPi with CumulusMX.
I have just tried it to take an old remote file https://www.komokaweather.com/weather/sunrecorder/Days30.png and download as Destination Filename c:\KomokaWeather\cumulusmxwll\Days30.png

2025-04-13 11:59:00.293 ProcessHttpFiles: Creating local http files
2025-04-13 11:59:00.293 ProcessHttpFiles: Done creating local http file tasks, not waiting for them to complete
2025-04-13 11:59:00.293 DownloadHttpFile: Downloading from https://www.komokaweather.com/weather/sunrecorder/Days30.png to c:\KomokaWeather\cumulusmxwll\Days30.png

The Days30.png file is now in my \local CumulusMX folder.
The download interval can be 1, 2, 5, 10, etc. minutes to 24hr.

I don't have current SRsunshine.dat file but that should work the same way.
 
Enjoy,
Paul

[davidmc36]

When I did it, it exported my processes to a plain text file.  I then used Task Scheduler to run:

"C:\Program Files (x86)\WinSCP\WinSCP.exe" /log="c:\zip\dayfile.log" /ini=nul /script="C:\ZIP\dayfile_upload.txt"

This is what I did to generate a txt file.
https://winscp.net/eng/docs/guide_automation

That should bypass all the PS bullscript.

Generation and manual transfer are not a problem.

I want it to run every few minutes or so automatically.

[davidmc36]

Hi David,
Sorry if I am requesting something that has already been posted... are the SRsunshine.dat and the 3 graph files created and saved on the PC that is running the Sunrecorder?  Can these files be FTP to your website from the PC?

If so and those files are on a website then you can use the CMX Dashboard HTTP Files settings to download the SRsunshine.dat file to your RPi with CumulusMX.
I have just tried it to take an old remote file https://www.komokaweather.com/weather/sunrecorder/Days30.png and download as Destination Filename c:\KomokaWeather\cumulusmxwll\Days30.png

2025-04-13 11:59:00.293 ProcessHttpFiles: Creating local http files
2025-04-13 11:59:00.293 ProcessHttpFiles: Done creating local http file tasks, not waiting for them to complete
2025-04-13 11:59:00.293 DownloadHttpFile: Downloading from https://www.komokaweather.com/weather/sunrecorder/Days30.png to c:\KomokaWeather\cumulusmxwll\Days30.png

The download interval can be 1, 2, 5, 10, etc. to 24hr minutes.

Enjoy,
Paul

I can SFTP directly to the required directory on the Pi from my PC. I never tried, but am sure the .png files could be also.

Getting the file moved anywhere on a schedule is the missing piece.

[PaulMy]

Getting the file moved anywhere on a schedule is the missing piece.

Does the CMX HTTP Files 1-minute interval download not do that?

Enjoy,
Paul

[PaulMy]

Getting the file moved anywhere on a schedule is the missing piece.

Run SR and CMX on the same PC

Enjoy,
Paul

[davidmc36]

Getting the file moved anywhere on a schedule is the missing piece.

Run SR and CMX on the same PC

Enjoy,
Paul

Getting the file moved anywhere on a schedule is the missing piece.

Does the CMX HTTP Files 1-minute interval download not do that?

Enjoy,
Paul

When I tried out CMx4 on same PC with SR it fetched a blank file. I was doing URL though and maybe Web Files will work OK. If necessary, I will look to put CMx4 and hope to get my records from Pi CMx3.

The ultimate goal though would be WinARM on Pi with everth housed there. I will need to get a Pi4 for that, zero is too slow to work on the desktop.

[davidmc36]

Gyvate mentioned Win11ARM in a PM. Is there any reason one would NOT buy a Pi 5? The 4 is cheap as chips but.......

I would SO like to host one device, headless, non-PC.

[davidmc36]

I will have to try importing my current data to CMx4 on the PC and see if that works at all......then pick up another Pi.

It's only a secondary webpage to my main site but it would be nice to have a working, fully functional, back-up.

[davidmc36]

So I did enough searching and putting info together that I figured out how to run the script in a loop with a defined pause. I send the .dat file to the Pi every 10 minutes.

I ordered up a Pi5 anyway. I want to take  swing at WinARM.