Author Topic: About to FLING FLING...  (Read 5595 times)

0 Members and 1 Guest are viewing this topic.

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
About to FLING FLING...
« on: August 06, 2016, 08:04:56 PM »
No, that's not a typo. I've about had it with FLING. Since upgrading to Win10 and 1and1's simultaneous move to force everyone to SFTP, I have not been able to update my website files with FLING as I had in the past.

The symptoms are that I can never get a connection to the server to even begin the upload. The connection just times out and the transfer aborts. Filezilla works just fine and I can access my website without any issues it's just that FLING won't connect. FLING works fine on all of my other websites that don't require SFTP connections. 1and1 Support has been totally useless and unable to help. To them, everything is working fine.

I have attempted to use every combination of options available on the FLING setup for SFTP (implied and explicit) to no avail.

Does anyone have any suggestions that I can try? I have the latest version of FLING (2.35). Is there some other software I could use that has similar functionality of FLING?
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: About to FLING FLING...
« Reply #1 on: August 06, 2016, 09:47:00 PM »
I'm running 2.35 also.

I had to change the properties on 'fling.exe' to use 'Windows 7' compatibility mode to get it to connect successfully :)

However.. if you do that, it may also engage the memory-munching non-pageable memory ..  I have to run a small perl script with the task scheduler to watch for that, then use the pskill tool to pop the service (it's the service that is doing it).  I reported the bug to NCH, but no word ever came back. :(

Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: About to FLING FLING...
« Reply #2 on: August 06, 2016, 11:34:16 PM »
Well, tried your suggestion but it didn't make any difference. I also used the Windows troubleshooter and it changed it to Win 8 compatibility mode without any positive results.

Now, I am realizing that VWS doesn't seem to be able to handle SFTP transfers either so this is becoming a more serious issue than I had first thought. I've searched through the latest VWS PDF Manual but it has no reference to SFTP at all.

WD seems to be working okay using SFTP. That may not be 100% correct but at least I do see some files getting through to the server which is better than it was earlier this afternoon.

As far as your memory-munching of non-pageable memory, does StartWatch have the capability to catch that particular parameter? I know it catches some kind of memory usage but not sure if is the non-pageable value. I have my VWS set to restart when StartWatch detects "Leak Protection - Close & restart if memory above 275 MB" but, I don't know what memory value that refers to. That particular setting, in the past, has been the only thing keeping my VWS running 24/7 for the past few years.

EDIT: Forgot to mention that I have sent an email to Ambient Support asking them if VWS is going to support SFTP in the near future now that many hosting companies are forcing customers to use SFTP. The answer will determine whether I stay with VWS or abandon it completely and go with Weather Display or some other software package.
« Last Edit: August 07, 2016, 12:29:10 AM by W3DRM »
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: About to FLING FLING...
« Reply #3 on: August 07, 2016, 04:47:44 PM »
Here is Ed's reply to my question regarding VWS using SFTP:

Quote
I am sorry but VWS does not support SFTP and there are no plans.
Regards,
Ed

Guess I'll have to find another way to get the job done. FLING is probably the best way, but I have to get it working first. I really hate to think that I have to manually set up all of the FTP transfers with FLING or some other software that VWS has routinely been handling for many years.
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: About to FLING FLING...
« Reply #4 on: August 07, 2016, 05:29:06 PM »
Don,

I have Fling handle the VWS uploads with two entries:

VWS-main  local: c:\vws\root  every 5 minutes
VWS-noaa local: c:\vws\data\noaa  every 5 minutes

For WeatherFlash, I select the HTTP method to update the site.

Then, no VWS FTP is needed :)
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline waterside

  • Member
  • *
  • Posts: 22
    • Crab Meadow Weather
Re: About to FLING FLING...
« Reply #5 on: August 07, 2016, 11:59:37 PM »
Many providers are slowly switching to forcing secure communication, either via FTPs or SFTP.  The concern is not so much over the actual data, but moreso over the authentication.  Standard ftp sends your password in clear-text, visible to anyone able to snoop on the connection.  The risk and ability to do so is a hotly-debated topic for another time.

The best option is for the native client to support this, but for cases where this either is not supported or does not work with one's particular provider (more common than you'd think), a "bridge" may be an option.

For SFTP, I have used Bitvise client as an ftp->sftp bridge.  The client is configured to connect to a provider using sftp, and the local application connects to the local bitvise client acting as a bridge using ftp.  Definitely not ideal, but is a potential option.


Offline W3DRM

  • Forecaster
  • *****
  • Posts: 3360
    • Emmett Weather
Re: About to FLING FLING...
« Reply #6 on: August 18, 2016, 12:13:31 AM »
Ken,

I have revised my VWS uploads so all of them are handled by Fling, as you suggested. However, my FTP uploads are still sporadic. Running a netstat grep, shows at most two simultaneous connections. I have checked this numerous times at various times of the day with the exact same results. Am I interpreting your posts to indicate that if I have an FTP problem from my one and only computer that uploads to my website that I should see those uploads in the netstat reports?

Is there any way to run something similar from the server side to see if I am getting FTP's from some other source than my laptop? 1and1 support says they don't see any indication that I am at or exceeding my FTP limit for my account. My only other thought is that there is something else uploading to my website that I don't know about. Filezilla from any of my computers fails constantly as do manual Fling uploads.

I do have FlightRadar24, Bloomsky and Blitzortung uploads taking place on my router but those uploads do not go to my server.

Still stumped and baffled...
Don - W3DRM - Emmett, Idaho --- Blitzortung ID: 808 --- FlightRadar24 ID: F-KBOI7
Davis Wireless VP2, WD 10.37s150,
StartWatch, VirtualVP, VPLive, Win10 Pro
--- Logitech HD Pro C920 webcam (off-line)
--- RIPE Atlas Probe - 32849

Offline ke9lz

  • De Pere - Athelstane Weather
  • Contributor
  • ***
  • Posts: 143
    • De Pere Weather
Re: About to FLING FLING...
« Reply #7 on: October 20, 2016, 10:02:41 AM »
Quote
However.. if you do that, it may also engage the memory-munching non-pageable memory ..  I have to run a small perl script with the task scheduler to watch for that, then use the pskill tool to pop the service (it's the service that is doing it).  I reported the bug to NCH, but no word ever came back. :(

Ken,  Would you be willing to share your perl script and how you set it up?  I to have been struggling to keep Fling up and running for longer than a couple of days and have also reported to NCH.

Thanks,  Steve
Steve (KE9LZ)
Athelstane Weather
Blitzortung 1177, 1565


Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: About to FLING FLING...
« Reply #8 on: October 20, 2016, 10:51:52 AM »
Quote
However.. if you do that, it may also engage the memory-munching non-pageable memory ..  I have to run a small perl script with the task scheduler to watch for that, then use the pskill tool to pop the service (it's the service that is doing it).  I reported the bug to NCH, but no word ever came back. :(

Ken,  Would you be willing to share your perl script and how you set it up?  I to have been struggling to keep Fling up and running for longer than a couple of days and have also reported to NCH.

Thanks,  Steve
Sure, here's the Perl script (I called it zap-fling.pl)
Code: [Select]
#!/usr/bin/perl
#
# use the pstools to find the current NP memory for fling, and optionally
# use pskill to zap it to release the held memory
#
# K. True - 30-Oct-2011
#
# Run via Windows Scheduler or Linux cron .. all output is to the log file(s)
#
# ---- configurable settings ----
$maxMem = 65392;  # amount in KB that we'll tolerate
$lookFor = 'fling';

$logsDir = "./fling-logs/"; # place to store the YYYYMMDD.txt

# ---- end of configurable settings --
#
$|=1; # no buffering of output

 $cur_time = time();
 @months = ("Jan","Feb","Mar","Apr","May","Jun","Jul","Aug","Sep","Oct","Nov","Dec");

 $log_filename = ISO_datestamp($cur_time) . ".txt";
 $nice_date = nice_datestamp($cur_time);

 open (OUT,">>${logsDir}${log_filename}");

# old-- open(PS,"pslist -m fling |") || die "..unable to run pslist $!\n";

#class Win32_PerfRawData_PerfProc_Process : Win32_PerfRawData
#{
#  string Caption;
#  uint32 CreatingProcessID;
#  uint64 ElapsedTime;
#  uint64 Frequency_Object;
#  uint64 Frequency_PerfTime;
#  uint64 Frequency_Sys100NS;
#  uint32 HandleCount;
#  uint32 IDProcess;
#  uint64 IODataOperationsPerSec;
#  uint64 IOOtherOperationsPerSec;
#  uint64 IOReadBytesPerSec;
#  uint64 IOReadOperationsPerSec;
#  uint64 IOWriteBytesPerSec;
#  uint64 IOWriteOperationsPerSec;
#  uint64 IODataBytesPerSec;
#  uint64 IOOtherBytesPerSec;
#  string Name;
#  uint32 PageFaultsPerSec;
#  uint64 PageFileBytes;
#  uint64 PageFileBytesPeak;
#  uint64 PercentPrivilegedTime;
#  uint64 PercentProcessorTime;
#  uint64 PercentUserTime;
#  uint32 PoolNonpagedBytes;
#  uint32 PoolPagedBytes;
#  uint32 PriorityBase;
#  uint64 PrivateBytes;
#  uint32 ThreadCount;
#  uint64 Timestamp_Object;
#  uint64 Timestamp_PerfTime;
#  uint64 Timestamp_Sys100NS;
#  uint64 VirtualBytes;
#  uint64 VirtualBytesPeak;
#  uint64 WorkingSet;
#  uint64 WorkingSetPeak;
#};
$wmicArgs = "Name,IDProcess,VirtualBytes,WorkingSet,PrivateBytes,PageFaultsPerSec,PoolNonpagedBytes,PoolPagedBytes";

#open(PS,"wmic path Win32_PerfFormattedData_PerfProc_Process WHERE Name=\"$lookFor\" GET $wmicArgs |") || die "..unable to run wmic $!\n";
open(PS,"wmic path Win32_PerfFormattedData_PerfProc_Process GET $wmicArgs |") || die "..unable to run wmic $!\n";


while (<PS>)
{
  #print $_;
  $rec = $_;
# IDProcess  Name   PageFaultsPersec  PoolNonpagedBytes  PoolPagedBytes  PrivateBytes  VirtualBytes  WorkingSet 
  ($PID,$name,$Faults,$NonP,$Page,$Priv,$VM,$WS) = split(/\s+/);

#old  ($name,$PID,$VM,$WS,$Priv,$Faults,$NonP,$Page) = split(/\s+/);
  next unless $WS =~ m/\d+/;
  next unless $name =~ m/fling/;
 
  #print STDOUT "$name\t$PID\t$NonP\n";
  print OUT "$nice_date\t$rec";
  print STDOUT "$nice_date\t$rec";
  $NonP = int($NonP / 1024);
 
  if($NonP >= $maxMem) {
    print OUT "$nice_date\t$name PID=$PID pskill for $NonP > $maxMem KB\n";
    print STDOUT "$nice_date\t$name PID=$PID pskill for $NonP > $maxMem KB\n";
open(PK,"pskill $PID |") || die "..unable to run pskill $!\n";
while (<PK>) {
    print OUT "$nice_date\t$_";
    print STDOUT "$nice_date\t$_";
 
}
    print OUT "$nice_date\t-------------------------------------------\n";
    print STDOUT "$nice_date\t-------------------------------------------\n";

  }
}

# ------  end of main program -----

sub nice_datestamp {
    my $d = shift;
   
    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($d);
    my $nicedate = sprintf(
        "%02d\-%s\-%04d %02d\:%02d\:%02d",
        $mday,$months[$mon],$year+1900,$hour,$min,$sec);
    return("$nicedate");

}

sub ISO_datestamp {
    my $d = shift;
   
    my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime($d);
    my $nicedate = sprintf("%04d%02d%02d",$year+1900,$mon+1,$mday);
    return("$nicedate");

}
You'll have to have installed Perl from activestate.com (link here) and the pskill utility from Microsoft SystemTools here.

The script uses the built-in wmic command to find/list the memory used by fling, and if it's larger than 64MB, issue a pskill to kill off the process and thereby release the non-pagable memory.  Fling will start a new process when needed.

I use the Windows Scheduler to run the zap-fling.pl program every 15 minutes round the clock.  It logs the results in a yyyymmdd.txt file in the ./fling-logs/ directory.

Hope this helps...
Best regards,
Ken
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline ke9lz

  • De Pere - Athelstane Weather
  • Contributor
  • ***
  • Posts: 143
    • De Pere Weather
Re: About to FLING FLING...
« Reply #9 on: October 20, 2016, 07:34:02 PM »
Hi Ken,  Yes, this helps a lot!  Seems to have done the trick!  Thank you very much!

Steve
Steve (KE9LZ)
Athelstane Weather
Blitzortung 1177, 1565


Offline zmarfak

  • Contributor
  • ***
  • Posts: 135
    • Matar
Re: About to FLING FLING...
« Reply #10 on: February 04, 2018, 08:13:31 AM »
Hi,
got the same memory issue (see attachment, monitored by free PRTG)
problems started after installing fling when I checked on the date I started to use it
When I searched on the net this tread came up.
Apparently no change on the software since 2016

I just implemented the script of Ken
I will keep an eye on it.
 
Fling version 2.35 is running here on Microsoft Windows 10 Home, OS Version:  10.0.16299 , 16MB memory
Patrick
Davis Vantage Pro2 with a Meteobridge NANO SD and WL (6.04) on a Intel NUC 
https://www.matar.be

Offline GregJ

  • Forecaster
  • *****
  • Posts: 946
    • http://www.SkunkBayWeather.com
Re: About to FLING FLING...
« Reply #11 on: February 05, 2018, 02:10:05 PM »
I used Fling for years...  I kept 2.21 and it was rock solid.  Recently I had to make a change...  I had too many simultaneous FTP sessions and my webhost was blacklisting my IP because their firewall was interpreting my activity as an attack....  So, I consolidated all of my ftp activity into a single session that I run every 2 minutes.  I am using www.scriptftp.com.  It has been fantastic and very easy to customize over my network.  Great customer service during setup too....

Thought I'd Share,
Greg
Davis VP2 with VWS

"Everybody talks about the weather, but nobody does anything about it." -- Mark Twain

Offline zmarfak

  • Contributor
  • ***
  • Posts: 135
    • Matar
Re: About to FLING FLING...
« Reply #12 on: February 05, 2018, 04:52:26 PM »

  I am using www.scriptftp.com.  It has been fantastic and very easy to customize over my network.  Great customer service during setup too....

Thought I'd Share,
Greg


Thanks for the info
Patrick
Davis Vantage Pro2 with a Meteobridge NANO SD and WL (6.04) on a Intel NUC 
https://www.matar.be

Offline ConligWX

  • Forecaster
  • *****
  • Posts: 836
  • #conligwx
    • conligwx.org
Re: About to FLING FLING...
« Reply #13 on: February 05, 2018, 05:27:51 PM »
WinSCP also has command line utilities for ftp/ftps/sftp uploading/downloading and work very well.
Regards Simon
Davis Vantage Pro2 Plus (6162UK) • Daytime FARS • WeatherLink Live • AirLink • PurpleAir PA-II-SD • CumulusMX •


Offline carl357

  • Member
  • *
  • Posts: 1
    • Script FTP
Re: About to FLING FLING...
« Reply #14 on: February 06, 2018, 06:04:52 AM »
I used Fling for years...  I kept 2.21 and it was rock solid.  Recently I had to make a change...  I had too many simultaneous FTP sessions and my webhost was blacklisting my IP because their firewall was interpreting my activity as an attack....  So, I consolidated all of my ftp activity into a single session that I run every 2 minutes.  I am using www.scriptftp.com.  It has been fantastic and very easy to customize over my network.  Great customer service during setup too....

Thought I'd Share,
Greg

Thanks Greg. I am the guy behind ScriptFTP and have just realized of this conversation.

Yes, the plain old FTP is being deprecated by many providers such as 1and1, Amazon web services etc. From the point of view of the network administrator is a pain to manage, specially because it is not secure and has a strange way to handle ports to transfer files.

Hopefully SFTP is now very common and most FTP clients as ScriptFTP support it. This is a small script (for ScriptFTP) that synchronizes a remote folder from a folder in your computer (uploads new or modified files).

Code: [Select]
OPENHOST("sftp://ftp.myhost.com","myuser","mypassword")
SYNC("C:\my_folder","/my_remote_folder",UPLOAD,SUBDIRS)
CLOSEHOST

As you can see ScriptFTP is meant to be easy to understand and, unlike WinSCP which also supports scripting, it is built from ground up to support scripting

If any of you have any questions on how to configure ScriptFTP to work as Fling (or anything else) feel free to ask me anything on the website chat at ScriptFTP.com. I will also stay tuned on this forum thread  :grin: ;) ;)





Offline zmarfak

  • Contributor
  • ***
  • Posts: 135
    • Matar
Re: About to FLING FLING...
« Reply #15 on: December 20, 2018, 07:25:16 AM »
Hi
I received an email on 4th december of NCH software with fling 3.00 link (beta), about memory leaks
As an answer to a message of me some times ago
Seems to me it's solved in this version.

I haven't checked on their website if this version is already available
 
Patrick
Davis Vantage Pro2 with a Meteobridge NANO SD and WL (6.04) on a Intel NUC 
https://www.matar.be

Offline ke9lz

  • De Pere - Athelstane Weather
  • Contributor
  • ***
  • Posts: 143
    • De Pere Weather
Re: About to FLING FLING...
« Reply #16 on: December 20, 2018, 10:47:56 AM »
I got the same email and did download it to find that my software key didn't work with the fixed version.  They require you to pay for a new key.  I decided to stay with Kens fix which has been working flawlessly.
Steve (KE9LZ)
Athelstane Weather
Blitzortung 1177, 1565


Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: About to FLING FLING...
« Reply #17 on: December 20, 2018, 11:47:00 AM »
I'd done the $29 upgrade today from V2.35 (with the leaky nonpaged memory issue) to V3.00, and activated the new version.  Installed (after I did a backup of the old fling directory), and it is running fine.  It's not eating nonpaged memory any more on Win10-Pro !!

For insurance, I'm still leaving the zap-fling.pl script running via Scheduler at 15 minute intervals, just to be safe.

I think the major change of adding SFTP support is a reason to do the upgrade.. be sure to get the upgrade pricing ($29 for the pro version).
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

Offline Cutty Sark Sailor

  • WxElement panel
  • Forecaster
  • *****
  • Posts: 3393
    • Frankfort Weather - TwinHollies WeatherCenter
Re: About to FLING FLING...
« Reply #18 on: December 20, 2018, 08:12:45 PM »
Yep, V3.0 cures the non-paged pool issue... smooth as silk...
as Ken says,
I'd done the $29 upgrade today from V2.35 (with the leaky nonpaged memory issue) to V3.00, and activated the new version.  Installed (after I did a backup of the old fling directory), and it is running fine.  It's not eating nonpaged memory any more on Win10-Pro !!

For insurance, I'm still leaving the zap-fling.pl script running via Scheduler at 15 minute intervals, just to be safe.

I think the major change of adding SFTP support is a reason to do the upgrade.. be sure to get the upgrade pricing ($29 for the pro version).
Especially if you ever think of SFTP...do the upgrade...  Good to Go
The new version also cured a small  glitch reading one folder that was a tad slower writing at times, for some reason,... no more error on that one... was no biggie, it'd catch it on the next scan, .. just irritating to see the 'error at times... Tip:   export your 'Folder List' just in case... so you can import it if need b
also a good idea to export it and save it somewhere.
 


Offline ke9lz

  • De Pere - Athelstane Weather
  • Contributor
  • ***
  • Posts: 143
    • De Pere Weather
Re: About to FLING FLING...
« Reply #19 on: December 21, 2018, 12:59:11 PM »
Quote
I think the major change of adding SFTP support is a reason to do the upgrade..

Guess I didn't realize that this was also changed with the addition to the memory fix.  For that I agree it is worth the upgrade.  Now that you guys had a few days of use with it is the memory leak for sure fixed?
Steve (KE9LZ)
Athelstane Weather
Blitzortung 1177, 1565


Offline saratogaWX

  • Administrator
  • Forecaster
  • *****
  • Posts: 9257
  • Saratoga, CA, USA Weather - free PHP scripts
    • Saratoga-Weather.org
Re: About to FLING FLING...
« Reply #20 on: December 21, 2018, 01:27:44 PM »
Yep.. still fixed.  Nonpaged memory about 49KB instead of growing 4KB each second until I zap it at 16MB.

Yes.. do the upgrade!
Ken True/Saratoga, CA, USA main site: saratoga-weather.org
Davis VP1+ FARS, Blitzortung RED, GRLevel3, WD, WL, VWS, Cumulus, Meteobridge
Free weather PHP scripts/website templates - update notifications on Twitter saratogaWXPHP

 

anything