Skip to content

New minor version - Fix for speedup and Compatibility from JRE 6 to JRE 11 and other improvements or fixes
Compare
Choose a tag to compare
@fredericBregier fredericBregier released this 30 Sep 12:44
· 7 commits to v3.5 since this release
v3.5.1
2b6b4f2

This minor update contains several improvements:

Speedups:

  • Remove or decrease extra sleep command while there are no more needed
  • Digest by packet is optimized (mode being SENDMD5 or RECVMD5, while digest could be any from CRC32, ADLER32, MD5, MD2, SHA1, SHA256, SHA384, SHA512)
  • Reduce the Local Digest computation when not needed (while Global Digest is done)
  • Cache DB access for Business, Host and Rule only
    • Note, this is deactivated if the server is in "MultipleMonitor" mode (since Cache can be not coherent).
  • Improve TaskRunner for specific case of Update on rank update (only rank and stop date are updated, not all fields).
  • Reduction of latency

Fixes:

  • Bad programming practice corrections
  • Improvement of documentation
  • Improvement of Rest V2 Service to include information from FileMonitoring
  • Improvement of logs (performances and adapted level)
  • Upgrade of external dependencies

New versions for JRE 6, 8 and 11 separately provided from now:

  • JRE 6 to JRE 11 compatibility, including 3 kind of packages:
    • Waarp*.jar for JRE6,
    • Waarp*-jre8.jar for JRE8
    • Waarp*-jre11.jar for JRE11

General recommendations for speedup
If one has a very limited resource server and wants to ignore all consistency checks, then :

  • using transfer with no MD5 mode will allow to improve about 10% to 20% the speed of transfer.
  • disabling final local check (localdigest to False) will allow to improve about 20% more to the speed of transfer (since optimization is already there).
  • disabling global check (globaldigest to False) will allow to improve about 25% more to the speed of transfer.

So globally, one can improve up to 50% (we were able to reach 110 MBPS or 880 MbPS).

However, it is not recommended to have no check at all during transfers.
So we recommend to at least active global digest (globaldigest to True or unset as default value), which will bring about 30% of improvements of speed (compared to all options actives).

Other recommendations

  • Java arguments: use -Xms2048m -Xmx2048m
  • XML configuration:
    • serverthread to nb of core (default) or less if you want to limit the CPU usage
    • clientthread to 10 * serverthread (default) or less if you want to limit the CPU usage
    • ignore or set usefastmd5 to False (default) (not anymore efficient)
    • digest to 7 (SHA-512 while MD5 being the default) or if you accept more collisions, 2 for MD5 but really more efficient (about 50%)
    • ignore or set globaldigest to True (default) (consistency check)
    • localdigest to False (True is the default) (optional consistency check, this fix limits greatly its impact however)
    • ignore or set runlimit to 1000 (default) or less if you want to limit the CPU usage (concurrency of submitted transfers, do not set to lower than 10)
    • blocksize to a multiple of 16KB (64KB being the default, but do not set more than 256KB) (the higher, the smaller the number of packets)
    • memorylimit to a value acceptable for HTTP and REST services (default 1GB, could be set to less)
    • sessionlimit and globallimit to an acceptable bandwidth limitation (for instance 100 MBPS), default being no limit (note: limit is in Bytes per second and not bits per second)
    • ignore or set usenio to False (default) (no more improvement with true)
    • Use CPU limitation to adjust bandwidth automatically to prevent CPU high consuming over 60% (here for the example since 0.6) (global bandwidth is adapted automatically)
      • usecpulimit to True (default False, as for usejdkcpulimit that could be ignored),
      • cpulimit to a value less than 1.0 (0.6 for instance),
      • connlimit still to 0 (default, unlimited):
Assets 8
Loading