SiteMap
Page Bottom  References from Other Sources

APL timing tests 5.2 to 6.0

I recently began using apl version 6.0.1 and was delighted to discover a major improvement in the execution speed of my major application when comparing it with apl version 5.2.11. After doing many timing tests, it appears that the major improvement is in tieing/untieing files which my app was doing "on the fly". Inspired by this major improvement in execution speed, and also deciding that my file system is sufficiently stable, I decided to change my use of files so that I reference tie numbers, and through quad-mf testing I discovered other ways to improve speed as well. The result is that testing lots of datasets that used to require 122 minutes now requires only 25 minutes, an overall reduction of 80% in execution time. Here are my conclusions about how this improvement was made possible.

  1. My first happy discovery was an improvement from 122 minutes to 79 minutes just by moving from 5.2 to 6.0. That's a 35% reduction in execution time.

I learned that in the 122 minute test that 55% of cpu usage was in Davin's cfDir & cfIO functions which I was using with explicit file names rather than tie numbers.

initial speedup from moving from apl 5.211 to apl 6.01

cpuaplvwssecondsminutespages/pagedatetime%cfcfalgos
PIV-24005211QKW_HTM200608277320122489115020060901000
           
PIV-24006010QKW_HTM2006082747407948919720060901000
PIV-24006010QKW_HTM20060827481380.2248919820060907.130.510824582355
amd64nm6010QKW_HTM20060827504084489110320060902.220.555127982242
amd64nm6010QKW_HTM20060827210535.0848904320060904.150.31966731432

move from PIV-2400 chip to AMD 64 bit chip

cpuaplvwssecondsminutespages/pagedatetime%cfcfalgos
amd64nm5211QKW_HTM552092489111320060902.13000
amd64nm5211QKW_HTM171028.5489135200609030.34435891121
amd64nm5211QKW_HTM184630.7751253620060919.18000
amd64nm5211QKW_HTM189031.550983720060920.10.30915841306
amd64nm5211QKW_HTM25204250984920060920.20.407610271493
amd64nm5211QKW_HTM198733.1250983920060920.21000
amd64nm5211QKW_HTM226737.7850984420060921.07000
amd64nm5211QKW_HTM176029.3350983520060921.07000
amd64nm5211QKW_HTM153625.650983020060921.080.3847591945
amd64nm5211QKW_HTM161726.9550983220060921.13000
           
PIV-24006010QKW_HTM24604148915020060901000
PIV-24006010QKW_HTM253242.248915220060904.050.19865032029
PIV-24006010QKW_HTM255342.5548915220060904.130.20395212032
PIV-24006010QKW_HTM283747.2848915820060904.150.27237732064
PIV-24006010QKW_HTM283147.1848995820060907.16000
PIV-24006010QKW_HTM279446.5751245520060912.1000
PIV-24006010QKW_HTM269544.9251245320060912.140.449212111484
PIV-24006010QKW_HTM263343.8851245120060912.19000
PIV-24006010QKW_HTM262143.6851255120060915.16000
PIV-24006010QKW_HTM237239.5350984720060921.10.18634421930
amd64nm6010QKW_HTM552092489111320060902.19000
amd64nm6010QKW_HTM175229.2489136200609030.33755911161
amd64nm6010QKW_HTM17402948913620060904.130.34816061134
amd64nm6010QKW_HTM189931.6551243720060912.17000
amd64nm6010QKW_HTM184730.7851243620060912.170.30785691278
amd64nm6010QKW_HTM179329.8851243520060912.18000
amd64nm6010QKW_HTM178229.751253520060915.15000
amd64nm6010QKW_HTM182430.451253620060917.16000
amd64nm6010QKW_HTM156126.0250983120060921.090.3699577984

eliminate conversion of files from .sf to .cf

cpuaplvwssecondsminutespages/pagedatetime%cfcfalgos
amd64nm5211PVMM18603148913820060901000
amd64nm5211PVMM167027.8348913420060904.110.35415911079
amd64nm5211PVMM153725.6250983020060921.190.3694568969
amd64nm5211PVMM147624.650982920060923.18000
amd64nm5211PVMM151725.2850983020060924.070.3801577940
amd64nm5211PVMM149224.8751022920061002.19000
amd64nm5211PVMM165527.5851063220061011.09000
amd64nm5211PVMM167627.9351063320061011.12000
amd64nm5211PVMM219536.5851064320061012.21000
amd64nm5211PVMM228338.0551064520061013.15000
amd64nm5211PVMM215835.9751064220061013.18000
amd64nm5211PVMM217536.2551064320061013.190.45539901185
amd64nm5211PVMM226137.6851144420061021.16000
amd64nm5211PVMM183530.5851013620061214.150.3446311204
           
PIV-24006010PVMM22803848914720060901000
PIV-24006010PVMM262843.848915420060904.060.22775982030
PIV-24006010PVMM260043.3348915320060904.120.22445832017
PIV-24006010PVMM256542.7548995220060907.18000
PIV-24006010PVMM198633.150983920060924.070.20834141572
amd64nm6010PVMM16802848913420060901000
amd64nm6010PVMM17402948913620060902.15000
amd64nm6010PVMM195832.6348914020060904.11000
amd64nm6010PVMM171228.5348913520060904.120.34615931119
amd64nm6010PVMM158226.3750983120060923.17000
amd64nm6010PVMM159926.6550983120060924.060.36355811018
amd64nm6010PVMM219836.6351064320061013.16000
amd64nm6010PVMM217836.351064320061013.170.43369441234
amd64nm6010PVMM191731.9551023820061214.180.32626251292
amd64nm6010PVMM188031.3351023720061215.1000

confirm with imgindex.w3

cpuaplvwssecondsminutespages/pagedatetime%cfcfalgos
amd64nm5211IMGINDEX267044.5241911020060919.2000
           
PIV-24006010IMGINDEX7244120.73241929920060915.19000

  1. Dell =
  2. HP =
  3. 5211 =
  4. 6010 =
  5. tieing =
  6. tn+conv =
  7. tn only =
  8. seconds =
  9. minutes =
  10. pages =
  11. date =
  12. CF % =
  13. CF sec =
  14. algo sec =

One other thing I noticed that is a change in apl behavior. With version 5.2, if you tie a file using its name in mixed case and then you tie it again with the name in all lower case, you will find that it is tied twice with two different tie numbers. With version 6.01, you'll find that it untied the first tie when it did the second tie, so now the first tie number is invalid. This was discovered using Colossal Files.

horizontal line
to home page e-mail Page Top