Document: burnmark.abs File Group: Classic Benchmarks Creation Date: 15 April 1997 Revision Date: Title: High Level Language Test Programs Keywords: RELIABILITY TESTING BURN-IN PERFORMANCE Abstract: This document contains details of general purpose test programs that can help to verify that PCs are fault free and also provide performance information. Note: This suite is still under test and should be treated as beta test software. Please submit feedback directly directly to Roy, or as a posting in Section 12. WARNING: The programs have been tested on various PCs including 486, Pentium and Pentium Pro systems. They have also been run via Windows 95 and NT 4.0. The DOS versions have also been run directly in the associated DOS mode and when booted as DOS only with MS DOS 6.2. However, it is inevitable that they will not run properly in all circumstances or occasionally may cause certain systems to hang. Likely reasons for failure include hardware or software incompatibilities besides hardware faults. The programs were originally designed as stress tests and to identify intermittent failures. In a weakened hardware situation or out of specification environment, the tests could provide the final straw to cause physical failure. RUN AT YOUR OWN RISK Contributor: Roy_Longbottom@compuserve.com HIGH LEVEL LANGUAGE TEST PROGRAMS 0. SUMMARY CPR4xxx.EXE and CDK1xxx.EXE are C++ versions of programs that were used for exercising processors and disks during acceptance trials of computers in a bygone age. The programs have been compiled to run via DOS (xxxxDOS.EXE), Windows 95 and NT (xxxx95NT.EXE) and OS/2 (xxxxOS2.EXE). With appropriate parameters, CDK1 can also be used for testing memory and network data transfers. Performance information is supplied as a supplement. Running details are logged into text files ChhmmssZ.TXT and DhhmmssZ.TXT where hhmmss is based on the time of day and Z is an optional letter to avoid file errors on multi-tasking. These and the disk data files are normally written in the directory containing the EXE codes. The programs can be run with manual input parameters or with extensions to an EXE command line for running in batch mode with no manual intervention. Manual input parameters for the disk test are optional client/server names, megabytes written and number of read passes. The processor test has a parameter for running time in minutes after the first pass (1 pass takes about 5 minutes on a 486DX2/66 to 40 seconds on a 200 MHz Pentium Pro). 0.1 FAST START Create a new directory BURNMARK on uncompressed drive C: Note free disk space. DO NOT RUN DISK TEST IF INSUFFICIENT FREE CAPACITY. Copy programs CDK1xxx.EXE, CPR4xxx.EXE and BAT files into this directory (DOS versions also DOS4GW.EXE). Windows varieties - close all other applications - display BURNMARK directory - click on EXE icon. DOS - reboot in direct DOS mode with minimum drivers e.g. for DOS 6.2, CONFIG.SYS one line DEVICE=HIMEM.SYS - AUTOEXEC.BAT one line PROMPT $p$g and an optional second line to run tests in memory e.g. SMARTDRV.EXE /X 8192 128 (for 8 MB cache buffer) - include SMARTDRV.EXE on the boot disk. Change directory to C:\BURNMARK, type CDK1DOS or CPR4DOS Disk test press ENTER in response to server name and client name File size - enter megabytes for memory test one quarter of main memory size (DOS less than SMARTDRV value). Number of read passes - enter 4 Repeat with file size up to at least main memory capacity. On Windows with variable disk cache size, some experimentation will be required to find the breakpoint between memory and disk testing (watch the access light on the disk). CPU test After first pass, enter 0 (zero) for additional running time - one further pass will be executed. Batch test Execute BATCHDOS.BAT or click on BATCHWIN.BAT or BATCHOS2.CMD to run the disk test with 4, 8, 16 and 32 megabytes with four read passes and CPU test for one pass plus 5 minutes. Multi-task Click on MULTIWIN.BAT or MULTIOS2.CMD to run two sets of programs multi-tasked: Two disk tests 8 MB each with 4 read passes, results file suffix A & B and a CPU test 1 pass plus 3 minutes. Disk test 8 MB with 16 read passes and two CPU tests of 2+ minutes with result file suffix C and D. Note: the number of repeat passes of the CPU test is derived from the running time of the first pass. With multi-tasking, this time will be inaccurate and variable. Examine all the DhhmmssZ.TXT and ChhmmssZ.TXT result log files to ensure that the tests ran error free. On the disk test, in the event of certain errors, the data files DhhmmssZ.TM1, TM2, TM3 and TM4 may still be present. These should be erased manually. 1. INTRODUCTION In 1958, the UK Government formed the Technical Support Unit (TSU), recruiting engineers from the telecommunications service, to evaluate computers and provide advice on procurement. TSU eventually became part of the existing Central Computer and Telecommunications Agency (CCTA). I joined TSU in 1960, initially mainly concentrating on reliability studies. One of the most popular functions of TSU was to design and supervise acceptance trials of new computers at Government Departments and, later, for centrally funded systems at universities. In the early days, tests were based on running manufacturers’ diagnostic programs for 20 to 40 hours rather than the USA concept of 30 day user tests. The shorter tests were used as users rarely had working jobs to run and program development activity was not considered to provide an adequate “reliability” test. From 1968 to 1970, I was responsible for all acceptance trials with some 20 staff. Then, with a reorganisation, my duties changed to appraisal and acceptance trials of scientific systems (until 1980 when I became a performance specialist). In 1970, the major question was how could we test systems with operating systems particularly considering multi-programming capabilities. The situation was somewhat rosier on scientific systems, at least for testing the CPU. However, the available programs tended to have fixed run times and produced vast quantities of difficult to check printed output. It was then that I decided to become a programmer (rather than pretending to be a manager) and spent most of the day on a teletype using the “Call 360” service to produce a suite of Fortran based test programs. The design objective of the programs was that they should be self checking or with minimum (or easy to check) output and have parameters so they could run for as long as was required on any computer. The programs were released for use in 1971. The results of these activities were FOPR00 to FOPR04 processor tests, FODK00 to FODK03 for disks, with others for magnetic tape units, line printers, card readers, card punches, paper tape readers and paper tape punches. These provided the core for effective acceptance trials of hundreds of computer systems (including commercial ones) for many years. The Fortran listings and details of acceptance test procedures are given in my book “Computer System Reliability”, Wiley, 1980. The book also contains details of the acceptance trial procedures and an analysis of the (then) most recent 400 tests. Of the processor tests, FOPR02 and FOPR04 were the most successful in identifying obscure faults. These mainly concentrate on floating point calculations and carry out calculations using the full number range. The formulae used produce a result of approximately 1.0, the results from each pass being multiplied together so that errors would be carried forward to a single line result. FOPR02 calculates the binomial expansion (p + q)^n, where p = 1 - q. It was usual to repeat the calculations a few times to check for consistency. This program identified numerous floating point hardware faults and was occasionally used for trouble shooting purposes by more than one supplier as it was better for reproducing intermittent faults than their functional diagnostic tests. FOPRO4 has eight sections which use a different set of Fortran functions. To simplify checking, an integer sum check is calculated which can have a vastly different value for a slight difference in the floating point result. This program also identified numerous hardware faults but its main claim to fame was showing that new hardware or compiler variations could produce different results. As shown later, running the same .exe program on 486 and Pentium based PCs shows some differences in computed results. The most successful test, as far as breaking disk drives is concerned, was FODK01. This generates up to ten files and reads blocks from the files in a random order. The data comprises repetitive binary patterns designed to reproduce pattern conscious faults. The program identified design faults on the LAN based disk sub-system of the CDC Cyber 205 supercomputer. It also caused the IBM MVT Operating System to read the wrong files. 2. NEW VERSIONS FOR PCs C++ implementations of FODK01 and FOPR04 have been produced, the latter with an extra section carrying out the same calculations as FOPR02. These have been compiled with Watcom C/C++ to run on PCs either directly from DOS or via Windows 95 or NT or OS/2. If a PC is behaving in a peculiar manner or crashing, it is useful to have programs that can run for a reasonable period to, at least, establish that the hardware appears to be working properly. I have had two cases where variants of FODK01 implementations were of use. The first case was on an 80386 based PC which crashed occasionally and frequently would not boot. The program could invariably cause the disk to freeze after running for a period. Re-seating the I/O card fixed the problem. The second case was more frequent but intermittent system crashes on a Pentium after adding an extra 8 Mbytes of non-parity memory. The program showed conclusively that data in the memory was being corrupted. The tests have facilities to input run time parameters interactively or as run switches following an EXE command. On Windows 95 and NT, the programs can be multi-tasked via a BAT file or a CMD file with OS/2. The disk test can be used to test local disks, network disks or memory (as used by a disk cache). 3. PROGRAM CDK1DOS, CDK195NT and CDK1OS2 The program writes four files each in multiples of 335,872 bytes. The program block size used is 512 words or 2048 bytes. Each block has a different bit pattern varying from one bit set in each word, through repeating bit patterns such as 11001100 to all 1's or 0's. An example of information displayed with manual input is: DK01 C++ Disk Test by Roy Longbottom Mon Mar 17 16:47:41 1997 Job D170614Z Copyright © 1997, Roy Longbottom, All rights reserved Based on FODK01 a TSU Acceptance Trials Program from another era Enter server computer name: Pentium Pro Enter client computer name: P100 With the minimum parameters, four files are written each 0.33 Megabytes. The program block size used is 512 words or 2048 bytes. Each block has a different bit pattern as displayed later. The block has a 3 word header for file number (1-4), write repeat (>1 for longer files), pattern number (1-164), then 509 words of the pattern. On reading, files are selected in a random order. The data is checked and any errors are displayed Input parameters are provided for file length and number of read passes. File Size - Enter total Megabytes (minimum 1.33) 16 Disk F Free disk space 1164.5 Mbytes will use 15.4 Enter number of read passes 2 Patterns Generated Number Pattern Or Number Pattern Or 1 0 0 2 1 1 3 2 2 4 4 4 5 8 8 6 16 10 7 32 20 8 64 40 9 128 80 10 256 100 11 512 200 12 1024 400 13 2048 800 14 4096 1000 15 8192 2000 16 16384 4000 17 32768 8000 18 65536 10000 19 131072 20000 20 262144 40000 21 524288 80000 22 1048576 100000 23 2097152 200000 24 4194304 400000 25 8388608 800000 26 16777216 1000000 27 33554432 2000000 28 67108864 4000000 29 134217728 8000000 30 268435456 10000000 31 536870912 20000000 32 1073741824 40000000 33 1 1 34 5 5 35 21 15 36 85 55 37 341 155 38 1365 555 39 5461 1555 40 21845 5555 41 87381 15555 42 349525 55555 43 1398101 155555 44 5592405 555555 45 22369621 1555555 46 89478485 5555555 47 357913941 15555555 48 1431655765 55555555 49 3 3 50 51 33 51 819 333 52 13107 3333 53 209715 33333 54 3355443 333333 55 53687091 3333333 56 858993459 33333333 57 7 7 58 455 1c7 59 29127 71c7 60 1864135 1c71c7 61 119304647 71c71c7 62 15 f 63 3855 f0f 64 986895 f0f0f 65 252645135 f0f0f0f 66 31 1f 67 31775 7c1f 68 32537631 1f07c1f 69 63 3f 70 258111 3f03f 71 127 7f 72 2080895 1fc07f 73 255 ff 74 16711935 ff00ff 75 511 1ff 76 1023 3ff 77 2047 7ff 78 4095 fff 79 8191 1fff 80 16383 3fff 81 32767 7fff 82 65535 ffff 83 -1 ffffffff 84 -2 fffffffe 85 -3 fffffffd 86 -5 fffffffb 87 -9 fffffff7 88 -17 ffffffef 89 -33 ffffffdf 90 -65 ffffffbf 91 -129 ffffff7f 92 -257 fffffeff 93 -513 fffffdff 94 -1025 fffffbff 95 -2049 fffff7ff 96 -4097 ffffefff 97 -8193 ffffdfff 98 -16385 ffffbfff 99 -32769 ffff7fff 100 -65537 fffeffff 101 -131073 fffdffff 102 -262145 fffbffff 103 -524289 fff7ffff 104 -1048577 ffefffff 105 -2097153 ffdfffff 106 -4194305 ffbfffff 107 -8388609 ff7fffff 108 -16777217 feffffff 109 -33554433 fdffffff 110 -67108865 fbffffff 111 -134217729 f7ffffff 112 -268435457 efffffff 113 -536870913 dfffffff 114 -1073741825 bfffffff 115 -2 fffffffe 116 -6 fffffffa 117 -22 ffffffea 118 -86 ffffffaa 119 -342 fffffeaa 120 -1366 fffffaaa 121 -5462 ffffeaaa 122 -21846 ffffaaaa 123 -87382 fffeaaaa 124 -349526 fffaaaaa 125 -1398102 ffeaaaaa 126 -5592406 ffaaaaaa 127 -22369622 feaaaaaa 128 -89478486 faaaaaaa 129 -357913942 eaaaaaaa 130 -1431655766 aaaaaaaa 131 -4 fffffffc 132 -52 ffffffcc 133 -820 fffffccc 134 -13108 ffffcccc 135 -209716 fffccccc 136 -3355444 ffcccccc 137 -53687092 fccccccc 138 -858993460 cccccccc 139 -8 fffffff8 140 -456 fffffe38 141 -29128 ffff8e38 142 -1864136 ffe38e38 143 -119304648 f8e38e38 144 -16 fffffff0 145 -3856 fffff0f0 146 -986896 fff0f0f0 147 -252645136 f0f0f0f0 148 -32 ffffffe0 149 -31776 ffff83e0 150 -32537632 fe0f83e0 151 -64 ffffffc0 152 -258112 fffc0fc0 153 -128 ffffff80 154 -2080896 ffe03f80 155 -256 ffffff00 156 -16711936 ff00ff00 157 -512 fffffe00 158 -1024 fffffc00 159 -2048 fffff800 160 -4096 fffff000 161 -8192 ffffe000 162 -16384 ffffc000 163 -32768 ffff8000 164 -65536 ffff0000 0.16 seconds for data generation and display 4 Data files closed each 4030464 bytes (data*12) Mon Mar 17 17:06:42 1997 27.25 seconds for writing files at 0.56 Mbytes/second 4 Data files closed Mon Mar 17 17:07:13 1997 31.14 seconds for read pass 1 of 2 at 0.49 Mbytes/second 4 Data files closed Mon Mar 17 17:07:43 1997 29.55 seconds for read pass 2 of 2 at 0.52 Mbytes/second Files erased End of program - results are in file D170614Z.txt 3.1 CDK1 EXE PARAMETERS Three parameters are can be included for total megabytes, number of read passes and, optionally, a letter that will create different output files for programs starting at the same time when multi-tasking. Examples of command lines to run the programs are: CDK1DOS 16 2 Dos version, 4 files of about 4 MB, 2 read passes CDK195NT 24 8 G NT/W95 version, 24 MB, 8 reads, add G to O/P file name. Examples of running via a network drive (CDK195NT.EXE on unit F, TEST directory) are: F:\TEST\CDK195NT 32 5 Downloads program and runs using client disk F: CD \TEST\ CDK195NT 32 5 Downloads program using files on server F disk 3.2 CDK1 OUTPUT FILES The results files have the format DHHMMSSZ.TXT where HH, MM, SS represent the hour, minute and second when the job started. Z is the default last letter which is replaced by any that are included in the command line. Similarly the four data files are DHHMMSSZ.TM1, TM2, TM3 and TM4. Results files and data files should be in the directory from which the program is run. Data files are erased at the end of the program. 3.3 CDK1 - WHAT IT TESTS Testing disks - In order to ensure that the data is read from disk, the file size must be greater than any cache size. On Windows 95 and NT, the cache size grows until all memory is used. In this case, the safe bet is to use an input megabyte parameter greater than the physical memory. Testing memory - The program cannot test specific memory locations but it can exercise memory accessing mechanisms and quite a lot of RAM. Some experimentation is required to determine the maximum memory that can be tested. The main reason for producing data transfer speeds in this program is to identify whether memory or disk is being tested but watching the disk access light on the CPU is as good a guide as any. Examples of memory test results are given later. Testing networks - With the program on a disk on a different system on a network, clicking on the icon will cause the program to be downloaded but the files will be written on the server. This tests the client CPU and memory (up to the cache size limit) and the network. The data must be greater than the maximum client cache size to test the network. If the server has a greater memory capacity (and with an appropriate Operating System), an optimum file size can be chosen to exercise just the network and client CPU with the data residing in the server cache. 3.4 CDK1 PERFORMANCE MEASUREMENTS The program is designed to exercise hardware not as a performance test. The logical block size written to disk is only 2K bytes. The files are written serially. They are also read serially but blocks are chosen from the four files on a randomised basis. The program also compares every word read, using a lot of processor time on a slow CPU. Actual reading speeds depend very much on the disk hardware and system buffering arrangements. The unknown variable of reading ahead with multiple files can produce unexpected results - my 200MHz Pentium Pro system with what should be a superior disk and Windows NT 4.0 achieves a disk transfer speed of about half that on my P5/100 with Windows 95 (but it is a lot faster when data is cached). It is useful to look at system performance monitors when running the tests, particularly when testing networks. This can confirm that data is being transferred across the network and can show the CPU load on the server for reading disks and transferring data. With this test, the CPU loading for reading from cache and transmitting the data can be appallingly high - 42% CPU utilisation on the p6/200/NT4 PC for transferring data at 0.8 Mbytes per second via the LAN. 3.5 CDK1 COMPRESSED FILES The data is not really suitable for testing compressed disks as the blocks containing a single bit pattern are good candidates for compression, obtaining a compression ratio of greater than 20. This provides misleading performance information. On a test via Windows 3.1, the data in cache also appeared to remain compressed, producing some confusion on available memory capacity. 3.6 CDK1 MULTI-TASKING With an appropriate Operating System, the program can be multi-tasked by clicking on the icon a few times, entering the data up to number of read passes, then entering the value for each window in turn. Or a BAT file can be used. START CDK195NT 4 15 L Starts first job and goes to next command START CDK195NT 4 15 M Starts second job (after a few seconds?) Remember that the letter (L or M or whatever) must be included to permit correct operation and avoid confusing error messages. The BAT file can also start any other program, including a network test. A series of batches can be run by omitting “START” for the last test of a batch or by using START /WAIT CDK1.. With multi-tasking there is less control on forcing the system to read from cache. 3.7 CDK1 EXAMPLE OUTPUT (results files) Example 1 - D110732Z.TXT Disk reading Dell XPS Pro, 200 MHz P6, 32 MB memory, 256K cache, Quantum Fireball TM2110A 2.1 GB disk, 128K cache. CPU utilisation was 15%. Disk heads appeared to be thrashing about. ############################################## Disk test dk01 in C++ by Roy Longbottom Job D110732Z Server ppro Client ppro Disk C Free disk space 1164.3 Mbytes will use 28.2 0.22 seconds for data generation and display File writing starting on Tue Mar 18 11:07:50 1997 4 files written each 7389184 bytes (data*22) 8.57 seconds writing files at 3.29 MB/sec Tue Mar 18 11:07:59 1997 39.67 seconds read pass 1 at 0.71 MB/sec Tue Mar 18 11:08:39 1997 50.15 seconds read pass 2 at 0.56 MB/sec Tue Mar 18 11:09:30 1997 66.58 seconds read pass 3 at 0.42 MB/sec Tue Mar 18 11:10:37 1997 46.51 seconds read pass 4 at 0.61 MB/sec Tue Mar 18 11:11:23 1997 Finished Files Erased on Tue Mar 18 11:11:23 1997 Example 2 - D140355Z.TXT Same system as above 20 MB parameter, 100 read passes. Note the speed up when data is all in memory. Also note that the speed for each pass is displayed but, in the output file, full details are only displayed for the first ten read passes. All other passes are summarised with a single line. ############################################## Disk test dk01 in C++ by Roy Longbottom Job D140355Z Server ppro Client ppro Disk C Free disk space 1164.5 Mbytes will use 19.2 0.23 seconds for data generation and display File writing starting on Tue Mar 18 14:04:16 1997 4 files written each 5038080 bytes (data*15) 4.04 seconds writing files at 4.76 MB/sec Tue Mar 18 14:04:20 1997 10.78 seconds read pass 1 at 1.78 MB/sec Tue Mar 18 14:04:31 1997 2.61 seconds read pass 2 at 7.36 MB/sec Tue Mar 18 14:04:34 1997 0.39 seconds read pass 3 at 49.28 MB/sec Tue Mar 18 14:04:34 1997 0.42 seconds read pass 4 at 45.76 MB/sec Tue Mar 18 14:04:35 1997 0.39 seconds read pass 5 at 49.28 MB/sec Tue Mar 18 14:04:35 1997 0.39 seconds read pass 6 at 49.28 MB/sec Tue Mar 18 14:04:36 1997 0.38 seconds read pass 7 at 50.58 MB/sec Tue Mar 18 14:04:36 1997 0.37 seconds read pass 8 at 51.94 MB/sec Tue Mar 18 14:04:36 1997 0.40 seconds read pass 9 at 48.05 MB/sec Tue Mar 18 14:04:37 1997 0.37 seconds read pass 10 at 51.94 MB/sec Tue Mar 18 14:04:37 1997 90 more passes - total time given 34.11 seconds 1729.7 MB at 50.71 MB/sec Tue Mar 18 14:05:11 1997 Finished Files Erased on Tue Mar 18 14:05:11 1997 Example 3 - D101653Z.TXT Escom 100 MHz Pentium, 16 MB memory, 256K cache, Seagate ST5850A 850 MB 256K cache, Windows 95. Reading from disk (uncompressed G drive). ############################################## Disk test dk01 in C++ by Roy Longbottom Job D101653Z Server p100 Client p100 Disk G Free disk space 48.7 Mbytes will use 19.2 0.68 seconds for data generation and display File writing starting on Tue Mar 18 10:17:05 1997 4 files written each 5038080 bytes (data*15) 12.53 seconds writing files at 1.53 MB/sec Tue Mar 18 10:17:17 1997 15.11 seconds read pass 1 at 1.27 MB/sec Tue Mar 18 10:17:32 1997 15.05 seconds read pass 2 at 1.28 MB/sec Tue Mar 18 10:17:47 1997 15.99 seconds read pass 3 at 1.20 MB/sec Tue Mar 18 10:18:03 1997 14.80 seconds read pass 4 at 1.30 MB/sec Tue Mar 18 10:18:18 1997 14.71 seconds read pass 5 at 1.31 MB/sec Tue Mar 18 10:18:33 1997 Finished Files Erased on Tue Mar 18 10:18:34 1997 Example 3 - D094009Z.TXT P100 running with files on Pentium Pro with data transfer over LAN. The 16 MB data was mainly read from the Pro memory cache. The CPU utilisation on the Pentium Pro was 42%. ############################################## Disk test dk01 in C++ by Roy Longbottom Job D094009Z Server ppro Client p100 Disk F Free disk space 1164.5 Mbytes will use 15.4 0.65 seconds for data generation and display File writing starting on Tue Mar 18 09:40:30 1997 4 files written each 4030464 bytes (data*12) 26.85 seconds writing files at 0.57 MB/sec Tue Mar 18 09:40:57 1997 25.57 seconds read pass 1 at 0.60 MB/sec Tue Mar 18 09:41:22 1997 27.21 seconds read pass 2 at 0.57 MB/sec Tue Mar 18 09:41:50 1997 26.40 seconds read pass 3 at 0.58 MB/sec Tue Mar 18 09:42:16 1997 19.91 seconds read pass 4 at 0.77 MB/sec Tue Mar 18 09:42:36 1997 16.40 seconds read pass 5 at 0.94 MB/sec Tue Mar 18 09:42:52 1997 16.64 seconds read pass 6 at 0.92 MB/sec Tue Mar 18 09:43:09 1997 17.17 seconds read pass 7 at 0.90 MB/sec Tue Mar 18 09:43:26 1997 16.59 seconds read pass 8 at 0.93 MB/sec Tue Mar 18 09:43:43 1997 21.25 seconds read pass 9 at 0.72 MB/sec Tue Mar 18 09:44:04 1997 23.77 seconds read pass 10 at 0.65 MB/sec Tue Mar 18 09:44:28 1997 Finished Files Erased on Tue Mar 18 09:44:28 1997 4. PROGRAM CPR4DOS, CPR495NT and CPR4OS2 This program carries out nine sets of computations based on known formulae where the result of one calculation has a theoretical value of 1.0. The calculations are repeated (mainly 500,000 times) using a wide range of numbers for variables. The results are multiplied together so any errors are likely to be carried forward to the final answer. Most calculations are in double precision. In order to facilitate easy visual checking, an integer sumcheck is calculated, designed to show a significantly different number for a small difference in a floating point result. An example of information displayed with manual input is: PR04 C++ Functions Test Copyright © Roy Longbottom 1997 Based on FOPR04 a TSU Acceptance Trials Program from another era Example Sumcheck Pentium 80486DX2 Functions Answer Sumcheck Sin, Cos 0.99999999999445120 -5532 -5532 -5532 Exp, Tanh 1.00000000000012900 5776 5776 5776 Log, Exp 0.99999999999111260 -6174 -6174 -6789 Log10, Sqrt 1.00000000000002800 3440 3440 4263 Complex 0.99996001225587480 5038 5038 4551 Imag, Real 1.00000000000051100 4756 4756 4756 Max, Min 0.99999999999253160 -6210 -6210 -6210 Atan, Fmod 0.99999999999363160 -5941 -5941 -5508 Binomial 1.00000000000094100 4574 4574 4574 Time taken 83.70 seconds Start at Thu Mar 20 09:30:25 1997 End at Thu Mar 20 09:31:49 1997 Enter minimum run time minutes to check for consistent results 1 Extra pass 1 of 1 Functions Answer Sumcheck Sin, Cos 0.99999999999445120 -5532 Result OK Exp, Tanh 1.00000000000012900 5776 Result OK Log, Exp 0.99999999999111260 -6174 Result OK Log10, Sqrt 1.00000000000002800 3440 Result OK Complex 0.99996001225587480 5038 Result OK Imag, Real 1.00000000000051100 4756 Result OK Max, Min 0.99999999999253160 -6210 Result OK Atan, Fmod 0.99999999999363160 -5941 Result OK Binomial 1.00000000000094100 4574 Result OK Time taken 84.10 seconds 0.48 seconds difference from first run Start at Thu Mar 20 09:32:30 1997 End at Thu Mar 20 09:33:54 1997 ************************* 0 errors so far (see output file C092417.TXT) Finished 4.1 CPR4 EXE PARAMETERS Two parameters are can be included for running time in minutes (after the first pass) and a letter that will create different output files for programs starting at the same time when multi-tasking. Examples of command lines to run the programs are: CPR4DOS 10 Dos version run for 10 minutes CPR495NT 60 R NT/W95 version, run for an hour, add R to results file name. 4.2 CPR4 MULTI-TASKING As with the disk tests, the CPU tests can be run concurrently under a multi-tasking Operating System. An example of running three versions with different suffixes for the log files is via a BAT file with: START CPR495NT 20 A START CPR495NT 20 B START CPR495NT 20 C The running time of 20 minutes may not be accurate as the number of passes required is derived from the time taken by the first pass. With multi- tasking this will be variable. If all programs start within a few seconds of each other and are given equal share of the CPU time, in principle, the total time for the batch should be around 20 minutes. 4.3 CPR4 RESULTS FILES These have the format CHHMMSSZ.TXT, the same as CDK1, except starting with C. Again, the Z is replaced by letters included in the command line. The file contains the first results as above, run times for extra passes and any standard output from any routines where a different numeric answer is produced. A typical output following the floating point results is: Ending at Thu Apr 10 11:05:35 1997 Time taken 39.79 seconds Extra pass 1 of 16 Start at Thu Apr 10 11:06:30 1997 Time taken 40.60 secs 0.81 secs difference from first run Extra pass 1 of 16 End at Thu Apr 10 11:07:11 1997 Extra pass 2 of 16 Start at Thu Apr 10 11:07:11 1997 Time taken 41.06 secs 1.27 secs difference from first run Extra pass 2 of 16 End at Thu Apr 10 11:07:52 1997 . . . . . . Extra pass 16 of 16 Start at Thu Apr 10 11:19:01 1997 Time taken 39.88 secs 0.09 secs difference from first run Extra pass 16 of 16 End at Thu Apr 10 11:19:41 1997 Finished 0 errors 4.4 EXAMPLE RUNNING TIMES Typical running times per pass are as follows: Pentium Pro 200 MHz 40 seconds Pentium 100 MHz 84 seconds 80486DX2 66 MHz 298 seconds 4.5 DIFFERENT OUTPUT The program can produce slightly different results on different CPUs and (deliberately) this is magnified out of all proportion by the sumcheck. These different results cannot be regarded as incorrect, only calculations to a different accuracy level. They are probably caused by differences in rounding on certain instructions. The main issue is that the program should produce consistent results on a given CPU. Examples of differences between 486 and Pentium CPUs are: 80486DX2 Pentium & Pentium Pro Log, Exp 0.99999999999111480 -6789 0.99999999999111260 -6174 Log10, Sqrt 1.00000000000002900 4263 1.00000000000002800 3440 Complex 0.99996001225587020 4551 0.99996001225587480 5038 Atan, Fmod 0.99999999999361820 -5508 0.99999999999363160 -5941 Note that 64 bit double precision numbers are accurate to 15 - 16 significant decimal digits (the print out is for 17 - 18 digits, hence 0’s at the end). Temporary results are 80 bits with an accuracy of 19 significant decimal digits. The results are produced by multiplying together numerous inaccurate calculations of 1.0. As the final answers should be 1.0, the produced results are accurate to between 4 and 14 decimal digits. Inaccuracy due to hardware peculiarities is beyond this natural level.