[ The PC Guide | Systems and Components Reference Guide | The Processor | Processor Architecture and Operation | Processor Performance ]

Processor Benchmarking

Benchmarking is the process of evaluating and comparing devices or systems. Processors are frequently benchmarked, in the hopes of coming up with a single number that can capture the value of the CPU and let it be easily compared to others. The value of benchmarking is that it allows this comparison. The problem with it is that there are so many ways to do it, and the answers different benchmarks give about the same processors aren't always consistent.

There is a difference between benchmarking processors and benchmarking whole systems. System benchmarking attempts to evaluate "real world" performance of the whole system, which involves much more than just the processor. Processor benchmarks show isolated performance of processors relative to one another.

Even in just looking at processors, there are many types of benchmarks, which can result in many different scores. Since the processor doesn't operate in a vacuum, the scores will always depend to some extent on the system that is used when performing the benchmark. Some benchmarks are more able to isolate the processor's performance from impact by other system components than others. In addition, benchmarks that are based on Windows or other multitasking operating systems, are subject to accuracy problems induced by how the software is set up, and what other programs might be in operation when they are run.

Some benchmarks can only be used by doing relative comparisons; since the absolute number that is produced depends on the configuration, it can realistically only be used to compare two processors when used in the same exact configuration (much the way the "P rating" calculation is done.) An absolute number from this sort of benchmark can only be given as an average of scores from many system configurations used to run it, or only given with also providing information about the configuration used. Other benchmarks provide roughly the same score for the processor across a wide array of system configurations.

In addition, the benchmark's score depends on what types of code it is executing to do the test. Is it 16-bit or 32-bit code? Many loops or just a few? There are many different types of code and some architectures are better than others at certain types. The power of some of the newer processors is understated when they are tested using older benchmarks (such as the original Norton SI index.) Also remember that benchmarks typically contain some floating-point code, which will mean older processors with no floating point unit will get lower scores.

Here are some relatively common processor tests, which I use on this site to compare processor performance. In doing so I am not claiming that these are necessarily the best measures, but I believe that combined they provide a reasonable picture of processor speed without getting overly bogged down in system-level dependencies:

• iCOMP: This is an acronym for the Intel COmparative Microprocessor Performance index. This is the "official" benchmark that Intel has chosen to use to state the relative performance of its processors. It is in fact a blending of the results from several other benchmarks. Intel used this rating to produce values for processors ranging from the 386SX-16 to the Pentium 166, before replacing it with iCOMP 2.0.
• iCOMP 2.0: This is Intel's revised iCOMP benchmark, used for Pentium 75 and later processors. It too is an amalgam of several other benchmarks (including SI32 and CPUmark32). It focuses more on 32-bit performance than the original iCOMP index, and also partially incorporates a multimedia benchmark. The focus on 32-bit tasks and multimedia make this test give higher scores to Pentium Pro and MMX-capable processors, relative to the "classic" Pentium.
• Norton SI: This is one of the earliest universal benchmarks, created by Peter Norton of Norton Utilities fame and used as part of the Norton System Information (sysinfo) utility. This benchmark has been around for years, and in fact is one of the few for which scores are available even for older processors. It is however an outdated benchmark, and really doesn't tell the whole story when it comes to modern software being run on modern processors. Still though, its universality and the fact that it gives a quick-and-easy performance benchmark relative to the first IBM PC (which is defined to be 1.0) keeps it in use still today. It is sometimes called "SI 8.0" after the last DOS version of the Norton Utilities to use it, although earlier versions tend to return the same values.
• Norton SI32: This is an updated version of the original Norton SI index and produces scores that are to a different scale than the old benchmark. It operates under Windows 95 and uses 32-bit code to perform its assessment. Scores are relative to the 386SX-16, which is defined as 1.0. Note that this benchmark does depend to some extent on the system that it is put into, so two systems with the same processor are likely to yield different results for this test.
• CPUmark32: These is one of two processor benchmarks (the other being CPUmark16) that are part of the Ziff-Davis Benchmark Operation (ZDBOp) WinBench benchmark set. They test performance executing 32-bit and 16-bit code, respectively. They are widely used and are somewhat dependent on particular machine setup.

Next: Performance Benchmarks for Specific Processors

Home  -  Search  -  Topics  -  Up