tecCHANNEL Benchmark Suite

14.03.2002 von Michael Eckert und THOMAS LOPATIC 
Details given by manufacturers often are very optimistic and do not always correspond to reality. Who wants to gain informations on the actual performance of one's hardware can now check it exactly with the tecCHANNEL Benchmark Suite.

If a PC does not run smoothly there is a variety of potential sources for the experienced problems. For example some components may not be configured optimally or may even be defective. Does a user have the impression that his computer is very slow, the PC may suffer from messed up BIOS settings which reduce for instance the CPU cache or memory performance. Still, a check of any performance results given by manufacturers is advisable anyway, not only in case of problems.

At present our tecCHANNEL Benchmark Suite contains three hardware-related test programs. tecBench is the graphical interface integrating the two benchmarks tecHD and tecCD/DVD for evaluating mass storage devices. tecMem, the third program of the trio, is an independent program with its own graphical user interface tailored to its needs. tecMem determines the available bandwidth for data transfers between the CPU and the different levels of the memory hierarchy (caches, main memory).

The tecCHANNEL editorial staff uses this software for nearly all its hardware tests. After two years of development we can now put a free version of our low-level benchmarks at your disposal.

Safety Notes

For precise hardware-related measurements the tecCHANNEL Benchmark Suite has to access the hardware as directly as possible. Part of it is thus implemented as a driver. A failure of this driver component, however, does usually not only result in a crash of the benchmarks but also in a crash or re-boot of the operating system. Hence, a driver failure will potentially bring about far more dramatic consequences - for example loss of data because of inconsistency in the file system - than problems with classic applications.

So please only use the tecCHANNEL Benchmark Suite if you are fully aware of the risk and the potential effects involved in using the software. tecCHANNEL, IDG publishers and the programming team do not accept any responsibilities for potential damages by running the tecCHANNEL Benchmark Suite.

Before starting the benchmarks you should close all applications. This absolutely has to be done for safety reasons, otherwise any unsaved data would be at risk in case of a program crash.

Moreover, while running the test other programs could access the mass storage device that is being evaluated, and this would lead to inaccurate results. At the very start tecBench and tecMem remind you of all this via dialogue box. It is absolutely recommended to test only unused drives with tecBench. It is recommendable to boot Windows from a separate hard drive and to integrate the drive that is to be tested into the system as a second, empty hard drive.

tecMem severely interferes with the operating system. That is why you should use tecMem even more cautiously than the tecBench software.

Do only start the programs of the tecCHANNEL Benchmark Suite on computers exclusively reserved for hardware tests. A test run on a PC containing important data therefore has to be ruled out. If any malfunction of the test-PC may put other parts of the system at risk - for example in a network - you also better do not use the tecCHANNEL Benchmark Suite.

Installation

The tecCHANNEL Benchmark Suite can only be run under Windows NT 4.0 with service pack 6, Windows 2000 and Windows XP. Apart from that you must have administrative privileges. tecMem requires at least 64 MBytes of main memory. Please do closely follow the safety notes of the software as well as those in this article.

The tecCHANNEL Benchmark Suite is shipped as an MSI file. It can comfortably be installed with the Windows Installer and can be deleted by means of the Control Panel. The Windows Installer is an essential part of Windows 2000 and Windows XP. A version of the Windows Installer for Windows NT 4.0 with service pack 6 is available on the Microsoft website.

A double click on the MSI file starts the installation. The data needed is copied and two entries are inserted in the start menu

If you want to carry out an upgrade of the tecCHANNEL Benchmark Suite please make sure you de-installed the older version beforehand. The de-installation works in conformity with Windows via its Control Panel.

tecBench

tecBench combines two benchmarks under one graphic interface:

tecHD as well as tecCD measures data transfer rates achieved while reading sequentially from a given media and access times observable while accessing a series of pseudo-randomly chosen sectors of a given medium.

tecBench Operating Instructions I

Via "Help" menu you will find a link to this German (Alt+G) as well as to this English (Alt+O) version of the on-line help for tecBench. The on-line help will call up a corresponding site of www.tecChannel.de.

The graphic interface of tecBench is split into three property pages. In the property page "Selection" you choose the mass storage device that is to be evaluated via a list box, in this box tecBench lists the logical drives available ("A:" to "Z:") and the recognised physical drives ("Phys0", "Phys1", ...):

For each recognised mass storage device tecBench indicates whether it will evaluate it with tecHD or tecCD. The decision for one of the two benchmarks depends on the features for the drive given by the operating system and is made automatically.

tecBench Operating Instructions II

A double click on one of the drives or a click on the start button starts the benchmark run and activates the property page "Benchmark". While running the test the results are shown in the upper graph "Transfer rate". In the bar chart ("Transfer rate") on the left below you find a table that summarizes the measured minimal ("Min", yellow), average ("Avg", blue), and maximum ("Max", green) transfer rate.

The second test run adds the burst rate ("Burst", red) to the bar chart. The burst rate is the maximum transfer rate that is possible during read access from the drive's cache. The third benchmark run determines the drive's access times and shows them in the bar chart on the right ("Access time").

Measurements with tecHD determine the average access time within the drive's first 500 MBytes ("500M", yellow) and for the complete drive ("Avg", blue). Benchmark runs with tecCD principally use the complete media/drive and show the measured minimal ("Min", yellow), average ("Avg", blue), and maximum ("Max", green) access time. Via the Export button you can save the graph and the bar charts as BMP image files. With the Reference button the average transfer rate and the access time are added to the reference database.

The third property page "Evaluation" releases its functionality not until the test is done. Here the average transfer rate and the average access time of the current benchmark run can be compared with the results from the reference database. The Add button inserts another record from the reference database in the two charts, the Remove button removes it. Via Export button also these charts can be saved as image files.

tecMem

tecMem measures the effectively used memory bandwidth between the CPU's Load/Store unit and the different levels of the memory hierarchy (L1, L2 cache and RAM). The basic idea here is to read the same memory block again and again for many iterations. If the memory block is small enough, for example 1 KByte, then its content is contained completely in the L1 cache after the first iteration, and all future iterations can be served from the L1 cache. Therefore the bandwidth between the Load/Store unit and the L1 cache is measured.

If you have a block of 128 KBytes for example, L1 caches of current CPUs are no longer sufficient, i.e. too small, for the complete amount of data. The caching algorithm always buffers different areas of the 128 KBytes block, the cache-hits in the L1 cache become rare. In this case, the L2 cache comes to fruition which can still take up the complete block. Now the bandwidth between the Load/Store unit and the L2 cache makes a major contribution to the performance and is measurable.

The same goes analogously for even bigger blocks, for example of 4 MBytes. This amount of data does typically not even fit into the L2 cache. The L2 cache-hits then also become rare and the bandwidth between the Load/Store unit and the main memory can be measured.

tecMem requires a minimum of 64 MBytes of main memory. The benchmark does not work in multi-processor systems or in Windows configurations supporting more than 4 GBytes main memory by means of Physical Address Extension (PAE). In PAE systems though the parameter "/NOPAE" in the "boot.ini" should remedy this and allow a perfect test.

tecMEM Operating Instruction

With a mouse click on the start button the benchmark run begins. Depending on the CPU tecMem measures the bandwidth with 32 bits, 64 bits MMX and 128 bits SSE transfer instructions. During the benchmark run the system seems to freeze, the mouse pointer only moves jerkily. That is a driver effect, the driver closes down the whole operating system during each measurement of a given block size and only wakens the OS shortly while changing the block size.

After the benchmark run you can change with the 32 bits, 64 bits and 128 bits buttons among the measured bandwidths of the 32 bits, 64 bits MMX and 128 bits SSE commands. You will find the program in the 32 bits window when the measurements are done.

All results are shown in the graph "Memory Performance". The x-axis gives the size of the used block, on the y-axis you see the bandwidth in MBytes/s. Below the graph and the buttons there is a text field where you can find a table with the measured bandwidths of the different block sizes and commands.

With the Export button you can save the presently shown graph as BMP image file.

tecBench Functionality

The two partial tecBench programs - tecHD and tecCD/DVD - open the drive that is to be tested with CreateFile() using the flags FILE_FLAG_NO_BUFFERING. For measuring the transfer rate the benchmarks look at the complete capacity of the drive as if it consists of 100 single successive sections of identical size. Starting at the beginning of the sections tecHD/tecCD reads a data block of 9600/3200 KBytes and then goes on with the next one. The positioning at the beginning of a section is carried out by means of Set FilePointer(), the reading of the data blocks by ReadFile(). tecHD splits up the read process in 150 ReadFile() calls with partial blocks of 64 KBytes. tecCD/DVD characteristically uses 50 calls.

For determining access times tecHD and tecCD read in pseudo-randomly selected sectors with SetFilePointer() and ReadFile(). In addition to the positioning time of the write/read head the benchmarks measure the latency between the positioning of the head and the actual reading of the sector. tecHD reads 1000 different sectors, tecCD is limited to only 200 sectors because of its higher access times. The burst rate is measured by reading the same sectors again and again. For this tecHD reads the same 64 KBytes block over 1000 iterations. tecCD/DVD does not measure any burst rates.

tecMem Functionality

At the very start tecMem registers its driver in the Service Control Manager. After the test it is de-registered and with that completely removed from the system. During the measurments the benchmark masks all interrupts.

The driver puts the virtual addressing in a fixed configuration that is optimized on the benchmark. It provides 16 MBytes physical RAM between 48 and 64 MBytes at a "fixed" virtual address. 8 MBytes of these 16 MBytes serve as source buffer, the other 8 MBytes as target buffer.

In case of 32 bits the optimized machine commands LODSD, STOSD and MOVSD are used for Load, Store and Move. For 64 bits Loads and Stores tecMem utilizes the MOVQ command from the MMX command set. The 128 bits tests are running with the SSE instruction MOVDQA. Load commands operate on the source buffer, Stores on the target buffer. Moves use both buffers while reading data from the source buffer and writing data into the target buffer.

In case the CPU supports it 4 MBytes Pages are used to relieve the Translation Look-Aside Buffer (TLB, a kind of cache for the necessary address conversion tables). For further relief Global Pages get switched off. And in addition, the driver activates Write Back Caching for used memory areas. (mec/bmu)

Download

The Software is designed exclusively for use on computers solely dedicated to hardware evaluation and which, therefore, in particular neither provide critical functionality nor store critical information. Use of the Software is permitted solely in such environment for purposes of hardware evaluation.

Please note that the use of the Software implies risks, which are beyond our control. For technical reasons, the Software needs to seriously interfere with the operation of your computer system. Running the Software may cause damage to or loss of data or other unpredictable problems. This risk is substantially higher than the risk involved in running classic applications. Therefore, the Software may be used only by experts that fully understand the implied risks.

For Windows NT 4.0 with Service Pack 6 / Windows Installer, Windows 2000, Windows XP only. Download: tecCHANNEL Benchmark Suite version 020218