In 1999 Microsoft funded a benchmark comparing Windows NT Server 4.0 with Linux 2.2.2. The result hit the Linux community hard: Windows appeared to be 2.5 times faster at file serving and 3.7 times faster as a web server. The methodology and Mindcraft, the company that ran the tests, were immediately attacked. Two months later the benchmark was repeated with Red Hat engineers present and under the supervision of PC Week Labs. Windows still won. This time, the result could no longer be dismissed simply because Microsoft had paid for the original test.

The key points of the 1999 Linux vs. Windows benchmark in 30 seconds

Microsoft paid Mindcraft to compare Windows NT Server with Linux 2.2.2.
The first test strongly favored Windows and was criticized for uneven tuning.
PC Week Labs later repeated the benchmark with Red Hat, Microsoft, and Mindcraft involved.
Windows still came out ahead, although some of the original problems disappeared.
The episode helped expose real SMP and networking bottlenecks that Linux developers fixed over the following months.

The story looks very different from 2026, when Linux dominates huge parts of server infrastructure, cloud computing, and supercomputing. But that position was far from guaranteed in the late 1990s.

Linux 2.2 had only recently arrived, and one of its major promises was improved support for symmetric multiprocessing, or SMP. Four-processor servers were becoming relevant in enterprise workloads, while Microsoft had already spent years optimizing Windows NT for that kind of hardware.

That was the environment in which Mindcraft published its benchmark.

Microsoft paid for the test, and Linux performed badly

Mindcraft published the comparison on April 13, 1999. The test system was a Dell server with four Pentium II Xeon processors and four Fast Ethernet network interfaces.

Windows NT Server 4.0 was compared with Red Hat Linux 5.2 updated to the Linux 2.2.2 kernel. Samba was used for file-serving tests and Apache for HTTP workloads.

The report itself disclosed that Microsoft Corporation sponsored the testing.

That did not automatically make the results false, but there were legitimate technical reasons to question the comparison.

Windows had received workload-specific tuning. Each network interface was associated with a processor, and several networking parameters had been adjusted.

The Linux setup was far less optimized. It relied on a very new kernel, Apache and Samba with comparatively limited tuning, and a Linux 2.2 networking stack that was still struggling to scale efficiently when several CPUs and several network cards were active at the same time.

The published result was brutal: Windows NT delivered roughly 2.5 times the file-server performance and 3.7 times the web-server performance of Linux.

The backlash was immediate.

Alan Cox, one of the most important Linux kernel developers at the time, criticized the methodology and the fact that Linux specialists had not had equivalent access to the test system. Others raised the same concern: Microsoft had funded the study, provided tuning expertise for Windows, and Linux had not been configured by engineers with the same depth of knowledge.

That criticism was reasonable.

But the controversy took an unexpected turn.

Red Hat got access to the lab, and Windows still won

Bruce Weiner, president of Mindcraft, responded by proposing an open rematch.

The first version of the offer still did not satisfy the Linux side. Alan Cox argued that the testing should be carried out by an independent organization, while Red Hat president Bob Young suggested using a recognized benchmarking lab.

That organization became PC Week Labs, part of Ziff-Davis.

In June 1999 representatives from Red Hat, Microsoft, Penguin Computing, and Mindcraft gathered for a new round of testing. This time, Linux specialists were allowed to tune their own system.

The setup was much harder to dismiss as propaganda.

The new benchmark corrected several criticisms raised after the April test, and some results changed. Linux no longer collapsed under certain heavy workloads, and at lighter loads its performance was much more competitive.

But under higher network load and on a multiprocessor configuration, Windows NT still remained ahead.

That changed the discussion.

The first Mindcraft benchmark had methodological problems, but the independent retest showed that those problems did not explain the entire performance gap.

Linux really did have scalability bottlenecks.

The bottleneck was inside the kernel

One of the key problems appeared in the way Linux 2.2 handled network traffic on SMP systems.

The benchmark machine had four network cards and four processors. That was exactly the type of configuration capable of exposing code that worked well on a single CPU but failed to scale when several processors tried to process packets simultaneously.

Analysis during and after the retest showed significant time being spent around synchronize_bh(), related to Linux’s old bottom half mechanism for deferred interrupt processing.

The practical problem was that parts of the networking workload were still effectively serialized.

Several network adapters could deliver packets at the same time, but important sections of packet processing could not scale across all four CPUs efficiently.

This is a classic systems problem: adding more processors does not help if they all have to wait for the same lock or global queue.

Linux developers later reworked these areas aggressively.

One major change was softnet, a redesign of the network processing architecture that entered the Linux 2.3 development series in early 2000. Among other things, it moved toward per-CPU networking structures so multiple processors could process network traffic in parallel.

Another bottleneck involved the Big Kernel Lock and poll().

poll() is used by servers to determine which file descriptors have pending activity. On systems handling large numbers of connections, forcing multiple CPUs through a global lock simply to perform that check could become another point of serialization.

In April 2000, developers discussed removing that global locking requirement from poll(), another example of how Linux was gradually eliminating the shared bottlenecks that limited SMP scalability.

The Linux engineers could tune the system, but they could not tune away the kernel

One of the most telling parts of the 1999 retest was what happened when Red Hat engineers were given control of the machine.

They adjusted Linux themselves.

The results barely moved.

In the file-serving benchmark, Mindcraft’s Linux setup achieved around 127.2 megabits per second. Red Hat’s own tuning brought that to roughly 127.7 megabits per second.

That is a difference of only about 0.4%.

The same thing happened on the web benchmark. Red Hat engineers did not suddenly discover a configuration switch that doubled performance.

They even replaced Apache with the Zeus web server to determine whether Apache itself was the bottleneck.

The result remained around the same limit.

That was a strong signal that the problem sat below the application layer.

Profiling pointed back toward the kernel.

In other words, the Linux community had initially assumed that poor tuning explained most of the result. Once its own engineers had access to the system, that explanation became much harder to maintain.

A year later, the numbers looked completely different

By 2000, the Linux networking and SMP landscape was already changing rapidly.

It is important not to turn later benchmark results into a direct one-to-one rematch, because the software, hardware, and benchmark methodology had changed.

But the direction of travel was striking.

In April 2000, a Dell PowerEdge 6400 with four 700 MHz Pentium III Xeon processors, Windows 2000 Advanced Server, and IIS 5.0 produced a certified SPECweb99 score of 1,598.

A few months later, another Dell PowerEdge 6400 with four 700 MHz Pentium III Xeons running Red Hat Linux 6.2 and the TUX web server achieved 4,200 SPECweb99.

SPECweb99 on Dell PowerEdge 6400/700	Result
Windows 2000 Advanced Server + IIS 5.0	1,598
Red Hat Linux 6.2 + TUX 1.0	4,200

That does not prove that Linux had become universally 2.6 times faster than Windows.

TUX was a highly optimized web server that ran partly inside the kernel, and the two configurations were not identical in every respect.

But it does show something important: the scalability ceiling exposed in 1999 was no longer the same.

Linux had changed.

The benchmark did not prove that Windows was “better than Linux”

It is tempting to turn the story into a simple revenge narrative.

Microsoft funded a benchmark. Linux lost. The community got angry. Developers fixed the kernel. Linux later posted much stronger numbers.

The real lesson is more useful than that.

Mindcraft tested one workload, one hardware configuration, specific software versions, and a particular set of tuning choices.

Microsoft sponsorship, uneven tuning, and flaws in the first test fully justified scrutiny.

But the independent retest also showed that those issues did not explain everything.

Linux had genuine scalability problems.

The most important ones were buried in places most users never saw: networking internals, SMP synchronization, deferred interrupt handling, global locks, and shared queues.

That is what makes the episode still relevant for sysadmins and developers today.

A benchmark can be biased in its setup and still expose a real technical weakness. It can be sponsored by a competitor and still contain useful data. Criticizing methodology is necessary, but reproducing the result, profiling the system, and finding where the CPU cycles are actually going is usually more productive than arguing about who paid for the test.

In 1999 the Linux community did both.

First, it argued.

Then it opened the profiler.

The second part is what changed the kernel.