HT 3.0 Spec Released

AMD led HyperTransport Consortium just released the HT 3.0 standard. The new features include:

1) 2.6GHZ clock, up to 20.8GB/s (166.4 Gb/s) per link bandwidth
2) AC mode allows links up to 1 meter length for chassis to chassis connections.
3) Auto link splitting between 2x8 or 1x16 links
4) Hot plugging

The extended HT link length allows one to connect multiple individual Opteron servers into higher order SMP boxes. Fujitsu-Siemens have a 8 way Opteron server which is made of four tighly packed 2P Opteron blades connected via HT. Now, one can have Opteron servers farther away forming larger SMP machines. The additional latency due to 1 meter wire length is less than 4ns. A 32P SMP machine can be built by linking 8 four way 1U servers. Since you can hot plug the HT links, you can even add and remove servers (CPUs) on the fly. This is like a cluster, but you are building a true SMP box, running a single instance of a standard SMP OS within a single memory space. This will revolutionize server computing! SUN and IBM better hurry and do more Opteron.

HT 3.0 is 100% backward compatible with previous generations of HT, which means an old NF4 chipset can be hooked onto a newer Opteron with HT 3.0, even though it's less efficient.

I expect AMD's Rev F Opterons to utilize HT 3.0, and Broadcom to supply a lot of the server chipsets (HT bridges with a whole bunch of features integrated).

Currently, an Opteron CPU has three 1GHZ HT links, all of which can be coherent (Opteron 8xx). A coherent HT link can be used for I/O or IPC, but a non-coherent link can only be used for I/O. A next generation socket 1207 Opteron may have more HT links, coupled with the increase in bandwidth, the next generation Opteron servers will be super strong. Even assmuming a Rev F Opteron has three HT links and one DDR2 800MHZ memory controller, the total bandwidth for a 1P Opteron will be 3* 20.8 + 12.8 = 75.2 GB/s. The total bandwidth for a 2P opteron will be 4*20.8 + 2 * 12.8 = 108.8 GB/s.

In comparison, a 1P Woodcrest will have a total bandwidth of 10.2GB/s with a 1333MHZ FSB (64 bit), not enough to pump DDR2 800MHZ (128 bit).