Supercomputing on Windows Clusters Experience an…….pptVIP

Supercomputing on Windows Clusters: Experience and Future Directions Andrew A. Chien CTO, Entropia, Inc. SAIC Chair Professor Computer Science and Engineering, UCSD National Computational Science Alliance Invited Talk, USENIX Windows, August 4, 2000 Overview Critical Enabling Technologies The Alliance’s Windows Supercluster Design and Performance Other Windows Cluster Efforts Future Terascale Clusters Entropia External Technology Factors Microprocessor Performance Micros: 10MF - 100 MF - 1GF - 3GF - 6GF (2001?) = Memory system performance catching up (2.6 GB/s 21264 memory BW) Killer Networks LAN: 10Mb/s - 100Mb/s - ? SAN: 12MB/s - 110MB/s (Gbps) - 1100MB/s - ? Myricom, Compaq, Giganet, Intel,... Network bandwidths limited by system internal memory bandwidths Cheap and very fast communication hardware Rich Desktop Operating Systems Environments Desktop (PC) operating systems now provide richest OS functionality best program development tools broadest peripheral/driver support broadest application software/ISV support Critical Enabling Technologies Critical Enabling Technologies Cluster management and resource integration (“use like” one system) Delivered communication performance IP protocols inappropriate Balanced systems Memory bandwidth I/O capability The HPVM System Goals Enable tightly coupled and distributed clusters with high efficiency and low effort (integrated solution) Provide usable access thru convenient standard parallel interfaces Deliver highest possible performance and simple programming model Delivered Communication Performance Early 1990’s, Gigabit testbeds 500Mbits (~60MB/s) @ 1 MegaByte packets IP protocols not for Gigabit SAN’s Cluster Objective: High performance communication to small and large messages Performance Balance Shift: Networks faster than I/O, memory, processor Fast Messages Design Elements User-level network access Lightweight protocols flow control, reliable delivery tightly-coupled link, buffer, and I/O bus management