Cluster/Supercomputer definition
Cluster/Supercomputer. A supercomputer is an aggregation of nodes connected through a high-speed network with a specialised topology. We can find different network topologies (i.e., how the nodes are connected), such as a 2D or 3D torus or hypercube. The kind of network topology will determine the number of hops that a message will need to reach its destination or communication bottlenecks. A supercomputer usually includes a distributed file system to offer a unified view of the cluster from the user point of view. The parallelism that can be used at the supercomputer level is a distributed memory approach. In this case, different processes can run in different nodes of the cluster and communicate through the interconnect network when necessary. We have seen all the computational elements that form a supercomputer from a hierarchical point of view. All the levels explained above also include different levels of storage that are organised in a hierarchy too. Starting from the core, we can find the registers where the operands of the instructions that will be executed are stored. Usually included also in the core or CPU, we can find the first level of cache. This is the smallest and fastest one; it is common that it is divided into two parts: one to store instructions (L1i) and another to store data (L1d). The second level of cache (L2) is bigger, still fast, and placed close to the core too. A common configuration is that the third level of cache (L3) is shared at the socket and L1 and L2 are private to the core, but any combination is possible. The main memory can be of several gigabytes (GB) and much slower than the caches. It is shared among the different processors of the node, but as we have explained before it can have a non-uniform memory access (NUMA), meaning that it is divided in pieces among the different sockets. At the supercomputer level, we find the disk that can store petabytes of data. For instance BSC's MareNostrum IV has a total capacity of 24.6 petabytes, EPCC's ▇▇▇▇▇▇ 4.4 petabytes and SURFsara's Cartessius 7.7 petabytes. At this level, the three patterns reveal all their complexity and variety. Monolithic applications stress the OpenMP/MPI interconnects in a relatively uniform way, with eventually load balance problems which can be addressed with careful programming and/or smart middleware. Coupled applications have the same problems, but adding the difficulties coming heterogeneous architectures on which we can deploy the different components of a coupled a...