In computer science, spatial architectures are a kind of computer architecture leveraging many collectively coordinated and directly communicating processing elements (PEs) to quickly and efficiently run highly parallelizable kernels. The "spatial" term comes from processing element instances being typically arranged in an array or grid, both logically and in the silicon design. Their most common workloads consist of matrix multiplications, convolutions, or, in general, tensor contractions. As such, spatial architectures are often used in AI accelerators. The key goal of a spatial architecture is to reduce the latency and power consumption of running very large kernels through the exploitation of scalable parallelism and data reuse. Consider a kernel, i.e. a function to be applied to several inputs, expressed as one or more loops; this means distributing its computations between processing elements while ensuring that their data dependencies land either within the same element or the same region of elements. While spatial architectures can be designed or programmed to support different algorithms, each workload must then be mapped onto the processing elements using specialized dataflows. Formulating a mapping involves the assignment of each operation to a processing element and the scheduling of the ensuing data movements. All tuned to maximize data parallelism and reuse. Spatial architectures are classifiable as a SPMD (or single function multiple data) array processor, in that each processing element runs the same operations on a different subset of data, yet they are still programmed through a single mapping. The architecture of an individual processing element can then itself belong to any Flynn class. In particular, spatial architectures are well suited for applications whose dataflow exhibits producer-consumer relationships (e.g., parallel reduce) or can leverage efficient data sharing among a region of PEs. Spatial architectures can typically be found as hardware accelerators in heterogeneous systems, under the broader category of manycore processor.
Design details Core element of spatial architecture is its multidimensional array of processing elements. Each processing element is simple, namely a multiply-and-accumulate functional unit, a stripped-down core, or application-specific logic. Processing elements are then connected with each other and the memory hierarchy through busses or a network on chip, or even asynchronous logic. The memory hierarchy is explicitly managed and may consist of multiple on-chip buffers, like register files, scratchpads, and FIFOs, backed by large off-chip DRAM and non-volatile memories. The number of processing elements, interconnect bandwidth, and amount of on-chip memory vary widely between designs and target applications. From thousands of processing elements and tens of megabytes of memory for high-performance computing to tens of elements and a few kilobytes for the edge. The key performance metrics for a spatial architecture are its consumed energy and latency when running a given workload. Due to technology and bandwidth limitations, the energy and latency required to access larger memories, like DRAM, dominate those of computation, being hundreds of times more than what's needed for storage near processing elements. That's why a spatial architecture's memory hierarchy is intended to localize most repeated value accesses on faster and more efficient on-chip memories, exploiting data reuse to minimize costly accesses.
Data reuse
The mechanisms that enable reuse in spatial architectures are multicast and reduction. Reuse can be further classified as spatial and temporal. Spatial architectures' interconnects can support spatial multicast as one read from an outer memory being used for multiple writes to inner instances, and spatial reduction, where reads from inner memories are accumulated in a single outer write. These can be implemented either with direct element-to-element forwarding, like in systolic arrays, or on the interconnect during memory accesses. Temporal reuse occurs when the same value is retained in a memory while being read (multicast) and/or updated in place (reduction) multiple times without it being re-fetched from another memory. Consider the case of kernels that can be computed with parallel ALU-like processing elements, such as matrix multiplications and convolutions. Direct inter-processing-element communication can be used effectively for passing partial sums to achieve spatially distributed accumulation, or sharing the same input data for parallel computation without repeated accesses to outer memories.
The amount of data reuse that can be exploited is a property of the kernel being run, and can be inferred by analyzing its data dependencies. When the kernel can be expressed as a loop nest, reuse arises from subsequent iterations accessing, in part, the same values. This overlap is a form of access locality and constitutes a reuse opportunity for spatial architectures often called "stationarity". For kernels presenting affine transformations of indices, like convolutions and, more generally, stencil patterns, the partial overlap arising from the sliding window of the computation also yields a reuse opportunity, taking the name of "ghost zone" or "halo". Naturally, spatial architectures are more effective the more reuse opportunities are present. At the same time, limited hardware resources mean that not all opportunities can be leveraged at once, requiring proper planning of the computation to exploit the most effective ones.
Mapping computations To run a kernel on a spatial architecture a mapping must be constructed, detailing how the execution will unfold. Mapping a workload to a spatial architecture requires binding each of its computations to a processing element and then scheduling both computations and the data movements required to support them. A good choice of mapping is crucial to maximize performance. The starting point for a mapping is the loop nest representation of a kernel. To leverage parallelism and data reuse simultaneously, iterations must be divided between processing elements while taking care that inter-iteration data dependencies are handled by the same element or neighborhood of elements.
… excerpt ends here. Continue reading the full article.
