Anteru.net Website Review

Looks like this became a small series about GPU compute, and this week I'm going to wrap it up by discussing execution units and caches. I Caches and execution units, where should we start? Let's take the execution units first. If you're familiar with modern CPU design, you know that a CPU is not a scalar thing processing one instruction at a time. A modern CPU units across the floating point and integer blocks. GPUs also take advantage of multiple issue ports, but not in the same way as CPUs. On a CPU, instructions get executed out-of-order, and some of them speculatively at that. This is not feasible on a GPU. The whole out-of-order machinery with register renaming requires even more registers, and GPUs already have tons of registers (a Vega10 GPU has for instance a whopping 16 MiB of registers on the die.) Not to mention that speculative execution increases power usage and that's already a heavily limiting factor for GPUs running very wide workloads. Finally, GPU programs don't look like CPU programs to start with -- out-of-order, speculation, prefetching and more is all great if you're executing GCC, but not for ye old pixel shader. That said, it totally makes sense to issue memory requests while the SIMD is busy working on data, or execute scalar instructions concurrently. So how can we get those advantages without going out-of-order? The advantage we have on a GPU over a CPU is that there is a ton of work in flight. Just as we take advantage for this for hiding memory latency, we can also exploit this for scheduling. On a CPU, we look ahead on a single instruction stream and try to find independent instructions. On a GPU, we already have tons of independent instruction streams. The easiest way to get instruction-level parallelism is to simply have different units for different instruction types, and issue accordingly. Turns out, that's exactly how GCN is set up, with a few execution ports per CU: can send one instruction to up to five of them per cycle. There are some special-case instructions which are handled in the dispatcher