Go's New SIMD Package Loses to a For Loop Below 256 Elements

Go 1.27 ships an experimental package that lets ordinary Go code use SIMD instructions (AVX2 on one machine, NEON on another, a software fallback on whatever’s left) without the programmer writing a single line of assembly or branching on GOARCH.¹ The announcement post is long on design and short on one thing: it contains no benchmark numbers at all. The closest it comes is a qualitative goal, “near-asm-performance”, plus a warning that reduction operations like summing a vector aren’t supported yet.¹

So I installed Go 1.27, wrote the dot product from the announcement post almost verbatim, and measured it on real hardware. The headline result isn’t the one the lane count would predict.

What the package actually does

simd, loosely modeled on Google’s Highway C++ library, solves a problem Highway solved for C++ years earlier: every SIMD instruction set has a different native vector width, and code written for one doesn’t run on another without a rewrite.¹² amd64 alone has three widths (128/256/512-bit across AVX, AVX2, AVX512); arm64’s NEON is fixed at 128 bits; RISC-V’s vector extension can be anywhere from 128 to 65,536 bits, decided at hardware design time, not compile time.¹

Go’s answer is to delete the width from the type system. simd.Float32s is a vector of float32s (however many the current CPU’s registers hold), not a Float32x8 or Float32x16. The package only exposes operations available on every supported platform, and fills any gaps with software emulation, so the same source compiles and runs correctly everywhere; it’s just faster on hardware that has real support.¹

Here’s the dot product, using the package’s actual API, verified against Go 1.27’s real simd docs rather than just the blog post’s prose:

func dotSIMD(x, y []float32) float32 {
    var acc simd.Float32s
    n := acc.Len() // lane count: whatever this CPU's vectors hold
    var i int
    for i = 0; i+n <= len(x); i += n {
        u := simd.LoadFloat32s(x[i : i+n])
        v := simd.LoadFloat32s(y[i : i+n])
        acc = u.MulAdd(v, acc)
    }
    buf := make([]float32, n)
    acc.Store(buf) // no ReduceSum in Go 1.27, sum the lanes yourself
    var sum float32
    for _, f := range buf {
        sum += f
    }
    for ; i < len(x); i++ { // tail: whatever didn't fill a full vector
        sum += x[i] * y[i]
    }
    return sum
}

Building it requires GOEXPERIMENT=simd go build; the package isn’t part of a default build yet.¹ On the machine this ran on (an Intel Core Ultra 7 258V, amd64, no AVX512), simd.VectorBitSize() reports 256, simd.Emulated() reports false, and Float32s.Len() is 8. That’s real AVX2 hardware dispatch, not the software fallback.

The benchmark

Dot product of two float32 slices, five sizes spanning three orders of magnitude (16, 256, 4,096, 65,536, and 1,048,576 elements), scalar loop against the simd version above, five repetitions each on Go 1.27.0 / windows-amd64, correctness checked first against the scalar result on odd and even lengths including a zero-length input.³

At 16 elements (two full 8-lane vectors), the simd version is slower: 14.3 ns against the scalar loop’s 9.5 ns, a median across five reps. The fixed cost of assembling a vector, storing it back out to sum the lanes by hand, and handling the tail doesn’t pay for itself yet at that size. Cross into 256 elements and the picture flips hard: 2.74x. From there the speedup settles into a plateau (1.97x at 4,096, 1.99x at 65,536) before easing down to 1.83x at just over a million elements.

Why 8 lanes doesn’t mean 8x

That plateau sitting at roughly 2x, not 8x, isn’t a flaw in the benchmark. It’s what happens when the operation is memory-bound rather than compute-bound. The raw throughput numbers make this visible directly: the scalar loop holds steady at ~8.3 GB/s across every size from 256 elements up to a million, regardless of how much arithmetic is happening per byte. That’s a memory-bandwidth ceiling, not a CPU-arithmetic one: a dot product does one multiply-add per element and then throws the inputs away, so the bottleneck is how fast the two input slices can be streamed from memory, not how many lanes are available to multiply them. The simd version roughly doubles that figure, to ~16–17 GB/s, at every size where it wins. Vectorizing the arithmetic doesn’t remove the memory bottleneck; it just makes each fetch do more useful work per cache line before the next one has to arrive.

The size where the multiplier is highest, 256 elements at 2.74x, is also the size that fits trivially inside a 32–48 KB L1 cache (2 KB for both input slices combined). By a million elements, the two slices total 8 MB, large enough to spill past L2 into L3-scale territory on this chip, where both versions pay more for every fetch and the ratio between them compresses.⁴ Lane count sets a ceiling on how much a vectorized kernel could speed up; whether real code gets near that ceiling depends on whether the CPU can feed the vector unit fast enough to keep it busy, and for a streaming, one-op-per-element kernel like this, it usually can’t.

flowchart LR
A["Go source<br/>simd.Float32s"] --> B["Compiler AST rewrite"]
B --> C["func@simd128<br/>NEON / SSE path"]
B --> D["func@simd256<br/>AVX2 path"]
B --> E["func@simd0<br/>emulated fallback"]
F["Program startup:<br/>detect hardware once"] -->|"picks one"| C
F -->|"picks one"| D
F -->|"picks one"| E

That compile-time specialization is also why the API can afford to hide the width at all: the dispatch cost is paid once per process, not once per call, so acc.Len() inside a hot loop isn’t a branch; it’s a compile-time constant for whichever specialized copy of the function is actually running.¹

Trade-offs

This is one CPU’s numbers, and that’s the whole point of a portability layer. VectorBitSize() returned 256 here because this chip has AVX2 and not AVX512. A machine with AVX512 gets 16 float32 lanes instead of 8 from the identical source, and would likely show a different plateau; an arm64 NEON machine is fixed at 128 bits, 4 lanes, lower still. The API’s entire value proposition is that the source doesn’t change across those machines; the measured speedup will.

The package is explicitly experimental, and it shows. ReduceSum doesn’t exist in Go 1.27; every user of MulAdd-into-an-accumulator has to hand-roll the lane sum this post did, and the Go team has already said a proper reduction is coming in the next release.¹ Benchmark this again after that ships: the horizontal-sum boilerplate is exactly the kind of fixed cost that hurts most at small sizes.

The crossover point is a property of this operation, not of simd in general. A kernel that does more arithmetic per byte loaded (matrix multiply, a cryptographic round function, anything compute-bound rather than streaming) would push both the crossover size down and the plateau multiplier up, potentially much closer to the full lane count. Dot product is close to the worst case for showing off a wide vector unit, which is exactly why it’s a fair, unflattering first benchmark.

Measure before reaching for it in a hot path under 256 elements. The scalar-vs-vector crossover in this specific benchmark landed between 16 and 256 elements. Real code rarely operates on exactly round numbers, and the fixed cost that makes the small case lose (assembling a vector, no built-in reduction, a manual tail loop) is present in every call, whether the input is 10 elements or 10 million.

Takeaways

• The official announcement shipped no benchmark numbers. “Near-asm-performance” is a design goal, not a measurement. This post’s numbers come from running the announcement’s own example, not from anything Google or the Go team published.
• Below 256 elements, the plain scalar loop won outright. 9.5 ns against 14.3 ns, median of 5 runs, because assembling a vector and hand-summing its lanes has a fixed cost the tiny input can’t amortize.
• From 256 elements up, the speedup plateaus around 1.8–2.7x, not the 8x the CPU’s lane count would suggest, because dot product is memory-bandwidth-bound: both versions cap out near a fixed GB/s ceiling, and vectorizing the arithmetic doesn’t widen that pipe.
• ReduceSum doesn’t exist yet. Every accumulator pattern in Go 1.27 needs a hand-written lane sum, a real, measurable fixed cost that the Go team has already flagged as fixed in a future release.
• The API’s portability claim is real; its performance claim is hardware-specific by construction. The exact multiplier this post measured belongs to one AVX2 chip. The source code that produced it will also run on NEON, AVX512, or nothing at all, just differently, and only measurement will say how.

References