Netlify Made Its Edge Functions Heavier. They Got Five Times Faster.

Netlify’s edge functions run about a billion times a day.¹ Until this week, every one of those billion calls meant the function’s code left Netlify’s own network, traveled across the open internet to a separate hosted service, ran there, and traveled back. Netlify has now replaced that entire path with one that never leaves its own network, and runs the code inside a full virtual machine instead of the lightweight sandbox it used before. Median latency fell from a reported 25 to 40 milliseconds down to 5 to 6 milliseconds.¹

That result runs against the usual rule of thumb. A full virtual machine is normally the heavier, slower option. A sandbox that shares a process with other code is normally the lighter, faster one. Here the heavier option won, and the reason why turns out to matter more than the headline number.

What an edge function actually runs on

If “edge function” is new to you: it’s a small piece of code, usually a handful of lines of JavaScript, that a hosting provider runs on a server physically close to whoever is making the request, instead of on your application’s main server (often called the “origin”), which might be a continent away. Content delivery networks have cached static files this way for years. Edge functions extend the same idea to actual logic: redirecting a visitor based on their country, rewriting a URL, checking a cookie, before the request ever reaches your app. “Edge computing” is just the general name for doing that kind of work at the location nearest the request (the “edge” of the network) rather than in one central data center. The appeal is physics, not cleverness: a request that doesn’t have to cross an ocean comes back sooner.

Until this change, Netlify ran that code inside something called a V8 isolate. V8 is the JavaScript engine inside Chrome and Node.js, and in V8’s own documentation, “an isolate is a VM instance with its own heap.”² It’s a self-contained slice of memory and execution living inside one running V8 process, cheap enough that a single machine can host thousands of them. It’s the same mechanism that lets a browser run a hundred tabs from a hundred different websites without each tab being a fully separate program.

The trade-off is in where that isolation actually lives. Isolates share the process around them. Under normal operation one isolate can’t read another’s memory, but if a bug ever let code escape V8’s own sandboxing, that escape would be happening inside a process that other isolates, possibly belonging to other customers, were also running in.

The old path left Netlify’s own network

Here’s roughly what used to happen on every call:

flowchart LR
  C[Client request] --> E[Edge node<br/>matches the route]
  E -->|leaves Netlify's network| H[External hosted<br/>execution service]
  H --> V[V8 isolate<br/>runs the function]
  V -->|back across the internet| E2[Edge node]
  E2 --> C2[Client]

The edge node closest to the visitor did its job quickly: it matched the request against the site’s configured routes in a fraction of a millisecond. What came after that was the problem. Running the function meant handing the request to a separate, externally hosted service reached over the public internet, which added a real network round trip to every single call, on top of whatever the function itself took to run.

The new path never leaves home

The replacement runs functions inside Firecracker, an open-source hypervisor. A hypervisor is the layer of software that creates and manages virtual machines: it carves one physical computer into several smaller ones, each of which behaves as if it has the hardware entirely to itself. Firecracker builds what it calls “microVMs,” full virtual machines with their own kernel, stripped down and purpose-built to start in a few milliseconds rather than the tens of seconds a general-purpose virtual machine typically takes. It’s the same technology AWS built to run Lambda, built for the same problem: executing enormous numbers of short-lived, mutually distrusting pieces of code, cheaply and safely.³

flowchart LR
  C[Client request] --> E[Edge node<br/>TLS + route matching]
  E -->|stays inside Netlify's network| N[Compute node<br/>same network]
  N --> M[Firecracker microVM<br/>runs the function]
  M --> N2[Compute node]
  N2 --> E2[Edge node]
  E2 --> C2[Client]

The edge node still handles the parts it always did (ending the encrypted connection and matching the route), but the function itself now runs one short hop away, inside Netlify’s own infrastructure, instead of across the open internet.

What actually got faster, and why

When a company reports “median latency,” also written as p50, it means that half of all requests finished faster than that number and half finished slower. Netlify also reports that its p99 latency, the time taken by the slowest 1 percent of requests, improved by 47.4 percent. The p99 number matters separately from the median because it’s usually what a frustrated user experiences, even on a day when the median looks perfectly fine.

Cold starts, the rare case where a microVM has to be created and booted rather than reused from a warm pool, affect roughly 1.2 percent of requests and average about 9 milliseconds.¹ Netlify also reports that the microVM itself is created in under a millisecond and finishes booting in about 2 milliseconds at the p99 mark, fast enough that even a cold request barely shows it.

It’s worth being precise about where the speedup actually came from, because the obvious story (full virtual machines beat lightweight isolates) is the wrong one. Firecracker microVMs are, in every traditional sense, heavier than V8 isolates: they have their own kernel, their own memory, and they take longer to start from nothing than an in-process sandbox does. The win here came almost entirely from deleting a network round trip to an external service, something that had nothing to do with isolates or virtual machines as technologies. If Netlify had kept that same network hop and only swapped in a heavier sandbox at the far end, the result would likely have been slower, not five times faster.

A stronger wall, and why Netlify wanted one anyway

Netlify’s own blog post states the isolation argument plainly: “V8 isolates, no matter their name, do not provide this level of isolation.”¹ The level it’s pointing to is the difference between sharing a process and not sharing one at all.

In practice this means every Netlify “deploy” (one version of a site’s code and its environment variables) gets its own microVM identity. A compromised deployment can misbehave inside its own virtual machine, but it has no shared process to climb into in order to reach a neighbor’s code or data. That’s a real security upgrade, and it’s also the kind of upgrade that’s hard to sell on its own, since most days nothing goes wrong either way. Pairing it with a five-times latency win is what makes the trade-off easy to justify internally, even though the two benefits came from different parts of the redesign.

Trade-offs worth noticing

• These numbers are Netlify’s own, published on Netlify’s own blog. No outside party has independently reproduced them yet, so treat them the way any vendor-reported benchmark deserves to be treated: directionally credible, not independently verified.
• The resource limits that existed before the change (50 milliseconds of CPU time per request, 512MB of memory, 20MB of compressed code) are reportedly still in place. Netlify says it’s reviewing them now that a real virtual machine and filesystem sit underneath, but a review isn’t a shipped change.
• The lesson here is narrower than “full VMs beat lightweight sandboxes.” It only held because Firecracker was specifically engineered to start inside a few milliseconds, and because the actual cost all along was a network hop that had nothing to do with isolates or virtual machines as such. Swap a general-purpose virtual machine in for Firecracker and keep everything else the same, and the same redesign would almost certainly have made things slower.
• The migration happened without requiring any code changes from existing users, and at the same pricing. That’s worth noting precisely because it’s the part a vendor gets to fully control and announce on its own schedule, unlike the latency numbers, which at least come with some methodology attached.

Takeaways

• The headline number (five times faster) is true, but the obvious explanation for it is wrong. Firecracker microVMs are heavier than V8 isolates by every normal measure. The speedup came from removing a network round trip, not from the new sandboxing technology being inherently quicker.
• “Lightweight” and “heavyweight” describe where the isolation lives, not how fast the whole system runs. A heavier isolation boundary paid off here only because it was paired with the removal of a much bigger, unrelated bottleneck.
• Security and performance improved together, but for separate reasons. Giving each deployment its own kernel and memory closes a real sharing risk that isolates can’t close by design; that upgrade would have been worth doing even without the latency win attached to it.
• Self-reported vendor benchmarks are a starting point, not a verdict. Netlify’s numbers are specific and plausible given the architecture change they describe, but nobody outside Netlify has measured them yet.
• The generalizable idea is “remove the actual bottleneck,” not “always choose the heavier technology.” Here the actual bottleneck was a trip across the open internet. A different system with a different bottleneck would need a different fix, even if the fix looks superficially similar on a slide.

References