Migrating the GitHub Copilot runtime to Rust, using Copilot

以下正文同步自 GitHub Blog，版权归原站所有，已转换为易读排版。

The GitHub Copilot CLI, GitHub Copilot app, and GitHub Copilot SDK are all backed by the Copilot agent runtime, an agentic harness that can be embedded into applications and services. It was originally written in TypeScript on Node.js and the V8 JavaScript engine for what is now the GitHub Copilot cloud agent (CCA), and the runtime stayed on that stack as the runtime and its capabilities grew rapidly.

That has now changed. Using the GitHub Copilot app and the Copilot CLI, we completely rewrote the runtime into more than 800,000 lines of production Rust. AI agents wrote most of the code, spanning 128 pull requests that landed in main and shipped incrementally rather than waiting for a single cutover at the end. The few inevitable regressions were discovered and fixed quickly along the way, while the performance of the runtime improved by orders of magnitude. A project that would have taken a whole team of developers a year or two before agents was now completed primarily by a single developer, in only a few months, all while the rest of the team continued to greatly expand the runtime’s capabilities and reach.

Why we needed to port

The Copilot agent runtime isn’t just the engine behind the Copilot CLI. It backs a growing set of Microsoft, GitHub, and ecosystem solutions, for each of which AI support is, architecturally, a shell around the same runtime plus whatever customizations that solution needs. This includes not only the GitHub Copilot CLI and the GitHub Copilot app, but also the latest releases of VS Code, Visual Studio, CCA, Copilot Code Review (CCR), Copilot Cowork, Copilot Studio, and Excel and Outlook and PowerPoint and Word and… it goes on.

These are very different products, and none of them wants to or should need to implement everything that goes into a production agent harness. They want all of the intelligence, security, reliability, and performance, and they want it shared so that a fix in one place fixes it in all of them. Most of the products listed in the previous paragraph initially implemented their own agent loop, but have since replaced it with the GitHub Copilot SDK, which is the entry point to the Copilot agent runtime. Doing so enables them to focus on their core business value and leave the details to the runtime. That’s all the more important given the pace of the industry and the employed agent loop needing to stay always best-of-breed in the face of intense competition.

So, shared runtime, good. The problem was the nature of the thing being shared.

If we look at the CLI, it’s logically a terminal UI (TUI) on top of an agent loop. As it happened, the whole stack was implemented in TypeScript, using Node.js as the framework and V8 for the execution engine, with Ink and React for UI. That’s a respectable choice for a TUI application; TypeScript and Node.js are broadly accessible and enable very rapid application development. And for the needs of a console application, the performance implications in terms of startup, responsiveness, throughput, and memory consumption are also reasonable. They are, unfortunately, much less reasonable when you think about that implementation being used in other environments, with other constraints, with demands for things like fast startup and excellent server density due to low memory overhead.

The architecture of the CLI and its runtime also contributed to challenges here. The whole industry is running extremely fast, and in that context, really bright people make decisions for delivery speed and market reach. The Copilot CLI was initially written and shipped quickly, and in doing so, the TUI and the runtime were fairly intertwined rather than separated into discrete layers. Then when an SDK was needed for programmatic access to that runtime, without clear separation of the layers, a pragmatic decision was made to layer the SDK on top of the CLI, even though logically you’d expect the inverse architecture. Rather than only being accessible via commands provided by the user at the command line, the CLI was updated with a mode where it could be run headless, reading similar commands from stdin and writing responses to stdout. A JSON-RPC protocol could then be used to marshal function calls from an external process to and from the CLI. The SDK could then be embedded in arbitrary consuming programs, which would spawn a CLI process to host the agent loop out-of-process, with the SDK calling functions in the remote process via this JSON-RPC mechanism. Neat. Fast to get out the door. Flexible. But not great for the performance (startup, memory, throughput) and reliability of those consuming applications. Creating a new CopilotClient from the SDK meant spawning another process:

const client = new CopilotClient();

await client.start(); // spawns the CLI as a subprocess

const session = await client.createSession({

/* ... */

});

The process would need to launch and host Node and V8. It meant parsing the significant amount of JavaScript produced from the TypeScript code in the CLI, generating bytecode for it, and potentially optimizing hot code in later JIT tiers. It meant all the memory overhead associated with V8. It meant inheriting Node’s threading model, which by default pushes us towards a model of all CPU-bound work being serialized. And it meant forced out-of-process communication just to make function calls. It meant every SDK consumer, in every language, ships Node.js or a bundled binary containing V8. It meant the C#, Python, Go, Java, and Rust SDKs all paid for a whole second language runtime per client, on the order of 100 MB of working set minimum, for a runtime their application otherwise had no use for. It meant every event, every message, and every abstracted session file system read and write was pushed across a process boundary. It meant a crash in Node took the session with it. And it meant anyone deploying this had, at a minimum, two processes to supervise, monitor, and debug.

Instead, we wanted a runtime:

- that does not include the TUI, that’s its own library the TUI and other applications and services can be properly layered on top of cleanly.

- implemented in a language with minimal dependencies and minimal overhead.

- implemented in a way that it can be cleanly embedded in-process rather than being forced out-of-process.

- implemented in a language with top characteristics around performance and scalability and reliability.

- implemented in a language that’s great for interop, such that it can be used cleanly by all six Copilot SDK language versions (C#, TypeScript, Python, Rust, Go, Java) with that stack’s foreign function interface (FFI) mechanism.

- implemented with a tool chain that provides a more modern security posture, with less supply chain risk and greater support for correct-by-construction code.

For all those reasons, as well as softer reasons (such as team experience and industry direction), we chose Rust. This is in no way a claim that every large TypeScript program should become Rust. Our requirements emphasized embedding through a C ABI, low startup and steady-state overhead, and predictable resource use. Rust made those goals possible, at the expense of other complications, e.g. we had to represent lifetimes and shared state explicitly (the lifecycle regressions discussed later highlight the implications of that). The right target language legitimately varies from application to application.

There were then two key related tasks undertaken:

- Separating the TUI-specific code from the runtime, so that the former is layered strictly on top of the latter, and more specifically layered strictly on top of the SDK’s public surface area. Today, the CLI still calls directly into runtime internals in several places; moving it fully onto the SDK’s surface area is ongoing work.

- Porting that runtime layer to 100% Rust, resulting in a pure native binary exposing a C ABI for in-process consumption by all the language front-ends and a stdin/stdout-based or socket-based server for when out-of-process is still desired.

This post primarily covers the second: porting the runtime to Rust.

What it looked like before

The initial porting plan in early May 2026 estimated the runtime at roughly 130,000 lines of TypeScript. For scoping purposes, this initial measurement was reasonably accurate, but, as it turned out, also wildly misleading, in two key ways. Concurrent with porting:

- Pieces still wrapped up in the TUI layer were being pushed down to the runtime layer. Entire components and significant percentages of code initially ignored in the estimates were then later considered relevant to porting.

- Pull requests contributing significant amounts of new TypeScript were constantly raising the amount of TypeScript in the repo. Tens of agentically assisted developers merging hundreds of pull requests per week.

Everything factored in, I estimate approximately 430,000 lines of production TypeScript ended up passing through the port. Those same factors also made it hard to see progress along the way: until close to the end, production TypeScript volume appeared to be holding relatively steady, if not increasing slightly, as porting kept pace with incoming work.

This is confused further because there was also incoming Rust code, separate from the port, over the timeframe; early in the porting effort, incoming code was more likely to be dominated by TypeScript, whereas later in the effort, it was more likely to be dominated by Rust.

During the port, the runtime took in ~300,000 production lines of TypeScript and shed ~430,000, while ~1,200,000 production Rust lines entered and ~365,000 left. In other words, the apparent stability of the TypeScript line in the above graph was actually hiding significant amounts of TypeScript churn.

In-place porting strategy

That chart also highlights an important aspect of how the port was done: in place.

There are two main approaches to a rewrite of this scale:

- Big bang. The new Rust runtime is developed as a complete alternative and then swapped in all at once when it’s ready. Such a big-bang cutover has two variations. a. Stop the world. Everyone ceases other work on the main branch while the rewrite happens, with the rewrite being done in main. b. Parallel development. The rewrite happens in a feature branch while work continues in the main branch, with the rewrite constantly trying to keep up with and merging in changes from the main branch.

- In place. This is done as a component-by-component port, where the runtime is incrementally rewritten one piece at a time. Such an in-place approach also has two variations. a. Atomic replacement. Each piece is flipped atomically from TypeScript to Rust, with interop between the remaining TypeScript and the new Rust providing continuity. Over time, less and less of the production runtime is TypeScript, and more and more is Rust, until one day, there’s no more TypeScript, only Rust. b. A/B. Rather than deleting components as they’re ported, both the TypeScript and the Rust components are maintained as hot-swappable options, with the TypeScript being deleted once confidence has plateaued.

We went with option 2a, for a variety of reasons:

- No one experiences work stoppage. The main branch continues to be active. Every developer not directly involved in the port gets to keep on keepin’ on, impacted only when a pull request they may have in flight for a prolonged period of time happens to touch code that gets ported concurrently, in which case they need to rebase and have their agents help port just their in-flight changes.

- The runtime’s main branch is always shippable. Each pull request replaces the existing TypeScript implementation with a thin shim that calls into Rust, and deletes the old code in one atomic change. The new code is immediately exercised, in-situ.

- The rewrite is incremental and reviewable. Each pull request ports a single component or slice, so the scope of change is smaller and the diff is easier to review, w

正文由 FLUX 从来源站点 RSS 同步，内容未经改写；遇到排版缺失或需要图片、视频时请以原文为准。