GitHub says retry bug pushed Copilot token traffic to 100K RPS

What caused the August 17 GitHub Copilot outage?

GitHub says Central US load balancers saturated after an Istio sidecar reached its concurrency limit and an autoscaling policy watched the host service but not the sidecar limit. The failure cascaded through four HAProxy nodes, while a latent VS Code retry bug pushed Copilot Token Service traffic from a normal 7-9K to 70-100K requests per second.

Official incident readback

GitHub says the incident ran from 13:28 to 21:15 UTC on August 17, lasting 7 hours 47 minutes. At peak, web and API error rates were approximately 20%, while archive and raw-content download errors reached approximately 50%. Most services recovered by 16:36 UTC, Actions remained degraded until approximately 18:03, and Copilot Token Service fully recovered by 21:02. These are GitHub's service-level figures, not an affected-user count.

The failure chain GitHub described

GitHub says a new traffic peak saturated load balancers in its Central US datacenter. An Istio sidecar pod reached its concurrency limit, while a misconfigured autoscaling policy monitored the host service but not the sidecar's limit. The failure cascaded until four HAProxy nodes exhausted their flow limits, degrading the gateway authentication path. GitHub says optimistic retry logic then added traffic to already overloaded internal load balancers.

Why Copilot recovery took longer

After some traffic moved to Northern Virginia, GitHub says delayed replies from one internal endpoint exposed a latent retry bug in VS Code. That behavior amplified traffic by approximately 10 times and delayed recovery for Copilot Token Service. GitHub says the service normally handled 7-9K requests per second, but traffic rose to 70-100K RPS during the incident. The range describes token-service requests, not users or universally affected clients.

How GitHub stabilized the service

GitHub says it temporarily reduced gateway retry logic and returned HTTP 403 for inbound Copilot Token Service token requests at load balancers, then gradually restored traffic per site. It also says pausing HAProxy on four exhausted nodes produced broad recovery. Scraping attacks on codeload endpoints are listed only as a complicating factor, not the immediate cause or evidence of a breach.

What GitHub plans to change

GitHub lists follow-up work to correct autoscaling policies, audit Istio request and concurrency limits, review retry limits and backoff behavior, address the VS Code retry behavior, and improve load-balancer monitoring and regional failover safeguards. These are planned actions described by GitHub, not proof that recurrence has been prevented.

Developer takeaway

The incident record shows how server and client retries can prolong recovery after an infrastructure fault. This is a dated, resolved event rather than a current-status monitor, so readers with a present problem should check GitHub Status. The AI Guy did not reproduce the outage, inspect GitHub's infrastructure or test the mitigations.

Independent reporting and the bottom line

Neowin independently reported the same core chain and 7-9K to 70-100K RPS range from GitHub's published analysis. It did not independently inspect GitHub's infrastructure, so GitHub remains the controlling source for the cause, figures, mitigations and plans. The dated incident is resolved; the public analysis now replaces this page's earlier pending-cause state.

Related tools and pages

GitHub Copilot researched profileOpen related page AI coding agentsOpen related page Claude Code auto mode workflow updateOpen related page DeepSeek agent harness and model updateOpen related page

Official sources

GitHub official live status page ↗Checked 2026-08-19
GitHub official August 17 incident API and analysis ↗Checked 2026-08-19
Engadget independent outage report ↗Checked 2026-08-17
GeekWire independent outage report ↗Checked 2026-08-17
Neowin independent report on GitHub's published analysis ↗Checked 2026-08-19

This is researched analysis based on public product information. The AI Guy has not independently benchmarked the feature described here.

FAQ

Is GitHub Copilot down?

GitHub marked the August 17 incident resolved at 21:15 UTC. Check GitHub's live status page if you are experiencing a current problem because this article records a dated incident, not every individual session.

Is all of GitHub down?

No current whole-platform outage is established here. GitHub marked the critical August 17 incident resolved at 21:15 UTC; use the live status page for the current component state.

What caused the GitHub outage?

GitHub says Central US load balancers saturated after an Istio sidecar hit its concurrency limit and an autoscaling policy watched the host service but not the sidecar limit. The failure cascaded across four HAProxy nodes, while retry logic amplified the load.

Did 100,000 GitHub Copilot users lose access?

That is not established. GitHub's 70-100K figure is Copilot Token Service requests per second during the incident, compared with a normal 7-9K RPS. It is not an affected-user count.

Did The AI Guy reproduce the outage?

No. This is a researched status brief based on GitHub's official incident analysis and independent reporting from Engadget, GeekWire and Neowin. The AI Guy did not inspect GitHub's infrastructure or independently test recovery.