On November 9, 2026, VulnCheck's Patrick Garrity published a review of Project Glasswing, Anthropic's effort to find vulnerabilities in open source software with AI. After seven months, VulnCheck's own tracking attributes 442 CVEs to Anthropic, about 0.5% of the 93,450 CVEs published so far in 2026, and only two of them appear in VulnCheck's Known Exploited Vulnerabilities catalog. Anthropic's public ledger, updated on November 2, lists 37,966 findings, of which 217 are recorded as fixed.
AnthropicThe review is a single analyst's reading of public data, and Anthropic has its own account of its results. This post lays out what each side's numbers measure, why they are hard to compare directly, and what the gap says about the bottleneck in AI-assisted security.
The timeline in brief
| Date | Event |
|---|---|
| April 7 | Anthropic launches Project Glasswing, saying it had found thousands of additional high- and critical-severity vulnerabilities |
| April 15 | VulnCheck publishes its first post tracking Anthropic-attributed CVEs |
| May | Anthropic launches its disclosure ledger, reporting 23,019 findings and a 90.8% true positive rate |
| October 8 | Anthropic launches an opt-in OSS Scanner |
| November 2 | Latest ledger update: 37,966 findings, 94.2% claimed true positive rate |
| November 9 | VulnCheck publishes its November review |
What the ledger shows
Between VulnCheck's September 8 snapshot and the November 2 update, the totals moved unevenly.
| Metric | Sept 8 | Nov 2 | Change |
|---|---|---|---|
| Total findings | 26,153 | 37,966 | +11,813 |
| Made public | 2,736 | 7,349 | +4,613 |
| Fixed | 202 | 217 | +15 |
| Withdrawn | 245 | 419 | +174 |
| Claimed true positive rate | 91.4% | 94.2% | +2.8 points |
Anthropic's Glasswing ledger, 2 November 2026: from findings to fixes
Counts from the ledger as summarised by VulnCheck on 9 November 2026, per VulnCheck.
Counts are of ledger entries. Fixed means entries recorded as fixed. VulnCheck separately attributes 442 CVEs to Anthropic through its own tracking.
Findings grew by about 45% over the period, while the fixed count rose by 15, or roughly 7%. Garrity reads that as fixes stalling while duplicates and withdrawals grow. A more neutral reading is that discovery and remediation run at different speeds: finding a candidate flaw is cheap for a model, while confirming it, writing a patch and shipping a release each require a maintainer's time. Anthropic said of its scanner, in a post quoted by VulnCheck, that it is "bottlenecked on our human capacity to validate these findings".
Note also that VulnCheck's count of 442 attributed CVEs is more than twice the ledger's 217 fixed entries. The two figures measure different things (CVEs that carry Anthropic's credit, against ledger entries marked fixed), so neither contradicts the other, but together they suggest the ledger is not a complete record of outcomes.
Finding candidate bugs is cheap for a model. Confirming, fixing and releasing each still cost a maintainer's time.
Analysis of the discovery-versus-remediation gap
Why the numbers do not reconcile
Garrity's central point is that a 94.2% true positive rate is hard to square with the other columns. His review cites several data points:
- Severity. In a September analysis, Claude rated 91.5% of ledger findings as critical or high, while maintainers rated 51.3% that way.
- curl. The curl project received a batch of 45 reports from Anthropic, rated one medium and the rest low. Of the 34 that project lead Daniel Stenberg had published so far, 19 were marked not a vulnerability (55.9%), 6 were confirmed as low severity (17.6%), and 9 were awaiting assessment.
- Mythos in May. Five Mythos findings reportedly led to one confirmed vulnerability.
Claimed accuracy and severity against what maintainers recorded
Percentages from VulnCheck's review of Anthropic's ledger and the curl project's published reports, per VulnCheck.
Different measures from different samples: the ledger's claimed true positive rate, severity ratings in the ledger (September analysis), and curl's 34 published reports. They are not one population.
These samples are not one population, and it would be a mistake to average them. The ledger's true positive rate, the severity ratings and curl's reports cover different sets of findings, and curl's batch of 45 is small. There are also definitional questions the review itself raises: a "true positive" might mean the model correctly identified a real code defect, while a maintainer's "not a vulnerability" might mean the defect is real but not exploitable or not security-relevant. Those can both be true at once, which is part of why accuracy figures are hard to compare across parties.
Garrity also suggests the ledger may lack a feedback loop for triage outcomes, so that a claimed rate does not update when maintainers reject a report. He notes a possible bug that inflates the ledger's "disclosed" count and advises readers to verify it using the ledger's filter. These are his observations, not findings confirmed by Anthropic.
The OSS Scanner and who bears the cost
Anthropic's October 8 post, as quoted by VulnCheck, said the self-service scanner had found more than 29,000 candidate vulnerabilities, with about 6,000 manually reviewed and nearly 5,000 unverified reports sent to maintainers in bulk. Anthropic said those reports were fully model-generated, without human review or triage, and acknowledged that some may be wrong.
This is the structural issue. When a tool shifts effort from discovery to triage, the cost lands on the people with the least capacity to absorb it. Stenberg's choice to publish curl's reports gradually, "to soften the impact for the team" in VulnCheck's paraphrase, is an example of a maintainer managing that load. Garrity encourages other maintainers to publish low and medium findings publicly, as curl did, so that outcomes become visible.
Reading the evidence fairly
- Anthropic's case. AI is finding real bugs at a scale individual researchers would find hard to match, and 442 attributed CVEs is a substantial number for seven months. Anthropic has also published a cyber verification program with access tiers, indicating it is building process around the work.
- VulnCheck's case. The share of CVEs, the very low exploited count and the unreconciled accuracy and severity figures suggest the launch narrative has outrun the evidence. Garrity concludes that AI is a valuable vulnerability-finding tool but that the record does not match the launch framing.
- What both can be true at once. A tool can find many real bugs and also generate a large volume of low-value reports. The open question is the ratio, and the ratio depends on who is counting and what counts as a vulnerability.
A low count of known-exploited CVEs also needs context. KEV lists vulnerabilities with evidence of exploitation in the wild, and a bug found by a defender and fixed quietly may never be exploited, which is the intended outcome. A small KEV overlap is therefore not by itself evidence of low value, though it does mean exploitation-based measures cannot yet show impact.
What to take from this
For teams using AI to find or triage vulnerabilities, the practical lesson is to measure the whole pipeline, not the discovery step: confirmed issues, time to fix and reviewer hours per accepted finding. Comparing how several models rate the same report, as is possible in multi-model tools like Metir, is one low-cost way to check whether a severity claim is robust before it reaches a maintainer.
Sources:
Image credits
Header and in-body image: US Army National Guard cyber analyst working with an international partner, US Air National Guard photo, via Wikimedia Commons, public domain (US government work).