Calibration
The scoring constants are not taste. They come from measuring twenty widely used crates totalling roughly 190,000 lines of Rust, twenty widely used TypeScript packages totalling roughly 443,000 lines, and twenty widely deployed C projects totalling roughly 2.1 million lines.
Why a corpus
A rule that fires often across code the ecosystem considers good is far more likely to be measuring its own imprecision than measuring sloppiness. Every rate is read that way first, and only then as a statement about the code.
The corpus caught a class of bug that testing against this repository structurally could not. A rule asking whether an item is written pub, rather than whether anything outside the crate can reach it, looked correct here because every module in this project is pub mod. Against anyhow, whose modules are all private, it reported 48 findings and scored its documentation 22 out of 100. All 48 were wrong.
The constants
Each category's half_density is set so the corpus median scores 70.
| Category | Median density | half_density | Corpus median score |
|---|---|---|---|
rust | 12.04 | 28.1 | 70 |
typescript | 37.39 | 87.3 | 70 |
c | not calibrated | 20.0 nominal | no median exists |
doc | 2.20 | 5.1 | 70 |
filler | 0.46 | 1.1 | 70 |
A language family is calibrated against its own twenty packages. The doc and filler families read every language the engine parses, so they are calibrated against all forty.
They differ by more than an order of magnitude because the constructs each family names differ that much in how often they appear, not because one language is worse than another. Comparing two categories' scores to each other says nothing. Comparing one category against itself across two revisions is what the number is for.
| Category | Min | Median | Max | Standard deviation |
|---|---|---|---|---|
rust | 40 | 70 | 96 | 14.6 |
typescript | 29 | 70 | 94 | 18.2 |
doc | 8 | 70 | 100 | 29.4 |
filler | 32 | 70 | 100 | 20.3 |
What resolving the export graph changed
TypeScript used to call a name public when its own module exported it. Since nearly every internal module exports what its siblings import, that covered the whole internal surface. It now walks out from the entry point the manifest names.
| Before | After | |
|---|---|---|
doc/missing-on-public-item findings | 3,735 | 1,352 |
| Of those, confirmed | 0.4% | 42.3% |
| Projects carrying an overall score | 12 of 20 | 19 of 20 |
Two thirds of those findings were reporting on declarations no importer could name. Documentation figures from before that change are not comparable to figures after it, because the rule is measuring a different set of declarations.
What the C corpus proved
Nineteen of the twenty C projects carry no c score. Across the corpus, 5,111 findings in that category and 125 confirmed, which is 97.6 percent unconfirmed.
| Rule | Confirmed | Total |
|---|---|---|
c/missing-internal-linkage | 0 | 2,741 |
c/orphan-declaration | 0 | 842 |
c/unincluded-header | 0 | 123 |
c/file-scope-mutable-state | 94 | 1,284 |
c/definition-in-header | 31 | 121 |
The reason is the same everywhere. Real C invokes macros at file scope, and a macro can expand to a declaration, a call, or an include the analyzer never reads. One anywhere in a project puts every count of callers or declarations in doubt.
That measurement exposed a flaw older than C. A score is computed from confirmed findings alone, so a category the analyzer could barely read scored near perfect. Before this, curl scored 100 out of 100. A category now carries no score when fewer than a tenth of its findings are confirmed, and each language family is normalized against its own language rather than against every line in the tree.
curl now reports no overall score and names each category's reason. That is a worse-looking report and a truer one.
Aiming the median at 70 rather than 50 is deliberate. A scale reporting respected libraries as average is describing the tool's opinion of the ecosystem rather than the codebase in front of it.
What the TypeScript corpus caught
typescript/explicit-any fired 5,229 times across 19 of the 20 packages, about one finding every eighteen lines. Reading the sites showed why: most written any in these libraries is type-level machinery, where it is the only way to say what is meant.
| Written | What it does |
|---|---|
T extends any ? keyof T : never | how a conditional type distributes over a union |
ZodType<any, any, any> | a constraint accepting every instantiation |
class ZodError<T = any> | a default type parameter |
None of those discards checking on a value, and together they buried the ones that do. The rule now reports any only where it types something holding a value: an annotation on a variable, parameter, property, or return position, or the target of a cast. Array<any> on a variable still fires. That took it from 5,229 findings to 3,946, and what remains reads as true positives throughout.
typescript/single-implementor-interface went the other way, finding two instances in the whole corpus. It exempts exported interfaces, and in a published library almost every interface is exported. It has more to say about an application than a package, and a corpus made of packages cannot show that.
Rules that ship switched off
Four rules are disabled by default. Each stays available behind --enable, and each has its reason recorded in the catalog itself.
| Rule | Why it is off |
|---|---|
rust/single-instantiation-generic | Needs call sites the index cannot see; fired 6.74 per KLOC across 19 of 20 crates |
rust/single-caller-indirection | Syntax cannot separate a named concept from needless indirection |
doc/unresolved-reference | Links reach into dependencies the index does not read |
doc/contentless-banner | Accurate, but a matter of taste: 154 findings across exactly 3 of 20 crates |
Three of the four are off because project-local analysis cannot establish what they need. That is a statement about the analyzer's reach rather than about the rules, and it is the honest reason to keep the count visible rather than let it accumulate one rule at a time.
Rules that were narrowed
Measurement is also how the false positives were found.
| Rule | Was | Now | What changed |
|---|---|---|---|
doc/commented-out-code | 47 | 18 | Code introduced by prose is illustration |
doc/restating-comment | 39 | 33 | Comments above unsafe are exempt |
doc/documented-parameter-absent | 7 | 0 | A list must name a real parameter |
rust/over-exposed-visibility | 1.22/KLOC | 0.36/KLOC | Associated items are left to their type |
The restating-comment case is the one worth dwelling on. It was reporting this, from bytes:
// `len <= cap`.
unsafe { UninitSlice::from_raw_parts_mut(ptr.add(len), cap - len) }That comment states the invariant making the operation sound. It shares the words len and cap with the code because it must, and the claim itself appears nowhere in the code. A tool that tells you to delete your safety reasoning is worse on that point than one that says nothing.
Reproducing it
The corpus is a list of crate names and exact versions in calibration/, so a later run can measure the same code. Read scores.categories.*.breakdown.weighted_density out of each JSON report.
Scores are not comparable across a change that alters what a rule reports. The calibration record states which changes those were.