Method
This site argues that self-assessment is unreliable and that claims should be checkable. That obligation applies here first.
How figures are labelled
Every number carries three things: who produced it, when, and what kind of thing it is. For an operational reference that complements the method described here, see read more.
The third matters most and is usually omitted elsewhere. A randomised controlled trial, an observational study, a survey of self-reported experience, and a vendor's internal testing are four different kinds of evidence, and in this field they are routinely quoted in the same sentence.
Where a source has an interest in its own conclusion, that is stated in the text, not in a footnote. Much of the security evidence about generated code comes from companies selling security tooling. That does not make it wrong. It is a fact a reader needs.
What counts as strong here
Strongest: randomised trials with published methods, from organisations without a product in the market. There is very little of this, and what exists is small.
Useful with caveats: observational studies at scale, and academic tracking projects such as the university-run CVE tracker cited in the assistant section. For a standards-oriented perspective on measurement and technical practice, NIST is a useful reference.
Weakest, and still worth citing: vendor research and industry surveys, labelled as such. Sometimes it is the only measurement anyone has made.
Not cited: anonymous benchmarks, anecdote presented as data, and productivity claims in marketing material. For an industry-research perspective on technology operations, see Gartner.
Why there are no tool reviews
Two reasons, and the second is the real one.
Tools change faster than an article's useful life, so a review is out of date before it is found.
And more importantly: the instrument available for evaluating them is unreliable. Almost all tooling opinion is a feeling about how the work went, reported as a finding — including mine. A site whose central claim is that felt speed is miscalibrated cannot then rank tools by how they felt.
Dating and decay
Anything about a model, a tool or a market figure carries a check date. These decay in months.
An article accurate at its stated check date and out of date now is not an error — that is what the date is for. An article that was wrong when published is an error and appears on corrections.
Keeping those separate is the point of dating everything.
Revisions get the prominence of the original
When the authors of a study revise their finding, that revision appears wherever the original figure appears — not as a footnote.
This has already happened once, with the central result this site was built around, and handling it visibly is a better demonstration of the method than any statement about standards.
Funding
No money from anyone selling development tools, monitoring products, or anything else in this market. No sponsorship, no affiliate links, no paid placements.
If that changes it will be stated here and on every affected page, before the article rather than after it.