reference
Every signal
A finding has a kind, and the kind is what tells you which sort of bug you are looking at. These are the names — fanout and self_latency are precise and unguessable, so each one is written out rather than left to be inferred.
Every measured signal is compared the same way: this release against the one before it, at the same quantile, with a confidence that rises with both the size of the change and how many times it was observed. There is no threshold to configure, because the baseline is your own previous release rather than a number somebody guessed.
Measured
latencylatencyms per call
this operation takes longer end to end than it used to.
Where the fix usually is. End-to-end time includes everything it calls. Check the child breakdown first — if a child moved, fix that instead; this finding is the symptom.
Only 1 request in 20 was slower than this. It is the number your unluckiest users actually feel, which is why it matters more than an average.
Reported by Node, Python, Go, Ruby & Rails, PHP, Browser, React Native, Supabase.
self_latencyown codems per call
the time spent in this function itself got longer — its children did not.
Where the fix usually is. The work is in this function body: a loop over a list that grew, a synchronous parse, a sort, a regex, JSON of a payload that got bigger.
Time spent in this code itself, not in the database calls or other functions it triggers. High own-time means the slow part is here, not somewhere it called.
Reported by Node, Python, Go, Ruby & Rails, PHP.
cpumore CPUms of CPU per call
this function burns more processor time per call than it used to — it is doing more work, not waiting longer.
Where the fix usually is. Work added per call inside the function body: a copy or deepcopy, a serialization, a sort, a hash, a regex, a comprehension over something that grew. Wall-clock latency may barely have moved; this is what saturates a box under load. Measured only for Python services.
How much of this operation's own time was actually spent computing, rather than waiting. Work it triggered elsewhere is excluded. When this rises, the code is doing more per call — which a stopwatch on a quiet machine barely shows, and a busy one shows all at once.
Reported by Python.
blockedwaitingms of waiting per call
this operation spends longer waiting for its turn while doing exactly the same amount of work.
Where the fix usually is. Contention rather than code: a lock held across more work, a connection or thread pool with no free slots, a C extension holding the GIL, work moved inside a request that used to happen outside one. Every per-call number stays correct while this happens, so nothing else here catches it. Measured only for Python services.
Its own time that was not computing: queued behind a lock, waiting for a free connection or thread, or held up by something else in the same process. The work per call is unchanged — only the waiting for a turn grew.
Reported by Python.
rowsrowsrows per call
this query returns more rows than it used to.
Where the fix usually is. Almost always a missing or widened filter, a LIMIT that was removed, or a join that started multiplying. The row count moved without the SQL changing means the data grew into an unbounded query.
How many database records this reads each time it runs. If it reads thousands to show a handful, the filtering is happening in your code instead of in the query.
Reported by Node, Python, Go, Ruby & Rails, PHP.
fanoutN+1queries per call
this operation now issues more database calls per invocation — an N+1.
Where the fix usually is. A query inside a loop or a map over a result set. Fix by batching into one query (IN / join) or by hoisting the query out of the iteration.
How many separate database queries run each time this page or function is used. One is normal. Fourteen usually means a query is running inside a loop.
Reported by Node, Python, Go, Ruby & Rails, PHP.
round_tripsfetched in batchesround trips per read
one read now waits on the database many times instead of once.
Where the fix usually is. A MongoDB cursor fetching in batches. The driver takes the first 101 documents, then as much as fits in 16MB per round trip — so this tracks the *size* of the result, not the document count: 30,000 small documents is about 2 waits, and the same 30,000 carrying a 16KB field each is about 30. A jump here usually means the documents got bigger (check for a field that grew, or a projection that stopped being applied), or the result set did. Check for an explicit .batchSize() first: a low one is the usual cause — 30,000 documents at batchSize(100) is 300 sequential waits — and raising or removing it is the fix. Lowering it makes this worse. Not an N+1: the code made one call and the driver made the rest.
How many separate times this waits for the database to answer one read. MongoDB sends the first 101 documents, then as much as fits in 16MB each time after that — so the waiting grows with how many *bytes* come back, not how many documents. Reading 30,000 small documents takes about 2 waits; the same 30,000 documents once they carry a 16KB field each takes about 30. Usually fixed by asking for fewer fields or fewer documents. If the code sets an explicit batch size, check it: a small one is the most common cause, not the cure.
payloadpayloadbytes per call
the serialized result of this operation got bigger.
Where the fix usually is. A select that started returning more columns, an added include/expand, or an embedded blob. Check what the response actually needs.
How much data is sent to the browser each time. Large responses are slow on phones and on bad connections even when your server is fast.
Reported by Node, Python, Go, Ruby & Rails, PHP.
errorserrorserror rate
a larger fraction of calls are throwing.
Where the fix usually is. Compare against the release boundary: a step at the deploy is a code change, a ramp is usually a dependency or data condition.
How many times this was observed. More measurements mean the numbers below are less likely to be a fluke.
Reported by Node, Python, Go, Ruby & Rails, PHP, Browser, React Native, Supabase.
planquery plan
the database chose a different plan for this query.
Where the fix usually is. An index that stopped being used, or statistics that shifted as the table grew.
Postgres decides a strategy for each query — use an index, or read the whole table. When that decision changes, the query can become far slower without a single line of your code changing.
Reported by Node, Ruby & Rails, PHP.
repeated_queryrepeated querytimes per request
the identical query runs several times within one request.
Where the fix usually is. Two call sites fetching the same thing, or a helper called once per component. Fetch once and pass it down, or memoise per request.
Fetching a list, then running one more query for every item in it. Twenty items become twenty-one queries, and it gets slower as your data grows.
Reported by Node, Python, Go, Ruby & Rails, PHP.
overfetchover-fetchingrows per call
far more rows are fetched than the code appears to use.
Where the fix usually is. A filter or aggregation being done in application code that the database could do — or a full table read behind a .find() on the result.
The query pulls thousands of records but the response is tiny, which means almost all of them were read and then thrown away.
Reported by Node, Python, Go, Ruby & Rails, PHP.
unboundedno limitrows per call
this query has no upper bound on what it can return.
Where the fix usually is. It is correct today because the table is small. Add a LIMIT and pagination before the table decides for you.
A query with nothing capping how many records it can return. Fine while the table is small, and it fails the day it is not.
Reported by Node, Python, Go, Ruby & Rails, PHP.
new_errornew erroroccurrences
an error that did not occur in the previous release.
Where the fix usually is. Anchored to a deploy, so start with the diff for this operation.
recursionrecursionlevels deep
this operation calls itself, deeper than it should.
Where the fix usually is. A recursive resolver or tree walk with no depth limit and no memoisation.
This function runs itself again, over and over, before finishing. A few levels deep is normal; dozens means it is not stopping when it should.
Reported by Node, Python, Go, Ruby & Rails, PHP.
runawaycalled in a loopcalls per minute
this operation is being called far more often than anything triggers it.
Where the fix usually is. A polling interval, an effect with an unstable dependency, or a retry with no backoff.
Each individual run is unchanged — same speed, same data — but something is now calling it hundreds of times where it used to call it once.
Reported by Node, Python, Go, Ruby & Rails, PHP, Browser, React Native.
render_stormrender looprenders
a component re-renders many times for one interaction.
Where the fix usually is. State set during render, an object or array literal in a dependency array, or a context value rebuilt every render.
How many times the page redrew itself from one click. A handful is normal; hundreds means it is redrawing in a loop and will drain a phone battery.
Reported by Browser.
dead_interactiondoes nothingof clicks
users click this and nothing observable happens.
Where the fix usually is. A handler that was never attached, an early return, or a disabled control that still looks clickable.
Someone clicked a control and nothing followed — no data changed, no request was sent, no page moved. Usually a button whose handler was never wired up.
Reported by Browser.
client_errorbrowser errorof views
this error is being thrown in real users’ browsers.
Where the fix usually is. Stack frames point at bundled chunks in production; use the operation name and the message rather than the line number.
How many times this was observed. More measurements mean the numbers below are less likely to be a fluke.
Reported by Browser, React Native.
web_vitalpage speedms
a page-speed metric got worse for real users.
Where the fix usually is. Check what the release added to the critical path: a font, a blocking script, an image without dimensions.
How long until the biggest thing on the page — usually the main image or heading — has appeared. This is when the page stops looking empty.
Reported by Browser.
stuck_loadingnever loadsof loads never finish
a loading state is entered and never left.
Where the fix usually is. A promise that can reject without clearing the flag, or a request whose response never sets state because the component unmounted.
A spinner still on screen twelve seconds after it appeared. Usually a request that quietly failed inside a catch, or a loading flag that is set true and never set back.
Reported by Browser.
silent_emptyreturns nothing
The query runs and succeeds, and returns no rows where it used to return plenty. Nothing reports an error, so the page just renders blank — this is what a broken permission rule looks like from the outside..
Reported by Browser, React Native, Supabase.
collapsereturns less
This is handing back roughly half the data it used to, or less. If that was not deliberate, something is filtering out rows that somebody expects to see..
Reported by Node, Python, Go, Ruby & Rails, PHP.
vanishedno longer used
It was being used steadily until this release and has not been used once since. Usually the link, button, or redirect that led here stopped working..
Reported by Node, Python, Go, Ruby & Rails, PHP.
traffic_dropused far less
This is still being used, but a fraction as often, and its share of your traffic fell too — so it is not just a quiet period..
Reported by Node, Python, Go, Ruby & Rails, PHP.
auth_failuresaccess refused
The server is turning these away on permission grounds rather than failing. People see an empty page or a save that quietly does nothing..
Reported by Browser, React Native, Supabase.
missing_tenancydata not scoped
This reads a table of per-person data without saying whose rows it wants. Unless your database is filtering it for you, everyone gets everyone else's..
Reported by Node, Python, Go, Ruby & Rails, PHP.
Found by reading the code
These cannot be measured — a button whose handler was never attached emits nothing at all. They are ranked by how much production traffic the file they live in serves, which is what separates a dead control in your checkout from one in a page nobody loads.
dead_interactiondoes nothingof clicks
users click this and nothing observable happens.
Where the fix usually is. A handler that was never attached, an early return, or a disabled control that still looks clickable.
Someone clicked a control and nothing followed — no data changed, no request was sent, no page moved. Usually a button whose handler was never wired up.
Reported by Browser.
effect_loopeffect loophits per day
an effect writes state it also depends on — found by reading the code, not by measuring.
Where the fix usually is. The dependency array contains something the effect body sets. The number attached is production reach, not severity.
Code that runs after the page updates, changes something, and by changing it causes itself to run again. It never settles.
Found by reading the code at build time, not by measuring.
missing_cleanupnever cleaned uphits per day
a subscription, interval or listener is created without a teardown.
Where the fix usually is. Return a cleanup from the effect. The cost appears as growing memory and duplicate handlers, not as a slow call.
Code that runs after the page updates, changes something, and by changing it causes itself to run again. It never settles.
Found by reading the code at build time, not by measuring.
exposed_secretexposed key
A password-like key is written into code that gets downloaded by everyone who visits the site. Anyone who looks can copy it and use it as you..
Found by reading the code at build time, not by measuring.
insecure_transportunencrypted
This talks to another service over a plain, unencrypted connection. Anything travelling on it can be read or changed along the way..
Found by reading the code at build time, not by measuring.