API reference¶
The capture¶
QueryCapture ¶
Record every statement executed inside a block, on one or more connections.
Capture rides on connection.execute_wrapper(), which is Django's
documented hook and is what makes this compose rather than compete.
django_assert_num_queries counts through CaptureQueriesContext, which
works by flipping force_debug_cursor and reading queries_log; a
wrapper is a separate list on the connection and touches neither. So this
nests inside or around that assertion without changing what it sees --
a fact about the two mechanisms rather than a hope about them.
It also counts every execution, where queries_log is a bounded deque.
See LogCeiling for what that costs the other path and how this reports it.
with QueryCapture() as capture:
list(Author.objects.all())
for fingerprint, records in capture.by_fingerprint().items():
print(len(records), fingerprint, records[0].call_site)
records
property
¶
Every execution, in order. A snapshot, so it is safe to read mid-block.
ceilings
property
¶
One per captured connection, populated on exit.
exceeded_ceilings
property
¶
The connections whose block ran past what Django's query log can hold.
by_fingerprint ¶
Group the records by fingerprint, in the order each was first seen.
The grouping every face of this capture starts from. A group of more than one is a repeated statement; whether it is an N+1 also depends on the call stack, which the records carry.
from_capture_context
classmethod
¶
Build a capture from a django.test.utils.CaptureQueriesContext.
This is the object django_assert_num_queries yields, so a caller
already holding one can be served without rewriting the test. What comes
back is honestly degraded, and the gaps are the argument for capturing
separately rather than a limitation to work around:
- No call stacks. That context records
{"sql", "time"}per query and nothing else, so there is no frame to recover.call_siteisNoneon every record. - No ceiling.
ceilingsis empty, because a count taken from a rotated deque cannot report how much it lost -- the dropped entries are gone, and their number with them. This is the one thing that cannot be reconstructed after the fact at any price. - No parameter counts, and
manyisFalsethroughout: the SQL of anexecutemanyarrives there already rewritten to"N times: ...".
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
context
|
Any
|
Anything with |
required |
The record¶
QueryRecord
dataclass
¶
A single statement, its fingerprint and the stack that emitted it.
This is the artifact the whole package is built around, and it is public and documented from the first release on purpose: four separate faces read it -- the pytest plugin, a CI report, call-site attribution and a runtime budget middleware -- and two of them are still unwritten. A record kept private until they are would grow a private accessor per face instead of a shape. Attribution is the evidence that the bet paid: it reads this record and needed no field added to it.
The contract in 0.x is additive: fields may be added, never removed and
never given a new meaning. It is frozen at 1.0.
Two things are deliberately absent.
No parameters. Retaining them would pin every bound value for the length
of a capture -- a bulk_create of ten thousand rows arrives here as one
execution and ten thousand values -- and the runtime middleware face would
then be holding customer data in memory to answer a question about query
counts. param_count is what the diagnosis actually needs ("the IN
list had five hundred entries"), and the plan face runs EXPLAIN at
execution time, where the parameters are still in hand.
No duration. This package's argument is that a performance assertion
mentioning a number of milliseconds is a flaky test with extra steps, and a
field invites the assertion. Timing a query is a profiler's job;
django-silk does it well and is deliberately out of scope.
index
instance-attribute
¶
Position within the capture, counting from zero. Names a query in a report.
sql
instance-attribute
¶
The statement exactly as handed to the cursor, placeholders and all.
fingerprint
instance-attribute
¶
sql reduced to the part that repeats. See normalise_sql.
vendor
instance-attribute
¶
The backend's vendor string: sqlite, postgresql, mysql, oracle.
many
instance-attribute
¶
True when this arrived through executemany rather than execute.
param_count
instance-attribute
¶
How many bindings were passed -- rows, for executemany.
None when the parameters were not sized: None itself, as every
transaction control statement passes, or an iterator, which Django permits
and which cannot be measured without consuming it.
stack
class-attribute
instance-attribute
¶
The call stack, outermost first. Empty when reconstructed from a source that had none.
stack_truncated
class-attribute
instance-attribute
¶
True when the stack was deeper than the capture's limit and outer frames were dropped.
plan
class-attribute
instance-attribute
¶
What PostgreSQL said it would do with this statement, when it was asked.
None means nobody asked: an ordinary
:class:~django_query_contract.QueryCapture takes no plans, and neither does
a capture rebuilt from a CaptureQueriesContext. A
:class:~django_query_contract.PlanCapture puts a
:class:~django_query_contract.QueryPlan on every record it makes, including
the statements it declined to explain -- those carry a plan whose root is
None and whose refusal says why, so "not asked" and "asked and
declined" stay distinguishable.
This is the first field added under the additive contract, and it is what
the contract was for. The record has been public since 0.1.0 on the bet
that four faces would read it, and the docstring above has said since then
that the plan face runs EXPLAIN at execution time because that is where
the parameters still are. Adding the field changes nothing for a reader that
does not want it, which is the promise 0.x made.
call_site
property
¶
The innermost frame outside Django: the line that asked for this query.
None when the stack holds no such frame -- an empty stack, or one
truncated before it reached the caller. That is reported rather than
approximated with the innermost frame available, because "the query came
from django/db/models/query.py" is true of every query and tells a
reader nothing.
Django is the only package skipped. Anything else in the stack -- a REST framework, a factory library, a service layer -- did emit the query, and deciding that some libraries are more interesting than others is the kind of tuning this package exists without.
An :class:~django_query_contract.NPlusOne names its call site the same
way, through the same helper, so a finding and the records inside it can
never disagree about where they came from.
project_call_site
property
¶
The innermost frame that is the reader's own: the line to go and edit.
The companion to :attr:call_site, and here for the reason that one
names the helper it shares: an :class:~django_query_contract.NPlusOne
answers this the same way through the same walk, so a finding and the
records inside it cannot disagree about which frame is the project's.
None when no frame in the window is the reader's.
StackFrame
dataclass
¶
Where a query came from, at one level of the call stack.
Deliberately three plain strings and an integer rather than a reference to a live frame object: a captured stack outlives the call it describes, and holding frames would keep every local variable of every ORM call alive for the length of a test. It would also make the record unpickleable, which the CI-report face needs it not to be.
The source line is not read either. traceback.extract_stack opens and
caches the file for every frame it formats, and this runs once per query in
a suite that may execute hundreds of thousands of them.
The finding¶
NPlusOne
dataclass
¶
A repeated statement, and the one place it was repeated from.
The definition is the product. More than one execution with the same normalised SQL and the same call stack is an N+1 by construction: the same line of code ran the same statement again, with different data, instead of asking for the data once. There is no threshold, no rule about lazy loads and no confidence score, so there is nothing to tune and nothing to be wrong about.
That matters because of how the ground looks. Four Python N+1 detectors are
dead on PyPI -- nplusone (2018, and still what every blog post
recommends), django-query-capture (2022), django-nplusone (2020) and
django-explain (2016) -- and the most probable reason is that they
classified by rule. nplusone listens for lazy loads, which flags
legitimate code and misses real N+1s behind an explicit loop. A detector
people disable finds nothing at all.
The identity is the whole stack, not the call site. Two weaker keys were considered and both are rules about which frames matter, which is exactly the judgement that turns into a knob:
- The fingerprint alone would merge a loop in a view with a loop in a template that happen to emit the same SQL, and report one finding whose "call site" is whichever of the two was seen first.
- The innermost frame outside Django alone would merge two callers of one
helper.
get_books(author)called from two different loops is two defects with two fixes, and a report pointing at the helper points at the one line that is fine.
The whole stack cannot make either mistake, because two executions with identical stacks did run the identical code path. It also cannot split a loop: every iteration of a loop enters the query through the same frames at the same lines. The one shape it does split is recursion, where the same line is reached at several depths and each depth is its own finding -- true, a little noisy, and preferred over the alternative of pointing a reader at a line that is not where the fix goes.
The connection alias is not part of the identity, because the claim
above does not mention it: a loop that queries two databases from one line
is one loop with one fix. aliases reports the span instead of encoding
it, so the report says what happened without the rule acquiring a clause.
Where "the whole stack" stops being whole, stated rather than glossed.
The capture keeps the innermost stack_depth frames, so the identity is
really that window. Measured under pytest, a query issued from a test
function is 38 frames deep and 30 of them are the runner's own constant
preamble, so at the default depth of 25 the window reaches well past the
test function and the frames it drops cannot tell two call paths apart
anyway. It is an application whose own stack is deeper than the window that
can put two paths in one bucket -- and the error it makes is a merge: two
findings reported as one, never a repetition that did not happen.
The way to find out is to raise stack_depth and see whether the finding
splits, which is a second measurement rather than a flag.
:attr:stack_truncated cannot answer it and this docstring used to say it
could: under a test runner it is True of every capture at any workable
depth, so it is a constant and a constant tells a reader nothing. A merge is
the thing that stops being one when the window widens, and there is a test
that widens it.
Legitimate repetition is still a finding. A bulk_create batched into
a hundred inserts is one statement shape executed a hundred times from one
line, and there is no structural difference between that and a defect --
only an intention, which is not in the capture. So this reports it, and the
package refuses to fail anything on it: a finding is a diagnosis attached to
a failure somebody else's assertion already produced, or a list somebody
asked for. An exemption list would be the first tunable, and the first
tunable is how a detector starts being wrong.
fingerprint
instance-attribute
¶
The normalised SQL shared by every record here. See normalise_sql.
stack
instance-attribute
¶
The call stack shared by every record here, outermost first.
records
instance-attribute
¶
Every execution on this path, in capture order. Always at least two.
call_site
property
¶
The innermost frame outside Django: the line to go and look at.
None when the stack reaches no such frame, which in practice means
it was truncated below the caller. Reported rather than approximated,
for the reason QueryRecord.call_site gives.
project_call_site
property
¶
The innermost frame that is the reader's own: the line to go and edit.
call_site answers what asked for this query and is the innermost
frame outside Django, which is the right answer to that question and
sometimes a useless address. A query issued under transaction.atomic
used as a decorator reaches the database through contextlib; a
library that monkeypatches the ORM puts itself there instead. Both are
truthful, neither is a line anybody edits, and a savepoint loop reported
against contextlib.py reads like a bug in the standard library.
This is the other question -- where do I go and look -- and the two are the same frame whenever the innermost non-Django frame is already the reader's, which is the ordinary case.
None when no frame in the window is the reader's: a stack truncated
below the caller, or a repetition genuinely internal to a dependency.
Reported rather than approximated, for the reason call_site gives.
A consumer writing its own report needed this and had to build it, which put the definition of "our code" in two places that could disagree about one finding from one capture.
aliases
property
¶
The connections this ran on, in the order they were first seen.
Usually one. More than one means a single line queried more than one database, which the identity deliberately does not split on.
stack_truncated
property
¶
True when any execution here had frames dropped above the ones kept.
any, not "all of them equally", because the kept frames can match
while the dropped ones did not: a recursive walk reaches the same line
from a different depth each time.
Not printed per finding by either report, and that is on purpose. Under a test runner this is true of every capture at any workable depth -- the dropped frames are the runner's own -- so a caveat on every line would say nothing on any of them.
Which also means it is not the way to find out whether this finding
merged two call paths, though it was documented as one until 0.7.0. It
says the window was full, which it always is under pytest; it cannot say
whether the frames that fell outside it were the ones that mattered. The
answer to that is a second measurement -- raise stack_depth and see
whether the finding splits -- and it is the only answer there is. What
this is good for is a capture taken outside a test runner, where a full
window is news.
first_index
property
¶
Position in the capture of the first execution on this path.
The tie-break that makes an ordering by count total, so two runs
over the same capture list findings in the same order.
The attribution¶
Attribution
dataclass
¶
Every statement one line of code emitted, and that line.
Attribution asks a different question from detection, so it groups by a
different key, and that difference is the whole reason this type exists
beside :class:~django_query_contract.NPlusOne rather than inside it.
A finding asks what is one defect. Its identity is the whole call stack,
because two callers of one get_books(author) helper are two defects with
two fixes, and the helper's own line is the one line that is fine.
An attribution asks where did these statements come from. For those same two callers the honest answer is the helper's line: that is where the statements were emitted, and both call paths really did emit them there.
So this deliberately merges what a finding keeps apart, and that is safe only because it claims nothing about defects. A group of forty is not a finding of forty; it is forty statements and an address. Nothing here fails a test, nothing here says a loop was found, and nothing here is a rule about which repetitions count -- which is what makes the merge a convenience rather than the first tunable.
The call site is a display rule, and it stays one. It is picked by the
single rule the whole package shares -- the innermost frame that is not
inside Django, see :attr:~django_query_contract.QueryRecord.call_site --
so a record, a finding and an attribution can never disagree about where a
statement came from. That rule decides what is printed. It is not part of
any identity, and the reason it must not become part of one is written out
at :class:~django_query_contract.NPlusOne.
call_site is None for the one group that has no address: records
with no stack at all -- everything in a capture rebuilt from a
CaptureQueriesContext -- and records whose kept frames were all Django's
own. They are grouped rather than dropped, so the statements in a capture
and the statements in its attribution always add up. An attribution that
quietly lost the ones it could not place would be the silently incomplete
measurement this package exists to complain about.
call_site
instance-attribute
¶
The line that emitted every statement here. None when there was none to name.
records
instance-attribute
¶
Every statement from that line, in capture order. Always at least one.
fingerprints
property
¶
The distinct statement shapes emitted here, in the order first seen.
More than one is ordinary: a line that evaluates a queryset with a related object on it emits several shapes. It is also the other half of why an attribution is not a finding -- a finding is one shape by definition, and this can hold as many as the line produced.
aliases
property
¶
The connections these ran on, in the order they were first seen.
Usually one. More than one means a single line queried more than one database, which is worth printing because nothing else in the group would say so.
first_index
property
¶
Position in the capture of the first statement from this line.
The tie-break that makes an ordering by count total, so two runs
over one capture list the same attributions in the same order.
RelationAccess
dataclass
¶
How one table was reached, by which statements, and from which lines.
This is the milestone that was going to be index advice, and it is a
report instead. The plan for this package said the output people actually
want is "these twelve queries sequentially scanned a two-million-row table,
here are the CREATE INDEX statements", and that it falls straight out of
having plans plus call sites. It does not, and the reason is the same rule
that decided every other question here: a finding is a fact the server
states, or an equality over measurements, never a number somebody picked.
Three routes to an assertable version were tried against a real server and all three ended in a threshold.
- A sequential scan on a relation that another captured statement reaches
by index. That reads like a comparison between two measurements rather
than a cut-off, and it is not: two statements filtering different
columns of one table are not measuring the same thing. Measured against
a server -- one statement reached
testapp_orderthrough the foreign key index while another read it end to end for a predicate that kept all 100,000 of its rows, which is the correct plan and the one no index improves. Both halves of the rule hold; the index it would point at is on the other statement's column. - A filter whose
Rows Removed by Filterthe server itself counted. The number is PostgreSQL's, but the verdict is not: a five-row table discards four rows in exactly the shape a hundred-thousand-row table discards 99,999, and only a magnitude separates them. The count does not even order the candidates, because the read that discarded nothing is the whole-table read that was right to be a scan. - Emitting the
CREATE INDEXitself. That needs a column, and the only place a column can be got is PostgreSQL's rendered predicate -- an expression that would have to be parsed, in a package that declines a SQL parser for reasons written out at :func:~django_query_contract.normalise_sql, and whose text carries the bound value this package retains nowhere.
So what is here is every fact the decision needs and no decision: the table,
how PostgreSQL reached it, the predicate it applied with the values taken
out, how many rows it said it threw away, the lines that asked, and -- from
:attr:~django_query_contract.PlanCapture.relation_indexes -- the indexes
that already exist, in the server's own words. The reader supplies the part
that is a judgement. That is the same bargain
:attr:~django_query_contract.PlanNode.estimate_error struck: report the
two numbers, classify neither.
It is a grouping and not a detector, in the sense
:func:~django_query_contract.group_by_call_site sets out. Nothing here is
a finding, nothing fails on it, and there is no rule anywhere in it about
which read is the interesting one -- which is exactly why it is allowed to
put a relation's reads side by side, where a finding would not be.
:attr:records and :attr:nodes are parallel: nodes[i] is the node in
records[i]'s plan that read this relation. A record appears twice when
one plan read the table twice, which is what a self-join is.
relation
instance-attribute
¶
The table, under the name PostgreSQL printed in the plan.
records
instance-attribute
¶
The executions that read it, in capture order. At least one.
nodes
instance-attribute
¶
The node in each of those plans that did the reading.
count
property
¶
How many times this relation was read.
Reads, not statements: one plan reading a table twice counts twice, so the reads listed under a relation cannot outnumber the number printed beside its name.
first_index
property
¶
Position in the capture of the first statement that read this relation.
The tie-break on an ordering by :attr:count -- and, unlike
:attr:~django_query_contract.Attribution.first_index, it does not
make that order total on its own. An attribution's first statement
belongs to it alone; one statement reads several relations, so two
groups here can share one and tie on both keys. Plan order settles
those, which is deterministic and is the order EXPLAIN printed them
in.
call_sites
property
¶
The distinct lines these reads came from, in the order first seen.
Picked by the rule the whole package shares -- the innermost frame
outside Django, see
:attr:~django_query_contract.QueryRecord.call_site -- so a report can
never name one line here and a different line for the same statement
three blocks higher up. A None is a record whose kept frames were
all Django's own, reported rather than approximated.
indexes_used
property
¶
Every index PostgreSQL read this relation through, in the order first seen.
Resolved per node by :attr:~django_query_contract.PlanNode.indexes_used,
which walks down past the nodes PostgreSQL splits a bitmap read across.
Empty means every read here went to the table itself.
unindexed_reads
property
¶
The reads that reached this table without an index, in plan order.
The subset an index decision is about -- and the whole of what this package will say on the subject. Whether a read wanting an index is a problem depends on how much of the table it read, and that is the number this package does not pick.
conditions
property
¶
The distinct predicates applied to this relation, in the order first seen.
Shapes rather than predicates as printed: the value is taken out at
parse time, for the reasons
:attr:~django_query_contract.PlanNode.condition gives. That is what
lets twelve executions of one statement with twelve parameters appear
here as one entry rather than twelve.
most_rows_discarded
property
¶
The most rows any one read of this table threw away, and which read.
An argmax rather than a cut-off, which is the same device
:attr:~django_query_contract.QueryPlan.worst_estimate uses and for the
same reason: every set of reads has one that discarded the most, so
naming it introduces no number of ours. Whether that many is too many is
the judgement this class declines.
The number is the whole read and not one loop of it, which is
:attr:~django_query_contract.PlanNode.total_rows_removed_by_filter
rather than the count printed on the node. A read PostgreSQL split
across three processes is still one read of this table, and it discarded
everything the three of them discarded. Ranking on the printed number
instead would order two reads by how many workers the server happened to
start: measured on one statement over 1,200,000 rows, the parallel plan
prints 374,699 and the serial plan 1,124,098 for the same work.
None when no read here filtered at all. PostgreSQL emits
Rows Removed by Filter only where it applied one, so zero would be a
measurement it never made.
The growth curve¶
Growth ¶
Bases: Enum
The claim a growth assertion makes: an upper bound on how a count may grow.
A growth assertion runs one block against worlds of several sizes and asks
whether the number of statements it emitted kept its shape. This names the
shape, and both members are upper bounds, which is the decision worth
stating: LINEAR is satisfied by a block that turns out to be constant,
because a count that grows less than allowed is never the defect. Only
growing faster than the claim is.
Every rule here is exact integer arithmetic, and that is the whole design.
The alternative -- fitting a curve to the counts and deciding whether the
slope is near enough to zero -- needs a tolerance, a goodness-of-fit floor
and a rule for what counts as linear, which is three knobs where this has
none. A growth assertion that is itself flaky is worse than no growth
assertion, because it gets deleted and takes the idea with it. See
measure_query_growth for the rest of that argument.
The rules are stated over a pair of measurements rather than over the whole curve because both are transitive: equality is, and so is a non-increasing ratio. Checking consecutive pairs therefore decides the whole curve, and the pair that failed is the pair a failure message can point at.
CONSTANT
class-attribute
instance-attribute
¶
O(1): the same statements whatever the data. The common case, and exact.
LINEAR
class-attribute
instance-attribute
¶
O(N): at most proportionally more statements for proportionally more data.
Legitimate for genuine bulk work -- a bulk_create batched by row count,
an .iterator() walking pages -- and it has to be expressible, or a suite
with real batch work has no way to assert anything about it and asserts
nothing at all.
headline
property
¶
The first line of a failure: what the counts did, in one sentence.
permits ¶
Whether going from smaller to larger stayed inside this bound.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
smaller
|
GrowthPoint
|
The measurement at the lower factor. |
required |
larger
|
GrowthPoint
|
The measurement at the higher factor. |
required |
Returns:
| Type | Description |
|---|---|
bool
|
|
explain ¶
State the rule this pair broke, in the numbers that broke it.
Kept beside :meth:permits rather than in the formatter so the sentence
and the arithmetic cannot drift into describing different rules -- the
failure a reader would have no way to detect, because the only evidence
they have is the sentence.
QueryGrowth
dataclass
¶
What one block did at several scale factors.
The measurement, kept separate from the claim on purpose. measure_query_growth
produces one of these and makes no assertion at all; assert_query_growth
is a thin layer that measures and then compares against a
:class:~django_query_contract.Growth. That split is the same one the rest
of the package makes -- a capture is data, a finding is a reading of it, and
a report is a rendering -- and it is what lets the CI-report face plot a
curve without a test runner or a claim in the loop.
A measurement can be compared against more than one claim, which is
occasionally what you want: a block that fails CONSTANT and holds
LINEAR is bulk work, and a block that fails both is a defect.
points
instance-attribute
¶
One per scale factor, in the order they ran, which is ascending by factor.
counts
property
¶
The statement count at each factor, in the same order as factors.
first_violation ¶
The first consecutive pair of points that broke growth.
Consecutive pairs decide the whole curve, because both rules are transitive: if every step kept the count equal then all of them are equal, and if every step left the count-per-unit-of-data no higher than the step before then the whole curve did. Checking every pair instead would find the same violations and report a wider one, which names two worlds that are further apart and is therefore less use to a reader.
Returns:
| Type | Description |
|---|---|
tuple[GrowthPoint, GrowthPoint] | None
|
The lower and higher measurement of the first pair that broke the |
tuple[GrowthPoint, GrowthPoint] | None
|
claim, or |
GrowthPoint
dataclass
¶
A single point on a growth curve: one world, one run of the block.
The capture is kept whole rather than reduced to its count, and that is what makes a growth failure actionable instead of merely true. "Four statements at factor 1, one thousand and three at factor 10" says a count grew; the capture behind the larger point says which statement grew and from which line, through the same N+1 detector and the same report every other face of this package uses.
It also carries the ceiling. A growth run is the regime where Django's own
query log stops being able to count -- a per-row statement over a thousand
rows at a factor or two more is thousands of statements in one block -- so
the point where a growth curve gets interesting is the point where
assertNumQueries would have started under-reporting, and
capture.exceeded_ceilings says so.
Nothing retains a point beyond the assertion that made it. A capture at a
high factor is large -- one record per statement, with up to stack_depth
frames each -- so a growth measurement is a local value in the test that
asked for it, never stashed anywhere with a longer life.
factor
instance-attribute
¶
The scale factor this world was built at. The curve's x-axis.
rows
instance-attribute
¶
How many rows the world reported holding, or None when it reported none.
A diagnostic, not the x-axis: the caller passed the factor in and already
knows it, while the row count is what the database actually took. None
when the world yielded something that is not a whole number -- a
hand-written @contextmanager that simply yields is a legitimate
implementation of the seam and gives no number, so the report says how big
the world was only when it was told.
capture
instance-attribute
¶
Every statement the block executed in this world, closed and readable.
The plan¶
PlanCapture ¶
Bases: QueryCapture
A :class:~django_query_contract.QueryCapture that also asks for the plan.
from django_query_contract import PlanCapture, format_query_plans
with PlanCapture() as capture:
render_author_list()
print(format_query_plans(capture))
The plan is taken at execution time, and it has to be. This package
retains no parameters -- a bulk_create arrives as one execution and ten
thousand values, and holding them would mean keeping customer data in memory
to answer a question about query counts -- so a plan cannot be taken after
the fact from a record. The wrapper is the one moment the statement and its
bindings are both in hand, and that was written into
:class:~django_query_contract.QueryRecord from the first release, before
this class existed.
It is taken before the statement runs, for the same reason the record is written before it runs: a statement that raises is still a statement that was executed, and a plan dropped because the query failed would be missing from exactly the diagnosis that needed it.
EXPLAIN goes out on the driver connection, under the Django cursor
rather than through it, and that is what keeps composition intact. Django's
execute_wrapper and its queries_log both sit on the Python cursor, so
a plan taken through connection.cursor() would be captured by this very
wrapper -- endlessly -- and would also be counted by
django_assert_num_queries, which counts through queries_log. Measured
both ways against a real server: the raw cursor is seen by neither. The
package already documented that blind spot as a limitation of the capture;
here it is the mechanism.
What it costs, measured. Against PostgreSQL 16 on a shaped 400,000-row
world, a two-statement block took 8.9 ms on its own and 14.8 ms with plan
capture: about 1.7x, which is what running each statement twice buys.
With analyze=False it was 9.1 ms, or 1.02x, because the server only plans.
ANALYZE is the default anyway, because a plan with no measurement in it
cannot produce a finding, and a plan capture that can produce no finding is
the vacuous pass this package exists to refuse.
unanalyzed_relations
property
¶
The tables in these plans that PostgreSQL has never gathered statistics for.
The other half of "never pass vacuously", and the half a vendor check cannot reach. Refusing SQLite covers the backend with no planner. This covers the backend that has one and was given nothing to reason with: rows loaded in a fixture and never analyzed leave the planner guessing from a default selectivity, and the plan it prints is confident and meaningless. Measured during this package's design: two million rows never analyzed produced a bitmap heap scan estimated at 10,000 rows for two predicates whose real answers were 30,298 and 1,959,743.
It says nothing about whether the tables are big enough, and cannot: "ten rows is too few" is a number, and a number is the knob this package refuses. What it can do is print the row count the planner is working from beside the plan and let a reader judge.
Empty before the block has finished, and empty on any capture that took no plans.
relation_indexes
property
¶
The indexes each relation in these plans already has, in PostgreSQL's own words.
Keyed by relation name, holding the CREATE INDEX statements the
server would write to build them -- pg_get_indexdef, unedited, so an
expression index, a partial index and a non-default operator class all
come out right without this package learning any of the three.
The one place index advice touched something real, and it is the half
that is a fact. The milestone this was written for wanted to emit the
CREATE INDEX statements a capture implies; that turned out to need a
threshold and a SQL parser, and was declined -- see
:class:~django_query_contract.RelationAccess. What survives is
printing the statements that already exist beside the filters that were
applied, so a reader can see which predicate nothing covers and decide
for themselves.
A relation is absent rather than empty when the catalogue said nothing about it, because an empty tuple would claim a table has no indexes and that is false for anything with a primary key.
Empty before the block has finished, and empty on any capture that took no plans.
refusal ¶
Why plans cannot be captured on these connections, or None.
Reads vendor off each connection and decides from the string, without
opening anything. That is deliberate and it is what keeps this package's
coverage gate on the portable matrix: a degradation path reachable only
by running the suite on the backend it refuses is a path the gating job
cannot see.
Public because the two callers need the same sentence delivered two ways.
__enter__ raises it, and the query_plans fixture skips with it.
QueryPlan
dataclass
¶
One statement's execution plan, or the stated reason there is not one.
Hangs off :attr:~django_query_contract.QueryRecord.plan, so a plan travels
with the statement it belongs to and with the call stack that emitted it.
That pairing is the whole point: an index recommendation is a plan plus an
address, and a plan in a list of its own would have to be joined back to the
statement by index before anything could be said about it.
A refusal is a value here, not an absence. EXPLAIN ANALYZE executes
the statement it is given, so this package runs it only on a statement that
is known not to change anything, and there are several ordinary reasons a
statement is skipped. Recording those as plan=None would make "nobody
asked for plans" and "we declined to explain this one" the same observation,
and a report could then only say nothing about either. So a skipped
statement carries a plan whose :attr:root is None and whose
:attr:refusal is the sentence saying why, and plan is None means one
thing only: this capture was not a
:class:~django_query_contract.PlanCapture.
root
instance-attribute
¶
The top of the plan tree. None when refusal says why there is none.
analyzed
instance-attribute
¶
Whether the plan carries measurements or only the planner's expectations.
False means EXPLAIN ran without ANALYZE: the shape of the plan is
real, every actual_rows is None, and no finding in this package can
be made from it. That is reported rather than left to be inferred from a
tree of Nones, because a plan with nothing to check against is exactly
the shape a vacuous pass would take.
refusal
class-attribute
instance-attribute
¶
Why this statement was not explained, in a sentence. None when it was.
nodes
property
¶
Every node in the tree, parents before children. Empty for a refusal.
worst_estimate
property
¶
How far out this plan's least accurate node was, and which node that is.
An argmax rather than a cut-off, and that is deliberate: every measured
plan has exactly one node the planner was most wrong about, so naming it
introduces no number. Whether being wrong by that much matters is left to
the reader, for the reasons
:attr:~django_query_contract.PlanNode.estimate_error sets out.
The factor comes back beside the node rather than being read off it again, so a caller cannot end up holding a node it has to re-test for a measurement this property already established it has.
None for a refusal and for a plan taken without ANALYZE, where
there is no actual to be wrong about.
from_explain
classmethod
¶
Build a plan from the JSON document EXPLAIN (FORMAT JSON) returns.
That document is a list holding a single object, whose Plan key is
the root node. The list is PostgreSQL's own shape and is unwrapped here
rather than by the caller, so the one place that knows the wire format is
the one place that reads it.
refused
classmethod
¶
A plan that was not taken, carrying the reason it was not.
PlanNode
dataclass
¶
A single step of a plan: what PostgreSQL chose, and what happened when it ran.
The fields are the subset of EXPLAIN (ANALYZE, BUFFERS, TIMING OFF,
FORMAT JSON) this package can say something true about, and every one of
them was read off a real server rather than off the documentation. What is
kept and what is dropped both follow one rule.
No timings are kept, and the TIMING OFF in that statement is the same
decision seen from the other side. This package's argument is that a
performance assertion mentioning a number of milliseconds is a flaky test
with extra steps, so a per-node duration would be a field inviting exactly
the assertion the package exists to refuse. Turning the instrumentation off
is therefore not a cost optimisation that happens to agree with the thesis;
it is the thesis, and the cost saving is the bonus -- ANALYZE with
timings on calls gettimeofday twice per row per node.
Rows are floats, not integers. Plan Rows is an integer today, but
Actual Rows became fractional in PostgreSQL 18 for a node under more
than one loop, where it is an average. Rounding it here to keep a tidier
type would make this record quietly lossy on a server that is already
shipping.
Row counts are per loop, and that is the one thing about this record that
changes meaning as a database grows. PostgreSQL divides a node's actual row
count by Actual Loops before printing it, and does the same to
Rows Removed by Filter. In a small world every node runs once, loops
is 1, and the printed number is the whole truth. In a big one the same
statement is handed to three processes, or the same scan is run once per
outer row, and the printed number becomes a share -- with nothing in the plan
announcing the change except the loop count nobody was reading.
Measured, and the pair is checked in: the same
SELECT COUNT(*) ... WHERE md5(reference) < %s over 1,200,000 rows reports
Rows Removed by Filter: 1124098 in one process and 374699 in three.
So :attr:total_actual_rows and :attr:total_rows_removed_by_filter are
here, and they are what a report and an assertion should read.
The estimate is not totalled, and refusing that is the harder half of the decision. The two multiplications are not the same one:
- Under a
Gather,loopscounts the processes that actually ran, while the planner divided its estimate byparallel_workersplus the fraction of a worker it credits the leader with -- 2.4 for two workers, against a loop count of 3. Measured on the pair above: 400,000 estimated serially, 166,667 on the parallel node, and 400,000 / 166,667 is 2.4 exactly. The divisor is not in the output, so no arithmetic here recovers it. - Under a nested loop,
loopsis the number of outer rows that arrived, which is a measurement. Multiplying a per-loop estimate by it produces a number the planner never predicted -- measured on an inner node estimating 60 rows over 1,260 loops, the product is 75,600, which is exactly what the join measured while the planner's own estimate for it was 400,020.
A total_estimated_rows would therefore be wrong under a Gather and
would agree with the measurement under a nested loop, which is worse: it
would read as perfect agreement on the plan the planner got most wrong.
node_type
instance-attribute
¶
Seq Scan, Nested Loop, Sort -- PostgreSQL's own name for the step.
relation
instance-attribute
¶
The table this node reads, when it reads one directly. None for a join or a sort.
index
instance-attribute
¶
The index this node reads, when it uses one. None for a sequential scan.
condition
instance-attribute
¶
The Filter this node applied, as a shape: the predicate without its values.
Half of what index advice is made of -- a filter over a relation, with the
rows it threw away, is the statement a CREATE INDEX would answer -- and
the reason the advice itself is declined is set out at
:class:~django_query_contract.RelationAccess.
PostgreSQL renders this predicate with the bound value spelled out, and
that value is taken back out here. A parameterised query against a real
server produces Filter: ((reference)::text = '601980.6826913885'::text),
so keeping the string verbatim would put a customer's data on a public
record for the length of a capture -- in a package that retains no
parameters anywhere else, and whose own refusal sentence tells the reader so
when it declines to quote a driver error. The rendering is put through
:func:~django_query_contract.normalise_sql, which is the same small list
of named rules the statement fingerprint is made with, so a value becomes
%s and the column, the operator and the casts survive.
That redaction is also what makes the predicate a group. With the value in it, one statement shape run with twelve parameters is twelve different conditions and no report could say the twelve executions did the same thing; without it, they are one.
estimated_rows
instance-attribute
¶
Plan Rows: how many rows per loop the planner expected this node to produce.
actual_rows
instance-attribute
¶
Actual Rows: how many it produced per loop. None when the plan was not analyzed.
loops
instance-attribute
¶
Actual Loops: how many times this node was executed. None without ANALYZE.
The divisor behind every other measurement on this record, and the field a reader of a small database never has to think about because it is 1 there.
parallel_aware
instance-attribute
¶
Parallel Aware: whether this node is one process's share of a parallel scan.
Why a loop count is not self-explanatory. More than one loop has two
quite different causes, and this is the only field that tells them apart. A
parallel-aware node ran once in each participating process, so its loops
is a count of processes and the work was divided; a node under a nested loop
ran once per outer row, so its loops is a measurement of the outer side
and the work was repeated. The totals are the same arithmetic either way --
see :attr:total_actual_rows -- but :attr:estimate_error is only
comparable in the second case, for the reason set out on it.
False for every node of a plan taken without ANALYZE as well, which
is correct rather than a default: parallelism is a property of the plan and
EXPLAIN prints Parallel Aware whether or not it measured anything.
rows_removed_by_filter
instance-attribute
¶
How many rows this node read and discarded, per loop, when it filtered.
Per loop, and therefore the number that reads as 374,699 on a scan that
discarded 1,124,098 rows because three parallel workers each did a third of
it. :attr:total_rows_removed_by_filter is the one to assert on.
sort_method
instance-attribute
¶
quicksort, external merge -- how a sort node sorted. None if it is not one.
sort_space_type
instance-attribute
¶
Memory or Disk: where a sort node's working space came from.
sort_space_used_kb
instance-attribute
¶
How much of that space it used, in kilobytes.
hash_batches
instance-attribute
¶
How many batches a hash node needed. More than one means it did not fit in work_mem.
Reads Hash Batches from a hash join and HashAgg Batches from a hash
aggregate, because they are the same fact under two names and a reader
asking "did this spill" should not have to know which node type produced it.
disk_usage_kb
instance-attribute
¶
Disk Usage: the temporary space a hash aggregate spilled, in kilobytes.
shared_hit_blocks
instance-attribute
¶
Buffer pages this node found in cache. None when BUFFERS was not asked for.
shared_read_blocks
instance-attribute
¶
Buffer pages this node had to read. The number a plan finding quotes as heap blocks.
children
instance-attribute
¶
The nodes feeding this one, in the order PostgreSQL listed them.
indexes_used
property
¶
The indexes PostgreSQL read this node's relation through. Empty means none.
Not the same question as :attr:index, and the difference is what
keeps a report from crying wolf. PostgreSQL splits a bitmap read
across two nodes: the Bitmap Heap Scan names the table and carries no
Index Name at all, while the Bitmap Index Scan beneath it names
the index and no table. Put a BitmapAnd between them -- two indexes
combined -- and the index is two levels down. A reading that looked only
at the node itself would report a table PostgreSQL reached through two
indexes as one it read end to end, which is the single worst thing this
report could say.
So it walks down, and stops at the next node that names a relation: that node is a different read of a different table, and its index belongs to it. Both payloads that pin this are a real server's output.
Order is the order PostgreSQL listed the nodes in, and duplicates are
possible in principle -- the same index reached twice under one
BitmapOr -- so a caller wanting a set should say so.
spilled_to_disk
property
¶
Whether this node ran out of work_mem and used the disk instead.
The one plan defect that needs no threshold of ours, because PostgreSQL
already applied its own. A sort that says Sort Space Type: Disk, a
hash join that needed more than one batch and a hash aggregate that
reports disk usage are all the server stating that the memory it was
given was not enough. The number that decided it is work_mem, which
belongs to the database being tested rather than to this package, so
there is nothing here to tune and nothing to be wrong about.
That also means the finding is a claim about a configuration as much as about a query, which is why nothing in this package fails a test on it.
total_actual_rows
property
¶
Every row this node produced, across all of its executions.
:attr:actual_rows multiplied by :attr:loops, which is the number a
reader means when they say "how many rows did this read return" and the
number an assertion written against a one-loop plan was really making a
claim about. Where loops is 1 it is :attr:actual_rows unchanged, so
adopting it costs nothing on the small worlds where the two agree.
It is a reconstruction and not a measurement, and the difference is
one row. PostgreSQL divides the count by the loop count and rounds
before printing, so multiplying back can be out by up to half a loop in
either direction: measured on a three-process scan that really produced
75,902 rows, the node says 25,301 and this says 75,903. The residue is
bounded by loops / 2 and it is stated here rather than hidden,
because the alternative -- an exact total -- is a number the server does
not print at all.
None without a measurement or without a loop count. A missing loop
count is not treated as 1: this record is public and a caller may have
built one from something that is not this parser, and turning a gap in
the input into a number is the move this package refuses everywhere else.
total_rows_removed_by_filter
property
¶
Every row this node read and discarded, across all of its executions.
The same multiplication as :attr:total_actual_rows, with the same
rounding residue, over the number a report quotes when it says how much
of a table a read threw away. That number is the one that moves most
alarmingly when a world gets big enough to be scanned in parallel, and
it moves downwards: a scan discarding 1,124,098 rows in one process
reports 374,699 in three.
None when this node applied no filter. PostgreSQL emits
Rows Removed by Filter only where it applied one, so a zero here
would be a measurement it never made.
estimate_error
property
¶
How many times out the planner's estimate turned out to be, at least 1.0.
Reported, never classified, and that distinction is the design. "The planner expected 20 rows and 20,323 arrived" is a fact about this plan. "An estimate more than 50 times out is a defect" is a policy about size, and a policy about size is the knob this package refuses everywhere else. So this is a number on a record, ordered by a report and read by a human; no code in this package turns it into a verdict.
There are also ordinary reasons for a large value that have nothing to do
with a defect. A node under a LIMIT stops early by design, so its
actual is meant to fall short of its estimate; measured on a plain
[:5] query against fifty rows, the scan under the limit reports an
estimate of 50 against an actual of 5. A rule that flagged that would cry
wolf on the first query anybody pointed it at.
On a parallel-aware node it is inflated, and by a bounded amount. The
two numbers are both per loop, but they were divided by different
denominators: the measurement by the processes that ran, the estimate by
parallel_workers plus the fraction of a worker the planner credits
the leader with. Measured, two workers means dividing the estimate by 2.4
and the measurement by 3, so a node the planner priced within 1% reports
a ratio of about 1.25. The inflation is at most that -- the two divisors
never differ by more than the leader's share -- so this stays the ratio
of the two numbers PostgreSQL printed, and :attr:parallel_aware is on
the record so a report can say which nodes it applies to. There is no
repair available: the divisor the planner used is not in the output.
Direction is not encoded, because both numbers are on the record and a
report prints them side by side. None when the plan was not analyzed:
without an actual there is nothing to be wrong about.
Zero actual rows are compared as one. PostgreSQL never estimates below one row itself, so one is the floor the planner's own arithmetic uses, and a ratio against zero is not a number.
from_explain
classmethod
¶
Build a node, and its children, from one Plan object of the JSON output.
Every key is read with a default because EXPLAIN emits a key only
when it applies: a sequential scan has no Index Name, a plan taken
without ANALYZE has no Actual Rows, and a node that never
filtered has no Rows Removed by Filter. Requiring any of them would
make this raise on the ordinary plan rather than on the unusual one.
PlanDefect ¶
Bases: Enum
What a plan finding accuses, and there are only two because only two qualify.
A finding here holds by construction, exactly as an N+1 does. The plan that produced this package listed four candidates: sequential scans over a row threshold, nested loops with a large inner, sorts spilling to disk, and planner estimates off by orders of magnitude from actual. Two of them are numbers wearing a description -- how many rows makes a sequential scan wrong, how large an inner is large -- and a number is a knob. The four dead N+1 detectors on PyPI are what a package of knobs looks like a few years later, so both are declined here rather than shipped with a default nobody would agree with.
The two that survived did so for different reasons, and the difference is worth keeping in view:
- a spill is a fact PostgreSQL asserts, using a threshold of its own
(
work_mem) that belongs to the database rather than to this package; - blindness is a fact about a pair of measurements, decided by equality and inequality, with no magnitude anywhere in it.
The estimate-versus-actual ratio itself is not here, and its absence is the
design rather than an omission. It is on every node as
:attr:~django_query_contract.PlanNode.estimate_error, ordered by the
report and read by a person: "the planner expected 20 and 20,323 arrived" is
a fact, while "more than fifty times out is a defect" is a policy about
size. This package reports the first and refuses to write the second.
PLANNER_BLIND
class-attribute
instance-attribute
¶
One statement shape, one estimate, and more than one truth.
Two or more executions of the same normalised SQL whose plans agree exactly
on how many rows the query would produce, and whose measured rows do not.
The planner cannot tell those executions apart; the data can. There is no
threshold in that sentence -- only == and != -- and it is the defect
the shaped-database dependency exists for.
Measured on a Zipf fan-out of 400,000 rows over 20,000 parents, joined
through the parent rather than through the foreign key column: a whale and a
tail row both estimated at 20 rows, against actuals of 20,323 and
6. Across a join PostgreSQL has only n_distinct for the edge, so it
hands every value of the join key the same average, and the average is the
one number that is wrong for both ends of a skewed distribution.
The identity is the fingerprint alone, and deliberately not the call
stack. That is the opposite choice from
:class:~django_query_contract.NPlusOne, and the rule behind both is the
same one: a finding is keyed on what the thing being accused can actually
see. An N+1 accuses your code, so it is keyed on your code's call path. This
accuses the planner, which is handed a statement and never hears about the
stack, so keying on the stack would split one blind spot into one finding per
line that happened to reach it.
SPILLED_TO_DISK
class-attribute
instance-attribute
¶
A sort, hash join or hash aggregate that did not fit in work_mem.
PostgreSQL says so itself -- Sort Space Type: Disk, more than one hash
batch, or a hash aggregate reporting disk usage -- so the threshold is
work_mem, which is the configuration of the database under test and not
a number chosen here.
That is also its limitation, stated rather than glossed: it is as much a finding about the server as about the query, and a suite whose CI database is configured differently from production will find different ones. Nothing in this package fails a test on it.
PlanFinding
dataclass
¶
A defect PostgreSQL's own output states, and the evidence for it.
:attr:records and :attr:nodes are parallel: nodes[i] is the node in
records[i]'s plan that the finding is about. One shape carries both
kinds, because both are "these executions, at these nodes" and giving each
kind its own class would make a report iterate two lists to say one thing.
A spill has one execution and the node that spilled.
:attr:~django_query_contract.PlanDefect.PLANNER_BLIND has two or more
executions and each of their root nodes, which is where a plan states how
many rows the query produces.
Nothing in this package fails a test on one of these. A finding is a
diagnosis printed under a failure somebody else's assertion produced, for
the reason :class:~django_query_contract.NPlusOne sets out at length: a
detector that fails builds is a detector that gets uninstalled.
records
instance-attribute
¶
The executions this finding is made of, in capture order. At least one.
nodes
instance-attribute
¶
The node in each of those executions' plans that the finding is about.
fingerprint
property
¶
The normalised SQL shared by every execution here. See normalise_sql.
first_index
property
¶
Position in the capture of the first execution involved.
The tie-break that makes an ordering total, so two runs over one capture list findings in the same order.
call_sites
property
¶
The distinct lines these executions came from, in the order first seen.
A tuple rather than a single frame, because the identity of
:attr:~django_query_contract.PlanDefect.PLANNER_BLIND is the statement
shape and not the call path -- the planner is handed SQL and never hears
about the stack -- so one finding can legitimately span several lines. A
None in here is a record whose kept frames were all Django's own,
reported rather than approximated for the reason
:attr:~django_query_contract.QueryRecord.call_site gives.
estimated_rows
property
¶
The row count the planner expected, which every node here agrees on.
For a spill there is one node and this is simply its estimate. For blindness the shared estimate is the finding: the grouping keys on it, so agreement is true by construction rather than by coincidence.
actual_rows
property
¶
What each execution actually produced at that node, in capture order.
Per loop like every other row count in this package, which is worth
saying here rather than leaving to be rediscovered. For
:attr:~django_query_contract.PlanDefect.PLANNER_BLIND it makes no
difference: those nodes are plan roots, a plan's root runs exactly once,
and a node that ran once is its own total. A spill is reported on
whichever node spilled, which can be the inner side of a join that ran
thousands of times, and there each number here describes one of those
executions -- :attr:~django_query_contract.PlanNode.total_actual_rows
on the node beside it is the whole of what that read produced.
float | None because that is what a node holds, and narrowing it here
would be this class asserting something about how it was built.
:func:~django_query_contract.find_plan_defects only ever makes a
finding out of analyzed plans, so every entry is a real measurement --
but this is a public record and a caller may hold one it built itself.
PlansUnsupported ¶
Bases: Exception
Plan capture was asked for on a connection that cannot produce a plan.
Raised by :class:~django_query_contract.PlanCapture on entry, before a
single statement has run, and carrying the sentence that names what was
refused, which connection, and what that connection actually is.
It raises rather than degrading, and that is the whole point of the class.
Every other honest degradation in this package reports and carries on: a
capture rebuilt from a CaptureQueriesContext says it has no stacks, a
block above the query-log ceiling says the count is wrong. Those are still
measurements. A plan capture on SQLite would not be a degraded measurement,
it would be an empty one, and an empty one is indistinguishable from a
healthy one -- so an assertion over it passes because the backend could not
check it, which is the exact failure this package exists to expose.
The pytest face turns this into a skip rather than an error, through the
query_plans fixture: a test that never ran is honest, and a test that ran
against a database with no planner is not.
The ceiling¶
LogCeiling
dataclass
¶
The arithmetic behind a query count that stops being true.
django.test.utils.CaptureQueriesContext -- which is what
assertNumQueries and pytest-django's django_assert_num_queries both
count with -- records len(connection.queries_log) on entry and on exit
and returns the slice between them. But queries_log is a
deque(maxlen=connection.queries_limit), 9000 by default, and once it
rotates those two absolute indices no longer point at what they did.
Measured against Django 6.1, with the count that context manager reports:
| Already in the log | Queries in the block | Reported |
|---|---|---|
| 0 | 8999 | 8999 |
| 0 | 9001 | 9000 |
| 8990 | 100 | 10 |
| 9000 | 5 | 0 |
The last row is the one that matters: five real queries, a reported count of
zero, and django_assert_max_num_queries(1) passing. Django does emit a
UserWarning when the log is full, so it is not perfectly silent -- but a
warning in the summary beside a green test is not what a reader takes from a
passing assertion, and the regime this happens in, thousands of queries in
one block, is precisely the N+1-at-scale case worth catching.
The capture in this package counts executions through
connection.execute_wrapper, which has no bound at all. This class is how
it says so instead of quietly being right where the other is wrong.
limit
instance-attribute
¶
connection.queries_limit. None means the log is unbounded and there is no ceiling.
log_length_at_enter
instance-attribute
¶
len(connection.queries_log) when the capture opened, itself capped at limit.
executions
instance-attribute
¶
Statements this capture counted. Unbounded, and therefore the true number.
headroom_at_enter
property
¶
How many more entries the log could hold when the capture opened.
Named for the moment it describes, because it never moves. A capture
reads len(connection.queries_log) on the way in and never again, so
this number is fixed for the length of the block: four thousand
statements later it still reports the room there was at the start.
That is the right number rather than a stale one -- it is what
:attr:visible needs, and it is the only one obtainable. Django writes a
statement to the log only when the debug cursor is on, and a capture
counts every execution through execute_wrapper whether it is or not,
so how much of the log this block consumed cannot be derived from either
end. What was wrong was the name, sitting beside a field already spelled
log_length_at_enter.
None when the log is unbounded.
visible
property
¶
What a CaptureQueriesContext opened at the same moment would report.
exceeded
property
¶
True when a count taken from Django's query log would be too low.
QueryLogCeilingWarning ¶
Bases: UserWarning
A block executed more statements than connection.queries_log can hold.
A warning rather than a failure, because this package asserts nothing: the
assertion belongs to django_assert_num_queries and this only says when
its arithmetic stopped being reliable. It is a warning rather than a report
section because the dangerous case is a test that passed -- five real
queries counted as zero -- and a section on a passing test is printed only
when someone asks for it.
Django emits a UserWarning of its own when the log is full, but it says
only that the log truncated; it does not say what the count should have been
or that an assertion just read a wrong number off it.
Functions¶
normalise_sql ¶
Reduce a statement to the shape it shares with its repeats.
normalise_sql
cached
¶
Return the fingerprint of sql: the part that repeats across executions.
Two executions with the same fingerprint ran the same statement with different data. Paired with the call stack, that is the definition of an N+1 -- which is why this is a small list of named, reversible-in-the-head rules rather than a hash. Every one of them can be pointed at, argued with and tested, and the record keeps the original SQL beside the fingerprint so a report never has to be believed on the strength of the normaliser alone.
A real SQL parser was considered and does not fit, for three independent
reasons. The SQL that reaches execute_wrapper carries Django's %s
placeholders -- the backend's own paramstyle is applied below the wrapper --
and %s is a syntax error to a PostgreSQL grammar, so a parser cannot
read the input at all without the parameters being substituted back in,
which is the opposite of what a fingerprint is for. It would also be
PostgreSQL-only in a package whose count and growth assertions are meant to
work on any backend. And it costs roughly fifteen times a regex pass, per
query, in a suite that may run hundreds of thousands.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
sql
|
str
|
The statement as handed to |
required |
Returns:
| Type | Description |
|---|---|
str
|
The normalised statement, with whitespace collapsed. |
find_n_plus_one ¶
Read a capture back as N+1 findings.
find_n_plus_one ¶
Group records into N+1 findings, most repeated first.
The whole of the detector, and it is short because the definition is: bucket
every execution by (fingerprint, stack) and keep the buckets holding more
than one. There is no threshold to configure -- "more than once from the same
place" is what the word means -- and no rule that decides some repetitions
are interesting and others are not. See :class:NPlusOne for why the key is
the whole stack rather than the call site, and for why a legitimate batched
write is reported like any other repetition.
A record with no call stack is not considered. It cannot be: the identity
is half stack, and a record rebuilt from a CaptureQueriesContext has
none. Bucketing those together would say "these ran from one place" on the
strength of knowing nothing about where any of them ran, which is a false
positive manufactured out of a gap in the input. They are skipped, and
format_capture_report says how many were skipped rather than reporting a
clean bill of health it did not earn.
Takes any iterable of records, so it reads a
:class:~django_query_contract.QueryCapture directly, a slice of one, or
the records of a single connection.
from django_query_contract import QueryCapture, find_n_plus_one
with QueryCapture() as capture:
render_author_list()
for finding in find_n_plus_one(capture):
print(finding.count, finding.call_site, finding.fingerprint)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[QueryRecord]
|
The executions to group. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
NPlusOne
|
The findings, ordered by |
|
...
|
first appeared in the capture. That second key is what makes the order |
|
total |
tuple[NPlusOne, ...]
|
no two findings share a first execution, so two runs over one |
tuple[NPlusOne, ...]
|
capture produce the same list and a report never reshuffles. |
group_by_call_site ¶
Read a capture back as the lines its statements came from.
group_by_call_site ¶
Group records by the line that emitted them, busiest line first.
"These forty statements came from these three lines" -- the other axis of the grouping the N+1 detector does, and the one that has an answer for every statement rather than only for a repeated one. Until this existed, a capture would name a call site only where a finding rendered one, so a failed count assertion with no N+1 in it named no lines at all.
It is called group_ and not find_, and the difference is not
cosmetic. :func:~django_query_contract.find_n_plus_one finds defects and
its key is the whole call stack. This finds nothing: it re-files the same
statements under the line that emitted them, which merges call paths a
finding deliberately keeps apart. Both are true at once, and
:class:~django_query_contract.Attribution sets out why the merge is safe
here and would be wrong there.
Every record ends up in exactly one group, including the ones with no
call site. A record with no stack, or whose kept frames were all Django's
own, joins the single group whose call_site is None. That group is
ordered like any other, on its size -- there is no rule here about which
group is the interesting one, and in a capture rebuilt from a
CaptureQueriesContext it is genuinely the headline: nothing in it can be
placed.
Takes any iterable of records, so it reads a
:class:~django_query_contract.QueryCapture directly, a slice of one, or
the records of a single connection.
from django_query_contract import QueryCapture, group_by_call_site
with QueryCapture() as capture:
render_author_list()
for attribution in group_by_call_site(capture):
print(attribution.count, attribution.call_site)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[QueryRecord]
|
The executions to attribute. |
required |
Returns:
| Type | Description |
|---|---|
Attribution
|
The attributions, ordered by |
...
|
line's first statement appeared in the capture. That second key is what |
tuple[Attribution, ...]
|
makes the order total: no two groups share a first statement, so two |
tuple[Attribution, ...]
|
runs over one capture produce the same list and a report never |
tuple[Attribution, ...]
|
reshuffles. |
group_by_relation ¶
Read a capture's plans back as the tables they touched.
group_by_relation ¶
Group the plan nodes in records by the table each one read.
"These twelve statements read orders, eleven of them without an index,
and here is what PostgreSQL threw away" -- the third axis of the same
capture, after the call path a defect repeats on and the line a statement
came from. It is the reader the index-advice milestone became once the
advice itself turned out to need a threshold; the argument is at
:class:~django_query_contract.RelationAccess.
It is called group_ and not find_, and that is load bearing here
more than anywhere. This finds nothing and accuses nothing. It re-files
plan nodes under the table they read, which lets a report put a sequential
read of one table beside an indexed read of the same table -- a juxtaposition
that would be a false accusation if anything claimed it meant something,
because two statements filtering different columns are not two measurements
of one thing.
Only nodes that name a relation are grouped. A join, a sort and an
aggregate read no table of their own, and inventing one for them would put a
row in this report that no CREATE INDEX could ever answer. Unlike
:func:~django_query_contract.find_plan_defects, plans taken without
ANALYZE are kept: which table a plan reads is on the plan whether or not
it was measured, and only the discarded-row counts go missing.
Takes any iterable of records, so it reads a
:class:~django_query_contract.PlanCapture directly, a slice of one, or the
records of a single connection. A record with no plan -- every record from
an ordinary :class:~django_query_contract.QueryCapture -- contributes
nothing, so calling this on a capture that took no plans returns nothing
rather than raising.
from django_query_contract import PlanCapture, group_by_relation
with PlanCapture() as capture:
render_dashboard()
for access in group_by_relation(capture):
print(access.relation, access.count, len(access.unindexed_reads))
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[QueryRecord]
|
The executions to read. |
required |
Returns:
| Type | Description |
|---|---|
RelationAccess
|
The accesses, ordered by |
...
|
relation's first statement appeared, and then -- for the two relations |
tuple[RelationAccess, ...]
|
one statement read -- by the order the plan listed them. |
tuple[RelationAccess, ...]
|
Deliberately not ordered by rows discarded, which is the order a |
tuple[RelationAccess, ...]
|
reader would find most useful and is exactly why it is refused: ranking |
tuple[RelationAccess, ...]
|
tables by how badly they want an index is the judgement this package |
tuple[RelationAccess, ...]
|
declines to make, and a sort key is a quiet way of making it anyway. A |
tuple[RelationAccess, ...]
|
count of measurements ranks nothing. There is a test that pulls the two |
tuple[RelationAccess, ...]
|
orders apart, because a fixture where they agree would pass either way. |
capture_stack ¶
Walk the live call stack into plain frame records.
capture_stack ¶
Return the innermost depth frames outside this package, and whether more existed.
Ordered outermost-first, the way a traceback reads. The innermost frames
are the ones kept when the stack is deeper than depth, because those are
where the information is: between a cursor execution and the test function
sit twenty-odd Django frames, and the call site this package exists to name
is just outside them. Truncating from the other end would drop it.
The boolean is the second half of that bargain. A truncated stack can hide the call site entirely, and this package's whole argument is that a measurement which quietly stops being true is worse than one that refuses -- so truncation is reported rather than absorbed.
Returns:
| Type | Description |
|---|---|
tuple[StackFrame, ...]
|
The frames and a flag that is |
bool
|
available beyond |
in_project_tree ¶
Whether a captured frame is code the reader can edit.
in_project_tree ¶
Whether this frame is code the reader can edit, rather than a dependency.
A display rule, and it must stay one. Which frames matter is exactly the judgement that becomes a knob, and a knob in a detector's identity is how the four dead N+1 detectors came to cry wolf -- so no finding is created, merged, dropped or renamed by this. It decides the order two findings are printed in and nothing else.
The question it exists for is not the one a finding answers. A finding says this statement repeated from this path, and every statement is in scope for that. A run-wide listing says what should I go and fix, and inherits an ordering -- raw repetition count -- under which any library that loops outranks every defect in the project. Measured on a consumer's suite: 158 findings, and none of the ones it had room to print were in the application.
The rule is the working directory minus installed packages, and it is deliberately that crude: no project root setting, no package name, nothing to configure and so nothing to be wrong about. A frame with no filename we can place is treated as not the project's, which is the safe direction -- it keeps an unplaceable finding out of the section a reader is told to act on.
relative_to_cwd ¶
Render a call site against the working directory.
relative_to_cwd ¶
Shorten a rendered call site to a path relative to the working directory.
Only when it is under it: os.path.relpath will happily walk out of the
tree with a row of .. segments, which is longer than the absolute path
and harder to read.
Here for the same reason as the frame choice above. Two renderings name a call site now -- a finding's block and an attribution's -- and a reader shown one path abbreviated and the other absolute would reasonably wonder whether they were the same file.
format_capture_report ¶
Turn a capture into the paragraph a reader needs under a failed assertion.
format_capture_report ¶
format_capture_report(
capture: QueryCapture,
*,
max_findings: int = 5,
max_call_sites: int = 5,
max_sql: int = 160,
) -> str
Describe what a capture saw, worst N+1 first.
This is the diagnosis half of the bargain with django_assert_num_queries:
that fixture builds its message inline and calls pytest.fail(), so there
is no hook inside it and no reason to want one. The user keeps writing the
assertion they already write, and a failure gains this underneath it.
It names an N+1 rather than merely reporting repetition, which is what it
did before the detector existed. The claim is safe to make because of how
the finding is defined -- more than one execution of one statement shape
from one call stack, with no threshold and no rule about lazy loads -- and
because nothing here fails a test on one. See
:class:~django_query_contract.NPlusOne.
A :class:~django_query_contract.PlanCapture gains a fourth block underneath
all of it, describing what the planner did with the statements it explained.
An ordinary capture has no plans and prints none, so one function still
answers "what did this block do" whichever capture a caller is holding.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
capture
|
QueryCapture
|
A closed capture. |
required |
max_findings
|
int
|
How many findings to list before summarising the rest. Applied to N+1 findings and, separately, to plan findings. |
5
|
max_call_sites
|
int
|
How many lines to name when attributing the statements no finding accounted for. |
5
|
max_sql
|
int
|
Where to cut a long statement. The record keeps the whole thing. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The report, without a trailing newline. Empty when there is nothing to |
str
|
say -- no statements and no ceiling crossed. There is deliberately no |
str
|
early return for that case: an explicit guard here was provably dead, |
str
|
because both halves below already produce nothing from nothing. |
format_n_plus_one ¶
Render one N+1 finding as the lines a reader can act on.
format_n_plus_one ¶
Describe finding as an indented block, call site first.
Call site first because that is the whole point. "42 similar queries" with no address is the output people turn off; the line that ran the loop is the line somebody edits, so it goes above the SQL rather than under it.
Both reports in this package render a finding through here -- the section under a failed query-count assertion and the end-of-run listing -- so the two cannot drift into describing the same finding differently.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
finding
|
NPlusOne
|
The finding to describe. |
required |
max_sql
|
int
|
Where to cut a long statement. The records keep the whole thing. |
160
|
label
|
str
|
Where the finding came from, when the caller is listing findings from more than one block. Omitted entirely when empty, so the block reads the same in a report that has only one. |
''
|
Returns:
| Type | Description |
|---|---|
str
|
The block, indented, without a trailing newline. |
format_n_plus_one_summary ¶
List every N+1 found across a run, worst first.
format_n_plus_one_summary ¶
format_n_plus_one_summary(
findings: Mapping[str, Sequence[NPlusOne]],
*,
max_findings: int = 20,
max_sql: int = 160,
) -> str
Describe findings gathered from several blocks, most repeated first.
The listing behind the pytest plugin's --n-plus-one, written as a plain
function so the CI-report face can use it with no test runner in the loop.
The keys are whatever names the blocks -- node ids, for that plugin.
Findings are not merged across blocks, and that is deliberate. The same call site reached from two tests is two findings here, because the identity of a finding is the whole call stack and two tests are two stacks. Merging them would need a second grouping rule -- "these are the same really" -- which is the sort of judgement this package is built without. The listing is ordered instead, so the worst one is the first thing on the screen whether it came from one block or twenty.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
findings
|
Mapping[str, Sequence[NPlusOne]]
|
Each block's findings, keyed by a name for the block. |
required |
max_findings
|
int
|
How many to print before summarising the rest. |
20
|
max_sql
|
int
|
Where to cut a long statement. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The listing, without a trailing newline. Never empty: with nothing to |
str
|
report it says so, because no N+1 anywhere is the answer somebody |
str
|
asked this question to get, and a blank screen does not give it to them. |
format_attributions ¶
Render call-site attributions as the lines a reader can act on.
format_attributions ¶
format_attributions(
attributions: Sequence[Attribution], *, max_sites: int = 5, max_sql: int = 160
) -> str
Describe attributions as indented blocks, busiest line first.
A block and no heading, the way :func:format_n_plus_one renders a finding,
because the caller is what knows why it is printing these: the section under
a failed query-count assertion introduces them as the statements no call
path repeated, and a report face would introduce them as something else
again. A heading here would be printed under the caller's own.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
attributions
|
Sequence[Attribution]
|
The groups to describe, already ordered -- |
required |
max_sites
|
int
|
How many lines to name before summarising the rest. |
5
|
max_sql
|
int
|
Where to cut a long statement. The records keep the whole thing. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The blocks, indented, without a trailing newline. Empty for an empty |
str
|
sequence, so a caller can print a heading only when there is something |
str
|
under it. |
assert_query_growth ¶
The growth assertion: a query count keeps its shape as the data grows.
assert_query_growth ¶
assert_query_growth(
world: ScaleWorld,
block: Callable[[], object],
*,
growth: Growth = Growth.CONSTANT,
factors: Sequence[int] = DEFAULT_FACTORS,
using: str | Iterable[str] | None = None,
stack_depth: int = DEFAULT_STACK_DEPTH,
warm_up: Callable[[], object] | None = None,
) -> QueryGrowth
Assert that block's query count keeps growth as the world gets bigger.
from django_query_contract import assert_query_growth
def test_the_listing_does_not_grow(world):
assert_query_growth(world, lambda: render_author_list())
world is asked for a hundred rows and then a thousand, the block runs in
each, and the two statement counts have to be equal. If they are not, the
failure names both counts, the rule they broke and the statement that grew,
with the line it came from.
This is not a count assertion, and it must not become one.
django_assert_num_queries is the count assertion: it is typed, it
handles connection= and using= and a custom note, and it yields the
captured queries. This package ships no second one and there is deliberately
no way to spell a fixed count here -- the claim is about how a count
changes, which is why fewer than two factors is refused rather than
treated as a count of one world. The two compose: assert the count with
theirs and the shape of it with this, and a failure of either is diagnosed
by the same capture.
What is new here is the growth claim itself. Ruby has had it since
n_plus_one_control -- run the code at several scale factors, assert the
count is O(1) -- and no Python package does. It catches what a fixed
count cannot: a listing asserted at three queries against three fixture rows
is asserted at three queries against a defect that costs one query per row,
because at three rows the loop and the prefetch look the same. A growth
assertion asks the only question that separates them.
The capture is opened inside the world, never around it, which is what
makes this a function rather than a recipe. See measure_query_growth for
the measured reason -- a world's own loader emits statements that grow with
the factor, so a capture wrapped around the build reports the loader's curve
as the block's.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
world
|
ScaleWorld
|
How to make the world be a given size. |
required |
block
|
Callable[[], object]
|
What to measure. Called once per factor, with no arguments. |
required |
growth
|
Growth
|
The bound to hold the count to. |
CONSTANT
|
factors
|
Sequence[int]
|
The sizes to measure, strictly ascending, at least two. |
DEFAULT_FACTORS
|
using
|
str | Iterable[str] | None
|
Which connections to capture. Every configured one by default. |
None
|
stack_depth
|
int
|
Frames kept per statement. |
DEFAULT_STACK_DEPTH
|
warm_up
|
Callable[[], object] | None
|
Run once inside the first world before the first measurement
and not captured, for a block whose first run fills a per-process
cache. |
None
|
Returns:
| Type | Description |
|---|---|
QueryGrowth
|
The measurement, so a passing test can go on to read the curve or the |
QueryGrowth
|
captures behind it -- the same courtesy |
QueryGrowth
|
does by yielding its |
Raises:
| Type | Description |
|---|---|
AssertionError
|
If the curve broke |
ValueError
|
If |
measure_query_growth ¶
Run one block against several sizes of world and record what it cost.
measure_query_growth ¶
measure_query_growth(
world: ScaleWorld,
block: Callable[[], object],
*,
factors: Sequence[int] = DEFAULT_FACTORS,
using: str | Iterable[str] | None = None,
stack_depth: int = DEFAULT_STACK_DEPTH,
warm_up: Callable[[], object] | None = None,
) -> QueryGrowth
Run block once in each sized world and return the curve.
The measurement half of the growth assertion, and the half with no claim in
it: this says what happened, assert_query_growth says whether that was
allowed, and the two are separate so a CI report can plot a curve without
asserting anything.
from django_query_contract import measure_query_growth
measured = measure_query_growth(world, lambda: render_author_list())
print(measured.factors, measured.counts)
The capture is opened inside the world, and that is the point of this
function existing rather than a recipe. Building a world runs statements
of its own -- on any backend without COPY, which is most of them, one
insert per batch of rows -- so a capture wrapped around world(factor)
counts the loader's statements along with the block's, and those grow with
the factor. Measured against django-data-shape and reported by its own
author: a two-table world captured from outside runs 8 statements at factor
1 and 17 at factor 10 on SQLite, flat at 9 on PostgreSQL where COPY does
not pass through Django's cursor wrapper at all. A harness reading that
curve reports a confident O(N) for a block that is O(1) -- the
harness measuring its own loader and calling it the subject.
So the harness owns the capture. A caller hands over a world and a block and
never writes QueryCapture at all, which is what makes the mistake
unavailable rather than merely discouraged. There is deliberately no
parameter for passing a capture in.
Why two points and an exact comparison, rather than a fitted curve. A fit would use every measurement and give a slope and a goodness of fit, and then need three thresholds to turn those into a verdict: how near zero is flat, how linear is linear, how good a fit has to be before the answer is believed. Each is a knob, each is wrong for somebody, and a growth assertion that fails once a fortnight for reasons nobody can reproduce is deleted -- taking with it the one assertion in this package that no other Python package makes. Comparing counts instead is integer arithmetic on integers that were counted rather than estimated: it is exact, it cannot be flaky about anything except the block itself, and every part of a failure can be printed. It is cruder, and crude is the correct trade here.
What the factors mean. Factor 1 is the world the declaration describes, so the declaration should be the smallest world that still means something. A hundred rows against a thousand is the regime this is for; the two-million-row database that makes a query plan realistic is a different assertion with a different cost, and it does not vary a factor at all.
The one way this can still be flaky, and its fix. A block whose first
run populates a per-process cache -- a content-type lookup, a memoised
settings read -- emits one statement more at the factor that ran first, and
a suite where an earlier test happened to fill that cache passes while a
suite that runs this test alone fails. That is the warm_up argument, and
the usual value for it is block itself.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
world
|
ScaleWorld
|
How to make the world be a given size:
|
required |
block
|
Callable[[], object]
|
What to measure. Called once per factor, with no arguments, and its return value is ignored. |
required |
factors
|
Sequence[int]
|
The sizes to measure, strictly ascending, at least two. |
DEFAULT_FACTORS
|
using
|
str | Iterable[str] | None
|
Which connections to capture, as |
None
|
stack_depth
|
int
|
Frames kept per statement. The knob that matters most here:
a growth run captures the block once per factor, so the largest
world sets the cost and a block that is genuinely |
DEFAULT_STACK_DEPTH
|
warm_up
|
Callable[[], object] | None
|
Run once inside the first world, before the first measurement,
and not captured. For a block whose first run fills a per-process
cache. |
None
|
Returns:
| Type | Description |
|---|---|
QueryGrowth
|
The curve, one :class: |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
format_query_growth ¶
Render a growth curve as the paragraph a reader needs under a failed claim.
format_query_growth ¶
format_query_growth(
measured: QueryGrowth, growth: Growth, *, max_findings: int = 3, max_sql: int = 160
) -> str
Describe measured as a curve, and say whether it kept growth.
This is the message assert_query_growth fails with, written as a plain
function so the CI-report face can render a curve that nobody asserted on.
It renders a holding curve too, and deliberately: a growth measurement is
worth printing when it passes, and a formatter that only had words for
failure would leave the reporting face to invent its own.
A growth failure has to name the counts at each factor, or a reader cannot tell a defect from noise -- "the count grew" is the same sentence for a hundredfold N+1 and for one extra statement from a cache that filled on the first run. So the curve is a table, and under it the rule that was broken, stated in the numbers that broke it.
Under that is the capture from the higher of the two factors that failed,
rendered by format_capture_report -- the same report that appears under
a failing django_assert_num_queries, which means the N+1 that explains
the growth is described exactly as it would be anywhere else in this
package, and the reader gets the call site rather than only the arithmetic.
The claim is required and has no default, which is worth saying because
the obvious reading of "render this measurement" is that a measurement is
enough. It is not: one curve reads as a pass against
:attr:~django_query_contract.Growth.LINEAR and a failure against
:attr:~django_query_contract.Growth.CONSTANT, and the headline sentence has
to say which. Defaulting to either would put a claim nobody made into a
report, which is the whole failure mode this package is about.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
measured
|
QueryGrowth
|
The curve. |
required |
growth
|
Growth
|
The claim to judge it against. Positional, and required -- see above. |
required |
max_findings
|
int
|
How many N+1 findings to list from the failing world. |
3
|
max_sql
|
int
|
Where to cut a long statement. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The report, without a trailing newline. |
find_plan_defects ¶
Read a capture's plans back as findings.
find_plan_defects ¶
Name the defects the captured plans state, blindness first.
The whole of the reader, and it is short for the same reason
:func:~django_query_contract.find_n_plus_one is: both defects are
definitions rather than heuristics. One is a fact PostgreSQL printed; the
other is an equality and an inequality over a pair of measurements. See
:class:~django_query_contract.PlanDefect for why the other two candidates
-- a sequential scan over a row threshold and a nested loop with a large
inner -- are declined, and for why the estimate-versus-actual ratio is
reported on every node instead of being classified here.
Only measured plans are considered. Without ANALYZE a plan carries
the planner's expectations and nothing to check them against, so neither
defect is decidable from one. Those statements are skipped rather than
passed, and :func:~django_query_contract.format_query_plans says how many
were skipped rather than reporting a clean bill of health it did not earn.
Takes any iterable of records, so it reads a
:class:~django_query_contract.PlanCapture directly, a slice of one, or the
records of a single connection. A record with no plan -- every record from an
ordinary :class:~django_query_contract.QueryCapture -- contributes nothing,
so calling this on a capture that took no plans returns nothing rather than
raising.
from django_query_contract import PlanCapture, find_plan_defects
with PlanCapture() as capture:
render_author_list()
for finding in find_plan_defects(capture):
print(finding.defect, finding.count, finding.actual_rows)
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
records
|
Iterable[QueryRecord]
|
The executions to read. |
required |
Returns:
| Type | Description |
|---|---|
PlanFinding
|
The findings, blindness first and then spills, each kind in capture |
...
|
order. The order across the two kinds is presentation and not a ranking: |
tuple[PlanFinding, ...]
|
a spilled sort and a blind estimate have no common scale, and inventing |
tuple[PlanFinding, ...]
|
one to sort them by would be the first knob. |
format_query_plans ¶
Turn captured plans into the paragraph a reader needs, and no verdict.
format_query_plans ¶
format_query_plans(
capture: PlanCapture,
*,
max_findings: int = 5,
max_estimates: int = 5,
max_relations: int = 5,
max_sql: int = 160,
) -> str
Describe what the planner did, what it got wrong, and what was not asked.
Five blocks, and the order is the argument. The findings come first because they are the only claims here that hold by construction. The estimate errors come after them and are explicitly not claims: they are two numbers PostgreSQL printed and the factor between them, ordered so the largest is visible, with no rule anywhere deciding which of them is a defect. The statements that carried no plan are counted last, so a reader can see that the numbers above do not cover everything.
The relations block sits between them and the count of what carried no plan,
and it is a third register again: neither a claim nor a number to be read,
but the material an index decision is made of, with the decision itself
declined in the text. See
:class:~django_query_contract.RelationAccess for why it is declined.
The block about relations with no statistics comes before all of it, because it can invalidate all of it: a plan over a table PostgreSQL has never analyzed is a guess, and a reader who does not know that will act on it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
capture
|
PlanCapture
|
A closed :class: |
required |
max_findings
|
int
|
How many findings to list before summarising the rest. |
5
|
max_estimates
|
int
|
How many statements to name in the estimate-error block. |
5
|
max_relations
|
int
|
How many tables to describe before counting the rest. |
5
|
max_sql
|
int
|
Where to cut a long statement. The record keeps the whole thing. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The report, without a trailing newline. Empty when no statement carried a |
str
|
plan -- which is every capture that is not a |
str
|
|
format_relation_access ¶
Print the evidence an index decision needs, and refuse to make the decision.
format_relation_access ¶
format_relation_access(
capture: PlanCapture, *, max_relations: int = 5, max_sql: int = 160
) -> str
Describe which tables these plans read, how, and what already indexes them.
The fourth reading of one capture, after the N+1 findings, the call sites and
the plans themselves. What it adds is the join between a plan and an address:
"testapp_order was read end to end eleven times, filtering
((status)::text = %s::text), discarding 199,000 rows, from
views.py:112" -- every clause of which is either PostgreSQL's own output
or this package's own call-site rule, and none of which is a recommendation.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
capture
|
PlanCapture
|
A closed :class: |
required |
max_relations
|
int
|
How many tables to describe before counting the rest. |
5
|
max_sql
|
int
|
Where to cut a long predicate or index definition. The records keep the whole thing. |
160
|
Returns:
| Type | Description |
|---|---|
str
|
The report, without a trailing newline. Empty when no statement in the |
str
|
capture carried a plan that read a table -- which is every ordinary |
str
|
class: |