server agent
Python
The only agent that can tell computing from waiting. Every other one reports that your code got slower and stops there — this one says which of the two happened, and they need opposite fixes.
- package
sixty-shon PyPI- runs on
- Python 3.8 or later. No dependencies.
- source
- packages/python
Installing it
Django, Flask, FastAPI or any WSGI/ASGI app: functions, HTTP routes and SQL, plus the CPU and waiting split only Python can measure.
The install is written as a prompt for the coding agent you already have open, not as a checklist for you. That is deliberate: it names what must be true when the install is finished rather than which files to edit, because where the code goes depends on the framework and putting it in the wrong place fails silently. An agent can read your repository and work that out; a paragraph on a documentation page cannot.
The same text is what install_sixty returns through the MCP server and what the collector serves at /v1/setup?kind=python. There is one copy of it.
the Python install, in full
Install the sixty agent in this Python service so its functions, HTTP routes
and database queries report to sixty.
1. Add sixty-sh to this project's runtime dependencies — the same place the
web framework is declared (pyproject.toml, requirements.txt, Pipfile), not
a dev or test group. It runs in production; that is the entire point. It
has no dependencies of its own.
2. Call sixty.init() once, as early in process startup as you can get it, and
before any database connection is opened. Put it at the top of the module
the deployed process actually starts — wsgi.py, asgi.py, main.py, manage.py
for a management command — not inside a function that runs per request.
3. Wrap the application so requests become operations. Work out which of these
this project is; do exactly one:
a. Flask — call instrument_flask(app) from "sixty.instrument.flask" after
the app and its routes exist. It wraps app.wsgi_app and names operations
by the matched url_rule.
b. Django — put "sixty.instrument.django.SixtyMiddleware" FIRST in the
MIDDLEWARE list, so the span covers the rest of the middleware rather
than sitting inside it. Operations are named by the route as written in
urls.py.
c. FastAPI, Starlette, Litestar, Quart, or anything else ASGI — wrap with
SixtyASGIMiddleware from "sixty.instrument.asgi".
d. Any other WSGI application — wrap the WSGI callable with SixtyMiddleware
from "sixty.instrument.wsgi".
4. Mark the functions worth measuring. This step is what turns "this endpoint
got slow" into "this function started issuing 14 queries", and skipping it
leaves the feed with routes and queries and nothing in between:
- Put @sixty.trace on the functions that do the work — the service layer,
the repository, whatever this project calls the code between the view and
the database. Not on view functions the middleware already covers.
- Or, for a module of them, call sixty.instrument_module(sys.modules[__name__])
at the bottom of the file; it wraps every public function that module
defines and leaves imported ones alone.
- Leave anything called hundreds of thousands of times a second alone. A
span costs a couple of microseconds, which is nothing next to a request
and everything next to a tight inner loop.
5. Set these environment variables wherever the service is deployed:
SIXTY_API_KEY = a secret key starting sixty_sk_ — ask me for it. Do not
invent one, and do not commit it.
SIXTY_SERVICE = my-app
SIXTY_ENDPOINT = https://ingest.sixty.sh
The release identifier is picked up automatically on Vercel, Render,
Railway, Fly, Heroku and GitHub Actions. If this deploys some other way,
set SIXTY_RELEASE to the commit SHA — without one, every measurement lands
in a single nameless bucket and no comparison can ever be made.
Constraints — correctness requirements, not style preferences:
- Do NOT change any application behaviour. This is instrumentation only: no
refactors, no reordering of business logic, no "while I was in here" fixes.
- Do NOT call init() at import time in a module that is also imported by test
collection or by a build step. Without a key it is inert, but a flush thread
started in a test runner is a surprise nobody asked for.
- Do NOT wrap generators or async generators with @sixty.trace. Their work
happens between next() calls, so the measurement would be of constructing an
object. Wrap whatever drains them.
- Do NOT add any analytics, user id, session id, or cookie to what is
reported. The agent is deliberately anonymous and must stay that way.
- Queries are instrumented through psycopg (2 and 3) automatically. If this
project talks to its database some other way, tell me rather than wiring
something up — measuring it may need work in the agent.
If this service runs under gunicorn, uwsgi or any pre-fork server, note how
many workers it runs: each one reports independently and the collector merges
them, which is correct, but it is worth knowing when you read the numbers.
When you are done, tell me which files you changed and what the deployed start
command now is, so I can confirm data is arriving.It needs a secret key — it starts sixty_sk_ and stays server-side. Mint one on the Settings page once you have signed in.
What it measures
| signal | unit | what it means |
|---|---|---|
rows | rows per call | this query returns more rows than it used to |
fanout | queries per call | this operation now issues more database calls per invocation — an N+1 |
latency | ms per call | this operation takes longer end to end than it used to |
self_latency | ms per call | the time spent in this function itself got longer — its children did not |
payload | bytes per call | the serialized result of this operation got bigger |
errors | error rate | a larger fraction of calls are throwing |
runaway | calls per minute | this operation is being called far more often than anything triggers it |
repeated_query | times per request | the identical query runs several times within one request |
overfetch | rows per call | far more rows are fetched than the code appears to use |
unbounded | rows per call | this query has no upper bound on what it can return |
recursion | levels deep | this operation calls itself, deeper than it should |
new_error | occurrences | an error that did not occur in the previous release |
missing_tenancy | — | This reads a table of per-person data without saying whose rows it wants. Unless your database is filtering it for you, everyone gets everyone else's. |
collapse | — | This is handing back roughly half the data it used to, or less. If that was not deliberate, something is filtering out rows that somebody expects to see. |
vanished | — | It was being used steadily until this release and has not been used once since. Usually the link, button, or redirect that led here stopped working. |
traffic_drop | — | This is still being used, but a fraction as often, and its share of your traffic fell too — so it is not just a quiet period. |
cpu | ms of CPU per call | this function burns more processor time per call than it used to — it is doing more work, not waiting longer |
blocked | ms of waiting per call | this operation spends longer waiting for its turn while doing exactly the same amount of work |
Where it hooks in
- Flask — instrument_flask(app) — operations are named by the matched url_rule.
- Django — SixtyMiddleware, first in MIDDLEWARE — named by the route as written in urls.py.
- FastAPI, Starlette, Litestar, Quart — SixtyASGIMiddleware, or any other ASGI app.
- Anything WSGI — Pyramid, Bottle, wsgiref, a framework nobody has written yet.
- Your own functions — @sixty.trace on the service layer, or instrument_module() for a whole module at once.
Databases
- psycopg — Versions 2 and 3, patched at the cursor. Rows, statement shape and attribution to the calling function.
What only this one does
- CPU per call — time.thread_time() is per-thread and costs about 100ns to read, and a synchronous span owns its thread for its whole duration — so the number is exact rather than apportioned.
- Waiting per call — Own time that was not computing: a lock held across more work, a pool with no free slot, a C extension holding the GIL. Every other per-call number stays correct while this happens, which is why nothing else catches it.
What it cannot do
- A span that sits across an await shares its thread with whatever else the loop ran, so it reports no CPU rather than an inflated one. An asyncio service gets CPU per synchronous function and per query, but not per request.
- Query plans are not captured. Postgres is there and the EXPLAIN would work; the agent does not issue one yet.
- psycopg is the only driver instrumented. SQLAlchemy on psycopg is measured, because the cursor underneath it is; asyncpg and MySQL drivers are not.
- Under gunicorn or uwsgi each worker reports independently and the collector merges them. That is correct, and worth knowing when you read a per-process number.
Configuration
Every agent reads the same four variables, and DRIFT_* still answers everywhere SIXTY_* does — the product was renamed and that name is not ours to retire from other people’s deployments.
SIXTY_API_KEY | Without it the agent stays inert and says so. It never guesses, never retries against an unknown endpoint, and never throws. |
|---|---|
SIXTY_SERVICE | What to call this service. Defaults to the project name where one is legible. |
SIXTY_RELEASE | The one that matters most. Picked up automatically on Vercel, Render, Railway, Fly, Heroku and GitHub Actions; set it to the commit SHA anywhere else. Without it every measurement lands in a single nameless bucket and no comparison can ever be made. |
SIXTY_ENDPOINT | Where to report. Defaults to http://localhost:4319, which is right on a laptop and wrong the moment the app is served to anyone else. |
The rest — flush interval, sample rate, what to instrument — is in the package’s own README, which is where it can stay true as the agent changes.