sixty

server agent

Python

The only agent that can tell computing from waiting. Every other one reports that your code got slower and stops there — this one says which of the two happened, and they need opposite fixes.

package
sixty-sh on PyPI
runs on
Python 3.8 or later. No dependencies.
source
packages/python

Installing it

Django, Flask, FastAPI or any WSGI/ASGI app: functions, HTTP routes and SQL, plus the CPU and waiting split only Python can measure.

The install is written as a prompt for the coding agent you already have open, not as a checklist for you. That is deliberate: it names what must be true when the install is finished rather than which files to edit, because where the code goes depends on the framework and putting it in the wrong place fails silently. An agent can read your repository and work that out; a paragraph on a documentation page cannot.

The same text is what install_sixty returns through the MCP server and what the collector serves at /v1/setup?kind=python. There is one copy of it.

the Python install, in full
Install the sixty agent in this Python service so its functions, HTTP routes
and database queries report to sixty.

1. Add sixty-sh to this project's runtime dependencies — the same place the
   web framework is declared (pyproject.toml, requirements.txt, Pipfile), not
   a dev or test group. It runs in production; that is the entire point. It
   has no dependencies of its own.

2. Call sixty.init() once, as early in process startup as you can get it, and
   before any database connection is opened. Put it at the top of the module
   the deployed process actually starts — wsgi.py, asgi.py, main.py, manage.py
   for a management command — not inside a function that runs per request.

3. Wrap the application so requests become operations. Work out which of these
   this project is; do exactly one:

   a. Flask — call instrument_flask(app) from "sixty.instrument.flask" after
      the app and its routes exist. It wraps app.wsgi_app and names operations
      by the matched url_rule.

   b. Django — put "sixty.instrument.django.SixtyMiddleware" FIRST in the
      MIDDLEWARE list, so the span covers the rest of the middleware rather
      than sitting inside it. Operations are named by the route as written in
      urls.py.

   c. FastAPI, Starlette, Litestar, Quart, or anything else ASGI — wrap with
      SixtyASGIMiddleware from "sixty.instrument.asgi".

   d. Any other WSGI application — wrap the WSGI callable with SixtyMiddleware
      from "sixty.instrument.wsgi".

4. Mark the functions worth measuring. This step is what turns "this endpoint
   got slow" into "this function started issuing 14 queries", and skipping it
   leaves the feed with routes and queries and nothing in between:

   - Put @sixty.trace on the functions that do the work — the service layer,
     the repository, whatever this project calls the code between the view and
     the database. Not on view functions the middleware already covers.
   - Or, for a module of them, call sixty.instrument_module(sys.modules[__name__])
     at the bottom of the file; it wraps every public function that module
     defines and leaves imported ones alone.
   - Leave anything called hundreds of thousands of times a second alone. A
     span costs a couple of microseconds, which is nothing next to a request
     and everything next to a tight inner loop.

5. Set these environment variables wherever the service is deployed:
      SIXTY_API_KEY  = a secret key starting sixty_sk_ — ask me for it. Do not
                       invent one, and do not commit it.
      SIXTY_SERVICE  = my-app
      SIXTY_ENDPOINT = https://ingest.sixty.sh

   The release identifier is picked up automatically on Vercel, Render,
   Railway, Fly, Heroku and GitHub Actions. If this deploys some other way,
   set SIXTY_RELEASE to the commit SHA — without one, every measurement lands
   in a single nameless bucket and no comparison can ever be made.

Constraints — correctness requirements, not style preferences:

- Do NOT change any application behaviour. This is instrumentation only: no
  refactors, no reordering of business logic, no "while I was in here" fixes.
- Do NOT call init() at import time in a module that is also imported by test
  collection or by a build step. Without a key it is inert, but a flush thread
  started in a test runner is a surprise nobody asked for.
- Do NOT wrap generators or async generators with @sixty.trace. Their work
  happens between next() calls, so the measurement would be of constructing an
  object. Wrap whatever drains them.
- Do NOT add any analytics, user id, session id, or cookie to what is
  reported. The agent is deliberately anonymous and must stay that way.
- Queries are instrumented through psycopg (2 and 3) automatically. If this
  project talks to its database some other way, tell me rather than wiring
  something up — measuring it may need work in the agent.

If this service runs under gunicorn, uwsgi or any pre-fork server, note how
many workers it runs: each one reports independently and the collector merges
them, which is correct, but it is worth knowing when you read the numbers.

When you are done, tell me which files you changed and what the deployed start
command now is, so I can confirm data is arriving.

It needs a secret key — it starts sixty_sk_ and stays server-side. Mint one on the Settings page once you have signed in.

What it measures

signalunitwhat it means
rowsrows per callthis query returns more rows than it used to
fanoutqueries per callthis operation now issues more database calls per invocation — an N+1
latencyms per callthis operation takes longer end to end than it used to
self_latencyms per callthe time spent in this function itself got longer — its children did not
payloadbytes per callthe serialized result of this operation got bigger
errorserror ratea larger fraction of calls are throwing
runawaycalls per minutethis operation is being called far more often than anything triggers it
repeated_querytimes per requestthe identical query runs several times within one request
overfetchrows per callfar more rows are fetched than the code appears to use
unboundedrows per callthis query has no upper bound on what it can return
recursionlevels deepthis operation calls itself, deeper than it should
new_erroroccurrencesan error that did not occur in the previous release
missing_tenancyThis reads a table of per-person data without saying whose rows it wants. Unless your database is filtering it for you, everyone gets everyone else's.
collapseThis is handing back roughly half the data it used to, or less. If that was not deliberate, something is filtering out rows that somebody expects to see.
vanishedIt was being used steadily until this release and has not been used once since. Usually the link, button, or redirect that led here stopped working.
traffic_dropThis is still being used, but a fraction as often, and its share of your traffic fell too — so it is not just a quiet period.
cpums of CPU per callthis function burns more processor time per call than it used to — it is doing more work, not waiting longer
blockedms of waiting per callthis operation spends longer waiting for its turn while doing exactly the same amount of work

Where it hooks in

  • Flaskinstrument_flask(app) — operations are named by the matched url_rule.
  • DjangoSixtyMiddleware, first in MIDDLEWARE — named by the route as written in urls.py.
  • FastAPI, Starlette, Litestar, QuartSixtyASGIMiddleware, or any other ASGI app.
  • Anything WSGIPyramid, Bottle, wsgiref, a framework nobody has written yet.
  • Your own functions@sixty.trace on the service layer, or instrument_module() for a whole module at once.

Databases

  • psycopgVersions 2 and 3, patched at the cursor. Rows, statement shape and attribution to the calling function.

What only this one does

  • CPU per calltime.thread_time() is per-thread and costs about 100ns to read, and a synchronous span owns its thread for its whole duration — so the number is exact rather than apportioned.
  • Waiting per callOwn time that was not computing: a lock held across more work, a pool with no free slot, a C extension holding the GIL. Every other per-call number stays correct while this happens, which is why nothing else catches it.

What it cannot do

  • A span that sits across an await shares its thread with whatever else the loop ran, so it reports no CPU rather than an inflated one. An asyncio service gets CPU per synchronous function and per query, but not per request.
  • Query plans are not captured. Postgres is there and the EXPLAIN would work; the agent does not issue one yet.
  • psycopg is the only driver instrumented. SQLAlchemy on psycopg is measured, because the cursor underneath it is; asyncpg and MySQL drivers are not.
  • Under gunicorn or uwsgi each worker reports independently and the collector merges them. That is correct, and worth knowing when you read a per-process number.

Configuration

Every agent reads the same four variables, and DRIFT_* still answers everywhere SIXTY_* does — the product was renamed and that name is not ours to retire from other people’s deployments.

SIXTY_API_KEYWithout it the agent stays inert and says so. It never guesses, never retries against an unknown endpoint, and never throws.
SIXTY_SERVICEWhat to call this service. Defaults to the project name where one is legible.
SIXTY_RELEASEThe one that matters most. Picked up automatically on Vercel, Render, Railway, Fly, Heroku and GitHub Actions; set it to the commit SHA anywhere else. Without it every measurement lands in a single nameless bucket and no comparison can ever be made.
SIXTY_ENDPOINTWhere to report. Defaults to http://localhost:4319, which is right on a laptop and wrong the moment the app is served to anyone else.

The rest — flush interval, sample rate, what to instrument — is in the package’s own README, which is where it can stay true as the agent changes.

The sixty Python agent — what it measures and how to install it