Skip to content

Architecture

Dirigent is one uv workspace of nineteen packages. This page says what each is for, what may depend on what, and where a new thing goes -- so the answer is a rule rather than a guess.

The tree

dirigent-common          value types and shared schemas                 -> nothing
dirigent-plugin          the block contract                             -> common
dirigent-client          wire schemas and the SDK                       -> common
dirigent-core            engine, scheduler, triggers, alerting          -> common, plugin, client
dirigent-server          the API and auth                               -> common, plugin, client, core, examples
dirigent-cli             the commands                                   -> common, plugin, client, core, server,
                                                                           blocks, block-execute, examples
dirigent-examples        the example corpus                             -> common, plugin, client
dirigent-block-base      log, report, validate, value, time, convert,
                         pipeline.run, and the log alert channel        -> common, plugin
dirigent-block-http      http.request, http.ready, webhook.post         -> common, plugin
dirigent-block-storage   copy, read, write, exists                      -> common, plugin
dirigent-block-execute   shell, docker, compose, build, checkout        -> common, plugin, block-http
dirigent-block-sql       sql.query, sql.execute                         -> common, plugin
dirigent-block-duckdb    the duckdb engine of the sql family            -> common, plugin, block-sql
dirigent-block-jq        transform.jq, map.jq, filter.jq                -> common, plugin
dirigent-block-queues    kafka and rabbitmq                             -> common, plugin
dirigent-block-parquet   convert.arrow, on pyarrow                      -> common, plugin
dirigent-blocks          the umbrella, and the alert channels           -> common, plugin, block-base,
                                                                           block-execute, block-http, block-jq,
                                                                           block-queues, block-sql, block-storage
dirigent-storage-s3      an adapter pack, and the shape of others       -> common, plugin
dirigent-testing         doubles and fixtures for testing a block       -> common, plugin

Two block packages are outside the umbrella and installed by name: dirigent-block-parquet, whose pyarrow outweighs the other eight together, and dirigent-block-duckdb, which registers on the sql family's own entry-point group rather than on the plugin group.

Edges point down and never back up. packages/dirigent-common/tests/test_dependency_tree.py asserts every one of them, from the metadata and from what the source actually imports -- because a package can declare the right thing and still reach past it. A line above is the most a package may depend on; several depend on less.

Where a new thing goes

What you are adding Where
A value a document writes and the engine reads -- a duration, a size, an entity code common
A schema more than one package must agree on -- a connection's shape, a health report common
Something a block author implements or is handed -- Operator, StepContext, a spec plugin
A request or response body, or a method on the SDK client
Anything that decides what runs, when, or what happened core
An HTTP route, or how a request is authenticated server
A command, or how one is rendered cli
A block anybody would want the dirigent-block-* family it belongs to
An engine or codec a family runs programs through -- duckdb, jq, arrow its own dirigent-block-* package
A block or backend for one external system an adapter pack
A double or fixture a block author writes tests against testing

The full naming taxonomy -- what dirigent-block-*, dirigent-storage-*, dirigent-notify-* and dirigent-<system> each mean -- is in the design document.

The question that decides it is who else needs this. A thing two packages need belongs below both, and common is what makes that possible: without something beneath plugin, a value type the SDK and the engine both need would have to live behind the block contract, and dirigent-client would depend on Operator to spell a duration.

Why the client does not depend on the plugin

An SDK talks to an API. It needs to say 30s and to address a pipeline by code, and it needs neither Operator nor Sensor nor StepContext. The value types it does need are in common, and the test asserts the edge to plugin stays absent.

Why an adapter pack does not depend on the standard library

HttpConnectionConfig and build_client live in common. An adapter pack for some external system needs the same base URL, auth, TLS and timeout fields that http.request uses, and a client that honours them -- and if the only way to get either were to depend on the standard library, a pack would either take all of it as a dependency or redefine the fields. Four packs doing the latter is four slightly different HTTP connections, which the UI renders as four slightly different forms, and four different ideas about whether a timeout applies to the connect or the read.

dirigent-block-http still owns the kind that registers it: the configuration and the client are shared, the registration is that family's. A pack that wants the kind itself depends on that one family rather than on everything.

What common deliberately does not hold

The helpers several packs would plausibly share -- the submit-probe-fetch shape, error classification from a response, remote status mapping, paging -- are not in common, and neither are status_class, is_success and ErrorClass. The rule is duplication first: a helper moves down when a second adapter pack repeats it, not when somebody guesses that one might. Build a pack watching for what the next one would copy, and lift that when it appears.