Design decisions¶
1. One binary, not a services split¶
Context. The obvious "enterprise" shape for a system with distinct domains (incidents, investigations, corrective actions, notifications, reporting) is to give each its own service.
Options considered. Split by domain into separate deployables behind an API gateway; or keep one deployable with domain boundaries enforced in code.
Choice. One deployable, domain boundaries as interfaces (interfaces/<domain>.go), not network boundaries.
Trade-off accepted. Gave up independent deployability and independent scaling per domain. Kept transactional consistency across domains that are tightly coupled anyway (an incident, its investigation, and its corrective actions are effectively one write unit), a single deploy pipeline, and no distributed-systems failure modes (partial deploys, network partitions between services) to design around.
Outcome. Held up. Nothing about current load or team structure has demanded independent scaling of one domain over another. See architecture.md for what would need to change before a split made sense.
2. Shared schema with a tenant_id column, not schema-per-tenant¶
Context. The system needed to support multiple organizations without cross-visibility, added after the system was already running single-tenant.
Options considered. Database-per-tenant; schema-per-tenant; shared schema with a discriminator column plus row-level security.
Choice. Shared schema, discriminator column, expand-and-contract rollout (nullable → backfilled → NOT NULL → scoped reads → planned RLS).
Trade-off accepted. Isolation depends on every query remembering to filter by tenant, until row-level security is switched on as a backstop. In exchange: no per-tenant migration fan-out, one connection pool, and a rollout that could be shipped in reversible stages against a live single-tenant system instead of a big-bang cutover.
Outcome. Working, but not finished — the read-scoping stage is still in progress and RLS isn't on yet. See multi-tenancy.md for exactly what's covered today versus planned.
3. Subdomain-based tenant resolution¶
Context. A request needs to know which tenant it belongs to before anything else can be scoped correctly — including, critically, password reset, where "enter your email" alone can't disambiguate which org's account to reset if two tenants share an email.
Options considered. A tenant identifier in a request header or query param; resolve tenant purely from the authenticated user's own record (works only for already-logged-in requests); resolve from subdomain.
Choice. Subdomain (acme.platform.example), with the JWT's own tenant claim cross-checked against it on every authenticated request, and a fallback to one default tenant while real subdomain DNS isn't yet in front of the deployment.
Trade-off accepted. Requires wildcard DNS and TLS in production, and CORS has to validate against a dynamic tenant registry instead of a static allowlist. In exchange, unauthenticated flows like password reset and self-service signup get a tenant for free, and a stolen token can't be replayed against a different tenant's subdomain.
Outcome. Held up so far; the fallback-to-default-tenant behavior is intentionally kept until real subdomain infrastructure exists, so today's single-host access keeps working unchanged.
4. Fire-and-forget goroutines for notifications — didn't hold up¶
Context. Sending an email shouldn't block the HTTP response for the action that triggered it (assigning an incident, scheduling an interview, and so on).
Options considered. Send the email synchronously in the request (simple, but ties response time to SMTP latency); a detached goroutine per notification; a durable queue with a worker pulling from it.
Choice, at the time. A detached goroutine per notification (go func() { ... }()), with any error just logged.
Trade-off accepted, and where it broke down. This looked free — no queue infrastructure, no worker process, immediate implementation. It wasn't free: there is no retry, no durability across a process restart, and no visibility when a send fails. A notification "sent" during a deploy or a transient SMTP hiccup is just gone, and nothing anywhere reflects that it didn't go out.
Outcome. Didn't work out. This is the clearest case in the whole system of a design that was fine at the very first pass and doesn't hold up under real operating conditions. See alerting.md for the mechanism and architecture.md for the durable-outbox alternative that should replace it.
5. Sequential reference numbers via a MAX+1 query hook — didn't hold up¶
Context. Incidents, hazards, and safety observations each need a human-readable sequential reference number (INC00001, HAZ00001, ...).
Options considered. A database sequence per record type; a MAX(reference_number) + 1 query inside the record's own pre-create hook; a dedicated per-tenant counter table.
Choice, at the time. The MAX + 1 query, run inside each model's BeforeCreate hook.
Trade-off accepted, and where it broke down. Simple, no extra table, worked fine under low concurrency and single-tenant. Two problems surfaced once tenancy was added: the query scans for the global max, not scoped to the current tenant, which both means it's slower than it needs to be as the table grows and — worse — it lets one tenant infer roughly how many records another tenant has created, the exact kind of cross-tenant inference the tenancy design otherwise goes out of its way to prevent. It's also a plain race under concurrent inserts: two requests can read the same max before either has committed.
Outcome. Recognized as a problem; a tenant_sequences table was built and is seeded per tenant during onboarding specifically to replace this — but the three hooks that generate reference numbers haven't been switched over to use it yet. Currently the least-finished piece of tenancy work in the system. See architecture.md, "what I'd change now."
6. SQL migrations ahead of GORM auto-migrate, not instead of it¶
Context. The tenancy rollout needed migration steps GORM's auto-migration can't express safely: backfilling a column across live data, building indexes without locking the table, promoting a column to NOT NULL only after every row is populated, dropping a now-redundant constraint.
Options considered. Do everything through GORM struct tags and auto-migrate; hand-write every migration and drop auto-migrate entirely; run both, SQL first.
Choice. Numbered, hand-written SQL migrations run first, with GORM's auto-migrate still running afterward for anything simpler.
Trade-off accepted. Two migration mechanisms to reason about instead of one. In exchange, the tenancy rollout could use exactly the primitives it needed (CREATE INDEX CONCURRENTLY, staged NOT NULL promotion, NOT VALID then VALIDATE CONSTRAINT to avoid long locks) without fighting an ORM that doesn't natively support running those safely against a live table.
Outcome. Held up. The tenancy migration was carried out in nine reversible stages against a database that stayed live the whole time.
7. Drop-on-full for the analytics logging queue¶
Context. Every request optionally gets logged into an analytics table, through a bounded in-memory queue with a small worker pool.
Options considered. Block the request if the queue is full; grow the queue unboundedly; drop the new entry and move on.
Choice. Drop the entry, protect request latency.
Trade-off accepted. Under sustained load the system silently loses some analytics rows. In exchange, a slow or backed-up analytics write path can never turn into a slow API — the trade is explicit and, for data whose purpose is aggregate usage reporting rather than a system of record, an easy one to accept.
Outcome. Held up, and correctly scoped — the same drop-on-full pattern would be the wrong choice for the alerting path (decision 4), where a lost entry is not an acceptable cost. The system doesn't apply it there, which is the right call, just made inconsistent by the fact that alerting has no queue at all rather than a differently-tuned one.