TFD Backoffice Runtime Architecture

Modular monolith, durable data, realtime operations, and private live/on-demand media pipelines

TFD Backoffice Runtime Architecture Modular monolith, durable data, realtime operations, and private live/on-demand media pipelines TFD Staff · Backoffice users · Architecture component TFD Staff Backoffice users AWS Cognito · Hosted UI + PKCE · Architecture component · Authentication AWS Cognito Hosted UI + PKCE Authentication Backoffice Frontend · React attendance + operations UI · Architecture component Backoffice Frontend React attendance + operations UI Live Analysis Capture · Canvas + MediaRecorder + microphone · Architecture component · Delivery Live Analysis Capture Canvas + MediaRecorder + microphone Delivery Business + Media Providers · AddPipe · Modal · Practice Better · CRM · Architecture component Business + Media Providers AddPipe · Modal · Practice Better · CRM Backoffice API + Cron · NestJS modular monolith processes · Architecture component · Business source of truth Backoffice API + Cron NestJS modular monolith processes Business source of truth PostgreSQL · State + Audit schema · Architecture component PostgreSQL State + Audit schema Redis · Streams · state · Pub/Sub · dead letters · Architecture component · Control + projection plane Redis Streams · state · Pub/Sub · dead letters Control + projection plane Zoom · Webhooks + Meeting SDK · Architecture component Zoom Webhooks + Meeting SDK Meeting Recorder · Sharded media capture · Architecture component Meeting Recorder Sharded media capture Meeting Streamer · Isolated playback share · Architecture component Meeting Streamer Isolated playback share Private S3 · Segments + normalized/final MP4 · Architecture component · Media plane Private S3 Segments + normalized/final MP4 Media plane Recording Finalizer · Visual remux + normalize + validate · Architecture component Recording Finalizer Visual remux + normalize + validate Analysis Renderer · Standalone Node + FFmpeg worker · Architecture component · No database access Analysis Renderer Standalone Node + FFmpeg worker No database access HTTPS PKCE HTTPS + Socket.IO start · pause · resume presigned Studio clips JWT identity Prisma adapters · signed webhooks signed webhooks AddPipe + YouTube media OAuth + meeting OBF media jobs v1 video-only MP4 + thumbnail stored clips · verified MP4 request ↔ result v1 Legend External Security Frontend Backend Database Message bus Cloud

Modular monolith

  • • All About You larger date ranges calculate the rate from matched confirmed VIP and attendance sums. The overview adds conversion_showed as the explicit eligible denominator; overall Scheduled/Showed retain their selected-period scope. Partial results explain excluded records. This read-time update uses existing Gold and requires no import or worker run.
  • • The same acquisition import selector also supports Consultations, dataset consultations and mapping consultations.v1. Integrations reads bounded unformatted A:H from yearly Consultations tabs; Sales publishes daily Gold in the existing transaction. The overview serves nullable month/week/day scheduled and net VIP metrics, source/quality metadata and explicit unsupported attribution. Monthly source precedence is established before date filters; no new worker or direct browser spreadsheet access.
  • • Identity, VIP/CRM, Sales, Marketing, Delivery, Engagement, Notifications, and Integrations remain explicit modules
  • • The acquisition import page selects Registration or All About You before Start import. Authenticated preview accepts dataset registration or all_about_you; source-specific review/publication and retry preserve that choice. All About You reads bounded unformatted A:I ranges from yearly tabs and corrects dates to the worksheet year. The existing Airtable conversion also projects canonical All About You Gold; no new worker. The card serves nullable metrics, source freshness and quality from saved Gold with monthly spreadsheet precedence and explicit unsupported-filter states.
  • • PostgreSQL owns durable business, audit, and long-running operation progress state
  • • Authorized Socket.IO snapshots are published only after durable progress changes
  • • Notification inbox writes publish schema-versioned invalidations through Redis Pub/Sub to the authenticated staff recipient room; HTTP remains the recovery path
  • • Role-aware notification preferences gate each eligible future inbox write before realtime publication
  • • Delivery, recorder, Engagement, and scheduled jobs emit non-blocking in-process facts; Notifications projects the assigned VIT StaffUser or active admins asynchronously
  • • Persisted inbox mutations emit a committed fact; the asynchronous realtime bridge handles local Socket.IO and Redis Pub/Sub fan-out
  • • Notification source facts are best effort across process termination; a future transactional outbox is required for guaranteed recovery
  • • Attendance calls recorder commands over authenticated HTTP and receives bot, participant, and capacity projections over Socket.IO
  • • Bot Management is the only editing surface for Delivery-owned ProgramRun Meeting ID assignments and optional configured expected session time; matching occurrences receive the IDs
  • • Its authorized dashboard projects monitored meeting status, duration anchors, online attendance participants, live camera counts, and active breakout-room participant counts from signed Zoom webhooks over Socket.IO
  • • A dedicated clinical-data import process runs one Practice Better batch every five minutes under Railway scheduling, independently of VIP and break jobs. The API queues scans and SQL checkpoints resume interrupted imports
  • • The five-minute clinical process handles explicit source-copy jobs as well as downloads; Clinical owns durable source versions and Integrations owns resumable checkpoints
  • • The Clinical frontend calls the Identity HTTP composition layer and ClinicalWorkspaceService inside the API. Body observations and summaries save synchronously with revisions; no new worker or realtime PHI is introduced
  • • Clinical exposes an explicitly authorized, VIP-scoped YouTube catalog from current copied note revisions. The frontend opens an official privacy-enhanced YouTube iframe only on staff selection; only a validated video id and origin referrer cross to YouTube, never VIP identifiers or note content. Existing recording permissions remain separate
  • • Sales Historical imports uses authorized HTTP history and review snapshots. The mounted staff page advances bounded Airtable capture pages after explicit start/resume; closing it leaves durable checkpoints. Acquisition remains a manual approved spreadsheet capture, with no new worker or provider secrets in the browser
  • • The existing complete Stripe import also captures Membership Silver and Gold atomically in Sales. Page-limited imports do not project; same-day identical captures are no-ops and corrections append revisions. Authorized /sales/overview/memberships reads saved aggregate Gold independently of Airtable and FBF; no new worker or schedule.
  • • Sales Overview shares inclusive UTC date bounds across cards. Membership keeps immutable snapshot and interval comparison rules. FBF1-FBF4 remain repeatable programs: reviewed worksheet names associate them with monthly editions without an extra mapping tab. The card labels registration, ad and affiliate figures as shared edition totals; monthly overlap selects whole reporting periods and no program delta is inferred. Confirmed single-program campaigns retain actual dates and same-program comparisons. A bounded organization-scoped Gold snapshot supplies saved reads. Unsupported attribution remains unavailable; context ownership and publication review are unchanged.
  • • Delivery owns revision-protected SessionOccurrence analysis roster exceptions. Program Run stores per-time VIT priorities, limits and overflow settings. Experience owners are allocated first, then ascending priority with balanced session counts, then overflow. Staff may change session-only VIT priorities, limits and membership, then recalculate suggestions or explicitly redistribute unstarted requests; active/completed work and projects/feedback are preserved. Assignment-change facts publish after commit.
  • • Integrations resolves recording VIP attendance links and phase through Delivery RecordingAttendanceContextService. Source, local recording date, phase and canonical VIP-name search filter the catalog before pagination; media remains private.
  • • Clinical versioned summaries own VIP Goals & Feedback Preference and Physical Limitations & Important Conditions. Clinical Records edits them; Studio tabs read the existing authorized Clinical HTTP surface without copying sensitive content to Delivery compositions.
  • • Integrations owns revision-aware session playback HTTP controls, immutable command-intent outbox snapshots and transactional control audit. Recovery serializes with mutations, retains pending commands until a live streamer lease exists, and validates stop events against the locked revision and fencing token. Passwords remain TTL-only Redis secrets; dead letters contain allowlisted envelope metadata. The separate Meeting Streamer runtime still requires the live provider spike.
  • • Read-only impersonation (ADR 0070) wraps the original Cognito access JWT with a target StaffUser selector for backend HTTP only. Identity verifies both users on every request and rejects writes. The browser preserves this mode through token refresh, shows a persistent exit banner, discards caches on entry/exit, and disables Socket.IO/chat and external OAuth actions. AuditEntry stores nullable impersonatorStaffId; no new runtime, secret, or independent login session is introduced.

Recorder control plane

  • • Redis Streams carries schema-versioned commands with short-lived meeting OBF authorization
  • • Integrations trims recorder events in atomic bounded batches after at least 24 hours, preserving unread and pending entries for every consumer group; both API and finalizer groups must exist
  • • Leases, TTL renewal, and fencing tokens protect recorder ownership
  • • Recorders retain fenced terminal ownership until Backoffice accepts the event and performs shard and lease cleanup
  • • Accepted failures allocate a replacement while intent is active or finalize after the last stopping shard
  • • Finalizer concurrency uses continuously replenished independent slots; one long participant remux cannot hold an otherwise available slot
  • • Backoffice persists participant capture-start evidence in PostgreSQL before registering the capture in Redis; Attendance distinguishes captured media still processing from participants with no capture evidence
  • • Projection events that fail five times move to a dead-letter stream only after the copy succeeds
  • • Connection failures remain durable timeline evidence; a higher desired revision may start a new bot attempt without erasing the prior failure
  • • An active desired assignment may recover a premature ended projection only when recorder instance, fencing token, and revision remain authoritative
  • • Unexpected shard exits enter reconnecting while desired state is active; one shard ending cannot terminate a meeting that still owns other shards
  • • A bot moved into a breakout room stops main-room capture, leaves the Zoom meeting, and reconnects to the configured main room while the SessionOccurrence intent remains active; it never records inside a breakout room
  • • Repeated commands for the same fenced shard reconcile targets in place; only its current assignment may fail or end that shard
  • • Zoom meeting start plus any participant join triggers one idempotent ALL-mode request after authorized StaffUser host validation; OBF prefers owner then co-host
  • • Recorder monitoring and automatic attendance share one resolution: Meeting ID identifies the Program Run, then Zoom start time selects the occurrence inside that run
  • • Authorized historical linking may explicitly select an attendance-enabled occurrence when its Zoom meeting ID is configured on the Program Run; host and time-window matching are bypassed. It persists ProgramSessionZoomLink evidence, then idempotently reprocesses participant-to-VIP attendance matching

Private media plane

  • • Media bytes never pass through Redis or the Backoffice API
  • • Recorder, finalizer, Session Analysis browser capture, and the Delivery renderer exchange media directly with private S3; movement lab browser mode stays local; explicit cloud mode uploads a private clip
  • • Zoom Live finalization downloads only visual HLS/fMP4 intermediates, remuxes without re-encoding, and stores hasAudio=false; AddPipe and Studio audio flows remain independent
  • • AddPipe webhooks persist inbox metadata and enqueue a versioned storage request; the finalizer downloads only bounded allowlisted HTTPS sources
  • • The AddPipe copy is H.264 320x180 at 15 fps with yuv420p, AAC 128 kbit/s, and faststart; Delivery availability waits for confirmed private storage
  • • Session Analysis resolves AddPipe stable video ids through authenticated private playback and never falls back to the provider URL
  • • Participant-to-VIP confirmation persists every stable video id on Delivery attendance and enriches Integrations recording metadata
  • • A stored participant-video fact immediately attempts idempotent attendance and Session Analysis reconciliation; confirmation-time and queue-read reconciliation recover either event order
  • • Session Analysis requests reuse all attendance video ids whether the request is created before or after finalization; the singular id remains a compatibility primary source
  • • Live Recording stores every Studio clip before a project version can queue one idempotent durable render job; Backoffice PostgreSQL alone owns progress, retry, cancellation, and recovery
  • • An opaque fingerprint of clip order, cuts, assets, duration, and microphone gain blocks duplicate completed renders until a saved clip changes
  • • Authorized staff download a completed MP4 from the Studio or project card through a short-lived attachment URL; private storage keys remain server-only
  • • Delivery publishes an authorized per-staff Socket.IO render snapshot only after persisting progress; reconnect reads PostgreSQL and disconnected clients fall back to HTTP polling
  • • The separately scaled Delivery FFmpeg worker has no database access; it consumes immutable render requests, applies order, cuts, and microphone gain, stores a private MP4, and returns result facts through Redis
  • • Source-video additions and ordered replacements cross the Integrations recording boundary, retain at least one source, and persist through Delivery optimistic locking
  • • The movement lab captures a staff-selected browser tab through explicit getDisplayMedia consent and MediaRecorder, then previews and detects movement locally under five-minute/100 MiB limits. Single-view 3D refinement runs MotionBERT-Lite through ONNX Runtime Web WASM and bounded sequence optimization in a cancellable browser worker. Versioned model shards are served as frontend static assets; captured, estimated and refined poses remain separate in browser memory. Browser-mode capture never enters the API, Redis or S3. Legacy staff-owned YouTube import routes and finalizer cleanup remain available but are not called by the lab UI.

Trust boundaries

  • • Cognito authenticates; StaffUser granular permissions authorize operational routes and matching UI actions, while direct role checks remain only for identity-bound or ownership rules
  • • ActiveCampaign credentials remain backend-only while Marketing performs reviewed tag reads and replacements
  • • OAuth connections are stored per Zoom user, allowing multiple staff in one Zoom account; OBF prefers the owner before eligible present co-hosts
  • • AddPipe redirects, hosts, sizes, durations, and processing time are bounded before media enters the private platform store

Audit P0 and P1 coverage

  • • Identity, Clinical, Sales, Delivery and Integrations retain their state; Audit receives bounded facts through the synchronous public contract
  • • Staff/RBAC, Clinical and attendance/evidence changes commit with Audit. Private media and sensitive VIP concessions await durable evidence before disclosure
  • • Integrations owns MediaDeletionJob: committed intent blocks playback, leased bounded retries remove private objects, completion and Audit commit together; terminal failure has an authorized repair path
  • • Audit HTTP queries use explicit independent read, restricted-metadata and export grants, server organization scope and validated unsigned pagination; every page reapplies authorization and query limits. No Audit secret is required. Retention is held without automatic purge or receipt expiry
  • • P1 adds manual VIP changes, VIT assignment decisions, Program Run creation, manual attendance and named soft archives; existing video decisions retain one P0 fact and owner histories remain independent

Optional cloud motion capture

  • • Delivery owns MotionCaptureJob in PostgreSQL, staff ownership checks and upload/result signing. Authenticated /motion-captures requests require motion_capture.generate explicitly assigned to the staff user; wildcard access alone is insufficient.
  • • The private Modal CPU gateway accepts backend proxy credentials and dispatches one L4 GPU worker on demand. No provider credential reaches the browser; no model execution depends on a developer Mac.
  • • Browser uploads directly to private S3. The GPU reads bounded signed sources and writes a versioned MHR70 skeleton plus a bounded mesh ZIP (faces, per-frame vertices, camera and timestamps) directly to S3. MotionCaptureJob retains an optional meshKey for backward compatibility. Delivery verifies both required artifacts before completion and signs owner-only downloads; media never passes through the API.
  • • MediaPipe locates one person within the selected full/left/right view; SAM estimates 3D. The frontend maps MHR70 joints to the existing anatomical rig, preserving atlas bone lengths and GPU muscle skinning.
  • • Start uses an atomic job claim; ambiguous dispatch fails without automatic retry. Polling updates durable completion. Stopping browser checks does not cancel already submitted GPU work.

Transcript review and processing — ADR 0069

  • • TranscriptWorkspaceModule composes owner services over authenticated /transcripts routes. PostgreSQL transcripts schema stores releases, derived halfvec chunks and leased index jobs; originals/candidates remain in Sales or Clinical.
  • • A ten-second in-process worker claims durable jobs with SKIP LOCKED, renewed leases and generation fencing. Audited remote batches use approved recipient profiles; local text indexing requires no provider.
  • • Source changes and revocation remove platform-owned derivatives atomically. External analysis creates schema-validated cited drafts only. Legacy identified Sales egress is separately gated and existing raw vectors are excluded.