Skip to content

🏛 Architecture

Khata follows a strict three-layer architecture with one-way data flow and no backflow between layers.

🧱 Layers

Layer Package(s) Responsibility
interface interfaces/ HTTP routes (OpenAPI handlers), CLI, SSE bus wiring, DTO mapping
domain domain/ Models, ports (abstract contract), services, orchestration, enrichment pipelines
infra infra/ Mongo repos, Redis cache, HTTP/email/Llama fetchers, OpenAI-compatible LLM pools

The pipelines/ package holds the hexa extraction runtime — the stateful, stage-driven extractors for PDF, CSV, XLSX, email, and llama sources. It sits outside the three-layer packages in pyproject.toml (packages = domain*, infra*, interfaces*) but is first-class code.

🔀 Data Flow

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
                         ┌────────────────────────────────────────┐
                         │              INTERFACES                 │
                         │  openapi handler  ·  sse bus  ·  cli    │
                         └──────────────────┬─────────────────────┘
                                            │ DTOs / commands
                         ┌──────────────────▼─────────────────────┐
                         │                DOMAIN                   │
                         │  models · ports · services (rules)      │
                         │  fetch_pipeline orchestrator            │
                         │  enrichment (entity / tags)  ← dagpipe  │
                         └──────────────────┬─────────────────────┘
                                            │ ports (abstract)
                         ┌──────────────────▼─────────────────────┐
                         │                INFRA                    │
                         │  mongo repos · redis · http · llm pools │
                         │  pdf/email/xlsx fetchers                │
                         └──────────────────┬─────────────────────┘
                                            │
                                  ┌─────────┴─────────┐
                                  ▼                   ▼
                              MongoDB              Redis / external

Interfaces speak DTOs/commands; the domain owns rules and orchestration; infra implements ports and talks to the outside world. No infra import may appear in the domain, and no domain import may appear in infra.

🧩 Wiring

Entrypoints (main.py, worker.py, init.py) build the object graph explicitly:

  • init() constructs the Mongo engine, repos, services, fetchers, cache store, the SSE bus, and a PipelineOrchestrator wiring, then returns a dict of named services.
  • Handler lookups go through get_svc(name) (e.g. "fetch_request_svc", "expense_svc", "ambiguity_svc").
  • main.py registers the OpenAPI app from openapi.yaml; worker.py polls for pending fetch requests; interfaces/fetch_request/cli drives the same services from the terminal.

🔁 Lifecycle Rules

  • One-way ports — repos implement (implements-style) the port interfaces declared in the domain; the domain never knows a concrete repo.
  • Canonical ids — documents are keyed by MongoDB ObjectId; cross references are populated with explicit filter-based x-fk resolution, never Mongo joins.
  • Caching discipline — repo reads go through an in-memory LRU (and an optional Redis mirror for HTTP); invalidation happens on writes via _invalidate_cache.
  • No god init — the domain has no plain __init__ modules with factories; facades.py and init.py hold the composition.

Components · Domain Resources · Extraction Pipelines · Running The Service · Development