WarpCore / Documentation

Data Model

SOURCE · docs/DATA_MODEL.md · synced from the engineering repo

WarpCore Data Model

The core domain entities and the persistence model behind WarpCore. The authoritative sources are the Rust domain types in crates/warpcore-models/src/ and the SQL DDL in migrations/. Where this document and those files disagree, the code and migrations win.


Two layers of state

WarpCore separates two concerns:

  1. The payment pipeline — the ISO 20022 → Kaspa settlement lifecycle, driven by a deterministic state machine and persisted in the core schema (migrations/001008).
  2. The platform engines — 13 institutional middleware engines (CBDC, FX, RTGS, …) plus the Dead Letter Queue, each an in-memory engine fronted by a durable PostgreSQL store (migrations/009025, plus the DLQ from migrations/004).

There are 25 SQL migrations in total. All amounts are exact decimals — NUMERIC(30,8) for money (FX rates NUMERIC(20,10); unsigned u64 securities quantities NUMERIC(40,0)) — mirroring rust_decimal::Decimal on the Rust side; no IEEE floats are used for money.


The payment domain

PaymentInstruction (core domain model)

The central entity, defined in crates/warpcore-models/src/domain.rs. Created from a parsed ISO 20022 message, enriched with compliance data, linked to a Kaspa transaction, and tracked through its lifecycle. Key fields:

FieldTypeNotes
idUuidWarpCore internal identifier (primary key)
uetrUetrSWIFT GPI Unique End-to-End Transaction Reference (UUID4), unique
message_typeMessageTypeOriginating ISO 20022 type (pacs.008, pacs.009, pain.001, …)
raw_xmlVec<u8>Original ISO 20022 XML bytes (stored for audit + hash verification)
message_hash[u8; 32]SHA-256 of raw_xml — the on-chain integrity anchor
amount / currencyDecimal / CurrencyCodeExact decimal amount; ISO 4217 currency
dtiOption<Dti>ISO 24165 Digital Token Identifier (if a digital asset)
debtor_bic / creditor_bicBicISO 9362 BICs
debtor_lei / creditor_leiOption<Lei>ISO 17442
debtor_iban / creditor_ibanOption<Iban>ISO 13616
debtor_name / creditor_nameOption<String>Extracted from <Dbtr><Nm> / <Cdtr><Nm>
beneficiary_addressOption<String>Resolved Kaspa address for second-wallet settlement
covenant_typeCovenantTypeSilverScript covenant (gated; see Toccata caveat)
compliance_ref / zkp_proofOption<Uuid> / Option<Vec<u8>>ZKP compliance proof reference + raw bytes
kaspa_txid / kaspa_daa_scoreOption<String> / Option<u64>Set on submission / confirmation (GHOSTDAG DAA)
travel_ruleOption<TravelRulePayload>FATF Travel Rule data (required at the configured USD threshold)
batch_idOption<Uuid>Groups payments from one pacs.008 with NbOfTxs > 1
stateTxStateCurrent lifecycle state
error_msgOption<String>Set on Rejected / Failed
status_reportsVec<String>Auto-generated pacs.002 status-report XML per transition
created_at / updated_atDateTime<Utc>

The message hash is computed as SHA-256(raw_xml) at construction time.

Payment state machine (TxState)

A deterministic, single-direction state machine. Transitions are validated by TxState::can_transition_to; only the edges below are legal.

Received ──▶ Validated ──▶ ComplianceChecked ──▶ Submitted ──▶ Confirming ──▶ Settled ──▶ Reported
   │             │                │
   └──▶ Rejected └──▶ Rejected    └──▶ Rejected            (Submitted/Confirming) ──▶ Failed
  • Happy path: Received → Validated → ComplianceChecked → Submitted → Confirming → Settled → Reported.
  • Pre-submission rejection: Received, Validated, and ComplianceChecked may each transition to Rejected.
  • Post-submission failure: Submitted and Confirming may transition to Failed.
  • Terminal states: Reported, Rejected, Failed. An invalid transition yields INVALID_STATE_TRANSITION.

Wire (Display) forms are screaming-snake-case: RECEIVED, VALIDATED, COMPLIANCE_CHECKED, SUBMITTED, CONFIRMING, SETTLED, REPORTED, REJECTED, FAILED.

Supporting domain types

  • TravelRulePayload — FATF Recommendation 16 data: originator/beneficiary name + account + LEI, VASP id, and a proof_hash ([u8; 32], SHA-256 of the encrypted TRISA payload).
  • Role — RBAC roles: Admin, Operator, Compliance, Auditor, Readonly.
  • ComplianceResult / ComplianceFlag, KaspaPayload (64-byte on-chain payload linking the ISO 20022 message to the settlement tx), and the standards newtypes (Bic, Lei, Iban, Dti, Uetr, CurrencyCode) in standards.rs.

Core schema (migrations/001008)

payments

The state-machine record. Mirrors PaymentInstruction. PK id (UUID), unique uetr. Holds the raw XML, message_hash, amount (NUMERIC(30,8)), the ISO-coded counterparty fields, kaspa_txid/kaspa_daa_score, compliance_ref/zkp_proof, travel_rule_hash, batch_id, state (default RECEIVED), error_msg, and — added by later migrations — beneficiary_address (002), debtor_name/creditor_name/status_reports (006), idempotency_key (003). Indexed on state, uetr, kaspa_txid, created_at, batch_id, and a partial unique index on idempotency_key.

messages

Inbound + outbound ISO 20022 XML store, keyed to a payment. direction constrained to INBOUND / OUTBOUND, with msg_hash (SHA-256) and a correlation_id linking request to response.

institutions

The BIC → Kaspa-address routing registry. PK bic (ISO 9362), unique lei, kaspa_address, name, ISO 3166-1 country, active.

dti_registry

ISO 24165 Digital Token Identifiers. PK dti (CHAR(10): 9 base + 1 Luhn-mod-36 check digit), asset_type constrained to native / token / stablecoin, network, issuer_lei, JSONB metadata, and onchain_hash / anchor_txid for the anchoring transaction.

zk_proofs

ZKP metadata per payment: circuit_id, proof_bytes, public_inputs (JSONB), commitment (SHA-256 of the proof), verification_key, verified flag.

compliance_alerts

Sanctions/AML alerts per payment. alert_type constrained to a fixed set (SANCTIONS_HIT, HIGH_RISK_JURISDICTION, STRUCTURING, VELOCITY_EXCEEDED, PEP_MATCH, ADVERSE_MEDIA, KYC_EXPIRED, TRAVEL_RULE_VIOLATION), severity (LOW/MEDIUM/HIGH/CRITICAL), workflow status (OPEN/REVIEWED/CLEARED/ESCALATED/CLOSED).

audit_log — tamper-evident hash chain

Append-only, hash-chained audit trail (ISO 27001 A.8.15 in intent). Each row stores prev_hash and row_hash = SHA-256(prev_hash || event data). Database RULEs reject UPDATE and DELETE (append-only at the DB level), and a stored function verify_audit_chain(start_id, end_id) walks the chain and reports the first break. Exposed via GET /v1/audit/log and POST /v1/audit/verify.

Idempotency tables

  • payments.idempotency_key (003) — original column-based mechanism.
  • idempotency_reservations (007) — the primary, concurrency-safe mechanism: a single INSERT … ON CONFLICT atomically claims a key or returns the existing payment_id, eliminating the check-then-create race. Both expire entries after 24 hours.

Persistence model: write-through + hydrate-on-boot

The platform engines run as in-memory state machines for speed, but every mutation is written through to PostgreSQL and the full state is rehydrated on startup, so engine state survives a process restart. The persistence adapters live in crates/warpcore-persistence/src/; each implements a *Persistence port defined by its engine crate (so the engine crates carry no SQL dependency).

Restart-survival is exercised by the durable-mode integration suite (scripts/test-persistence.sh — the *_state_survives_restart tests) gated by scripts/ci-local.sh. Test counts are not asserted here; the local gate is the only source of truth for the count, and all such figures are self-reported.

The 18 durable stores

13 platform engines, the DLQ, and four compliance/control stores (KYC subject registry + STR/SAR register + transaction-monitor + policy-engine) each have a durable store:

#Engine / storeModulePrincipal tables
1CBDCcbdc.rscbdc_tokens, cbdc_balances, cbdc_attestations, cbdc_holder_limits
2Liquidityliquidity.rs, pool.rsnostro_accounts, netting_cycles, netting_obligations, liquidity_pools, liquidity_pool_shares
3FXfx.rsfx_rates, pvp_settlements (+ FX full-state cols in 019)
4RTGSrtgs.rsrtgs_participants, rtgs_payments, gridlock_cycles
5Exchangeexchange.rsexchange_orders, exchange_trades
6Trade Financetrade_finance.rsletters_of_credit, invoices
7Identityidentity.rsinstitutional_dids, verifiable_credentials, bilateral_agreements
8Reconciliationrecon.rsrecon_records, recon_exceptions
9Programmableprogrammable.rspayment_programs
10Orchestrationorchestration.rspayment_batches, standing_orders
11Reportingreporting.rsregulatory_reports, report_schedules
12Bridge (HTLC)bridge.rsbridge_transfers (+ full HTLC state in 012)
13Securitiessecurities.rstokenised_securities, settlement_fails
14Dead Letter Queuedlq.rsdead_letter_queue (+ webhook_subscriptions)
15KYC subject registrykyc.rskyc_subjects, kyc_beneficial_owners (migration 022)
16STR / SAR registerstr_register.rsstr_register (migration 024)
17Transaction monitormonitoring.rstransaction-monitoring aggregates (migration 025)
18Policy enginepolicy.rspolicy-engine aggregates (migration 025)

Migrations 012025 add the remaining columns each engine needs to round-trip its complete in-memory state (not just the public-facing fields), so an in-flight bridge lock, a partial securities delivery, a netting cycle, etc., all resume cleanly after a restart.

Selected platform-engine tables

  • CBDC (009, 010): cbdc_tokens (supply/reserves ledger), cbdc_balances (per-holder), cbdc_attestations (reserve attestations), cbdc_holder_limits (per-DID balance / per-transaction caps).
  • Liquidity (009, 014): nostro_accounts, netting_cycles + netting_obligations (multilateral netting), liquidity_pools + liquidity_pool_shares (single-asset shared share ledger).
  • RTGS (009): rtgs_participants (positions + credit limits), rtgs_payments (priority queue), gridlock_cycles (resolution runs).
  • Bridge (009, 012): bridge_transfers — cross-chain HTLC with hash_lock, time_lock, and (012) lock_tx_kaspa, lock_tx_external, and the HTLC pre-image secret. Note: secret is bearer key material stored in plaintext; this is flagged for encryption-at-rest before mainnet.
  • Securities (020, 021): tokenised_securities (ISIN-keyed registry with CUSIP/SEDOL/CFI/MIC, total_supply as NUMERIC(40,0) for full u64 fidelity) and settlement_fails (CSDR fails lifecycle: quantities, fail_reason, status, retry/buy-in deadlines, penalty_bps, and reference_price for the CSDR cash-penalty calculation).
  • Reporting (009, 011): regulatory_reports (generated/filed reports with a content_hash) and report_schedules (recurring report definitions).