WarpCore / Documentation
Data Model
SOURCE · docs/DATA_MODEL.md · synced from the engineering repo
WarpCore Data Model
The core domain entities and the persistence model behind WarpCore. The
authoritative sources are the Rust domain types in
crates/warpcore-models/src/ and the SQL DDL
in migrations/. Where this document and those files disagree,
the code and migrations win.
Two layers of state
WarpCore separates two concerns:
- The payment pipeline — the ISO 20022 → Kaspa settlement lifecycle, driven
by a deterministic state machine and persisted in the core schema
(
migrations/001–008). - The platform engines — 13 institutional middleware engines (CBDC, FX,
RTGS, …) plus the Dead Letter Queue, each an in-memory engine fronted by a
durable PostgreSQL store (
migrations/009–025, plus the DLQ frommigrations/004).
There are 25 SQL migrations in total. All amounts are exact decimals —
NUMERIC(30,8) for money (FX rates NUMERIC(20,10); unsigned u64 securities
quantities NUMERIC(40,0)) — mirroring rust_decimal::Decimal on the Rust side;
no IEEE floats are used for money.
The payment domain
PaymentInstruction (core domain model)
The central entity, defined in
crates/warpcore-models/src/domain.rs.
Created from a parsed ISO 20022 message, enriched with compliance data, linked to
a Kaspa transaction, and tracked through its lifecycle. Key fields:
| Field | Type | Notes |
|---|---|---|
id | Uuid | WarpCore internal identifier (primary key) |
uetr | Uetr | SWIFT GPI Unique End-to-End Transaction Reference (UUID4), unique |
message_type | MessageType | Originating ISO 20022 type (pacs.008, pacs.009, pain.001, …) |
raw_xml | Vec<u8> | Original ISO 20022 XML bytes (stored for audit + hash verification) |
message_hash | [u8; 32] | SHA-256 of raw_xml — the on-chain integrity anchor |
amount / currency | Decimal / CurrencyCode | Exact decimal amount; ISO 4217 currency |
dti | Option<Dti> | ISO 24165 Digital Token Identifier (if a digital asset) |
debtor_bic / creditor_bic | Bic | ISO 9362 BICs |
debtor_lei / creditor_lei | Option<Lei> | ISO 17442 |
debtor_iban / creditor_iban | Option<Iban> | ISO 13616 |
debtor_name / creditor_name | Option<String> | Extracted from <Dbtr><Nm> / <Cdtr><Nm> |
beneficiary_address | Option<String> | Resolved Kaspa address for second-wallet settlement |
covenant_type | CovenantType | SilverScript covenant (gated; see Toccata caveat) |
compliance_ref / zkp_proof | Option<Uuid> / Option<Vec<u8>> | ZKP compliance proof reference + raw bytes |
kaspa_txid / kaspa_daa_score | Option<String> / Option<u64> | Set on submission / confirmation (GHOSTDAG DAA) |
travel_rule | Option<TravelRulePayload> | FATF Travel Rule data (required at the configured USD threshold) |
batch_id | Option<Uuid> | Groups payments from one pacs.008 with NbOfTxs > 1 |
state | TxState | Current lifecycle state |
error_msg | Option<String> | Set on Rejected / Failed |
status_reports | Vec<String> | Auto-generated pacs.002 status-report XML per transition |
created_at / updated_at | DateTime<Utc> |
The message hash is computed as SHA-256(raw_xml) at construction time.
Payment state machine (TxState)
A deterministic, single-direction state machine. Transitions are validated by
TxState::can_transition_to; only the edges below are legal.
Received ──▶ Validated ──▶ ComplianceChecked ──▶ Submitted ──▶ Confirming ──▶ Settled ──▶ Reported
│ │ │
└──▶ Rejected └──▶ Rejected └──▶ Rejected (Submitted/Confirming) ──▶ Failed
- Happy path:
Received → Validated → ComplianceChecked → Submitted → Confirming → Settled → Reported. - Pre-submission rejection:
Received,Validated, andComplianceCheckedmay each transition toRejected. - Post-submission failure:
SubmittedandConfirmingmay transition toFailed. - Terminal states:
Reported,Rejected,Failed. An invalid transition yieldsINVALID_STATE_TRANSITION.
Wire (Display) forms are screaming-snake-case: RECEIVED, VALIDATED,
COMPLIANCE_CHECKED, SUBMITTED, CONFIRMING, SETTLED, REPORTED,
REJECTED, FAILED.
Supporting domain types
TravelRulePayload— FATF Recommendation 16 data: originator/beneficiary name + account + LEI, VASP id, and aproof_hash([u8; 32], SHA-256 of the encrypted TRISA payload).Role— RBAC roles:Admin,Operator,Compliance,Auditor,Readonly.ComplianceResult/ComplianceFlag,KaspaPayload(64-byte on-chain payload linking the ISO 20022 message to the settlement tx), and the standards newtypes (Bic,Lei,Iban,Dti,Uetr,CurrencyCode) instandards.rs.
Core schema (migrations/001–008)
payments
The state-machine record. Mirrors PaymentInstruction. PK id (UUID), unique
uetr. Holds the raw XML, message_hash, amount (NUMERIC(30,8)), the
ISO-coded counterparty fields, kaspa_txid/kaspa_daa_score,
compliance_ref/zkp_proof, travel_rule_hash, batch_id, state (default
RECEIVED), error_msg, and — added by later migrations — beneficiary_address
(002), debtor_name/creditor_name/status_reports (006), idempotency_key
(003). Indexed on state, uetr, kaspa_txid, created_at, batch_id, and a
partial unique index on idempotency_key.
messages
Inbound + outbound ISO 20022 XML store, keyed to a payment. direction
constrained to INBOUND / OUTBOUND, with msg_hash (SHA-256) and a
correlation_id linking request to response.
institutions
The BIC → Kaspa-address routing registry. PK bic (ISO 9362), unique lei,
kaspa_address, name, ISO 3166-1 country, active.
dti_registry
ISO 24165 Digital Token Identifiers. PK dti (CHAR(10): 9 base + 1
Luhn-mod-36 check digit), asset_type constrained to
native / token / stablecoin, network, issuer_lei, JSONB metadata,
and onchain_hash / anchor_txid for the anchoring transaction.
zk_proofs
ZKP metadata per payment: circuit_id, proof_bytes, public_inputs (JSONB),
commitment (SHA-256 of the proof), verification_key, verified flag.
compliance_alerts
Sanctions/AML alerts per payment. alert_type constrained to a fixed set
(SANCTIONS_HIT, HIGH_RISK_JURISDICTION, STRUCTURING, VELOCITY_EXCEEDED,
PEP_MATCH, ADVERSE_MEDIA, KYC_EXPIRED, TRAVEL_RULE_VIOLATION),
severity (LOW/MEDIUM/HIGH/CRITICAL), workflow status
(OPEN/REVIEWED/CLEARED/ESCALATED/CLOSED).
audit_log — tamper-evident hash chain
Append-only, hash-chained audit trail (ISO 27001 A.8.15 in intent). Each row
stores prev_hash and row_hash = SHA-256(prev_hash || event data). Database
RULEs reject UPDATE and DELETE (append-only at the DB level), and a stored
function verify_audit_chain(start_id, end_id) walks the chain and reports the
first break. Exposed via GET /v1/audit/log and POST /v1/audit/verify.
Idempotency tables
payments.idempotency_key(003) — original column-based mechanism.idempotency_reservations(007) — the primary, concurrency-safe mechanism: a singleINSERT … ON CONFLICTatomically claims a key or returns the existingpayment_id, eliminating the check-then-create race. Both expire entries after 24 hours.
Persistence model: write-through + hydrate-on-boot
The platform engines run as in-memory state machines for speed, but every
mutation is written through to PostgreSQL and the full state is
rehydrated on startup, so engine state survives a process restart. The
persistence adapters live in
crates/warpcore-persistence/src/; each
implements a *Persistence port defined by its engine crate (so the engine
crates carry no SQL dependency).
Restart-survival is exercised by the durable-mode integration suite
(scripts/test-persistence.sh — the *_state_survives_restart tests) gated by
scripts/ci-local.sh. Test counts are not asserted here; the local gate is the
only source of truth for the count, and all such figures are self-reported.
The 18 durable stores
13 platform engines, the DLQ, and four compliance/control stores (KYC subject registry + STR/SAR register + transaction-monitor + policy-engine) each have a durable store:
| # | Engine / store | Module | Principal tables |
|---|---|---|---|
| 1 | CBDC | cbdc.rs | cbdc_tokens, cbdc_balances, cbdc_attestations, cbdc_holder_limits |
| 2 | Liquidity | liquidity.rs, pool.rs | nostro_accounts, netting_cycles, netting_obligations, liquidity_pools, liquidity_pool_shares |
| 3 | FX | fx.rs | fx_rates, pvp_settlements (+ FX full-state cols in 019) |
| 4 | RTGS | rtgs.rs | rtgs_participants, rtgs_payments, gridlock_cycles |
| 5 | Exchange | exchange.rs | exchange_orders, exchange_trades |
| 6 | Trade Finance | trade_finance.rs | letters_of_credit, invoices |
| 7 | Identity | identity.rs | institutional_dids, verifiable_credentials, bilateral_agreements |
| 8 | Reconciliation | recon.rs | recon_records, recon_exceptions |
| 9 | Programmable | programmable.rs | payment_programs |
| 10 | Orchestration | orchestration.rs | payment_batches, standing_orders |
| 11 | Reporting | reporting.rs | regulatory_reports, report_schedules |
| 12 | Bridge (HTLC) | bridge.rs | bridge_transfers (+ full HTLC state in 012) |
| 13 | Securities | securities.rs | tokenised_securities, settlement_fails |
| 14 | Dead Letter Queue | dlq.rs | dead_letter_queue (+ webhook_subscriptions) |
| 15 | KYC subject registry | kyc.rs | kyc_subjects, kyc_beneficial_owners (migration 022) |
| 16 | STR / SAR register | str_register.rs | str_register (migration 024) |
| 17 | Transaction monitor | monitoring.rs | transaction-monitoring aggregates (migration 025) |
| 18 | Policy engine | policy.rs | policy-engine aggregates (migration 025) |
Migrations 012–025 add the remaining columns each engine needs to round-trip
its complete in-memory state (not just the public-facing fields), so an
in-flight bridge lock, a partial securities delivery, a netting cycle, etc., all
resume cleanly after a restart.
Selected platform-engine tables
- CBDC (
009,010):cbdc_tokens(supply/reserves ledger),cbdc_balances(per-holder),cbdc_attestations(reserve attestations),cbdc_holder_limits(per-DID balance / per-transaction caps). - Liquidity (
009,014):nostro_accounts,netting_cycles+netting_obligations(multilateral netting),liquidity_pools+liquidity_pool_shares(single-asset shared share ledger). - RTGS (
009):rtgs_participants(positions + credit limits),rtgs_payments(priority queue),gridlock_cycles(resolution runs). - Bridge (
009,012):bridge_transfers— cross-chain HTLC withhash_lock,time_lock, and (012)lock_tx_kaspa,lock_tx_external, and the HTLC pre-imagesecret. Note:secretis bearer key material stored in plaintext; this is flagged for encryption-at-rest before mainnet. - Securities (
020,021):tokenised_securities(ISIN-keyed registry with CUSIP/SEDOL/CFI/MIC,total_supplyasNUMERIC(40,0)for fullu64fidelity) andsettlement_fails(CSDR fails lifecycle: quantities,fail_reason,status, retry/buy-in deadlines,penalty_bps, andreference_pricefor the CSDR cash-penalty calculation). - Reporting (
009,011):regulatory_reports(generated/filed reports with acontent_hash) andreport_schedules(recurring report definitions).