0.5.1-pre.002
This commit is contained in:
67
ks-store/CHANGELOG.md
Normal file
67
ks-store/CHANGELOG.md
Normal file
@@ -0,0 +1,67 @@
|
||||
<!-- file: ks-store/CHANGELOG.md -->
|
||||
<!-- version: 13 -->
|
||||
|
||||
# CHANGELOG — ks-store
|
||||
|
||||
## `0.5.1-pre.002`
|
||||
|
||||
- renomme `kb-store` en `ks-store` et `kb_store` en `ks_store` ;
|
||||
- aligne le target racine et les consommateurs, tout en conservant volontairement les tables `kb_sol_*` et les migrations SQL inchangées jusqu'à `0.5.3`.
|
||||
|
||||
## `0.5.0-pre.003`
|
||||
|
||||
### Documentation
|
||||
|
||||
- documente l’audit préparatoire `0.5.3` sans modifier le schéma ni les migrations `0001` à `0004` ;
|
||||
- formalise dans le TODO la caractérisation du schéma `0.4.8`, la séparation de `block_time` on-chain des temps d’acquisition/persistance, l’enveloppe de faits/provenance et la décision future sur les projections normalisées parallèles au journal générique de matérialisation.
|
||||
|
||||
## 0.4.6
|
||||
|
||||
- alignement de la crate sur la version fonctionnelle bot3 `0.4.6` ;
|
||||
- clôture des tâches de migration applicables et report explicite des évolutions ultérieures dans le TODO.
|
||||
|
||||
## 0.1.0-pre.074
|
||||
|
||||
- clôture de la tâche documentaire préalable à `0.4.6` après publication du guide PostgreSQL et stockage ;
|
||||
|
||||
## 0.1.0-pre.073
|
||||
|
||||
- ajout du guide transversal [`docs/guides/POSTGRES_STORAGE.md`](../docs/guides/POSTGRES_STORAGE.md) ;
|
||||
|
||||
## 0.1.0-pre.072
|
||||
|
||||
- reclassement du TODO selon les blocants avant `0.4.6`, les travaux `0.4.7`, les versions ultérieures et les dépendances conditionnelles.
|
||||
|
||||
## 0.1.0-pre.070
|
||||
|
||||
- enrichissement de `USAGE.md` avec plusieurs exemples couvrant les familles d’API publiques significatives.
|
||||
|
||||
## 0.1.0-pre.069
|
||||
|
||||
### Documentation
|
||||
|
||||
- création de `TODO.md`, `USAGE.md` et du changelog détaillé ;
|
||||
- réécriture du README autour des contrats consolidés et de `PostgresStore` ;
|
||||
- suppression de la section `Non publié`, incompatible avec le rôle chronologique du changelog ;
|
||||
- retrait des constats sans action du TODO et des travaux futurs du changelog ;
|
||||
- réécriture de `TODO.md` sous forme de tâches uniquement ;
|
||||
- correction des exemples de `USAGE.md` afin de ne pas utiliser l’opérateur `?`.
|
||||
|
||||
## 0.1.0
|
||||
|
||||
### Migré
|
||||
|
||||
- consolidation de `ks_store_core`, `ks_store_pg` et des contrats associés dans `ks-store` ;
|
||||
- migration des tables raw, observations, Core, ledger, événements décodés, couverture et matérialisations ;
|
||||
- migration des migrations SQL, diagnostics et requêtes de replay.
|
||||
|
||||
### Modifié
|
||||
|
||||
- adoption de Rust 2024 et des règles Khadhroony ;
|
||||
- maintien des modules internes privés derrière une façade publique contrôlée ;
|
||||
- erreurs structurées via `ks_core::Error` et transactions async via `sqlx`.
|
||||
|
||||
### Validation
|
||||
|
||||
- tests des DTO, pagination, filtres, schéma, migrations, diagnostics et atomicité ;
|
||||
- tests PostgreSQL réels optionnels pilotés par environnement.
|
||||
23
ks-store/Cargo.toml
Normal file
23
ks-store/Cargo.toml
Normal file
@@ -0,0 +1,23 @@
|
||||
# file: ks-store/Cargo.toml
|
||||
# version: 3
|
||||
|
||||
[package]
|
||||
name = "ks-store"
|
||||
version.workspace = true
|
||||
edition.workspace = true
|
||||
license.workspace = true
|
||||
publish.workspace = true
|
||||
|
||||
[dependencies]
|
||||
async-trait.workspace = true
|
||||
chrono.workspace = true
|
||||
ks-core = { path = "../ks-core" }
|
||||
ks-lib = { path = "../ks-lib" }
|
||||
serde.workspace = true
|
||||
serde_json.workspace = true
|
||||
sqlx.workspace = true
|
||||
tokio.workspace = true
|
||||
tracing.workspace = true
|
||||
|
||||
[lints]
|
||||
workspace = true
|
||||
52
ks-store/README.md
Normal file
52
ks-store/README.md
Normal file
@@ -0,0 +1,52 @@
|
||||
<!-- file: ks-store/README.md -->
|
||||
<!-- version: 4 -->
|
||||
|
||||
# ks-store
|
||||
|
||||
`ks-store` consolide les contrats de stockage et l’implémentation PostgreSQL de `khadhroony-bot3`.
|
||||
|
||||
## Périmètre
|
||||
|
||||
La crate expose :
|
||||
|
||||
- DTO et entités raw, observations, Core, décodage, matérialisation et ledger ;
|
||||
- traits async de repositories indépendants du backend ;
|
||||
- pagination et filtres bornés ;
|
||||
- `PostgresStore` et ses options de connexion ;
|
||||
- initialisation idempotente du schéma et migrations SQL ;
|
||||
- diagnostics read-only, healthchecks et résumés de replay ;
|
||||
- transactions atomiques pour extraction, décodage et matérialisation.
|
||||
|
||||
## Responsabilités
|
||||
|
||||
- garantir les invariants des données persistées ;
|
||||
- séparer contrats et implémentation PostgreSQL à l’intérieur de la crate consolidée ;
|
||||
- maintenir les tables canoniques `kb_sol_*` ;
|
||||
- fournir au pipeline des frontières async typées ;
|
||||
- protéger les opérations multi-tables par transactions ;
|
||||
- borner toutes les requêtes de diagnostic et de replay exposées.
|
||||
|
||||
## Hors périmètre
|
||||
|
||||
La crate ne décode pas les instructions, n’acquiert pas les transactions et ne décide pas quelle matérialisation exécuter. Elle persiste les résultats produits par `ks-lib` et orchestrés par `ks-pipeline`.
|
||||
|
||||
## API publique
|
||||
|
||||
Les consommateurs utilisent principalement `PostgresStore`, `PostgresStoreOptions`, les traits `*Store`, les DTO `*Insert`/`*Row`, les filtres de replay et les diagnostics. Les constantes de tables et fonctions de validation sont publiques pour les audits et outils d’administration.
|
||||
|
||||
## Relations
|
||||
|
||||
- dépend de `ks-core` pour les erreurs ;
|
||||
- utilise les contrats de replay de `ks-lib` ;
|
||||
- est consommée principalement par `ks-pipeline`, les scénarios de démonstration et le desktop.
|
||||
|
||||
## Statut
|
||||
|
||||
Le stockage consolidé dispose de tests unitaires étendus et de tests PostgreSQL optionnels pilotés par environnement. Les opérations réelles exigent un serveur PostgreSQL et l’application préalable des migrations.
|
||||
|
||||
## Documents
|
||||
|
||||
- [Utilisation](USAGE.md)
|
||||
- [Travaux restants](TODO.md)
|
||||
- [Historique](CHANGELOG.md)
|
||||
- [Architecture du stockage](../docs/architecture/STORAGE_ARCHITECTURE.md)
|
||||
15
ks-store/TODO.md
Normal file
15
ks-store/TODO.md
Normal file
@@ -0,0 +1,15 @@
|
||||
<!-- file: ks-store/TODO.md -->
|
||||
<!-- version: 7 -->
|
||||
|
||||
# TODO — ks-store
|
||||
|
||||
## Série `0.5.x`
|
||||
|
||||
- [ ] `0.5.3` — caractériser le schéma `0.4.8` par tests d'API externe et fixture de migration avant toute évolution SQL.
|
||||
- [ ] `0.5.3` — définir le contrat temporel distinguant `block_time` on-chain des timestamps d'acquisition et de persistance.
|
||||
- [ ] `0.5.3` — définir l'enveloppe de faits, provenance et index nécessaires aux futures projections trading/routing avant de créer des tables spécifiques.
|
||||
- [ ] `0.5.3` — décider si `kb_sol_mat_events` reste le journal générique parallèlement à des projections normalisées spécialisées.
|
||||
- [ ] PostgreSQL - auditer le dimensionnement du pool selon les charges réelles.
|
||||
- [ ] Résilience - décider et implémenter, si justifié, un retry borné pour les timeouts transitoires.
|
||||
- [ ] Administration - définir les outils supplémentaires prévus par le ROADMAP avant leur implémentation.
|
||||
- [ ] Historique - documenter et tester les futures opérations historiques ajoutées à la crate.
|
||||
163
ks-store/USAGE.md
Normal file
163
ks-store/USAGE.md
Normal file
@@ -0,0 +1,163 @@
|
||||
<!-- file: ks-store/USAGE.md -->
|
||||
<!-- version: 4 -->
|
||||
|
||||
# Utilisation de ks-store
|
||||
|
||||
## Connexion PostgreSQL
|
||||
|
||||
```rust
|
||||
let options = match ks_store::PostgresStoreOptions::new(
|
||||
database_url,
|
||||
10,
|
||||
10_000,
|
||||
true,
|
||||
) {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
println!("postgres endpoint: {}", options.masked_dsn());
|
||||
|
||||
let store = match ks_store::PostgresStore::connect(options).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
```
|
||||
|
||||
`PostgresStoreOptions::new` valide le DSN, le nombre de connexions et le timeout. Lorsque `auto_initialize_schema` vaut `true`, la connexion applique les schémas idempotents. Utiliser `masked_dsn` ou `mask_postgres_dsn` dans les logs ; ne jamais journaliser le DSN brut.
|
||||
|
||||
## Diagnostics
|
||||
|
||||
```rust
|
||||
let health = store.health_snapshot().await;
|
||||
let migrations = store.migration_snapshot().await;
|
||||
let backend = store.backend_diagnostics().await;
|
||||
|
||||
println!("health={:?}", health.status);
|
||||
println!("migration={:?}", migrations.status);
|
||||
println!("backend={:?}", backend.descriptor.backend);
|
||||
```
|
||||
|
||||
Les diagnostics de tables raw, Core et decode/materialization sont également disponibles par les méthodes `*_table_diagnostics`.
|
||||
|
||||
```rust
|
||||
let raw_tables = match store.raw_table_diagnostics().await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
for table in raw_tables {
|
||||
println!(
|
||||
"table={} domain={} exists={}",
|
||||
table.table_name,
|
||||
table.domain,
|
||||
table.exists
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Pagination bornée
|
||||
|
||||
```rust
|
||||
let first_page = ks_store::PageRequest::first_page();
|
||||
|
||||
let next_page = match ks_store::PageRequest::new(250, 250) {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
assert_eq!(first_page.limit, ks_store::DEFAULT_PAGE_SIZE);
|
||||
assert!(next_page.limit <= ks_store::MAX_PAGE_SIZE);
|
||||
```
|
||||
|
||||
## Contrats de repositories
|
||||
|
||||
Les traits publics principaux sont :
|
||||
|
||||
- `StoreHealthStore` ;
|
||||
- `RawTransactionStore` ;
|
||||
- `CoreTransactionStore` ;
|
||||
- `CoreExtractionStore` ;
|
||||
- `DecodePipelineStore` ;
|
||||
- `ProgramObservationStore` ;
|
||||
- `DecodedEventStore` ;
|
||||
- `MaterializedEventStore` ;
|
||||
- `ProcessingLedgerStore`.
|
||||
|
||||
Ils permettent au pipeline de dépendre d’un contrat async plutôt que d’une requête SQL directe.
|
||||
|
||||
## Replay et requêtes bornées
|
||||
|
||||
```rust
|
||||
let candidates = match store.replay_transaction_candidates(filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
for candidate in candidates {
|
||||
println!(
|
||||
"signature={} slot={}",
|
||||
candidate.signature,
|
||||
candidate.slot
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
let programs = match store.replay_program_summaries(program_filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
let entities = match store.replay_entity_summaries(entity_filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
println!("programs={}, entities={}", programs.len(), entities.len());
|
||||
```
|
||||
|
||||
Les filtres refusent les limites nulles ou supérieures aux bornes publiques. `PageRequest` impose également `DEFAULT_PAGE_SIZE` et `MAX_PAGE_SIZE`.
|
||||
|
||||
## Transactions atomiques
|
||||
|
||||
Les bundles `CoreExtractionBundle`, `DecodePersistenceBundle` et `MaterializationPersistenceBundle` regroupent les écritures qui doivent réussir ou être annulées ensemble. Les consommateurs ne doivent pas reproduire manuellement ces transactions avec des écritures isolées.
|
||||
|
||||
## Validation des noms de tables
|
||||
|
||||
```rust
|
||||
if let Err(error) = ks_store::validate_raw_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = ks_store::validate_core_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = ks_store::validate_decode_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
|
||||
assert!(ks_store::is_valid_solana_table_name(
|
||||
ks_store::CORE_TRANSACTIONS_TABLE_NAME
|
||||
));
|
||||
```
|
||||
|
||||
## Schéma et tables
|
||||
|
||||
Les constantes `RAW_STORE_TABLE_NAMES`, `CORE_STORE_TABLE_NAMES` et `DECODE_STORE_TABLE_NAMES` exposent les noms canoniques. Les fonctions `validate_*_table_names` et `is_valid_solana_table_name` servent aux audits et outils d’administration.
|
||||
|
||||
## Erreurs et invariants
|
||||
|
||||
Toutes les APIs utilisent `ks_core::Result`. Les DTO valident notamment les signatures, chemins, Program IDs, clés, montants, états de traitement et identités de ledger. Les erreurs de contrat peuvent être construites avec `storage_contract_error`.
|
||||
|
||||
## Tests instructifs
|
||||
|
||||
- les tests `*_rejects_*` documentent les invariants des DTO et filtres ;
|
||||
- `decoded_events_and_ledger_roll_back_together` vérifie l’atomicité ;
|
||||
- `optional_postgres_*_from_env` couvre les parcours réels lorsque l’environnement PostgreSQL est configuré ;
|
||||
- les tests de schéma vérifient les noms, index, contraintes et migrations.
|
||||
|
||||
## Limites
|
||||
|
||||
- backend actif : PostgreSQL ;
|
||||
- les tests réels sont optionnels sans DSN de test ;
|
||||
- la crate ne prend aucune décision de décodage ou de matérialisation.
|
||||
21
ks-store/maintenance/drop_raw_core_store.sql
Normal file
21
ks-store/maintenance/drop_raw_core_store.sql
Normal file
@@ -0,0 +1,21 @@
|
||||
-- file: ks-store/maintenance/drop_raw_core_store.sql
|
||||
-- version: 2
|
||||
|
||||
-- Drop raw and core Solana storage tables in reverse dependency order.
|
||||
-- This script is intended for local development and test databases only.
|
||||
-- It does not drop unrelated schemas, migrations, roles or configuration state.
|
||||
-- It intentionally avoids CASCADE so unexpected dependencies fail loudly.
|
||||
-- No custom PostgreSQL enum/domain types are owned by these 0.2.3/0.2.4 tables.
|
||||
|
||||
BEGIN;
|
||||
|
||||
DROP TABLE IF EXISTS kb_sol_core_balance_changes;
|
||||
DROP TABLE IF EXISTS kb_sol_core_logs;
|
||||
DROP TABLE IF EXISTS kb_sol_core_inner_instructions;
|
||||
DROP TABLE IF EXISTS kb_sol_core_instructions;
|
||||
DROP TABLE IF EXISTS kb_sol_core_account_keys;
|
||||
DROP TABLE IF EXISTS kb_sol_core_transactions;
|
||||
DROP TABLE IF EXISTS kb_sol_obs_transaction_observations;
|
||||
DROP TABLE IF EXISTS kb_sol_raw_transactions;
|
||||
|
||||
COMMIT;
|
||||
30
ks-store/maintenance/reset_derived_store_keep_raw.sql
Normal file
30
ks-store/maintenance/reset_derived_store_keep_raw.sql
Normal file
@@ -0,0 +1,30 @@
|
||||
-- file: ks-store/maintenance/reset_derived_store_keep_raw.sql
|
||||
-- version: 3
|
||||
|
||||
-- Reset every reconstructible Solana projection while preserving acquisition data.
|
||||
-- Intended for controlled local/research rebuilds only.
|
||||
-- Preserved tables:
|
||||
-- - kb_sol_raw_transactions
|
||||
-- - kb_sol_obs_transaction_observations
|
||||
-- No CASCADE is used: unexpected dependencies fail loudly.
|
||||
|
||||
BEGIN;
|
||||
TRUNCATE TABLE
|
||||
kb_sol_mat_events,
|
||||
kb_sol_decode_coverage_observations,
|
||||
kb_sol_decode_coverage_declarations,
|
||||
kb_sol_decode_events,
|
||||
kb_sol_ops_processing_ledger,
|
||||
kb_sol_core_balance_changes,
|
||||
kb_sol_core_logs,
|
||||
kb_sol_core_inner_instructions,
|
||||
kb_sol_core_instructions,
|
||||
kb_sol_core_account_keys,
|
||||
kb_sol_core_transactions
|
||||
RESTART IDENTITY;
|
||||
UPDATE kb_sol_raw_transactions
|
||||
SET
|
||||
processing_state = 'received',
|
||||
lifecycle_reason = NULL,
|
||||
updated_at = NOW();
|
||||
COMMIT;
|
||||
94
ks-store/migrations/0001_canonical_transaction_store.sql
Normal file
94
ks-store/migrations/0001_canonical_transaction_store.sql
Normal file
@@ -0,0 +1,94 @@
|
||||
-- file: ks-store/migrations/0001_canonical_transaction_store.sql
|
||||
-- version: 2
|
||||
|
||||
-- Current canonical transaction and acquisition observation store.
|
||||
-- This baseline contains only active tables and uses the current PostgreSQL schema.
|
||||
-- Historical 0.2.x upgrade logic was validated before this baseline was consolidated.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_raw_transactions (
|
||||
id BIGSERIAL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
canonical_json JSONB,
|
||||
canonical_json_hash TEXT,
|
||||
canonical_format_version INTEGER NOT NULL DEFAULT 1,
|
||||
retention_state TEXT NOT NULL DEFAULT 'full',
|
||||
processing_state TEXT NOT NULL DEFAULT 'received',
|
||||
lifecycle_reason TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_raw_transactions PRIMARY KEY (id),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_format_version_positive CHECK (canonical_format_version > 0),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_retention_state CHECK (retention_state IN ('full', 'compacted', 'archived', 'purged')),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_processing_state CHECK (processing_state IN ('received', 'core_extracted', 'decoded', 'materialized', 'failed')),
|
||||
CONSTRAINT ck_kb_sol_raw_transactions_full_has_json CHECK (retention_state <> 'full' OR canonical_json IS NOT NULL)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_raw_transactions_signature ON kb_sol_raw_transactions (signature);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_slot ON kb_sol_raw_transactions (slot);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_created_at ON kb_sol_raw_transactions (created_at);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_processing ON kb_sol_raw_transactions (processing_state);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_canonical_hash ON kb_sol_raw_transactions (canonical_json_hash) WHERE canonical_json_hash IS NOT NULL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_obs_transaction_observations (
|
||||
id BIGSERIAL,
|
||||
raw_transaction_id BIGINT,
|
||||
observation_key TEXT NOT NULL,
|
||||
signature TEXT,
|
||||
slot BIGINT,
|
||||
provider TEXT NOT NULL,
|
||||
endpoint_code TEXT,
|
||||
protocol TEXT NOT NULL,
|
||||
acquisition_method TEXT NOT NULL,
|
||||
origin TEXT NOT NULL,
|
||||
commitment TEXT,
|
||||
capture_session_id TEXT,
|
||||
filter_code TEXT,
|
||||
detected_at TIMESTAMPTZ,
|
||||
received_at TIMESTAMPTZ NOT NULL,
|
||||
normalized_at TIMESTAMPTZ,
|
||||
persisted_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
payload_size_bytes BIGINT,
|
||||
source_payload_hash TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'received',
|
||||
error_code TEXT,
|
||||
error_message TEXT,
|
||||
CONSTRAINT pk_kb_sol_obs_transaction_observations PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_obs_transaction_observations_raw_transaction FOREIGN KEY (raw_transaction_id) REFERENCES kb_sol_raw_transactions(id) ON DELETE SET NULL,
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_key_not_empty CHECK (length(btrim(observation_key)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_signature_not_empty CHECK (signature IS NULL OR length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_slot_non_negative CHECK (slot IS NULL OR slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_provider_not_empty CHECK (length(btrim(provider)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_endpoint_not_empty CHECK (endpoint_code IS NULL OR length(btrim(endpoint_code)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_protocol_not_empty CHECK (length(btrim(protocol)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_method_not_empty CHECK (length(btrim(acquisition_method)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_origin CHECK (origin IN ('live', 'backfill', 'replay', 'repair', 'migration')),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_commitment_not_empty CHECK (commitment IS NULL OR length(btrim(commitment)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_session_not_empty CHECK (capture_session_id IS NULL OR length(btrim(capture_session_id)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_filter_not_empty CHECK (filter_code IS NULL OR length(btrim(filter_code)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_payload_size_non_negative CHECK (payload_size_bytes IS NULL OR payload_size_bytes >= 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_hash_not_empty CHECK (source_payload_hash IS NULL OR length(btrim(source_payload_hash)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_status CHECK (status IN ('detected', 'received', 'normalized', 'persisted', 'failed', 'missing')),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_error_code_not_empty CHECK (error_code IS NULL OR length(btrim(error_code)) > 0),
|
||||
CONSTRAINT ck_kb_sol_obs_transaction_observations_error_message_not_empty CHECK (error_message IS NULL OR length(btrim(error_message)) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_obs_transaction_observations_key ON kb_sol_obs_transaction_observations (observation_key);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_signature ON kb_sol_obs_transaction_observations (signature) WHERE signature IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_slot ON kb_sol_obs_transaction_observations (slot) WHERE slot IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_provider_method ON kb_sol_obs_transaction_observations (provider, acquisition_method);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_received_at ON kb_sol_obs_transaction_observations (received_at);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_raw_transaction ON kb_sol_obs_transaction_observations (raw_transaction_id) WHERE raw_transaction_id IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_status ON kb_sol_obs_transaction_observations (status);
|
||||
196
ks-store/migrations/0002_core_store.sql
Normal file
196
ks-store/migrations/0002_core_store.sql
Normal file
@@ -0,0 +1,196 @@
|
||||
-- file: ks-store/migrations/0002_core_store.sql
|
||||
-- version: 2
|
||||
|
||||
-- Current normalized Solana core store.
|
||||
-- This migration uses the current PostgreSQL schema from the active profile.
|
||||
-- It must not create application schemas such as raw, core, obs, decode, mat or catalog.
|
||||
-- Object naming convention: pk_ for primary keys, fk_ for foreign keys, ux_ for unique indexes and ix_ for non-unique indexes.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_transactions (
|
||||
id BIGSERIAL,
|
||||
raw_transaction_id BIGINT,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
failed BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
err_json JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_transactions PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_transactions_raw_transaction FOREIGN KEY (raw_transaction_id) REFERENCES kb_sol_raw_transactions(id) ON DELETE SET NULL,
|
||||
CONSTRAINT ck_kb_sol_core_transactions_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_transactions_slot_non_negative CHECK (slot >= 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_transactions_signature
|
||||
ON kb_sol_core_transactions (signature);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_slot
|
||||
ON kb_sol_core_transactions (slot);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_created_at
|
||||
ON kb_sol_core_transactions (created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_account_keys (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
account_index INTEGER NOT NULL,
|
||||
account_key TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
writable BOOLEAN NOT NULL,
|
||||
signer BOOLEAN NOT NULL,
|
||||
executable BOOLEAN,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_account_keys PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_account_keys_transaction FOREIGN KEY (transaction_id) REFERENCES kb_sol_core_transactions(id) ON DELETE CASCADE,
|
||||
CONSTRAINT ck_kb_sol_core_account_keys_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_account_keys_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_account_keys_index_non_negative CHECK (account_index >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_account_keys_key_not_empty CHECK (length(btrim(account_key)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_account_keys_source CHECK (source IN ('static', 'loaded_writable', 'loaded_readonly'))
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_account_keys_sig_index
|
||||
ON kb_sol_core_account_keys (signature, account_index);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_account_keys_key
|
||||
ON kb_sol_core_account_keys (account_key);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_instructions (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
instruction_path TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL,
|
||||
accounts_json JSONB NOT NULL,
|
||||
payload_json JSONB,
|
||||
payload_json_hash TEXT,
|
||||
processing_state TEXT NOT NULL DEFAULT 'pending',
|
||||
processor_name TEXT,
|
||||
processor_version TEXT,
|
||||
lifecycle_reason TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_instructions PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_instructions_transaction FOREIGN KEY (transaction_id) REFERENCES kb_sol_core_transactions(id) ON DELETE CASCADE,
|
||||
CONSTRAINT ck_kb_sol_core_instructions_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_instructions_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_instructions_path_not_empty CHECK (length(btrim(instruction_path)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_instructions_program_not_empty CHECK (length(btrim(program_id)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_instructions_processing_state CHECK (processing_state IN ('pending', 'decoded', 'materialized', 'ignored', 'failed', 'replay_requested'))
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_instructions_sig_path
|
||||
ON kb_sol_core_instructions (signature, instruction_path);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_program
|
||||
ON kb_sol_core_instructions (program_id);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_slot
|
||||
ON kb_sol_core_instructions (slot);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_processing
|
||||
ON kb_sol_core_instructions (processing_state);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_inner_instructions (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
parent_instruction_path TEXT NOT NULL,
|
||||
instruction_path TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL,
|
||||
accounts_json JSONB NOT NULL,
|
||||
payload_json JSONB,
|
||||
payload_json_hash TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_inner_instructions PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_inner_instructions_transaction FOREIGN KEY (transaction_id) REFERENCES kb_sol_core_transactions(id) ON DELETE CASCADE,
|
||||
CONSTRAINT ck_kb_sol_core_inner_instructions_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_inner_instructions_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_inner_instructions_parent_not_empty CHECK (length(btrim(parent_instruction_path)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_inner_instructions_path_not_empty CHECK (length(btrim(instruction_path)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_inner_instructions_program_not_empty CHECK (length(btrim(program_id)) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_inner_instructions_sig_path
|
||||
ON kb_sol_core_inner_instructions (signature, instruction_path);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_parent
|
||||
ON kb_sol_core_inner_instructions (signature, parent_instruction_path);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_program
|
||||
ON kb_sol_core_inner_instructions (program_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_logs (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
log_index INTEGER NOT NULL,
|
||||
instruction_path TEXT,
|
||||
program_id TEXT,
|
||||
log_text TEXT,
|
||||
log_text_hash TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_logs PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_logs_transaction FOREIGN KEY (transaction_id) REFERENCES kb_sol_core_transactions(id) ON DELETE CASCADE,
|
||||
CONSTRAINT ck_kb_sol_core_logs_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_logs_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_logs_index_non_negative CHECK (log_index >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_logs_text_not_empty CHECK (log_text IS NULL OR length(btrim(log_text)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_logs_path_not_empty CHECK (instruction_path IS NULL OR length(btrim(instruction_path)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_logs_program_not_empty CHECK (program_id IS NULL OR length(btrim(program_id)) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_logs_sig_index
|
||||
ON kb_sol_core_logs (signature, log_index);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_program
|
||||
ON kb_sol_core_logs (program_id)
|
||||
WHERE program_id IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_path
|
||||
ON kb_sol_core_logs (signature, instruction_path)
|
||||
WHERE instruction_path IS NOT NULL;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_core_balance_changes (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
balance_change_index INTEGER NOT NULL,
|
||||
balance_kind TEXT NOT NULL,
|
||||
account_index INTEGER,
|
||||
account_key TEXT,
|
||||
mint TEXT,
|
||||
owner TEXT,
|
||||
pre_balance_json JSONB,
|
||||
post_balance_json JSONB,
|
||||
delta_json JSONB,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_core_balance_changes PRIMARY KEY (id),
|
||||
CONSTRAINT fk_kb_sol_core_balance_changes_transaction FOREIGN KEY (transaction_id) REFERENCES kb_sol_core_transactions(id) ON DELETE CASCADE,
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_signature_not_empty CHECK (length(btrim(signature)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_index_non_negative CHECK (balance_change_index >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_account_index_non_negative CHECK (account_index IS NULL OR account_index >= 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_kind CHECK (balance_kind IN ('native_lamports', 'token_amount')),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_account_not_empty CHECK (account_key IS NULL OR length(btrim(account_key)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_mint_not_empty CHECK (mint IS NULL OR length(btrim(mint)) > 0),
|
||||
CONSTRAINT ck_kb_sol_core_balance_changes_owner_not_empty CHECK (owner IS NULL OR length(btrim(owner)) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_balance_changes_sig_index
|
||||
ON kb_sol_core_balance_changes (signature, balance_change_index);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_account
|
||||
ON kb_sol_core_balance_changes (account_key)
|
||||
WHERE account_key IS NOT NULL;
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_mint
|
||||
ON kb_sol_core_balance_changes (mint)
|
||||
WHERE mint IS NOT NULL;
|
||||
|
||||
41
ks-store/migrations/0003_processing_ledger.sql
Normal file
41
ks-store/migrations/0003_processing_ledger.sql
Normal file
@@ -0,0 +1,41 @@
|
||||
-- file: ks-store/migrations/0003_processing_ledger.sql
|
||||
-- version: 2
|
||||
|
||||
-- Current processing ledger for canonical to core extraction and later pipeline stages.
|
||||
-- This migration uses the current PostgreSQL schema from the active profile.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_ops_processing_ledger (
|
||||
id BIGSERIAL,
|
||||
stage TEXT NOT NULL,
|
||||
processor_name TEXT NOT NULL,
|
||||
processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL,
|
||||
input_hash TEXT NOT NULL,
|
||||
status TEXT NOT NULL,
|
||||
attempt_count INTEGER NOT NULL DEFAULT 0,
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
error_code TEXT,
|
||||
error_message TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_ops_processing_ledger PRIMARY KEY (id),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_stage_not_empty CHECK (length(btrim(stage)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_processor_name_not_empty CHECK (length(btrim(processor_name)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_processor_version_not_empty CHECK (length(btrim(processor_version)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_input_key_not_empty CHECK (length(btrim(input_key)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_input_hash_not_empty CHECK (length(btrim(input_hash)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_status CHECK (status IN ('running', 'succeeded', 'failed')),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_attempt_count_non_negative CHECK (attempt_count >= 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_error_code_not_empty CHECK (error_code IS NULL OR length(btrim(error_code)) > 0),
|
||||
CONSTRAINT ck_kb_sol_ops_processing_ledger_error_message_not_empty CHECK (error_message IS NULL OR length(btrim(error_message)) > 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_ops_processing_ledger_identity
|
||||
ON kb_sol_ops_processing_ledger (stage, processor_name, processor_version, input_key);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_status
|
||||
ON kb_sol_ops_processing_ledger (stage, processor_name, status, updated_at);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_input_hash
|
||||
ON kb_sol_ops_processing_ledger (input_hash);
|
||||
132
ks-store/migrations/0004_decode_materialization_store.sql
Normal file
132
ks-store/migrations/0004_decode_materialization_store.sql
Normal file
@@ -0,0 +1,132 @@
|
||||
-- file: ks-store/migrations/0004_decode_materialization_store.sql
|
||||
-- version: 2
|
||||
|
||||
-- Common versioned decode, coverage and materialization store.
|
||||
-- This migration uses the current PostgreSQL schema from the active profile.
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_decode_events (
|
||||
id BIGSERIAL,
|
||||
processor_name TEXT NOT NULL,
|
||||
processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL,
|
||||
input_hash TEXT NOT NULL,
|
||||
event_key TEXT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
instruction_path TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL,
|
||||
protocol_code TEXT NOT NULL,
|
||||
surface_code TEXT NOT NULL,
|
||||
event_code TEXT NOT NULL,
|
||||
event_name TEXT NOT NULL,
|
||||
event_family TEXT NOT NULL,
|
||||
source_kind TEXT NOT NULL,
|
||||
confidence TEXT NOT NULL,
|
||||
proof_kind TEXT NOT NULL,
|
||||
proof_jsonb JSONB NOT NULL,
|
||||
payload_jsonb JSONB NOT NULL,
|
||||
transaction_failed BOOLEAN NOT NULL,
|
||||
transaction_error_jsonb JSONB,
|
||||
observation_committed BOOLEAN NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_decode_events PRIMARY KEY (id),
|
||||
CONSTRAINT ck_kb_sol_decode_events_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_decode_events_failed_not_committed CHECK (NOT transaction_failed OR NOT observation_committed)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_events_processor_input_event
|
||||
ON kb_sol_decode_events (processor_name, processor_version, input_key, event_key);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_signature_path
|
||||
ON kb_sol_decode_events (signature, instruction_path);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_program_surface
|
||||
ON kb_sol_decode_events (program_id, surface_code, event_code);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_family_commit
|
||||
ON kb_sol_decode_events (event_family, transaction_failed, observation_committed);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_declarations (
|
||||
id BIGSERIAL,
|
||||
processor_name TEXT NOT NULL,
|
||||
processor_version TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL,
|
||||
surface_code TEXT,
|
||||
entry_kind TEXT NOT NULL,
|
||||
entry_code TEXT NOT NULL,
|
||||
discriminator_hex TEXT,
|
||||
historical BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_decode_coverage_declarations PRIMARY KEY (id)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_declarations_identity
|
||||
ON kb_sol_decode_coverage_declarations (
|
||||
processor_name,
|
||||
processor_version,
|
||||
program_id,
|
||||
COALESCE(surface_code, ''),
|
||||
entry_kind,
|
||||
entry_code,
|
||||
COALESCE(discriminator_hex, '')
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_declarations_program
|
||||
ON kb_sol_decode_coverage_declarations (program_id, processor_name, processor_version);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_observations (
|
||||
id BIGSERIAL,
|
||||
processor_name TEXT NOT NULL,
|
||||
processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL,
|
||||
input_hash TEXT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
instruction_path TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL,
|
||||
surface_code TEXT,
|
||||
entry_code TEXT,
|
||||
discriminator_hex TEXT,
|
||||
status TEXT NOT NULL,
|
||||
recognized BOOLEAN NOT NULL,
|
||||
decoded_count INTEGER NOT NULL DEFAULT 0,
|
||||
materialized_count INTEGER NOT NULL DEFAULT 0,
|
||||
error_count INTEGER NOT NULL DEFAULT 0,
|
||||
transaction_failed BOOLEAN NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_decode_coverage_observations PRIMARY KEY (id),
|
||||
CONSTRAINT ck_kb_sol_decode_coverage_observations_slot_non_negative CHECK (slot >= 0),
|
||||
CONSTRAINT ck_kb_sol_decode_coverage_observations_counts_non_negative CHECK (decoded_count >= 0 AND materialized_count >= 0 AND error_count >= 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_observations_identity
|
||||
ON kb_sol_decode_coverage_observations (processor_name, processor_version, input_key);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_program_status
|
||||
ON kb_sol_decode_coverage_observations (program_id, status, transaction_failed);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_entry
|
||||
ON kb_sol_decode_coverage_observations (processor_name, processor_version, surface_code, entry_code);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS kb_sol_mat_events (
|
||||
id BIGSERIAL,
|
||||
processor_name TEXT NOT NULL,
|
||||
processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL,
|
||||
input_hash TEXT NOT NULL,
|
||||
output_key TEXT NOT NULL,
|
||||
source_event_key TEXT NOT NULL,
|
||||
source_decoder_name TEXT NOT NULL,
|
||||
source_decoder_version TEXT NOT NULL,
|
||||
source_decode_input_key TEXT NOT NULL,
|
||||
signature TEXT NOT NULL,
|
||||
slot BIGINT NOT NULL,
|
||||
materialized_family TEXT NOT NULL,
|
||||
payload_jsonb JSONB NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT NOW(),
|
||||
CONSTRAINT pk_kb_sol_mat_events PRIMARY KEY (id),
|
||||
CONSTRAINT ck_kb_sol_mat_events_slot_non_negative CHECK (slot >= 0)
|
||||
);
|
||||
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_mat_events_processor_input_output
|
||||
ON kb_sol_mat_events (processor_name, processor_version, input_key, output_key);
|
||||
CREATE INDEX IF NOT EXISTS ix_kb_sol_mat_events_signature_family
|
||||
ON kb_sol_mat_events (source_decoder_name, source_decoder_version, source_decode_input_key, signature, materialized_family);
|
||||
7
ks-store/src/constants.rs
Normal file
7
ks-store/src/constants.rs
Normal file
@@ -0,0 +1,7 @@
|
||||
// file: ks-store/src/constants.rs
|
||||
// version: 2
|
||||
|
||||
//! Local constants for the `ks-store` crate.
|
||||
|
||||
/// Canonical tracing target for this crate.
|
||||
pub(crate) const TRACING_TARGET: &str = "ks-store";
|
||||
152
ks-store/src/contracts.rs
Normal file
152
ks-store/src/contracts.rs
Normal file
@@ -0,0 +1,152 @@
|
||||
// file: ks-store/src/contracts.rs
|
||||
// version: 3
|
||||
|
||||
//! Backend-neutral storage contracts used by pipeline crates.
|
||||
|
||||
mod dto;
|
||||
mod entity;
|
||||
mod error;
|
||||
mod health;
|
||||
mod pagination;
|
||||
mod repository;
|
||||
|
||||
/// Core account key insert contract.
|
||||
pub use self::dto::CoreAccountKeyInsert;
|
||||
/// Core account key source category.
|
||||
pub use self::dto::CoreAccountKeySource;
|
||||
/// Core balance change insert contract.
|
||||
pub use self::dto::CoreBalanceChangeInsert;
|
||||
/// Core balance change kind.
|
||||
pub use self::dto::CoreBalanceChangeKind;
|
||||
/// Complete normalized core extraction write bundle.
|
||||
pub use self::dto::CoreExtractionBundle;
|
||||
/// Failure details persisted for one canonical to core extraction attempt.
|
||||
pub use self::dto::CoreExtractionFailure;
|
||||
/// Bounded canonical transaction selection filter for core extraction.
|
||||
pub use self::dto::CoreExtractionSelectionFilter;
|
||||
/// Core inner instruction insert contract.
|
||||
pub use self::dto::CoreInnerInstructionInsert;
|
||||
/// Core instruction insert contract.
|
||||
pub use self::dto::CoreInstructionInsert;
|
||||
/// Core instruction lifecycle mark request.
|
||||
pub use self::dto::CoreInstructionLifecycleMark;
|
||||
/// Core instruction processing state.
|
||||
pub use self::dto::CoreInstructionProcessingState;
|
||||
/// Core instruction replay filter contract.
|
||||
pub use self::dto::CoreInstructionReplayFilter;
|
||||
/// Core log insert contract.
|
||||
pub use self::dto::CoreLogInsert;
|
||||
/// Core transaction insert contract.
|
||||
pub use self::dto::CoreTransactionInsert;
|
||||
/// One machine-readable decoder coverage declaration row.
|
||||
pub use self::dto::DecodeCoverageDeclarationInsert;
|
||||
/// One observed coverage classification row owned by one decode attempt.
|
||||
pub use self::dto::DecodeCoverageObservationInsert;
|
||||
/// One row of aggregated decoder coverage diagnostics.
|
||||
pub use self::dto::DecodeCoverageSummaryRow;
|
||||
/// Failed decode attempt persisted in the common ledger.
|
||||
pub use self::dto::DecodeFailure;
|
||||
/// One processor-owned decoded observation row.
|
||||
pub use self::dto::DecodeObservationInsert;
|
||||
/// Atomic persistence bundle for one decoder and one contextual input.
|
||||
pub use self::dto::DecodePersistenceBundle;
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
pub use self::dto::DecodeSelectionFilter;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::dto::DecodedEventInsert;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::dto::InsertOutcome;
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
pub use self::dto::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
|
||||
/// Atomic persistence bundle for one materializer and one decoded observation.
|
||||
pub use self::dto::MaterializationPersistenceBundle;
|
||||
/// Bounded read-only materialized event selection.
|
||||
pub use self::dto::MaterializedEventFilter;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::dto::MaterializedEventInsert;
|
||||
/// One materialized output returned by a bounded query.
|
||||
pub use self::dto::MaterializedEventQueryRow;
|
||||
/// One processor-owned materialized output row.
|
||||
pub use self::dto::MaterializedOutputInsert;
|
||||
/// Stable processing ledger identity.
|
||||
pub use self::dto::ProcessingLedgerIdentity;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::dto::ProcessingLedgerMark;
|
||||
/// Stable processing ledger status.
|
||||
pub use self::dto::ProcessingLedgerStatus;
|
||||
/// Raw payload lifecycle mark request.
|
||||
pub use self::dto::RawPayloadLifecycleMark;
|
||||
/// Raw payload processing state.
|
||||
pub use self::dto::RawPayloadProcessingState;
|
||||
/// Raw payload retention state.
|
||||
pub use self::dto::RawPayloadRetentionState;
|
||||
/// Canonical raw Solana transaction insert contract.
|
||||
pub use self::dto::RawTransactionInsert;
|
||||
/// Store backend diagnostic contract.
|
||||
pub use self::dto::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::dto::StoreBackendKind;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::dto::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::dto::StoreMigrationStatus;
|
||||
/// Transaction acquisition observation insert contract.
|
||||
pub use self::dto::TransactionObservationInsert;
|
||||
/// Transaction acquisition observation origin.
|
||||
pub use self::dto::TransactionObservationOrigin;
|
||||
/// Transaction acquisition observation status.
|
||||
pub use self::dto::TransactionObservationStatus;
|
||||
/// Core account key SQL-like row contract.
|
||||
pub use self::entity::CoreAccountKeyRow;
|
||||
/// Core balance change SQL-like row contract.
|
||||
pub use self::entity::CoreBalanceChangeRow;
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
pub use self::entity::CoreInnerInstructionRow;
|
||||
/// Core instruction SQL-like row contract.
|
||||
pub use self::entity::CoreInstructionRow;
|
||||
/// Core log SQL-like row contract.
|
||||
pub use self::entity::CoreLogRow;
|
||||
/// Core transaction SQL-like row contract.
|
||||
pub use self::entity::CoreTransactionRow;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::entity::DecodedEventRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::entity::MaterializedEventRow;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::entity::ProcessingLedgerRow;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
pub use self::entity::RawTransactionRow;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
pub use self::entity::TransactionObservationRow;
|
||||
/// Storage error helper functions.
|
||||
pub use self::error::storage_contract_error;
|
||||
/// Store backend health snapshot.
|
||||
pub use self::health::StoreHealthSnapshot;
|
||||
/// Store backend health status.
|
||||
pub use self::health::StoreHealthStatus;
|
||||
/// Default repository page size.
|
||||
pub use self::pagination::DEFAULT_PAGE_SIZE;
|
||||
/// Maximum repository page size.
|
||||
pub use self::pagination::MAX_PAGE_SIZE;
|
||||
/// Page request contract for repository list operations.
|
||||
pub use self::pagination::PageRequest;
|
||||
/// Sort direction for repository list operations.
|
||||
pub use self::pagination::SortDirection;
|
||||
/// Canonical transaction to core extraction storage behavior.
|
||||
pub use self::repository::CoreExtractionStore;
|
||||
/// Core Solana storage behavior.
|
||||
pub use self::repository::CoreTransactionStore;
|
||||
/// Contextual instruction decode and materialization storage behavior.
|
||||
pub use self::repository::DecodePipelineStore;
|
||||
/// Decoded event storage behavior.
|
||||
pub use self::repository::DecodedEventStore;
|
||||
/// Materialized event storage behavior.
|
||||
pub use self::repository::MaterializedEventStore;
|
||||
/// Processing ledger storage behavior.
|
||||
pub use self::repository::ProcessingLedgerStore;
|
||||
/// Program observation storage behavior.
|
||||
pub use self::repository::ProgramObservationStore;
|
||||
/// Raw transaction storage behavior.
|
||||
pub use self::repository::RawTransactionStore;
|
||||
/// Store health storage behavior.
|
||||
pub use self::repository::StoreHealthStore;
|
||||
99
ks-store/src/contracts/dto.rs
Normal file
99
ks-store/src/contracts/dto.rs
Normal file
@@ -0,0 +1,99 @@
|
||||
// file: ks-store/src/contracts/dto.rs
|
||||
// version: 3
|
||||
|
||||
//! Backend-neutral DTO exports for storage repository contracts.
|
||||
|
||||
mod core;
|
||||
mod core_extraction;
|
||||
mod decode;
|
||||
mod event;
|
||||
mod ledger;
|
||||
mod raw;
|
||||
mod store;
|
||||
|
||||
/// Core account key insert contract.
|
||||
pub use self::core::CoreAccountKeyInsert;
|
||||
/// Core account key source category.
|
||||
pub use self::core::CoreAccountKeySource;
|
||||
/// Core balance change insert contract.
|
||||
pub use self::core::CoreBalanceChangeInsert;
|
||||
/// Core balance change kind.
|
||||
pub use self::core::CoreBalanceChangeKind;
|
||||
/// Core inner instruction insert contract.
|
||||
pub use self::core::CoreInnerInstructionInsert;
|
||||
/// Core instruction insert contract.
|
||||
pub use self::core::CoreInstructionInsert;
|
||||
/// Core instruction lifecycle mark request.
|
||||
pub use self::core::CoreInstructionLifecycleMark;
|
||||
/// Core instruction processing state.
|
||||
pub use self::core::CoreInstructionProcessingState;
|
||||
/// Core instruction replay filter contract.
|
||||
pub use self::core::CoreInstructionReplayFilter;
|
||||
/// Core log insert contract.
|
||||
pub use self::core::CoreLogInsert;
|
||||
/// Core transaction insert contract.
|
||||
pub use self::core::CoreTransactionInsert;
|
||||
/// Complete normalized core extraction write bundle.
|
||||
pub use self::core_extraction::CoreExtractionBundle;
|
||||
/// Failure details persisted for one canonical to core extraction attempt.
|
||||
pub use self::core_extraction::CoreExtractionFailure;
|
||||
/// Bounded canonical transaction selection filter for core extraction.
|
||||
pub use self::core_extraction::CoreExtractionSelectionFilter;
|
||||
/// Stable processing ledger identity.
|
||||
pub use self::core_extraction::ProcessingLedgerIdentity;
|
||||
/// Stable processing ledger status.
|
||||
pub use self::core_extraction::ProcessingLedgerStatus;
|
||||
/// One machine-readable decoder coverage declaration row.
|
||||
pub use self::decode::DecodeCoverageDeclarationInsert;
|
||||
/// One observed coverage classification row owned by one decode attempt.
|
||||
pub use self::decode::DecodeCoverageObservationInsert;
|
||||
/// One row of aggregated decoder coverage diagnostics.
|
||||
pub use self::decode::DecodeCoverageSummaryRow;
|
||||
/// Failed decode attempt persisted in the common ledger.
|
||||
pub use self::decode::DecodeFailure;
|
||||
/// One processor-owned decoded observation row.
|
||||
pub use self::decode::DecodeObservationInsert;
|
||||
/// Atomic persistence bundle for one decoder and one contextual input.
|
||||
pub use self::decode::DecodePersistenceBundle;
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
pub use self::decode::DecodeSelectionFilter;
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
pub use self::decode::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
|
||||
/// Atomic persistence bundle for one materializer and one decoded observation.
|
||||
pub use self::decode::MaterializationPersistenceBundle;
|
||||
/// Bounded read-only materialized event selection.
|
||||
pub use self::decode::MaterializedEventFilter;
|
||||
/// One materialized output returned by a bounded query.
|
||||
pub use self::decode::MaterializedEventQueryRow;
|
||||
/// One processor-owned materialized output row.
|
||||
pub use self::decode::MaterializedOutputInsert;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::event::DecodedEventInsert;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::event::InsertOutcome;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::event::MaterializedEventInsert;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::ledger::ProcessingLedgerMark;
|
||||
/// Raw payload lifecycle mark request.
|
||||
pub use self::raw::RawPayloadLifecycleMark;
|
||||
/// Raw payload processing state.
|
||||
pub use self::raw::RawPayloadProcessingState;
|
||||
/// Raw payload retention state.
|
||||
pub use self::raw::RawPayloadRetentionState;
|
||||
/// Canonical raw Solana transaction insert contract.
|
||||
pub use self::raw::RawTransactionInsert;
|
||||
/// Transaction acquisition observation insert contract.
|
||||
pub use self::raw::TransactionObservationInsert;
|
||||
/// Transaction acquisition observation origin.
|
||||
pub use self::raw::TransactionObservationOrigin;
|
||||
/// Transaction acquisition observation status.
|
||||
pub use self::raw::TransactionObservationStatus;
|
||||
/// Store backend diagnostic contract.
|
||||
pub use self::store::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::store::StoreBackendKind;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::store::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::store::StoreMigrationStatus;
|
||||
888
ks-store/src/contracts/dto/core.rs
Normal file
888
ks-store/src/contracts/dto/core.rs
Normal file
@@ -0,0 +1,888 @@
|
||||
// file: ks-store/src/contracts/dto/core.rs
|
||||
// version: 2
|
||||
|
||||
//! Core Solana storage DTOs.
|
||||
|
||||
/// Processing state for one normalized core instruction.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum CoreInstructionProcessingState {
|
||||
/// Instruction is available for replay or first processing.
|
||||
Pending,
|
||||
/// Instruction was decoded by at least one decoder version.
|
||||
Decoded,
|
||||
/// Instruction produced materialized outputs.
|
||||
Materialized,
|
||||
/// Instruction is intentionally skipped for the current pipeline policy.
|
||||
Ignored,
|
||||
/// Instruction processing failed and requires diagnostics.
|
||||
Failed,
|
||||
/// Instruction must be replayed even if a previous processor marked it.
|
||||
ReplayRequested,
|
||||
}
|
||||
|
||||
/// Source category for one normalized Solana account key.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum CoreAccountKeySource {
|
||||
/// Account key came from the static transaction message account keys.
|
||||
Static,
|
||||
/// Account key came from loaded writable address table entries.
|
||||
LoadedWritable,
|
||||
/// Account key came from loaded readonly address table entries.
|
||||
LoadedReadonly,
|
||||
}
|
||||
|
||||
/// Balance change family extracted from Solana transaction metadata.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum CoreBalanceChangeKind {
|
||||
/// Native lamports balance change.
|
||||
NativeLamports,
|
||||
/// SPL token balance change.
|
||||
TokenAmount,
|
||||
}
|
||||
|
||||
/// Core transaction insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreTransactionInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Whether the transaction failed on-chain.
|
||||
pub failed: bool,
|
||||
/// Optional raw error JSON extracted from transaction metadata.
|
||||
pub err_json: std::option::Option<serde_json::Value>,
|
||||
/// Optional canonical raw transaction row id used for lineage when available.
|
||||
pub raw_transaction_id: std::option::Option<i64>,
|
||||
}
|
||||
|
||||
impl CoreTransactionInsert {
|
||||
/// Builds a core transaction insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
failed: bool,
|
||||
err_json: std::option::Option<serde_json::Value>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let signature_result = validate_required_text(
|
||||
&signature_value,
|
||||
"core transaction signature must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = signature_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
failed,
|
||||
err_json,
|
||||
raw_transaction_id: std::option::Option::None,
|
||||
});
|
||||
}
|
||||
|
||||
/// Adds canonical raw transaction lineage to an already validated core transaction insert.
|
||||
pub fn with_raw_transaction_id(mut self, raw_transaction_id: i64) -> Self {
|
||||
self.raw_transaction_id = std::option::Option::Some(raw_transaction_id);
|
||||
return self;
|
||||
}
|
||||
}
|
||||
|
||||
/// Core account key insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreAccountKeyInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Stable account index after static and loaded keys are resolved.
|
||||
pub account_index: u32,
|
||||
/// Account public key as non-empty base58 text.
|
||||
pub account_key: std::string::String,
|
||||
/// Source category for this account key.
|
||||
pub source: CoreAccountKeySource,
|
||||
/// Whether the resolved account is writable for the transaction.
|
||||
pub writable: bool,
|
||||
/// Whether the resolved account signed the transaction.
|
||||
pub signer: bool,
|
||||
/// Whether the resolved account is executable when known.
|
||||
pub executable: std::option::Option<bool>,
|
||||
}
|
||||
|
||||
impl CoreAccountKeyInsert {
|
||||
/// Builds a core account key insert contract after minimal validation.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
account_index: u32,
|
||||
account_key: impl std::convert::Into<std::string::String>,
|
||||
source: CoreAccountKeySource,
|
||||
writable: bool,
|
||||
signer: bool,
|
||||
executable: std::option::Option<bool>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let account_key_value = account_key.into();
|
||||
let signature_result = validate_required_text(
|
||||
&signature_value,
|
||||
"core account key signature must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = signature_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let account_key_result =
|
||||
validate_required_text(&account_key_value, "core account key must not be empty");
|
||||
if let std::result::Result::Err(error) = account_key_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
account_index,
|
||||
account_key: account_key_value,
|
||||
source,
|
||||
writable,
|
||||
signer,
|
||||
executable,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Core instruction insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInstructionInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Instruction accounts as JSON, preserving unresolved forms when needed.
|
||||
pub accounts_json: serde_json::Value,
|
||||
/// Instruction payload JSON, preserving raw and partially decoded forms when needed.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Optional deterministic payload JSON hash.
|
||||
pub payload_json_hash: std::option::Option<std::string::String>,
|
||||
/// Initial processing state used by replay schedulers.
|
||||
pub processing_state: CoreInstructionProcessingState,
|
||||
}
|
||||
|
||||
impl CoreInstructionInsert {
|
||||
/// Builds a core instruction insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
instruction_path: impl std::convert::Into<std::string::String>,
|
||||
program_id: impl std::convert::Into<std::string::String>,
|
||||
accounts_json: serde_json::Value,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let instruction_path_value = instruction_path.into();
|
||||
let program_id_value = program_id.into();
|
||||
let validation_result = validate_instruction_identity(
|
||||
signature_value.as_str(),
|
||||
instruction_path_value.as_str(),
|
||||
program_id_value.as_str(),
|
||||
"core instruction",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
instruction_path: instruction_path_value,
|
||||
program_id: program_id_value,
|
||||
accounts_json,
|
||||
payload_json,
|
||||
payload_json_hash: std::option::Option::None,
|
||||
processing_state: CoreInstructionProcessingState::Pending,
|
||||
});
|
||||
}
|
||||
|
||||
/// Adds a deterministic payload JSON hash.
|
||||
pub fn with_payload_json_hash(
|
||||
mut self,
|
||||
payload_json_hash: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let value = payload_json_hash.into();
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core instruction payload hash must not be empty",
|
||||
));
|
||||
}
|
||||
self.payload_json_hash = std::option::Option::Some(value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
}
|
||||
|
||||
/// Core inner instruction insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInnerInstructionInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Parent top-level or inner instruction path.
|
||||
pub parent_instruction_path: std::string::String,
|
||||
/// Stable inner instruction path, for example `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Inner instruction accounts as JSON.
|
||||
pub accounts_json: serde_json::Value,
|
||||
/// Inner instruction payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Optional deterministic payload JSON hash.
|
||||
pub payload_json_hash: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl CoreInnerInstructionInsert {
|
||||
/// Builds a core inner instruction insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
parent_instruction_path: impl std::convert::Into<std::string::String>,
|
||||
instruction_path: impl std::convert::Into<std::string::String>,
|
||||
program_id: impl std::convert::Into<std::string::String>,
|
||||
accounts_json: serde_json::Value,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let parent_instruction_path_value = parent_instruction_path.into();
|
||||
let instruction_path_value = instruction_path.into();
|
||||
let program_id_value = program_id.into();
|
||||
let parent_result = validate_required_text(
|
||||
&parent_instruction_path_value,
|
||||
"core inner instruction parent path must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = parent_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let validation_result = validate_instruction_identity(
|
||||
signature_value.as_str(),
|
||||
instruction_path_value.as_str(),
|
||||
program_id_value.as_str(),
|
||||
"core inner instruction",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
parent_instruction_path: parent_instruction_path_value,
|
||||
instruction_path: instruction_path_value,
|
||||
program_id: program_id_value,
|
||||
accounts_json,
|
||||
payload_json,
|
||||
payload_json_hash: std::option::Option::None,
|
||||
});
|
||||
}
|
||||
|
||||
/// Adds a deterministic payload JSON hash.
|
||||
pub fn with_payload_json_hash(
|
||||
mut self,
|
||||
payload_json_hash: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let value = payload_json_hash.into();
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core inner instruction payload hash must not be empty",
|
||||
));
|
||||
}
|
||||
self.payload_json_hash = std::option::Option::Some(value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
}
|
||||
|
||||
/// Core log insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreLogInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Log index preserving transaction log order.
|
||||
pub log_index: u32,
|
||||
/// Optional instruction path resolved from invocation depth when known.
|
||||
pub instruction_path: std::option::Option<std::string::String>,
|
||||
/// Optional program id resolved from the log line or invocation context.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Original log text.
|
||||
pub log_text: std::string::String,
|
||||
/// Optional deterministic log text hash.
|
||||
pub log_text_hash: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl CoreLogInsert {
|
||||
/// Builds a core log insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
log_index: u32,
|
||||
instruction_path: std::option::Option<std::string::String>,
|
||||
program_id: std::option::Option<std::string::String>,
|
||||
log_text: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let log_text_value = log_text.into();
|
||||
let signature_result =
|
||||
validate_required_text(&signature_value, "core log signature must not be empty");
|
||||
if let std::result::Result::Err(error) = signature_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let log_text_result =
|
||||
validate_required_text(&log_text_value, "core log text must not be empty");
|
||||
if let std::result::Result::Err(error) = log_text_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let instruction_path_result = validate_optional_text(
|
||||
instruction_path.as_deref(),
|
||||
"core log instruction path must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = instruction_path_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let program_id_result = validate_optional_text(
|
||||
program_id.as_deref(),
|
||||
"core log program id must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = program_id_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
log_index,
|
||||
instruction_path,
|
||||
program_id,
|
||||
log_text: log_text_value,
|
||||
log_text_hash: std::option::Option::None,
|
||||
});
|
||||
}
|
||||
|
||||
/// Adds a deterministic log text hash.
|
||||
pub fn with_log_text_hash(
|
||||
mut self,
|
||||
log_text_hash: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let value = log_text_hash.into();
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core log text hash must not be empty",
|
||||
));
|
||||
}
|
||||
self.log_text_hash = std::option::Option::Some(value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
}
|
||||
|
||||
/// Core balance change insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreBalanceChangeInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Stable balance change index preserving extraction order.
|
||||
pub balance_change_index: u32,
|
||||
/// Balance change family.
|
||||
pub balance_kind: CoreBalanceChangeKind,
|
||||
/// Optional account index when available.
|
||||
pub account_index: std::option::Option<u32>,
|
||||
/// Optional account public key when available.
|
||||
pub account_key: std::option::Option<std::string::String>,
|
||||
/// Optional SPL token mint for token balances.
|
||||
pub mint: std::option::Option<std::string::String>,
|
||||
/// Optional owner public key for token balances.
|
||||
pub owner: std::option::Option<std::string::String>,
|
||||
/// Pre-balance JSON value preserving RPC representation.
|
||||
pub pre_balance_json: std::option::Option<serde_json::Value>,
|
||||
/// Post-balance JSON value preserving RPC representation.
|
||||
pub post_balance_json: std::option::Option<serde_json::Value>,
|
||||
/// Delta JSON value preserving integer or decimal-safe representation.
|
||||
pub delta_json: std::option::Option<serde_json::Value>,
|
||||
}
|
||||
|
||||
impl CoreBalanceChangeInsert {
|
||||
/// Builds a core balance change insert contract after minimal validation.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
balance_change_index: u32,
|
||||
balance_kind: CoreBalanceChangeKind,
|
||||
account_index: std::option::Option<u32>,
|
||||
account_key: std::option::Option<std::string::String>,
|
||||
mint: std::option::Option<std::string::String>,
|
||||
owner: std::option::Option<std::string::String>,
|
||||
pre_balance_json: std::option::Option<serde_json::Value>,
|
||||
post_balance_json: std::option::Option<serde_json::Value>,
|
||||
delta_json: std::option::Option<serde_json::Value>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let signature_result = validate_required_text(
|
||||
&signature_value,
|
||||
"core balance change signature must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = signature_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let account_key_result = validate_optional_text(
|
||||
account_key.as_deref(),
|
||||
"core balance change account key must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = account_key_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let mint_result = validate_optional_text(
|
||||
mint.as_deref(),
|
||||
"core balance change mint must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = mint_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let owner_result = validate_optional_text(
|
||||
owner.as_deref(),
|
||||
"core balance change owner must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = owner_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
balance_change_index,
|
||||
balance_kind,
|
||||
account_index,
|
||||
account_key,
|
||||
mint,
|
||||
owner,
|
||||
pre_balance_json,
|
||||
post_balance_json,
|
||||
delta_json,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Core instruction replay filter contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInstructionReplayFilter {
|
||||
/// Optional processing state to select, usually `Pending`, `Failed` or `ReplayRequested`.
|
||||
pub processing_state: std::option::Option<CoreInstructionProcessingState>,
|
||||
/// Optional program id filter.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Optional inclusive minimum slot.
|
||||
pub min_slot: std::option::Option<u64>,
|
||||
/// Optional inclusive maximum slot.
|
||||
pub max_slot: std::option::Option<u64>,
|
||||
}
|
||||
|
||||
impl CoreInstructionReplayFilter {
|
||||
/// Builds a replay filter after minimal validation.
|
||||
pub fn new(
|
||||
processing_state: std::option::Option<CoreInstructionProcessingState>,
|
||||
program_id: std::option::Option<std::string::String>,
|
||||
min_slot: std::option::Option<u64>,
|
||||
max_slot: std::option::Option<u64>,
|
||||
) -> ks_core::Result<Self> {
|
||||
if let std::option::Option::Some(program_id_value) = program_id.as_ref() {
|
||||
if program_id_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core instruction replay program id must not be empty when present",
|
||||
));
|
||||
}
|
||||
}
|
||||
if let (
|
||||
std::option::Option::Some(min_slot_value),
|
||||
std::option::Option::Some(max_slot_value),
|
||||
) = (min_slot, max_slot)
|
||||
{
|
||||
if min_slot_value > max_slot_value {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core instruction replay min slot must be lower than or equal to max slot",
|
||||
));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
processing_state,
|
||||
program_id,
|
||||
min_slot,
|
||||
max_slot,
|
||||
});
|
||||
}
|
||||
|
||||
/// Builds the default pending instruction replay filter.
|
||||
pub fn pending() -> Self {
|
||||
return Self {
|
||||
processing_state: std::option::Option::Some(CoreInstructionProcessingState::Pending),
|
||||
program_id: std::option::Option::None,
|
||||
min_slot: std::option::Option::None,
|
||||
max_slot: std::option::Option::None,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Core instruction lifecycle mark request.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInstructionLifecycleMark {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// New processing state.
|
||||
pub processing_state: CoreInstructionProcessingState,
|
||||
/// Optional processor name that produced the lifecycle transition.
|
||||
pub processor_name: std::option::Option<std::string::String>,
|
||||
/// Optional processor version that produced the lifecycle transition.
|
||||
pub processor_version: std::option::Option<std::string::String>,
|
||||
/// Optional reason visible in diagnostics.
|
||||
pub reason: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl CoreInstructionLifecycleMark {
|
||||
/// Builds a core instruction lifecycle mark after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
instruction_path: impl std::convert::Into<std::string::String>,
|
||||
processing_state: CoreInstructionProcessingState,
|
||||
processor_name: std::option::Option<std::string::String>,
|
||||
processor_version: std::option::Option<std::string::String>,
|
||||
reason: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let instruction_path_value = instruction_path.into();
|
||||
let signature_result = validate_required_text(
|
||||
&signature_value,
|
||||
"core instruction lifecycle signature must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = signature_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let instruction_path_result = validate_required_text(
|
||||
&instruction_path_value,
|
||||
"core instruction lifecycle path must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = instruction_path_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let processor_name_result = validate_optional_text(
|
||||
processor_name.as_deref(),
|
||||
"core instruction lifecycle processor name must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = processor_name_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let processor_version_result = validate_optional_text(
|
||||
processor_version.as_deref(),
|
||||
"core instruction lifecycle processor version must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = processor_version_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let reason_result = validate_optional_text(
|
||||
reason.as_deref(),
|
||||
"core instruction lifecycle reason must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = reason_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
instruction_path: instruction_path_value,
|
||||
processing_state,
|
||||
processor_name,
|
||||
processor_version,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_instruction_identity(
|
||||
signature: &str,
|
||||
instruction_path: &str,
|
||||
program_id: &str,
|
||||
label: &str,
|
||||
) -> ks_core::Result<()> {
|
||||
let signature_result =
|
||||
validate_required_text(signature, "instruction signature must not be empty");
|
||||
if let std::result::Result::Err(_error) = signature_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"{label} signature must not be empty"
|
||||
)));
|
||||
}
|
||||
let instruction_path_result =
|
||||
validate_required_text(instruction_path, "instruction path must not be empty");
|
||||
if let std::result::Result::Err(_error) = instruction_path_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"{label} path must not be empty"
|
||||
)));
|
||||
}
|
||||
let program_id_result =
|
||||
validate_required_text(program_id, "instruction program id must not be empty");
|
||||
if let std::result::Result::Err(_error) = program_id_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"{label} program id must not be empty"
|
||||
)));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_required_text(
|
||||
value: &str,
|
||||
message: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<()> {
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message.into()));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_optional_text(
|
||||
value: std::option::Option<&str>,
|
||||
message: &'static str,
|
||||
) -> ks_core::Result<()> {
|
||||
if let std::option::Option::Some(text_value) = value {
|
||||
if text_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn core_transaction_rejects_empty_signature() {
|
||||
let result = crate::CoreTransactionInsert::new(" ", 1, false, std::option::Option::None);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_transaction_accepts_optional_raw_lineage() {
|
||||
let result = crate::CoreTransactionInsert::new("abc", 1, false, std::option::Option::None);
|
||||
let input = match result {
|
||||
std::result::Result::Ok(value) => value.with_raw_transaction_id(7),
|
||||
std::result::Result::Err(error) => panic!("unexpected transaction error: {error}"),
|
||||
};
|
||||
assert_eq!(input.raw_transaction_id, std::option::Option::Some(7));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_account_key_rejects_empty_key() {
|
||||
let result = crate::CoreAccountKeyInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
0,
|
||||
" ",
|
||||
crate::CoreAccountKeySource::Static,
|
||||
false,
|
||||
false,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_instruction_rejects_empty_program_id() {
|
||||
let result = crate::CoreInstructionInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
"0",
|
||||
" ",
|
||||
serde_json::json!([]),
|
||||
serde_json::json!({}),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_instruction_defaults_to_pending() {
|
||||
let result = crate::CoreInstructionInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
"0",
|
||||
"program",
|
||||
serde_json::json!([]),
|
||||
serde_json::json!({}),
|
||||
);
|
||||
if let std::result::Result::Ok(input) = result {
|
||||
assert_eq!(crate::CoreInstructionProcessingState::Pending, input.processing_state);
|
||||
} else {
|
||||
panic!("core instruction insert should be valid");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_inner_instruction_rejects_empty_parent_path() {
|
||||
let result = crate::CoreInnerInstructionInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
" ",
|
||||
"0/0",
|
||||
"program",
|
||||
serde_json::json!([]),
|
||||
serde_json::json!({}),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_log_rejects_empty_text() {
|
||||
let result = crate::CoreLogInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
0,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
" ",
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn core_balance_change_rejects_empty_optional_mint() {
|
||||
let result = crate::CoreBalanceChangeInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
0,
|
||||
crate::CoreBalanceChangeKind::TokenAmount,
|
||||
std::option::Option::Some(0),
|
||||
std::option::Option::Some("account".to_string()),
|
||||
std::option::Option::Some(" ".to_string()),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_input_rejects_empty_key() {
|
||||
let result = crate::MdCoreInstructionReplayInput::new(
|
||||
" ",
|
||||
"abc",
|
||||
1,
|
||||
"0",
|
||||
"program",
|
||||
false,
|
||||
std::option::Option::None,
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
std::option::Option::Some(serde_json::json!({})),
|
||||
std::option::Option::None,
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_input_accepts_ordered_outer_instruction_array() {
|
||||
let result = crate::MdCoreInstructionReplayInput::new(
|
||||
"abc:2",
|
||||
"abc",
|
||||
1,
|
||||
"2",
|
||||
"program",
|
||||
false,
|
||||
std::option::Option::None,
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
std::option::Option::Some(serde_json::json!({"dataBase64": "AQ=="})),
|
||||
std::option::Option::Some("payload-hash".to_string()),
|
||||
serde_json::json!([
|
||||
{
|
||||
"instructionIndex": 0,
|
||||
"instructionPath": "0",
|
||||
"programId": "other",
|
||||
"payloadJson": {"dataBase64": "Ag=="},
|
||||
"payloadHash": "other-hash"
|
||||
},
|
||||
{
|
||||
"instructionIndex": 2,
|
||||
"instructionPath": "2",
|
||||
"programId": "program",
|
||||
"payloadJson": {"dataBase64": "AQ=="},
|
||||
"payloadHash": "payload-hash"
|
||||
}
|
||||
]),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
);
|
||||
let input = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected replay input error: {error}"),
|
||||
};
|
||||
assert_eq!(input.core_contract_version, 2);
|
||||
assert_eq!(
|
||||
input.outer_instructions_json.as_array().map(std::vec::Vec::len),
|
||||
std::option::Option::Some(2)
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_input_rejects_non_array_outer_instruction_context() {
|
||||
let result = crate::MdCoreInstructionReplayInput::new(
|
||||
"abc:0",
|
||||
"abc",
|
||||
1,
|
||||
"0",
|
||||
"program",
|
||||
false,
|
||||
std::option::Option::None,
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
std::option::Option::Some(serde_json::json!({"dataBase64": "AQ=="})),
|
||||
std::option::Option::None,
|
||||
serde_json::json!({"instructionIndex": 0}),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
serde_json::json!([]),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_filter_rejects_empty_program_id() {
|
||||
let result = crate::CoreInstructionReplayFilter::new(
|
||||
std::option::Option::Some(crate::CoreInstructionProcessingState::Pending),
|
||||
std::option::Option::Some(" ".to_string()),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn replay_filter_rejects_inverted_slots() {
|
||||
let result = crate::CoreInstructionReplayFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::Some(10),
|
||||
std::option::Option::Some(1),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn lifecycle_mark_rejects_empty_path() {
|
||||
let result = crate::CoreInstructionLifecycleMark::new(
|
||||
"signature",
|
||||
" ",
|
||||
crate::CoreInstructionProcessingState::Decoded,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
289
ks-store/src/contracts/dto/core_extraction.rs
Normal file
289
ks-store/src/contracts/dto/core_extraction.rs
Normal file
@@ -0,0 +1,289 @@
|
||||
// file: ks-store/src/contracts/dto/core_extraction.rs
|
||||
// version: 2
|
||||
|
||||
//! Canonical transaction to core extraction storage contracts.
|
||||
|
||||
/// Stable processing status for one extraction ledger entry.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum ProcessingLedgerStatus {
|
||||
/// Processing is currently running.
|
||||
Running,
|
||||
/// Processing completed successfully.
|
||||
Succeeded,
|
||||
/// Processing failed and may be retried.
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// Bounded selection filter for canonical transactions awaiting core extraction.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreExtractionSelectionFilter {
|
||||
/// Optional exact signatures selected by the operator.
|
||||
pub signatures: std::vec::Vec<std::string::String>,
|
||||
/// Optional inclusive minimum slot.
|
||||
pub min_slot: std::option::Option<u64>,
|
||||
/// Optional inclusive maximum slot.
|
||||
pub max_slot: std::option::Option<u64>,
|
||||
/// Optional raw processing state restriction.
|
||||
pub processing_state: std::option::Option<crate::RawPayloadProcessingState>,
|
||||
/// Optional program id previously resolved in core instructions.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Maximum number of canonical transactions returned.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl CoreExtractionSelectionFilter {
|
||||
/// Builds a validated extraction selection filter.
|
||||
pub fn new(
|
||||
signatures: std::vec::Vec<std::string::String>,
|
||||
min_slot: std::option::Option<u64>,
|
||||
max_slot: std::option::Option<u64>,
|
||||
processing_state: std::option::Option<crate::RawPayloadProcessingState>,
|
||||
program_id: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
if limit == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction selection limit must be greater than zero",
|
||||
));
|
||||
}
|
||||
if let (std::option::Option::Some(minimum), std::option::Option::Some(maximum)) =
|
||||
(min_slot, max_slot)
|
||||
{
|
||||
if minimum > maximum {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction minimum slot must not exceed maximum slot",
|
||||
));
|
||||
}
|
||||
}
|
||||
for signature in &signatures {
|
||||
if signature.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction signature filter must not contain empty values",
|
||||
));
|
||||
}
|
||||
}
|
||||
if program_id.as_deref().is_some_and(|value| return value.trim().is_empty()) {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction program id filter must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signatures,
|
||||
min_slot,
|
||||
max_slot,
|
||||
processing_state,
|
||||
program_id,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
|
||||
/// Builds a pending raw transaction selection.
|
||||
pub fn pending(limit: u32) -> ks_core::Result<Self> {
|
||||
return Self::new(
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::Some(crate::RawPayloadProcessingState::Received),
|
||||
std::option::Option::None,
|
||||
limit,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// Stable identity of one processor input in the processing ledger.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct ProcessingLedgerIdentity {
|
||||
/// Processing stage code.
|
||||
pub stage: std::string::String,
|
||||
/// Processor implementation name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Processor semantic version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable input key.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic input hash.
|
||||
pub input_hash: std::string::String,
|
||||
}
|
||||
|
||||
impl ProcessingLedgerIdentity {
|
||||
/// Builds a validated processing ledger identity.
|
||||
pub fn new(
|
||||
stage: impl std::convert::Into<std::string::String>,
|
||||
processor_name: impl std::convert::Into<std::string::String>,
|
||||
processor_version: impl std::convert::Into<std::string::String>,
|
||||
input_key: impl std::convert::Into<std::string::String>,
|
||||
input_hash: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let value = Self {
|
||||
stage: stage.into(),
|
||||
processor_name: processor_name.into(),
|
||||
processor_version: processor_version.into(),
|
||||
input_key: input_key.into(),
|
||||
input_hash: input_hash.into(),
|
||||
};
|
||||
if value.stage.trim().is_empty()
|
||||
|| value.processor_name.trim().is_empty()
|
||||
|| value.processor_version.trim().is_empty()
|
||||
|| value.input_key.trim().is_empty()
|
||||
|| value.input_hash.trim().is_empty()
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger identity fields must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(value);
|
||||
}
|
||||
}
|
||||
|
||||
/// Complete set of normalized rows produced from one canonical transaction.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreExtractionBundle {
|
||||
/// Source canonical raw transaction technical id.
|
||||
pub raw_transaction_id: i64,
|
||||
/// Processing ledger identity.
|
||||
pub ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
/// Core transaction row.
|
||||
pub transaction: crate::CoreTransactionInsert,
|
||||
/// Resolved account keys.
|
||||
pub account_keys: std::vec::Vec<crate::CoreAccountKeyInsert>,
|
||||
/// Top-level instructions.
|
||||
pub instructions: std::vec::Vec<crate::CoreInstructionInsert>,
|
||||
/// Inner instructions.
|
||||
pub inner_instructions: std::vec::Vec<crate::CoreInnerInstructionInsert>,
|
||||
/// Ordered transaction logs.
|
||||
pub logs: std::vec::Vec<crate::CoreLogInsert>,
|
||||
/// Native and token balance changes.
|
||||
pub balance_changes: std::vec::Vec<crate::CoreBalanceChangeInsert>,
|
||||
}
|
||||
|
||||
impl CoreExtractionBundle {
|
||||
/// Validates lineage and stable signature consistency across the bundle.
|
||||
pub fn validate(&self) -> ks_core::Result<()> {
|
||||
if self.raw_transaction_id <= 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction raw transaction id must be positive",
|
||||
));
|
||||
}
|
||||
if self.transaction.raw_transaction_id != std::option::Option::Some(self.raw_transaction_id)
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction transaction lineage does not match the raw transaction id",
|
||||
));
|
||||
}
|
||||
if self.transaction.signature != self.ledger_identity.input_key {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction ledger input key must equal the transaction signature",
|
||||
));
|
||||
}
|
||||
let signature = self.transaction.signature.as_str();
|
||||
for input in &self.account_keys {
|
||||
if input.signature != signature {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction account key signature mismatch",
|
||||
));
|
||||
}
|
||||
}
|
||||
for input in &self.instructions {
|
||||
if input.signature != signature {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction instruction signature mismatch",
|
||||
));
|
||||
}
|
||||
}
|
||||
for input in &self.inner_instructions {
|
||||
if input.signature != signature {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction inner instruction signature mismatch",
|
||||
));
|
||||
}
|
||||
}
|
||||
for input in &self.logs {
|
||||
if input.signature != signature {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction log signature mismatch",
|
||||
));
|
||||
}
|
||||
}
|
||||
for input in &self.balance_changes {
|
||||
if input.signature != signature {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction balance signature mismatch",
|
||||
));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
/// Failure details persisted when canonical to core extraction cannot complete.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreExtractionFailure {
|
||||
/// Source canonical raw transaction technical id.
|
||||
pub raw_transaction_id: i64,
|
||||
/// Processing ledger identity.
|
||||
pub ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
/// Stable machine-readable error code.
|
||||
pub error_code: std::string::String,
|
||||
/// Human-readable diagnostic message.
|
||||
pub error_message: std::string::String,
|
||||
}
|
||||
|
||||
impl CoreExtractionFailure {
|
||||
/// Builds a validated failure record.
|
||||
pub fn new(
|
||||
raw_transaction_id: i64,
|
||||
ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
error_code: impl std::convert::Into<std::string::String>,
|
||||
error_message: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let value = Self {
|
||||
raw_transaction_id,
|
||||
ledger_identity,
|
||||
error_code: error_code.into(),
|
||||
error_message: error_message.into(),
|
||||
};
|
||||
if value.raw_transaction_id <= 0
|
||||
|| value.error_code.trim().is_empty()
|
||||
|| value.error_message.trim().is_empty()
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"core extraction failure fields are invalid",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(value);
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn pending_filter_requires_positive_limit() {
|
||||
let result = crate::CoreExtractionSelectionFilter::pending(0);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn selection_filter_rejects_inverted_slots() {
|
||||
let result = crate::CoreExtractionSelectionFilter::new(
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(20),
|
||||
std::option::Option::Some(10),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
10,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ledger_identity_requires_input_hash() {
|
||||
let result = crate::ProcessingLedgerIdentity::new(
|
||||
"core_extraction",
|
||||
"canonical_to_core",
|
||||
"1",
|
||||
"signature",
|
||||
" ",
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
688
ks-store/src/contracts/dto/decode.rs
Normal file
688
ks-store/src/contracts/dto/decode.rs
Normal file
@@ -0,0 +1,688 @@
|
||||
// file: ks-store/src/contracts/dto/decode.rs
|
||||
// version: 5
|
||||
|
||||
//! Backend-neutral decode, coverage and materialization persistence DTOs.
|
||||
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
pub const MAX_MATERIALIZED_EVENT_QUERY_ROWS: u32 = 500;
|
||||
|
||||
/// Bounded read-only materialized event selection.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventFilter {
|
||||
/// Optional exact materializer processor name.
|
||||
pub processor_name: std::option::Option<std::string::String>,
|
||||
/// Optional exact materialized family code.
|
||||
pub materialized_family: std::option::Option<std::string::String>,
|
||||
/// Optional partial transaction signature.
|
||||
pub signature_contains: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl MaterializedEventFilter {
|
||||
/// Builds and validates a bounded materialized event filter.
|
||||
pub fn new(
|
||||
processor_name: std::option::Option<std::string::String>,
|
||||
materialized_family: std::option::Option<std::string::String>,
|
||||
signature_contains: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
if limit == 0 || limit > crate::MAX_MATERIALIZED_EVENT_QUERY_ROWS {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"materialized event query limit must be between 1 and {}",
|
||||
crate::MAX_MATERIALIZED_EVENT_QUERY_ROWS
|
||||
)));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
processor_name: crate::contracts::dto::decode::trim_optional_text(processor_name),
|
||||
materialized_family: crate::contracts::dto::decode::trim_optional_text(
|
||||
materialized_family,
|
||||
),
|
||||
signature_contains: crate::contracts::dto::decode::trim_optional_text(
|
||||
signature_contains,
|
||||
),
|
||||
limit,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// One materialized output returned by the common bounded query contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventQueryRow {
|
||||
/// Materializer processor name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Materializer processor version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable materializer input key.
|
||||
pub input_key: std::string::String,
|
||||
/// Stable processor-owned output key.
|
||||
pub output_key: std::string::String,
|
||||
/// Source decoded event key.
|
||||
pub source_event_key: std::string::String,
|
||||
/// Source decoder name.
|
||||
pub source_decoder_name: std::string::String,
|
||||
/// Source decoder version.
|
||||
pub source_decoder_version: std::string::String,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot.
|
||||
pub slot: u64,
|
||||
/// Materialized family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Typed materialized payload.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Creation timestamp rendered by the backend.
|
||||
pub created_at: std::string::String,
|
||||
/// Last replacement timestamp rendered by the backend.
|
||||
pub updated_at: std::string::String,
|
||||
}
|
||||
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeSelectionFilter {
|
||||
/// Explicit transaction signatures to select.
|
||||
pub signatures: std::vec::Vec<std::string::String>,
|
||||
/// Explicit instruction processing states to select.
|
||||
pub processing_states: std::vec::Vec<crate::CoreInstructionProcessingState>,
|
||||
/// Optional inclusive minimum slot.
|
||||
pub min_slot: std::option::Option<u64>,
|
||||
/// Optional inclusive maximum slot.
|
||||
pub max_slot: std::option::Option<u64>,
|
||||
/// Explicit program identifiers to select.
|
||||
pub program_ids: std::vec::Vec<std::string::String>,
|
||||
/// Explicit stable instruction paths to select.
|
||||
pub instruction_paths: std::vec::Vec<std::string::String>,
|
||||
/// Expands any incomplete instruction match to every instruction in the same signature.
|
||||
pub incomplete_signatures: bool,
|
||||
/// Maximum number of contextual inputs, or signatures when expansion is enabled.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl DecodeSelectionFilter {
|
||||
/// Builds a validated bounded decode selection filter.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new(
|
||||
signatures: std::vec::Vec<std::string::String>,
|
||||
processing_states: std::vec::Vec<crate::CoreInstructionProcessingState>,
|
||||
min_slot: std::option::Option<u64>,
|
||||
max_slot: std::option::Option<u64>,
|
||||
program_ids: std::vec::Vec<std::string::String>,
|
||||
instruction_paths: std::vec::Vec<std::string::String>,
|
||||
incomplete_signatures: bool,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
if limit == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode selection limit must be greater than zero",
|
||||
));
|
||||
}
|
||||
if min_slot.is_some() && max_slot.is_some() && min_slot > max_slot {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode selection minimum slot must not exceed maximum slot",
|
||||
));
|
||||
}
|
||||
let signatures_result = validate_text_list(&signatures, "decode selection signature");
|
||||
if let std::result::Result::Err(error) = signatures_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let program_ids_result = validate_text_list(&program_ids, "decode selection program id");
|
||||
if let std::result::Result::Err(error) = program_ids_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let paths_result =
|
||||
validate_text_list(&instruction_paths, "decode selection instruction path");
|
||||
if let std::result::Result::Err(error) = paths_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signatures,
|
||||
processing_states,
|
||||
min_slot,
|
||||
max_slot,
|
||||
program_ids,
|
||||
instruction_paths,
|
||||
incomplete_signatures,
|
||||
limit,
|
||||
});
|
||||
}
|
||||
|
||||
/// Builds the default pending, failed and replay-requested selection.
|
||||
pub fn actionable(limit: u32) -> ks_core::Result<Self> {
|
||||
return crate::DecodeSelectionFilter::new(
|
||||
std::vec::Vec::new(),
|
||||
std::vec![
|
||||
crate::CoreInstructionProcessingState::Pending,
|
||||
crate::CoreInstructionProcessingState::Failed,
|
||||
crate::CoreInstructionProcessingState::ReplayRequested,
|
||||
],
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::vec::Vec::new(),
|
||||
std::vec::Vec::new(),
|
||||
false,
|
||||
limit,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
/// One processor-owned decoded observation row.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeObservationInsert {
|
||||
/// Stable decode processor name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Stable decode processor version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable contextual input key.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic contextual input hash.
|
||||
pub input_hash: std::string::String,
|
||||
/// Stable event key within the processor and input.
|
||||
pub event_key: std::string::String,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot.
|
||||
pub slot: u64,
|
||||
/// Source instruction path.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Source program identifier.
|
||||
pub program_id: std::string::String,
|
||||
/// Stable protocol code.
|
||||
pub protocol_code: std::string::String,
|
||||
/// Stable surface code.
|
||||
pub surface_code: std::string::String,
|
||||
/// Stable event code.
|
||||
pub event_code: std::string::String,
|
||||
/// Stable event name.
|
||||
pub event_name: std::string::String,
|
||||
/// Stable event family code.
|
||||
pub event_family: std::string::String,
|
||||
/// Stable event source code.
|
||||
pub source_kind: std::string::String,
|
||||
/// Stable decoder confidence code.
|
||||
pub confidence: std::string::String,
|
||||
/// Stable proof kind code.
|
||||
pub proof_kind: std::string::String,
|
||||
/// Proof evidence JSON.
|
||||
pub proof_json: serde_json::Value,
|
||||
/// Typed decoded payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Whether the source transaction failed on-chain.
|
||||
pub transaction_failed: bool,
|
||||
/// Optional source transaction error JSON.
|
||||
pub transaction_error: std::option::Option<serde_json::Value>,
|
||||
/// Whether the observed state mutation was committed on-chain.
|
||||
pub observation_committed: bool,
|
||||
}
|
||||
|
||||
impl DecodeObservationInsert {
|
||||
/// Validates stable identities and failed transaction commit semantics.
|
||||
pub fn validate(&self) -> ks_core::Result<()> {
|
||||
let fields = [
|
||||
self.processor_name.as_str(),
|
||||
self.processor_version.as_str(),
|
||||
self.input_key.as_str(),
|
||||
self.input_hash.as_str(),
|
||||
self.event_key.as_str(),
|
||||
self.signature.as_str(),
|
||||
self.instruction_path.as_str(),
|
||||
self.program_id.as_str(),
|
||||
self.protocol_code.as_str(),
|
||||
self.surface_code.as_str(),
|
||||
self.event_code.as_str(),
|
||||
self.event_name.as_str(),
|
||||
self.event_family.as_str(),
|
||||
self.source_kind.as_str(),
|
||||
self.confidence.as_str(),
|
||||
self.proof_kind.as_str(),
|
||||
];
|
||||
if fields.iter().any(|value| return value.trim().is_empty()) {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded observation identity fields must not be empty",
|
||||
));
|
||||
}
|
||||
if self.transaction_failed && self.observation_committed {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"failed transaction decoded observations must not be committed",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
/// One machine-readable decoder coverage declaration row.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeCoverageDeclarationInsert {
|
||||
/// Stable decoder name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Stable decoder version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Exact Solana program identifier.
|
||||
pub program_id: std::string::String,
|
||||
/// Optional stable surface code.
|
||||
pub surface_code: std::option::Option<std::string::String>,
|
||||
/// Stable entry kind code.
|
||||
pub entry_kind: std::string::String,
|
||||
/// Stable instruction, event or discriminator code.
|
||||
pub entry_code: std::string::String,
|
||||
/// Optional normalized hexadecimal discriminator.
|
||||
pub discriminator_hex: std::option::Option<std::string::String>,
|
||||
/// Whether the entry is historical or deprecated.
|
||||
pub historical: bool,
|
||||
}
|
||||
|
||||
/// One observed coverage classification row owned by one decode attempt.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeCoverageObservationInsert {
|
||||
/// Stable decoder name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Stable decoder version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable contextual input key.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic contextual input hash.
|
||||
pub input_hash: std::string::String,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot.
|
||||
pub slot: u64,
|
||||
/// Source instruction path.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Source program identifier.
|
||||
pub program_id: std::string::String,
|
||||
/// Optional stable surface code.
|
||||
pub surface_code: std::option::Option<std::string::String>,
|
||||
/// Optional recognized entry code.
|
||||
pub entry_code: std::option::Option<std::string::String>,
|
||||
/// Optional discriminator.
|
||||
pub discriminator_hex: std::option::Option<std::string::String>,
|
||||
/// Stable decode terminal status code.
|
||||
pub status: std::string::String,
|
||||
/// Whether the processor recognized the input as compatible.
|
||||
pub recognized: bool,
|
||||
/// Number of decoded observations produced.
|
||||
pub decoded_count: u32,
|
||||
/// Number of materialized outputs produced immediately after decoding.
|
||||
pub materialized_count: u32,
|
||||
/// Number of decoder diagnostics classified as errors.
|
||||
pub error_count: u32,
|
||||
/// Whether the source transaction failed on-chain.
|
||||
pub transaction_failed: bool,
|
||||
}
|
||||
|
||||
/// Atomic persistence bundle for one decoder and one contextual input.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodePersistenceBundle {
|
||||
/// Processing ledger identity for the decode attempt.
|
||||
pub ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source instruction path.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Stable terminal status code.
|
||||
pub status: std::string::String,
|
||||
/// Optional stable machine-readable terminal error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional human-readable terminal error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
/// Processor-owned decoded observations.
|
||||
pub observations: std::vec::Vec<DecodeObservationInsert>,
|
||||
/// Coverage observation for this attempt.
|
||||
pub coverage: DecodeCoverageObservationInsert,
|
||||
}
|
||||
|
||||
impl DecodePersistenceBundle {
|
||||
/// Validates identities shared by every atomic decode output.
|
||||
pub fn validate(&self) -> ks_core::Result<()> {
|
||||
if self.ledger_identity.stage != "instruction_decode"
|
||||
|| self.signature.trim().is_empty()
|
||||
|| self.instruction_path.trim().is_empty()
|
||||
|| self.status.trim().is_empty()
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode persistence bundle identity is invalid",
|
||||
));
|
||||
}
|
||||
if !matches!(self.status.as_str(), "decoded" | "ignored" | "unsupported" | "failed") {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode persistence bundle status is unsupported",
|
||||
));
|
||||
}
|
||||
if self.coverage.processor_name != self.ledger_identity.processor_name
|
||||
|| self.coverage.processor_version != self.ledger_identity.processor_version
|
||||
|| self.coverage.input_key != self.ledger_identity.input_key
|
||||
|| self.coverage.input_hash != self.ledger_identity.input_hash
|
||||
|| self.coverage.signature != self.signature
|
||||
|| self.coverage.instruction_path != self.instruction_path
|
||||
|| self.coverage.status != self.status
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode coverage observation does not match bundle identity",
|
||||
));
|
||||
}
|
||||
let observation_count_result = u32::try_from(self.observations.len());
|
||||
let observation_count = match observation_count_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode observation count exceeds the supported range",
|
||||
));
|
||||
},
|
||||
};
|
||||
if self.coverage.decoded_count != observation_count
|
||||
|| (self.status == "decoded" && self.observations.is_empty())
|
||||
|| (self.status != "decoded" && !self.observations.is_empty())
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode status, coverage count and observations are inconsistent",
|
||||
));
|
||||
}
|
||||
let missing_decode_error = match (&self.error_code, &self.error_message) {
|
||||
(std::option::Option::Some(code), std::option::Option::Some(message)) => {
|
||||
code.trim().is_empty() || message.trim().is_empty()
|
||||
},
|
||||
_ => true,
|
||||
};
|
||||
if self.status == "failed" && missing_decode_error {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"failed decode bundle requires error code and message",
|
||||
));
|
||||
}
|
||||
for observation in &self.observations {
|
||||
let validation_result = observation.validate();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
if observation.processor_name != self.ledger_identity.processor_name
|
||||
|| observation.processor_version != self.ledger_identity.processor_version
|
||||
|| observation.input_key != self.ledger_identity.input_key
|
||||
|| observation.input_hash != self.ledger_identity.input_hash
|
||||
|| observation.signature != self.signature
|
||||
|| observation.instruction_path != self.instruction_path
|
||||
|| observation.transaction_failed != self.coverage.transaction_failed
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded observation does not match bundle identity",
|
||||
));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
/// Failed decode attempt persisted in the common ledger.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeFailure {
|
||||
/// Processing ledger identity.
|
||||
pub ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source instruction path.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Stable machine-readable error code.
|
||||
pub error_code: std::string::String,
|
||||
/// Human-readable diagnostic message.
|
||||
pub error_message: std::string::String,
|
||||
}
|
||||
|
||||
/// One processor-owned materialized output row.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedOutputInsert {
|
||||
/// Stable materializer name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Stable materializer version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable decoded observation source key.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic decoded observation input hash.
|
||||
pub input_hash: std::string::String,
|
||||
/// Stable output key within the materializer and input.
|
||||
pub output_key: std::string::String,
|
||||
/// Source decoded event key.
|
||||
pub source_event_key: std::string::String,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot.
|
||||
pub slot: u64,
|
||||
/// Stable materialized family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Typed business payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
}
|
||||
|
||||
/// Atomic persistence bundle for one materializer and one decoded observation.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializationPersistenceBundle {
|
||||
/// Processing ledger identity for the materialization attempt.
|
||||
pub ledger_identity: crate::ProcessingLedgerIdentity,
|
||||
/// Source decoder name owning the decoded observation.
|
||||
pub source_decoder_name: std::string::String,
|
||||
/// Source decoder version owning the decoded observation.
|
||||
pub source_decoder_version: std::string::String,
|
||||
/// Source contextual decode input key.
|
||||
pub source_decode_input_key: std::string::String,
|
||||
/// Source transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Source stable instruction path.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Stable terminal materializer status code.
|
||||
pub status: std::string::String,
|
||||
/// Optional stable machine-readable terminal error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional human-readable terminal error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
/// Processor-owned materialized outputs.
|
||||
pub outputs: std::vec::Vec<MaterializedOutputInsert>,
|
||||
}
|
||||
|
||||
impl MaterializationPersistenceBundle {
|
||||
/// Validates the materializer, source decoder and output identities.
|
||||
pub fn validate(&self) -> ks_core::Result<()> {
|
||||
if self.ledger_identity.stage != "event_materialization"
|
||||
|| self.source_decoder_name.trim().is_empty()
|
||||
|| self.source_decoder_version.trim().is_empty()
|
||||
|| self.source_decode_input_key.trim().is_empty()
|
||||
|| self.signature.trim().is_empty()
|
||||
|| self.instruction_path.trim().is_empty()
|
||||
|| self.status.trim().is_empty()
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialization persistence bundle identity is invalid",
|
||||
));
|
||||
}
|
||||
if !matches!(
|
||||
self.status.as_str(),
|
||||
"inserted" | "replaced" | "ignored" | "refused" | "failed"
|
||||
) {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialization persistence bundle status is unsupported",
|
||||
));
|
||||
}
|
||||
if (matches!(self.status.as_str(), "inserted" | "replaced") && self.outputs.is_empty())
|
||||
|| (matches!(self.status.as_str(), "ignored" | "refused" | "failed")
|
||||
&& !self.outputs.is_empty())
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialization status and outputs are inconsistent",
|
||||
));
|
||||
}
|
||||
let missing_materialization_error = match (&self.error_code, &self.error_message) {
|
||||
(std::option::Option::Some(code), std::option::Option::Some(message)) => {
|
||||
code.trim().is_empty() || message.trim().is_empty()
|
||||
},
|
||||
_ => true,
|
||||
};
|
||||
if self.status == "failed" && missing_materialization_error {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"failed materialization bundle requires error code and message",
|
||||
));
|
||||
}
|
||||
for output in &self.outputs {
|
||||
if output.output_key.trim().is_empty()
|
||||
|| output.source_event_key.trim().is_empty()
|
||||
|| output.materialized_family.trim().is_empty()
|
||||
|| output.processor_name != self.ledger_identity.processor_name
|
||||
|| output.processor_version != self.ledger_identity.processor_version
|
||||
|| output.input_key != self.ledger_identity.input_key
|
||||
|| output.input_hash != self.ledger_identity.input_hash
|
||||
|| output.signature != self.signature
|
||||
{
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialized output does not match bundle identity",
|
||||
));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
}
|
||||
|
||||
/// One row of aggregated decoder coverage diagnostics.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodeCoverageSummaryRow {
|
||||
/// Stable decoder name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Stable decoder version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Exact Solana program identifier.
|
||||
pub program_id: std::string::String,
|
||||
/// Optional stable surface code.
|
||||
pub surface_code: std::option::Option<std::string::String>,
|
||||
/// Stable entry code or unknown classifier.
|
||||
pub entry_code: std::string::String,
|
||||
/// Number of declared matching entries.
|
||||
pub declared_count: i64,
|
||||
/// Number of observed inputs.
|
||||
pub observed_count: i64,
|
||||
/// Number of recognized inputs.
|
||||
pub recognized_count: i64,
|
||||
/// Number of decoded observations.
|
||||
pub decoded_count: i64,
|
||||
/// Number of materialized outputs.
|
||||
pub materialized_count: i64,
|
||||
/// Number of errors.
|
||||
pub error_count: i64,
|
||||
/// Number of observations classified as unknown or unsupported.
|
||||
pub unknown_count: i64,
|
||||
/// Number of successful source transactions.
|
||||
pub successful_transaction_count: i64,
|
||||
/// Number of failed source transactions.
|
||||
pub failed_transaction_count: i64,
|
||||
}
|
||||
|
||||
fn validate_text_list(values: &[std::string::String], label: &str) -> ks_core::Result<()> {
|
||||
if values.iter().any(|value| return value.trim().is_empty()) {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!("{label} must not be empty")));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn trim_optional_text(
|
||||
value: std::option::Option<std::string::String>,
|
||||
) -> std::option::Option<std::string::String> {
|
||||
return value.and_then(|text| {
|
||||
let trimmed = text.trim();
|
||||
if trimmed.is_empty() {
|
||||
return std::option::Option::None;
|
||||
}
|
||||
return std::option::Option::Some(trimmed.to_string());
|
||||
});
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn actionable_filter_requires_positive_limit() {
|
||||
assert!(crate::DecodeSelectionFilter::actionable(0).is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn incomplete_signature_filter_preserves_signature_limit_semantics() {
|
||||
let result = crate::DecodeSelectionFilter::new(
|
||||
std::vec::Vec::new(),
|
||||
std::vec![crate::CoreInstructionProcessingState::Failed],
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::vec::Vec::new(),
|
||||
std::vec::Vec::new(),
|
||||
true,
|
||||
25,
|
||||
);
|
||||
let filter = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected filter error: {error}"),
|
||||
};
|
||||
assert!(filter.incomplete_signatures);
|
||||
assert_eq!(filter.limit, 25);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn selection_filter_rejects_inverted_slots() {
|
||||
let result = crate::DecodeSelectionFilter::new(
|
||||
std::vec::Vec::new(),
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(20),
|
||||
std::option::Option::Some(10),
|
||||
std::vec::Vec::new(),
|
||||
std::vec::Vec::new(),
|
||||
false,
|
||||
10,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn materialized_event_filter_is_bounded_and_trims_optional_text() {
|
||||
let result = crate::MaterializedEventFilter::new(
|
||||
std::option::Option::Some(" materializer.transaction.annotations ".to_string()),
|
||||
std::option::Option::Some(" transaction_annotation ".to_string()),
|
||||
std::option::Option::Some(" signature ".to_string()),
|
||||
crate::MAX_MATERIALIZED_EVENT_QUERY_ROWS,
|
||||
);
|
||||
let filter = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected filter error: {error}"),
|
||||
};
|
||||
assert_eq!(
|
||||
filter.processor_name.as_deref(),
|
||||
std::option::Option::Some("materializer.transaction.annotations")
|
||||
);
|
||||
assert_eq!(filter.signature_contains.as_deref(), std::option::Option::Some("signature"));
|
||||
assert!(
|
||||
crate::MaterializedEventFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
0,
|
||||
)
|
||||
.is_err()
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn failed_decoded_observation_cannot_be_committed() {
|
||||
let input = crate::DecodeObservationInsert {
|
||||
processor_name: "decoder".to_string(),
|
||||
processor_version: "1".to_string(),
|
||||
input_key: "signature:0".to_string(),
|
||||
input_hash: "hash".to_string(),
|
||||
event_key: "event".to_string(),
|
||||
signature: "signature".to_string(),
|
||||
slot: 1,
|
||||
instruction_path: "0".to_string(),
|
||||
program_id: "program".to_string(),
|
||||
protocol_code: "protocol".to_string(),
|
||||
surface_code: "surface".to_string(),
|
||||
event_code: "event".to_string(),
|
||||
event_name: "event".to_string(),
|
||||
event_family: "audit".to_string(),
|
||||
source_kind: "instruction".to_string(),
|
||||
confidence: "exact".to_string(),
|
||||
proof_kind: "exact_layout".to_string(),
|
||||
proof_json: serde_json::json!({}),
|
||||
payload_json: serde_json::json!({}),
|
||||
transaction_failed: true,
|
||||
transaction_error: std::option::Option::Some(serde_json::json!({})),
|
||||
observation_committed: true,
|
||||
};
|
||||
assert!(input.validate().is_err());
|
||||
}
|
||||
}
|
||||
143
ks-store/src/contracts/dto/event.rs
Normal file
143
ks-store/src/contracts/dto/event.rs
Normal file
@@ -0,0 +1,143 @@
|
||||
// file: ks-store/src/contracts/dto/event.rs
|
||||
// version: 2
|
||||
|
||||
//! Decoded and materialized event storage DTOs.
|
||||
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct InsertOutcome {
|
||||
/// Number of rows inserted by the repository call.
|
||||
pub inserted_count: u64,
|
||||
/// Number of rows updated by the repository call.
|
||||
pub updated_count: u64,
|
||||
/// Number of rows skipped by the repository call.
|
||||
pub skipped_count: u64,
|
||||
}
|
||||
|
||||
impl InsertOutcome {
|
||||
/// Builds an insert outcome from explicit counters.
|
||||
pub fn new(inserted_count: u64, updated_count: u64, skipped_count: u64) -> Self {
|
||||
return Self {
|
||||
inserted_count,
|
||||
updated_count,
|
||||
skipped_count,
|
||||
};
|
||||
}
|
||||
|
||||
/// Returns the sum of inserted, updated and skipped rows.
|
||||
pub fn total_count(&self) -> u64 {
|
||||
return self.inserted_count + self.updated_count + self.skipped_count;
|
||||
}
|
||||
}
|
||||
|
||||
/// Decoded event insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodedEventInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Protocol family code.
|
||||
pub protocol_code: std::string::String,
|
||||
/// Protocol surface code.
|
||||
pub surface_code: std::string::String,
|
||||
/// Canonical event code.
|
||||
pub event_code: std::string::String,
|
||||
/// Decoded payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
}
|
||||
|
||||
impl DecodedEventInsert {
|
||||
/// Builds a decoded event insert contract from the shared decoded model.
|
||||
pub fn from_model(
|
||||
event: &ks_lib::MdDecodedProtocolEvent,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
if event.signature.0.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded event signature must not be empty",
|
||||
));
|
||||
}
|
||||
if event.program_id.0.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded event program id must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: event.signature.0.clone(),
|
||||
slot: event.slot.0,
|
||||
instruction_path: event.instruction_path.0.clone(),
|
||||
program_id: event.program_id.0.clone(),
|
||||
protocol_code: event.protocol_code.0.clone(),
|
||||
surface_code: event.surface_code.0.clone(),
|
||||
event_code: event.event_code.0.clone(),
|
||||
payload_json,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Materialized event insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventInsert {
|
||||
/// Source transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Materialized event family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Materialized payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
}
|
||||
|
||||
impl MaterializedEventInsert {
|
||||
/// Builds a materialized event insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
materialized_family: impl std::convert::Into<std::string::String>,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let materialized_family_value = materialized_family.into();
|
||||
if signature_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialized event signature must not be empty",
|
||||
));
|
||||
}
|
||||
if materialized_family_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialized event family must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
materialized_family: materialized_family_value,
|
||||
payload_json,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn insert_outcome_total_counts_all_buckets() {
|
||||
let outcome = crate::InsertOutcome::new(1, 2, 3);
|
||||
assert_eq!(outcome.total_count(), 6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn materialized_event_rejects_empty_family() {
|
||||
let result = crate::MaterializedEventInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
" ",
|
||||
serde_json::json!({"kind": "trade"}),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
73
ks-store/src/contracts/dto/ledger.rs
Normal file
73
ks-store/src/contracts/dto/ledger.rs
Normal file
@@ -0,0 +1,73 @@
|
||||
// file: ks-store/src/contracts/dto/ledger.rs
|
||||
// version: 2
|
||||
|
||||
//! Processing ledger storage DTOs.
|
||||
|
||||
/// Processing ledger mark request contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct ProcessingLedgerMark {
|
||||
/// Processing stage name, for example `raw_ingest`, `core_extract` or `decode`.
|
||||
pub stage: std::string::String,
|
||||
/// Processing module name.
|
||||
pub module_name: std::string::String,
|
||||
/// Processing module version.
|
||||
pub module_version: std::string::String,
|
||||
/// Stable input key, usually a signature or notification id.
|
||||
pub input_key: std::string::String,
|
||||
}
|
||||
|
||||
impl ProcessingLedgerMark {
|
||||
/// Builds a processing ledger mark request after minimal validation.
|
||||
pub fn new(
|
||||
stage: impl std::convert::Into<std::string::String>,
|
||||
module_name: impl std::convert::Into<std::string::String>,
|
||||
module_version: impl std::convert::Into<std::string::String>,
|
||||
input_key: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let stage_value = stage.into();
|
||||
let module_name_value = module_name.into();
|
||||
let module_version_value = module_version.into();
|
||||
let input_key_value = input_key.into();
|
||||
if stage_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger stage must not be empty",
|
||||
));
|
||||
}
|
||||
if module_name_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger module name must not be empty",
|
||||
));
|
||||
}
|
||||
if module_version_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger module version must not be empty",
|
||||
));
|
||||
}
|
||||
if input_key_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger input key must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
stage: stage_value,
|
||||
module_name: module_name_value,
|
||||
module_version: module_version_value,
|
||||
input_key: input_key_value,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn ledger_mark_rejects_empty_input_key() {
|
||||
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", " ");
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ledger_mark_accepts_minimal_values() {
|
||||
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", "signature");
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
}
|
||||
609
ks-store/src/contracts/dto/raw.rs
Normal file
609
ks-store/src/contracts/dto/raw.rs
Normal file
@@ -0,0 +1,609 @@
|
||||
// file: ks-store/src/contracts/dto/raw.rs
|
||||
// version: 2
|
||||
|
||||
//! Canonical Solana transaction and acquisition observation storage DTOs.
|
||||
|
||||
/// Retention state for a canonical raw transaction payload.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum RawPayloadRetentionState {
|
||||
/// Full canonical payload is still present in the primary store.
|
||||
Full,
|
||||
/// Canonical payload was reduced to a compact audit representation.
|
||||
Compacted,
|
||||
/// Canonical payload was moved to an archive tier outside the primary hot store.
|
||||
Archived,
|
||||
/// Canonical payload was purged after derived data became authoritative enough.
|
||||
Purged,
|
||||
}
|
||||
|
||||
/// Processing state for a canonical raw transaction payload.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum RawPayloadProcessingState {
|
||||
/// Transaction was received but not extracted yet.
|
||||
Received,
|
||||
/// Generic Solana core data was extracted from the canonical transaction.
|
||||
CoreExtracted,
|
||||
/// Decoder outputs were produced from the canonical transaction or its core extraction.
|
||||
Decoded,
|
||||
/// Business projections were materialized from decoded or core data.
|
||||
Materialized,
|
||||
/// Processing failed and requires diagnostics or replay.
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// Origin of one transaction acquisition observation.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum TransactionObservationOrigin {
|
||||
/// Transaction was observed from a current live stream.
|
||||
Live,
|
||||
/// Transaction was acquired by an explicit historical backfill.
|
||||
Backfill,
|
||||
/// Transaction was replayed from an already captured source.
|
||||
Replay,
|
||||
/// Transaction was fetched to repair an acquisition gap.
|
||||
Repair,
|
||||
/// Observation was converted from a historical storage table.
|
||||
Migration,
|
||||
}
|
||||
|
||||
/// Technical status of one transaction acquisition observation.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum TransactionObservationStatus {
|
||||
/// A transaction candidate was detected before a complete payload was received.
|
||||
Detected,
|
||||
/// A source payload was received.
|
||||
Received,
|
||||
/// A source payload was normalized into the canonical transaction contract.
|
||||
Normalized,
|
||||
/// The observation and any linked canonical transaction were persisted.
|
||||
Persisted,
|
||||
/// Acquisition or normalization failed.
|
||||
Failed,
|
||||
/// The source reported or implied a transaction that was temporarily unavailable.
|
||||
Missing,
|
||||
}
|
||||
|
||||
/// Canonical raw Solana transaction insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct RawTransactionInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Canonical source-independent transaction document.
|
||||
pub canonical_json: serde_json::Value,
|
||||
/// Optional deterministic digest of the canonical document.
|
||||
pub canonical_json_hash: std::option::Option<std::string::String>,
|
||||
/// Positive version of the canonical transaction contract.
|
||||
pub canonical_format_version: u32,
|
||||
}
|
||||
|
||||
impl RawTransactionInsert {
|
||||
/// Builds a canonical raw transaction insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
canonical_json: serde_json::Value,
|
||||
canonical_format_version: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
if signature_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"canonical transaction signature must not be empty",
|
||||
));
|
||||
}
|
||||
if canonical_format_version == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"canonical transaction format version must be greater than zero",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
canonical_json,
|
||||
canonical_json_hash: std::option::Option::None,
|
||||
canonical_format_version,
|
||||
});
|
||||
}
|
||||
|
||||
/// Builds a storage insert from the source-independent canonical transaction model.
|
||||
pub fn from_canonical(transaction: &ks_lib::MdCanonicalTransaction) -> ks_core::Result<Self> {
|
||||
let canonical_json_result = transaction.to_canonical_json();
|
||||
let canonical_json = match canonical_json_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let hash_result = transaction.canonical_json_hash();
|
||||
let hash = match hash_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let insert_result = Self::new(
|
||||
transaction.primary_signature.clone(),
|
||||
transaction.slot,
|
||||
canonical_json,
|
||||
transaction.format_version,
|
||||
);
|
||||
let insert = match insert_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
return insert.with_canonical_json_hash(hash);
|
||||
}
|
||||
|
||||
/// Adds a precomputed deterministic canonical document hash.
|
||||
pub fn with_canonical_json_hash(
|
||||
mut self,
|
||||
canonical_json_hash: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let hash_value = canonical_json_hash.into();
|
||||
if hash_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"canonical transaction hash must not be empty",
|
||||
));
|
||||
}
|
||||
self.canonical_json_hash = std::option::Option::Some(hash_value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
}
|
||||
|
||||
/// Lightweight transaction acquisition observation insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct TransactionObservationInsert {
|
||||
/// Stable observation key used for deduplication.
|
||||
pub observation_key: std::string::String,
|
||||
/// Optional known canonical transaction row id.
|
||||
pub raw_transaction_id: std::option::Option<i64>,
|
||||
/// Optional transaction signature when known.
|
||||
pub signature: std::option::Option<std::string::String>,
|
||||
/// Optional transaction slot when known.
|
||||
pub slot: std::option::Option<u64>,
|
||||
/// Provider code.
|
||||
pub provider: std::string::String,
|
||||
/// Optional endpoint code from the active configuration.
|
||||
pub endpoint_code: std::option::Option<std::string::String>,
|
||||
/// Source protocol.
|
||||
pub protocol: std::string::String,
|
||||
/// Source acquisition method.
|
||||
pub acquisition_method: std::string::String,
|
||||
/// Observation origin.
|
||||
pub origin: TransactionObservationOrigin,
|
||||
/// Optional Solana commitment.
|
||||
pub commitment: std::option::Option<std::string::String>,
|
||||
/// Optional capture session identifier.
|
||||
pub capture_session_id: std::option::Option<std::string::String>,
|
||||
/// Optional configured filter code.
|
||||
pub filter_code: std::option::Option<std::string::String>,
|
||||
/// Optional first detection timestamp.
|
||||
pub detected_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Timestamp at which the source payload was received locally.
|
||||
pub received_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Optional timestamp at which canonical normalization completed.
|
||||
pub normalized_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Optional uncompressed source payload size in bytes.
|
||||
pub payload_size_bytes: std::option::Option<u64>,
|
||||
/// Optional digest of the source-specific payload without retaining that payload.
|
||||
pub source_payload_hash: std::option::Option<std::string::String>,
|
||||
/// Observation status.
|
||||
pub status: TransactionObservationStatus,
|
||||
/// Optional machine-readable error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional diagnostic error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl TransactionObservationInsert {
|
||||
/// Builds a lightweight transaction observation after validating required source metadata.
|
||||
pub fn new(
|
||||
observation_key: impl std::convert::Into<std::string::String>,
|
||||
provider: impl std::convert::Into<std::string::String>,
|
||||
protocol: impl std::convert::Into<std::string::String>,
|
||||
acquisition_method: impl std::convert::Into<std::string::String>,
|
||||
origin: TransactionObservationOrigin,
|
||||
received_at: chrono::DateTime<chrono::Utc>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let observation_key_value = observation_key.into();
|
||||
let provider_value = provider.into();
|
||||
let protocol_value = protocol.into();
|
||||
let acquisition_method_value = acquisition_method.into();
|
||||
let validation_result = validate_required_observation_texts(
|
||||
observation_key_value.as_str(),
|
||||
provider_value.as_str(),
|
||||
protocol_value.as_str(),
|
||||
acquisition_method_value.as_str(),
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
observation_key: observation_key_value,
|
||||
raw_transaction_id: std::option::Option::None,
|
||||
signature: std::option::Option::None,
|
||||
slot: std::option::Option::None,
|
||||
provider: provider_value,
|
||||
endpoint_code: std::option::Option::None,
|
||||
protocol: protocol_value,
|
||||
acquisition_method: acquisition_method_value,
|
||||
origin,
|
||||
commitment: std::option::Option::None,
|
||||
capture_session_id: std::option::Option::None,
|
||||
filter_code: std::option::Option::None,
|
||||
detected_at: std::option::Option::None,
|
||||
received_at,
|
||||
normalized_at: std::option::Option::None,
|
||||
payload_size_bytes: std::option::Option::None,
|
||||
source_payload_hash: std::option::Option::None,
|
||||
status: TransactionObservationStatus::Received,
|
||||
error_code: std::option::Option::None,
|
||||
error_message: std::option::Option::None,
|
||||
});
|
||||
}
|
||||
|
||||
/// Links the observation to a known canonical transaction row id.
|
||||
pub fn with_raw_transaction_id(mut self, raw_transaction_id: i64) -> ks_core::Result<Self> {
|
||||
if raw_transaction_id <= 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"transaction observation raw transaction id must be greater than zero",
|
||||
));
|
||||
}
|
||||
self.raw_transaction_id = std::option::Option::Some(raw_transaction_id);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Adds the transaction signature and optional slot carried by the source.
|
||||
pub fn with_transaction_identity(
|
||||
mut self,
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: std::option::Option<u64>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
if signature_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"transaction observation signature must not be empty",
|
||||
));
|
||||
}
|
||||
self.signature = std::option::Option::Some(signature_value);
|
||||
self.slot = slot;
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Adds an endpoint code after validating non-empty optional text.
|
||||
pub fn with_endpoint_code(
|
||||
mut self,
|
||||
endpoint_code: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let endpoint_code_value = endpoint_code.into();
|
||||
let validation_result = validate_optional_text(
|
||||
endpoint_code_value.as_str(),
|
||||
"transaction observation endpoint code must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
self.endpoint_code = std::option::Option::Some(endpoint_code_value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Adds an optional commitment value after validation.
|
||||
pub fn with_commitment(
|
||||
mut self,
|
||||
commitment: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let commitment_value = commitment.into();
|
||||
let validation_result = validate_optional_text(
|
||||
commitment_value.as_str(),
|
||||
"transaction observation commitment must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
self.commitment = std::option::Option::Some(commitment_value);
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Adds optional capture session and filter codes after validation.
|
||||
pub fn with_capture_context(
|
||||
mut self,
|
||||
capture_session_id: std::option::Option<std::string::String>,
|
||||
filter_code: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let session_result = validate_optional_owned_text(
|
||||
capture_session_id.as_ref(),
|
||||
"transaction observation capture session id must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = session_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let filter_result = validate_optional_owned_text(
|
||||
filter_code.as_ref(),
|
||||
"transaction observation filter code must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = filter_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
self.capture_session_id = capture_session_id;
|
||||
self.filter_code = filter_code;
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Adds detection and normalization timestamps.
|
||||
pub fn with_timings(
|
||||
mut self,
|
||||
detected_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
normalized_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
) -> Self {
|
||||
self.detected_at = detected_at;
|
||||
self.normalized_at = normalized_at;
|
||||
return self;
|
||||
}
|
||||
|
||||
/// Adds source payload size and hash metadata without retaining the source payload.
|
||||
pub fn with_payload_metadata(
|
||||
mut self,
|
||||
payload_size_bytes: std::option::Option<u64>,
|
||||
source_payload_hash: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let hash_result = validate_optional_owned_text(
|
||||
source_payload_hash.as_ref(),
|
||||
"transaction observation source payload hash must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = hash_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
self.payload_size_bytes = payload_size_bytes;
|
||||
self.source_payload_hash = source_payload_hash;
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
|
||||
/// Replaces the current observation status.
|
||||
pub fn with_status(mut self, status: TransactionObservationStatus) -> Self {
|
||||
self.status = status;
|
||||
return self;
|
||||
}
|
||||
|
||||
/// Adds an acquisition error and marks the observation as failed.
|
||||
pub fn with_error(
|
||||
mut self,
|
||||
error_code: impl std::convert::Into<std::string::String>,
|
||||
error_message: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let error_code_value = error_code.into();
|
||||
let code_result = validate_optional_text(
|
||||
error_code_value.as_str(),
|
||||
"transaction observation error code must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = code_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let message_result = validate_optional_owned_text(
|
||||
error_message.as_ref(),
|
||||
"transaction observation error message must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = message_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
self.error_code = std::option::Option::Some(error_code_value);
|
||||
self.error_message = error_message;
|
||||
self.status = TransactionObservationStatus::Failed;
|
||||
return std::result::Result::Ok(self);
|
||||
}
|
||||
}
|
||||
|
||||
/// Canonical raw transaction lifecycle mark request.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct RawPayloadLifecycleMark {
|
||||
/// Physical canonical raw table name using the `kb_sol_<domain>_<name>` convention.
|
||||
pub raw_table_name: std::string::String,
|
||||
/// Stable canonical raw row key, currently the transaction signature.
|
||||
pub raw_row_key: std::string::String,
|
||||
/// Retention state to record for the canonical payload.
|
||||
pub retention_state: RawPayloadRetentionState,
|
||||
/// Processing state to record for the canonical payload.
|
||||
pub processing_state: RawPayloadProcessingState,
|
||||
/// Optional reason visible in diagnostics.
|
||||
pub reason: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl RawPayloadLifecycleMark {
|
||||
/// Builds a canonical raw payload lifecycle mark after minimal validation.
|
||||
pub fn new(
|
||||
raw_table_name: impl std::convert::Into<std::string::String>,
|
||||
raw_row_key: impl std::convert::Into<std::string::String>,
|
||||
retention_state: RawPayloadRetentionState,
|
||||
processing_state: RawPayloadProcessingState,
|
||||
reason: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let raw_table_name_value = raw_table_name.into();
|
||||
let raw_row_key_value = raw_row_key.into();
|
||||
if raw_table_name_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"raw lifecycle table name must not be empty",
|
||||
));
|
||||
}
|
||||
if raw_row_key_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"raw lifecycle row key must not be empty",
|
||||
));
|
||||
}
|
||||
let reason_result = validate_optional_owned_text(
|
||||
reason.as_ref(),
|
||||
"raw lifecycle reason must not be empty when present",
|
||||
);
|
||||
if let std::result::Result::Err(error) = reason_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
raw_table_name: raw_table_name_value,
|
||||
raw_row_key: raw_row_key_value,
|
||||
retention_state,
|
||||
processing_state,
|
||||
reason,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn validate_required_observation_texts(
|
||||
observation_key: &str,
|
||||
provider: &str,
|
||||
protocol: &str,
|
||||
acquisition_method: &str,
|
||||
) -> ks_core::Result<()> {
|
||||
let values = [
|
||||
(observation_key, "transaction observation key must not be empty"),
|
||||
(provider, "transaction observation provider must not be empty"),
|
||||
(protocol, "transaction observation protocol must not be empty"),
|
||||
(
|
||||
acquisition_method,
|
||||
"transaction observation acquisition method must not be empty",
|
||||
),
|
||||
];
|
||||
for (value, message) in values {
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_optional_text(value: &str, message: &str) -> ks_core::Result<()> {
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_optional_owned_text(
|
||||
value: std::option::Option<&std::string::String>,
|
||||
message: &str,
|
||||
) -> ks_core::Result<()> {
|
||||
if let std::option::Option::Some(text) = value {
|
||||
if text.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn raw_transaction_rejects_empty_signature() {
|
||||
let result = crate::RawTransactionInsert::new(" ", 1, serde_json::json!({"ok": true}), 1);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_transaction_rejects_zero_format_version() {
|
||||
let result = crate::RawTransactionInsert::new("abc", 1, serde_json::json!({"ok": true}), 0);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
fn canonical_transaction_fixture() -> ks_lib::MdCanonicalTransaction {
|
||||
return ks_lib::MdCanonicalTransaction {
|
||||
format_version: ks_lib::MD_CANONICAL_TRANSACTION_FORMAT_VERSION,
|
||||
primary_signature: "2Ana1pUpv2ZbMVkwF5FXapYeBEjdxDatLn7nvJkhgTSXbs59SyZSx866bXirPgj8QQVB57uxHJBG1YFvkRbFj4T".to_string(),
|
||||
slot: 9,
|
||||
block_time: std::option::Option::None,
|
||||
version: ks_lib::MdCanonicalTransactionVersion::Legacy,
|
||||
signatures: std::vec![
|
||||
"2Ana1pUpv2ZbMVkwF5FXapYeBEjdxDatLn7nvJkhgTSXbs59SyZSx866bXirPgj8QQVB57uxHJBG1YFvkRbFj4T".to_string(),
|
||||
],
|
||||
message: ks_lib::MdCanonicalTransactionMessage {
|
||||
header: ks_lib::MdCanonicalMessageHeader {
|
||||
num_required_signatures: 1,
|
||||
num_readonly_signed_accounts: 0,
|
||||
num_readonly_unsigned_accounts: 1,
|
||||
},
|
||||
static_account_keys: std::vec![
|
||||
"11111111111111111111111111111111".to_string(),
|
||||
"ComputeBudget111111111111111111111111111111".to_string(),
|
||||
],
|
||||
recent_blockhash: "11111111111111111111111111111111".to_string(),
|
||||
instructions: std::vec![ks_lib::MdCanonicalCompiledInstruction {
|
||||
program_id_index: 1,
|
||||
account_indexes: std::vec![0],
|
||||
data_base64: "AQ==".to_string(),
|
||||
stack_height: std::option::Option::Some(1),
|
||||
}],
|
||||
address_table_lookups: std::vec::Vec::new(),
|
||||
loaded_addresses: ks_lib::MdCanonicalLoadedAddresses::default(),
|
||||
},
|
||||
metadata: std::option::Option::Some(ks_lib::MdCanonicalTransactionMetadata {
|
||||
status: ks_lib::MdCanonicalTransactionStatus::Success,
|
||||
error: std::option::Option::None,
|
||||
fee: 5000,
|
||||
pre_balances: std::vec![10000, 1],
|
||||
post_balances: std::vec![5000, 1],
|
||||
inner_instructions: std::vec::Vec::new(),
|
||||
log_messages: std::vec::Vec::new(),
|
||||
pre_token_balances: std::vec::Vec::new(),
|
||||
post_token_balances: std::vec::Vec::new(),
|
||||
rewards: std::vec::Vec::new(),
|
||||
return_data: std::option::Option::None,
|
||||
compute_units_consumed: std::option::Option::Some(100),
|
||||
cost_units: std::option::Option::None,
|
||||
}),
|
||||
};
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_transaction_builds_from_canonical_model() {
|
||||
let transaction = canonical_transaction_fixture();
|
||||
let result = crate::RawTransactionInsert::from_canonical(&transaction);
|
||||
let insert = match result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("canonical insert failed: {error}"),
|
||||
};
|
||||
assert_eq!(insert.signature, transaction.primary_signature);
|
||||
assert_eq!(insert.slot, transaction.slot);
|
||||
assert_eq!(
|
||||
insert.canonical_format_version,
|
||||
ks_lib::MD_CANONICAL_TRANSACTION_FORMAT_VERSION
|
||||
);
|
||||
assert_eq!(insert.canonical_json_hash.as_deref().map(|value| return value.len()), Some(64));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_transaction_accepts_canonical_payload() {
|
||||
let result = crate::RawTransactionInsert::new("abc", 1, serde_json::json!({"ok": true}), 1);
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transaction_observation_rejects_empty_provider() {
|
||||
let result = crate::TransactionObservationInsert::new(
|
||||
"obs:1",
|
||||
" ",
|
||||
"solana_http",
|
||||
"getTransaction",
|
||||
crate::TransactionObservationOrigin::Backfill,
|
||||
chrono::Utc::now(),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transaction_observation_accepts_signatureless_failure_candidate() {
|
||||
let result = crate::TransactionObservationInsert::new(
|
||||
"obs:1",
|
||||
"helius",
|
||||
"solana_http",
|
||||
"getTransaction",
|
||||
crate::TransactionObservationOrigin::Repair,
|
||||
chrono::Utc::now(),
|
||||
);
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn raw_lifecycle_rejects_empty_reason_when_present() {
|
||||
let result = crate::RawPayloadLifecycleMark::new(
|
||||
"kb_sol_raw_transactions",
|
||||
"signature",
|
||||
crate::RawPayloadRetentionState::Full,
|
||||
crate::RawPayloadProcessingState::Received,
|
||||
std::option::Option::Some(" ".to_string()),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
125
ks-store/src/contracts/dto/store.rs
Normal file
125
ks-store/src/contracts/dto/store.rs
Normal file
@@ -0,0 +1,125 @@
|
||||
// file: ks-store/src/contracts/dto/store.rs
|
||||
// version: 2
|
||||
|
||||
//! Store backend diagnostic DTOs.
|
||||
|
||||
/// Store backend kind contract.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreBackendKind {
|
||||
/// PostgreSQL backend.
|
||||
Postgres,
|
||||
/// SQLite backend retained for tests and legacy imports.
|
||||
Sqlite,
|
||||
/// In-memory backend used by offline tests.
|
||||
Memory,
|
||||
/// Backend is not known.
|
||||
Unknown,
|
||||
}
|
||||
|
||||
/// Store migration status contract.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreMigrationStatus {
|
||||
/// Migration status is not known yet.
|
||||
Unknown,
|
||||
/// Store has no migration table or migration history yet.
|
||||
NotInitialized,
|
||||
/// Store migrations are current.
|
||||
Current,
|
||||
/// Store has pending migrations.
|
||||
Pending,
|
||||
/// Store migration history is inconsistent.
|
||||
Drift,
|
||||
/// Store migration check failed.
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// Store backend diagnostic contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreBackendDescriptor {
|
||||
/// Store backend kind.
|
||||
pub backend_kind: StoreBackendKind,
|
||||
/// Human-readable backend label.
|
||||
pub backend_label: std::string::String,
|
||||
/// Masked DSN or connection descriptor safe for diagnostics.
|
||||
pub masked_dsn: std::option::Option<std::string::String>,
|
||||
/// Current PostgreSQL schema or equivalent namespace when known.
|
||||
pub current_schema: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl StoreBackendDescriptor {
|
||||
/// Builds a store backend descriptor after minimal validation.
|
||||
pub fn new(
|
||||
backend_kind: StoreBackendKind,
|
||||
backend_label: impl std::convert::Into<std::string::String>,
|
||||
masked_dsn: std::option::Option<std::string::String>,
|
||||
current_schema: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let backend_label_value = backend_label.into();
|
||||
if backend_label_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"store backend label must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
backend_kind,
|
||||
backend_label: backend_label_value,
|
||||
masked_dsn,
|
||||
current_schema,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreMigrationSnapshot {
|
||||
/// Current migration status.
|
||||
pub status: StoreMigrationStatus,
|
||||
/// Last applied migration identifier when known.
|
||||
pub current_version: std::option::Option<std::string::String>,
|
||||
/// Pending migration identifiers when known.
|
||||
pub pending_versions: std::vec::Vec<std::string::String>,
|
||||
/// Optional human-readable diagnostic message.
|
||||
pub message: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl StoreMigrationSnapshot {
|
||||
/// Builds a migration snapshot from explicit values.
|
||||
pub fn new(
|
||||
status: StoreMigrationStatus,
|
||||
current_version: std::option::Option<std::string::String>,
|
||||
pending_versions: std::vec::Vec<std::string::String>,
|
||||
message: std::option::Option<std::string::String>,
|
||||
) -> Self {
|
||||
return Self {
|
||||
status,
|
||||
current_version,
|
||||
pending_versions,
|
||||
message,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn store_descriptor_rejects_empty_label() {
|
||||
let result = crate::StoreBackendDescriptor::new(
|
||||
crate::StoreBackendKind::Postgres,
|
||||
" ",
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn migration_snapshot_keeps_pending_versions() {
|
||||
let snapshot = crate::StoreMigrationSnapshot::new(
|
||||
crate::StoreMigrationStatus::Pending,
|
||||
std::option::Option::None,
|
||||
std::vec![std::string::String::from("0001")],
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert_eq!(snapshot.pending_versions.len(), 1);
|
||||
}
|
||||
}
|
||||
32
ks-store/src/contracts/entity.rs
Normal file
32
ks-store/src/contracts/entity.rs
Normal file
@@ -0,0 +1,32 @@
|
||||
// file: ks-store/src/contracts/entity.rs
|
||||
// version: 3
|
||||
|
||||
//! Backend-neutral SQL-like entity exports for storage adapters.
|
||||
|
||||
mod core;
|
||||
mod event;
|
||||
mod ledger;
|
||||
mod raw;
|
||||
|
||||
/// Core account key SQL-like row contract.
|
||||
pub use self::core::CoreAccountKeyRow;
|
||||
/// Core balance change SQL-like row contract.
|
||||
pub use self::core::CoreBalanceChangeRow;
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
pub use self::core::CoreInnerInstructionRow;
|
||||
/// Core instruction SQL-like row contract.
|
||||
pub use self::core::CoreInstructionRow;
|
||||
/// Core log SQL-like row contract.
|
||||
pub use self::core::CoreLogRow;
|
||||
/// Core transaction SQL-like row contract.
|
||||
pub use self::core::CoreTransactionRow;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::event::DecodedEventRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::event::MaterializedEventRow;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::ledger::ProcessingLedgerRow;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
pub use self::raw::RawTransactionRow;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
pub use self::raw::TransactionObservationRow;
|
||||
166
ks-store/src/contracts/entity/core.rs
Normal file
166
ks-store/src/contracts/entity/core.rs
Normal file
@@ -0,0 +1,166 @@
|
||||
// file: ks-store/src/contracts/entity/core.rs
|
||||
// version: 2
|
||||
|
||||
//! Core Solana SQL-like entities.
|
||||
|
||||
/// Core transaction SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreTransactionRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Whether the transaction failed on-chain.
|
||||
pub failed: bool,
|
||||
/// Optional canonical raw transaction row id used for lineage when available.
|
||||
pub raw_transaction_id: std::option::Option<i64>,
|
||||
/// Optional raw error JSON extracted from transaction metadata.
|
||||
pub err_json: std::option::Option<serde_json::Value>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Last update timestamp.
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core account key SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreAccountKeyRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Parent core transaction technical key.
|
||||
pub transaction_id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Stable account index after static and loaded keys are resolved.
|
||||
pub account_index: i32,
|
||||
/// Account public key as non-empty base58 text.
|
||||
pub account_key: std::string::String,
|
||||
/// Source category for this account key.
|
||||
pub source: crate::CoreAccountKeySource,
|
||||
/// Whether the resolved account is writable for the transaction.
|
||||
pub writable: bool,
|
||||
/// Whether the resolved account signed the transaction.
|
||||
pub signer: bool,
|
||||
/// Whether the resolved account is executable when known.
|
||||
pub executable: std::option::Option<bool>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core instruction SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInstructionRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Parent core transaction technical key.
|
||||
pub transaction_id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Instruction accounts as JSON, preserving unresolved forms when needed.
|
||||
pub accounts_json: serde_json::Value,
|
||||
/// Instruction payload JSON while it is still retained in the hot store.
|
||||
pub payload_json: std::option::Option<serde_json::Value>,
|
||||
/// Optional digest of the instruction payload after compaction or purge.
|
||||
pub payload_json_hash: std::option::Option<std::string::String>,
|
||||
/// Current processing state used by instruction-level replay.
|
||||
pub processing_state: crate::CoreInstructionProcessingState,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Last lifecycle update timestamp.
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInnerInstructionRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Parent core transaction technical key.
|
||||
pub transaction_id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Parent top-level or inner instruction path.
|
||||
pub parent_instruction_path: std::string::String,
|
||||
/// Stable inner instruction path, for example `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Inner instruction accounts as JSON.
|
||||
pub accounts_json: serde_json::Value,
|
||||
/// Inner instruction payload JSON while it is retained in the hot store.
|
||||
pub payload_json: std::option::Option<serde_json::Value>,
|
||||
/// Optional digest of the inner instruction payload after compaction or purge.
|
||||
pub payload_json_hash: std::option::Option<std::string::String>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core log SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreLogRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Parent core transaction technical key.
|
||||
pub transaction_id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Log index preserving transaction log order.
|
||||
pub log_index: i32,
|
||||
/// Optional instruction path resolved from invocation depth when known.
|
||||
pub instruction_path: std::option::Option<std::string::String>,
|
||||
/// Optional program id resolved from the log line or invocation context.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Original log text while it is retained in the hot store.
|
||||
pub log_text: std::option::Option<std::string::String>,
|
||||
/// Optional digest of the log text after compaction or purge.
|
||||
pub log_text_hash: std::option::Option<std::string::String>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core balance change SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreBalanceChangeRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Parent core transaction technical key.
|
||||
pub transaction_id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Stable balance change index preserving extraction order.
|
||||
pub balance_change_index: i32,
|
||||
/// Balance change family.
|
||||
pub balance_kind: crate::CoreBalanceChangeKind,
|
||||
/// Optional account index when available.
|
||||
pub account_index: std::option::Option<i32>,
|
||||
/// Optional account public key when available.
|
||||
pub account_key: std::option::Option<std::string::String>,
|
||||
/// Optional SPL token mint for token balances.
|
||||
pub mint: std::option::Option<std::string::String>,
|
||||
/// Optional owner public key for token balances.
|
||||
pub owner: std::option::Option<std::string::String>,
|
||||
/// Pre-balance JSON value preserving RPC representation.
|
||||
pub pre_balance_json: std::option::Option<serde_json::Value>,
|
||||
/// Post-balance JSON value preserving RPC representation.
|
||||
pub post_balance_json: std::option::Option<serde_json::Value>,
|
||||
/// Delta JSON value preserving integer or decimal-safe representation.
|
||||
pub delta_json: std::option::Option<serde_json::Value>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
46
ks-store/src/contracts/entity/event.rs
Normal file
46
ks-store/src/contracts/entity/event.rs
Normal file
@@ -0,0 +1,46 @@
|
||||
// file: ks-store/src/contracts/entity/event.rs
|
||||
// version: 2
|
||||
|
||||
//! Decoded and materialized event SQL-like entities.
|
||||
|
||||
/// Decoded event SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodedEventRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Protocol family code.
|
||||
pub protocol_code: std::string::String,
|
||||
/// Protocol surface code.
|
||||
pub surface_code: std::string::String,
|
||||
/// Canonical event code.
|
||||
pub event_code: std::string::String,
|
||||
/// Decoded payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Materialized event SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Source transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Materialized event family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Materialized payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
37
ks-store/src/contracts/entity/ledger.rs
Normal file
37
ks-store/src/contracts/entity/ledger.rs
Normal file
@@ -0,0 +1,37 @@
|
||||
// file: ks-store/src/contracts/entity/ledger.rs
|
||||
// version: 2
|
||||
|
||||
//! Processing ledger SQL-like entities.
|
||||
|
||||
/// Processing ledger SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct ProcessingLedgerRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Processing stage name.
|
||||
pub stage: std::string::String,
|
||||
/// Processor implementation name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Processor semantic version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable input key, usually a transaction signature.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic input hash.
|
||||
pub input_hash: std::string::String,
|
||||
/// Current processing status.
|
||||
pub status: crate::ProcessingLedgerStatus,
|
||||
/// Number of started attempts for this processor identity.
|
||||
pub attempt_count: i32,
|
||||
/// Optional start timestamp for the latest attempt.
|
||||
pub started_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Optional finish timestamp for the latest terminal attempt.
|
||||
pub finished_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Optional machine-readable error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional diagnostic error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Last update timestamp.
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
78
ks-store/src/contracts/entity/raw.rs
Normal file
78
ks-store/src/contracts/entity/raw.rs
Normal file
@@ -0,0 +1,78 @@
|
||||
// file: ks-store/src/contracts/entity/raw.rs
|
||||
// version: 2
|
||||
|
||||
//! Canonical Solana transaction and acquisition observation SQL-like entities.
|
||||
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct RawTransactionRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Canonical source-independent transaction document while retained in the hot store.
|
||||
pub canonical_json: std::option::Option<serde_json::Value>,
|
||||
/// Optional deterministic digest of the canonical document.
|
||||
pub canonical_json_hash: std::option::Option<std::string::String>,
|
||||
/// Version of the canonical transaction document contract.
|
||||
pub canonical_format_version: i32,
|
||||
/// Current raw payload retention state.
|
||||
pub retention_state: crate::RawPayloadRetentionState,
|
||||
/// Current processing state derived from this canonical transaction.
|
||||
pub processing_state: crate::RawPayloadProcessingState,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Last lifecycle update timestamp.
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct TransactionObservationRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Optional linked canonical transaction row id.
|
||||
pub raw_transaction_id: std::option::Option<i64>,
|
||||
/// Stable observation key used for deduplication.
|
||||
pub observation_key: std::string::String,
|
||||
/// Optional transaction signature when known.
|
||||
pub signature: std::option::Option<std::string::String>,
|
||||
/// Optional transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: std::option::Option<i64>,
|
||||
/// Provider code, for example `helius`, `triton` or `legacy_unknown`.
|
||||
pub provider: std::string::String,
|
||||
/// Optional endpoint code from the active configuration.
|
||||
pub endpoint_code: std::option::Option<std::string::String>,
|
||||
/// Acquisition protocol, for example `solana_http`, `solana_websocket` or `yellowstone_grpc`.
|
||||
pub protocol: std::string::String,
|
||||
/// Acquisition method, for example `getTransaction`, `transactionSubscribe` or `transactions`.
|
||||
pub acquisition_method: std::string::String,
|
||||
/// Acquisition origin category.
|
||||
pub origin: crate::TransactionObservationOrigin,
|
||||
/// Optional Solana commitment.
|
||||
pub commitment: std::option::Option<std::string::String>,
|
||||
/// Optional capture session identifier.
|
||||
pub capture_session_id: std::option::Option<std::string::String>,
|
||||
/// Optional configured filter code.
|
||||
pub filter_code: std::option::Option<std::string::String>,
|
||||
/// Optional first detection timestamp.
|
||||
pub detected_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Timestamp at which the source payload was received locally.
|
||||
pub received_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Optional timestamp at which canonical normalization completed.
|
||||
pub normalized_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Timestamp at which the observation was persisted.
|
||||
pub persisted_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Optional uncompressed source payload size in bytes.
|
||||
pub payload_size_bytes: std::option::Option<i64>,
|
||||
/// Optional digest of the source-specific payload without retaining that payload.
|
||||
pub source_payload_hash: std::option::Option<std::string::String>,
|
||||
/// Current observation status.
|
||||
pub status: crate::TransactionObservationStatus,
|
||||
/// Optional machine-readable error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional diagnostic error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
}
|
||||
27
ks-store/src/contracts/error.rs
Normal file
27
ks-store/src/contracts/error.rs
Normal file
@@ -0,0 +1,27 @@
|
||||
// file: ks-store/src/contracts/error.rs
|
||||
// version: 2
|
||||
|
||||
//! Backend-neutral storage error helpers.
|
||||
|
||||
/// Creates a storage contract error with a stable code.
|
||||
pub fn storage_contract_error(code: &str, message: &str) -> ks_core::Error {
|
||||
if code.trim().is_empty() {
|
||||
return ks_core::Error::db(message);
|
||||
}
|
||||
return ks_core::Error::new(code, message);
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn storage_error_preserves_non_empty_code() {
|
||||
let error = crate::storage_contract_error("store_contract", "invalid value");
|
||||
assert_eq!(error.code(), "store_contract");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn storage_error_falls_back_to_db_for_empty_code() {
|
||||
let error = crate::storage_contract_error(" ", "invalid value");
|
||||
assert_eq!(error.code(), "db");
|
||||
}
|
||||
}
|
||||
58
ks-store/src/contracts/health.rs
Normal file
58
ks-store/src/contracts/health.rs
Normal file
@@ -0,0 +1,58 @@
|
||||
// file: ks-store/src/contracts/health.rs
|
||||
// version: 2
|
||||
|
||||
//! Backend-neutral health contracts for storage implementations.
|
||||
|
||||
/// Store backend health status.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreHealthStatus {
|
||||
/// Health is not known yet.
|
||||
Unknown,
|
||||
/// Backend is reachable and usable.
|
||||
Healthy,
|
||||
/// Backend is reachable but not fully usable.
|
||||
Degraded,
|
||||
/// Backend is not usable.
|
||||
Unhealthy,
|
||||
}
|
||||
|
||||
/// Store backend health snapshot.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreHealthSnapshot {
|
||||
/// Stable backend code such as `postgres` or `sqlite`.
|
||||
pub backend: std::string::String,
|
||||
/// Current health status.
|
||||
pub status: StoreHealthStatus,
|
||||
/// Optional human-readable diagnostic message.
|
||||
pub message: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl StoreHealthSnapshot {
|
||||
/// Builds a store health snapshot after minimal validation.
|
||||
pub fn new(
|
||||
backend: impl std::convert::Into<std::string::String>,
|
||||
status: StoreHealthStatus,
|
||||
message: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let backend_value = backend.into();
|
||||
if backend_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"store health backend must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self { backend: backend_value, status, message });
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn health_snapshot_rejects_empty_backend() {
|
||||
let result = crate::StoreHealthSnapshot::new(
|
||||
" ",
|
||||
crate::StoreHealthStatus::Unknown,
|
||||
std::option::Option::None,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
74
ks-store/src/contracts/pagination.rs
Normal file
74
ks-store/src/contracts/pagination.rs
Normal file
@@ -0,0 +1,74 @@
|
||||
// file: ks-store/src/contracts/pagination.rs
|
||||
// version: 2
|
||||
|
||||
//! Backend-neutral pagination and sorting contracts for repository operations.
|
||||
|
||||
/// Default page size for repository list operations.
|
||||
pub const DEFAULT_PAGE_SIZE: u16 = 100;
|
||||
/// Maximum page size for repository list operations.
|
||||
pub const MAX_PAGE_SIZE: u16 = 1000;
|
||||
|
||||
/// Sort direction for repository list operations.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum SortDirection {
|
||||
/// Sort values in ascending order.
|
||||
Asc,
|
||||
/// Sort values in descending order.
|
||||
Desc,
|
||||
}
|
||||
|
||||
/// Page request contract for repository list operations.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PageRequest {
|
||||
/// Maximum number of rows to return.
|
||||
pub limit: u16,
|
||||
/// Zero-based row offset.
|
||||
pub offset: u64,
|
||||
}
|
||||
|
||||
impl PageRequest {
|
||||
/// Builds a page request after minimal bounds validation.
|
||||
pub fn new(limit: u16, offset: u64) -> ks_core::Result<Self> {
|
||||
if limit == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"page limit must be greater than zero",
|
||||
));
|
||||
}
|
||||
if limit > crate::MAX_PAGE_SIZE {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"page limit exceeds maximum page size",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self { limit, offset });
|
||||
}
|
||||
|
||||
/// Builds the default first page request.
|
||||
pub fn first_page() -> Self {
|
||||
return Self {
|
||||
limit: crate::DEFAULT_PAGE_SIZE,
|
||||
offset: 0,
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn page_request_rejects_zero_limit() {
|
||||
let result = crate::PageRequest::new(0, 0);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn page_request_rejects_limit_above_maximum() {
|
||||
let result = crate::PageRequest::new(crate::MAX_PAGE_SIZE + 1, 0);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn first_page_uses_default_limit() {
|
||||
let request = crate::PageRequest::first_page();
|
||||
assert_eq!(request.limit, crate::DEFAULT_PAGE_SIZE);
|
||||
assert_eq!(request.offset, 0);
|
||||
}
|
||||
}
|
||||
237
ks-store/src/contracts/repository.rs
Normal file
237
ks-store/src/contracts/repository.rs
Normal file
@@ -0,0 +1,237 @@
|
||||
// file: ks-store/src/contracts/repository.rs
|
||||
// version: 2
|
||||
|
||||
//! Storage trait definitions shared by concrete stores.
|
||||
|
||||
/// Store health storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait StoreHealthStore {
|
||||
/// Reads the backend descriptor visible to diagnostics.
|
||||
async fn backend_descriptor(&self) -> ks_core::Result<crate::StoreBackendDescriptor>;
|
||||
|
||||
/// Reads the current health snapshot.
|
||||
async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot>;
|
||||
|
||||
/// Reads the current migration snapshot when the backend supports migrations.
|
||||
async fn migration_snapshot(&self) -> ks_core::Result<crate::StoreMigrationSnapshot>;
|
||||
}
|
||||
|
||||
/// Canonical raw transaction and acquisition observation storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait RawTransactionStore {
|
||||
/// Returns true when a canonical transaction signature is already stored.
|
||||
async fn has_raw_transaction_signature(
|
||||
&self,
|
||||
signature: &ks_lib::MdSignature,
|
||||
) -> ks_core::Result<bool>;
|
||||
|
||||
/// Returns true when a transaction observation key is already stored.
|
||||
async fn has_transaction_observation_key(&self, observation_key: &str)
|
||||
-> ks_core::Result<bool>;
|
||||
|
||||
/// Stores one canonical source-independent transaction payload.
|
||||
async fn insert_raw_transaction(
|
||||
&self,
|
||||
input: &crate::RawTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores one lightweight transaction acquisition observation.
|
||||
async fn insert_transaction_observation(
|
||||
&self,
|
||||
input: &crate::TransactionObservationInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Updates canonical raw transaction retention and processing metadata.
|
||||
async fn mark_raw_payload_lifecycle(
|
||||
&self,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Core Solana storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait CoreTransactionStore {
|
||||
/// Stores one normalized core transaction.
|
||||
async fn insert_core_transaction(
|
||||
&self,
|
||||
input: &crate::CoreTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores normalized core account keys.
|
||||
async fn insert_core_account_keys(
|
||||
&self,
|
||||
inputs: &[crate::CoreAccountKeyInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores normalized core instructions.
|
||||
async fn insert_core_instructions(
|
||||
&self,
|
||||
inputs: &[crate::CoreInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores normalized core inner instructions.
|
||||
async fn insert_core_inner_instructions(
|
||||
&self,
|
||||
inputs: &[crate::CoreInnerInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores normalized core logs.
|
||||
async fn insert_core_logs(
|
||||
&self,
|
||||
inputs: &[crate::CoreLogInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Stores normalized core balance changes.
|
||||
async fn insert_core_balance_changes(
|
||||
&self,
|
||||
inputs: &[crate::CoreBalanceChangeInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Lists normalized core instructions selected for replay or first processing.
|
||||
async fn list_core_instructions_for_replay(
|
||||
&self,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::CoreInstructionRow>>;
|
||||
|
||||
/// Lists replay inputs with instruction context, logs, balances and account keys.
|
||||
async fn list_core_instruction_replay_inputs(
|
||||
&self,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>>;
|
||||
|
||||
/// Updates one normalized core instruction lifecycle state.
|
||||
async fn mark_core_instruction_lifecycle(
|
||||
&self,
|
||||
mark: &crate::CoreInstructionLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Canonical transaction to core extraction storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait CoreExtractionStore {
|
||||
/// Lists canonical raw transactions selected for core extraction.
|
||||
async fn list_raw_transactions_for_core_extraction(
|
||||
&self,
|
||||
filter: &crate::CoreExtractionSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>>;
|
||||
|
||||
/// Returns true when the same processor version already succeeded for the same input hash.
|
||||
async fn is_core_extraction_current(
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool>;
|
||||
|
||||
/// Atomically replaces one signature core graph and marks the processing ledger as succeeded.
|
||||
async fn persist_core_extraction(
|
||||
&self,
|
||||
bundle: &crate::CoreExtractionBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Persists one failed extraction attempt and marks the canonical raw transaction as failed.
|
||||
async fn mark_core_extraction_failed(
|
||||
&self,
|
||||
failure: &crate::CoreExtractionFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Contextual instruction decode and materialization storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait DecodePipelineStore {
|
||||
/// Lists contextual core instructions selected for a bounded decode campaign.
|
||||
async fn list_decode_inputs(
|
||||
&self,
|
||||
filter: &crate::DecodeSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>>;
|
||||
|
||||
/// Returns true when the same processor version already succeeded for the same input hash.
|
||||
async fn is_decode_current(
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool>;
|
||||
|
||||
/// Replaces declarations owned by one decoder version.
|
||||
async fn persist_decode_coverage_declarations(
|
||||
&self,
|
||||
declarations: &[crate::DecodeCoverageDeclarationInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Atomically persists decoded observations, coverage and the common ledger.
|
||||
async fn persist_decode_result(
|
||||
&self,
|
||||
bundle: &crate::DecodePersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Persists a failed decode attempt and marks the source instruction as failed.
|
||||
async fn mark_decode_failed(
|
||||
&self,
|
||||
failure: &crate::DecodeFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Atomically persists materialized outputs and the common ledger.
|
||||
async fn persist_materialization_result(
|
||||
&self,
|
||||
bundle: &crate::MaterializationPersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Reads aggregated machine-readable coverage diagnostics.
|
||||
async fn list_decode_coverage_summary(
|
||||
&self,
|
||||
processor_name: std::option::Option<&str>,
|
||||
processor_version: std::option::Option<&str>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>>;
|
||||
|
||||
/// Lists bounded materialized outputs for read-only application views.
|
||||
async fn list_materialized_events(
|
||||
&self,
|
||||
filter: &crate::MaterializedEventFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>>;
|
||||
}
|
||||
|
||||
/// Program observation storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait ProgramObservationStore {
|
||||
/// Stores program observations.
|
||||
async fn store_observations(
|
||||
&self,
|
||||
observations: &[ks_lib::MdProgramObservation],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Decoded event storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait DecodedEventStore {
|
||||
/// Stores decoded protocol events.
|
||||
async fn store_decoded_events(
|
||||
&self,
|
||||
events: &[crate::DecodedEventInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Materialized event storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait MaterializedEventStore {
|
||||
/// Stores materialized business events.
|
||||
async fn store_materialized_events(
|
||||
&self,
|
||||
events: &[crate::MaterializedEventInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Processing ledger storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait ProcessingLedgerStore {
|
||||
/// Marks an input as processed for a module version.
|
||||
async fn mark_processed(
|
||||
&self,
|
||||
mark: &crate::ProcessingLedgerMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Returns true when an input was already processed for a module version.
|
||||
async fn is_processed(&self, mark: &crate::ProcessingLedgerMark) -> ks_core::Result<bool>;
|
||||
}
|
||||
257
ks-store/src/lib.rs
Normal file
257
ks-store/src/lib.rs
Normal file
@@ -0,0 +1,257 @@
|
||||
// file: ks-store/src/lib.rs
|
||||
// version: 5
|
||||
|
||||
//! Backend-neutral storage contracts and PostgreSQL implementation.
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
#![forbid(unsafe_code)]
|
||||
|
||||
mod constants;
|
||||
mod contracts;
|
||||
mod postgres;
|
||||
|
||||
/// Canonical tracing target for storage operations.
|
||||
pub(crate) use self::constants::TRACING_TARGET;
|
||||
|
||||
/// Core account key insert contract.
|
||||
pub use self::contracts::CoreAccountKeyInsert;
|
||||
/// Core account key SQL-like row contract.
|
||||
pub use self::contracts::CoreAccountKeyRow;
|
||||
/// Core account key source category.
|
||||
pub use self::contracts::CoreAccountKeySource;
|
||||
/// Core balance change insert contract.
|
||||
pub use self::contracts::CoreBalanceChangeInsert;
|
||||
/// Core balance change kind.
|
||||
pub use self::contracts::CoreBalanceChangeKind;
|
||||
/// Core balance change SQL-like row contract.
|
||||
pub use self::contracts::CoreBalanceChangeRow;
|
||||
/// Complete normalized core extraction write bundle.
|
||||
pub use self::contracts::CoreExtractionBundle;
|
||||
/// Failure details persisted for one canonical to core extraction attempt.
|
||||
pub use self::contracts::CoreExtractionFailure;
|
||||
/// Bounded canonical transaction selection filter for core extraction.
|
||||
pub use self::contracts::CoreExtractionSelectionFilter;
|
||||
/// Canonical transaction to core extraction storage behavior.
|
||||
pub use self::contracts::CoreExtractionStore;
|
||||
/// Core inner instruction insert contract.
|
||||
pub use self::contracts::CoreInnerInstructionInsert;
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
pub use self::contracts::CoreInnerInstructionRow;
|
||||
/// Core instruction insert contract.
|
||||
pub use self::contracts::CoreInstructionInsert;
|
||||
/// Core instruction lifecycle mark request.
|
||||
pub use self::contracts::CoreInstructionLifecycleMark;
|
||||
/// Core instruction processing state.
|
||||
pub use self::contracts::CoreInstructionProcessingState;
|
||||
/// Core instruction replay filter contract.
|
||||
pub use self::contracts::CoreInstructionReplayFilter;
|
||||
/// Core instruction SQL-like row contract.
|
||||
pub use self::contracts::CoreInstructionRow;
|
||||
/// Core log insert contract.
|
||||
pub use self::contracts::CoreLogInsert;
|
||||
/// Core log SQL-like row contract.
|
||||
pub use self::contracts::CoreLogRow;
|
||||
/// Core transaction insert contract.
|
||||
pub use self::contracts::CoreTransactionInsert;
|
||||
/// Core transaction SQL-like row contract.
|
||||
pub use self::contracts::CoreTransactionRow;
|
||||
/// Core Solana storage behavior.
|
||||
pub use self::contracts::CoreTransactionStore;
|
||||
/// Default repository page size.
|
||||
pub use self::contracts::DEFAULT_PAGE_SIZE;
|
||||
/// One machine-readable decoder coverage declaration row.
|
||||
pub use self::contracts::DecodeCoverageDeclarationInsert;
|
||||
/// One observed coverage classification row owned by one decode attempt.
|
||||
pub use self::contracts::DecodeCoverageObservationInsert;
|
||||
/// One row of aggregated decoder coverage diagnostics.
|
||||
pub use self::contracts::DecodeCoverageSummaryRow;
|
||||
/// Failed decode attempt persisted in the common ledger.
|
||||
pub use self::contracts::DecodeFailure;
|
||||
/// One processor-owned decoded observation row.
|
||||
pub use self::contracts::DecodeObservationInsert;
|
||||
/// Atomic persistence bundle for one decoder and one contextual input.
|
||||
pub use self::contracts::DecodePersistenceBundle;
|
||||
/// Contextual instruction decode and materialization storage behavior.
|
||||
pub use self::contracts::DecodePipelineStore;
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
pub use self::contracts::DecodeSelectionFilter;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::contracts::DecodedEventInsert;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::contracts::DecodedEventRow;
|
||||
/// Decoded event storage behavior.
|
||||
pub use self::contracts::DecodedEventStore;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::contracts::InsertOutcome;
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
pub use self::contracts::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
|
||||
/// Maximum repository page size.
|
||||
pub use self::contracts::MAX_PAGE_SIZE;
|
||||
/// Atomic persistence bundle for one materializer and one decoded observation.
|
||||
pub use self::contracts::MaterializationPersistenceBundle;
|
||||
/// Bounded read-only materialized event selection.
|
||||
pub use self::contracts::MaterializedEventFilter;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::contracts::MaterializedEventInsert;
|
||||
/// One materialized output returned by a bounded query.
|
||||
pub use self::contracts::MaterializedEventQueryRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::contracts::MaterializedEventRow;
|
||||
/// Materialized event storage behavior.
|
||||
pub use self::contracts::MaterializedEventStore;
|
||||
/// One processor-owned materialized output row.
|
||||
pub use self::contracts::MaterializedOutputInsert;
|
||||
/// Page request contract for repository list operations.
|
||||
pub use self::contracts::PageRequest;
|
||||
/// Stable processing ledger identity.
|
||||
pub use self::contracts::ProcessingLedgerIdentity;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::contracts::ProcessingLedgerMark;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::contracts::ProcessingLedgerRow;
|
||||
/// Stable processing ledger status.
|
||||
pub use self::contracts::ProcessingLedgerStatus;
|
||||
/// Processing ledger storage behavior.
|
||||
pub use self::contracts::ProcessingLedgerStore;
|
||||
/// Program observation storage behavior.
|
||||
pub use self::contracts::ProgramObservationStore;
|
||||
/// Raw payload lifecycle mark request.
|
||||
pub use self::contracts::RawPayloadLifecycleMark;
|
||||
/// Raw payload processing state.
|
||||
pub use self::contracts::RawPayloadProcessingState;
|
||||
/// Raw payload retention state.
|
||||
pub use self::contracts::RawPayloadRetentionState;
|
||||
/// Canonical raw Solana transaction insert contract.
|
||||
pub use self::contracts::RawTransactionInsert;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
pub use self::contracts::RawTransactionRow;
|
||||
/// Raw transaction storage behavior.
|
||||
pub use self::contracts::RawTransactionStore;
|
||||
/// Sort direction for repository list operations.
|
||||
pub use self::contracts::SortDirection;
|
||||
/// Store backend diagnostic contract.
|
||||
pub use self::contracts::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::contracts::StoreBackendKind;
|
||||
/// Store backend health snapshot.
|
||||
pub use self::contracts::StoreHealthSnapshot;
|
||||
/// Store backend health status.
|
||||
pub use self::contracts::StoreHealthStatus;
|
||||
/// Store health storage behavior.
|
||||
pub use self::contracts::StoreHealthStore;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::contracts::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::contracts::StoreMigrationStatus;
|
||||
/// Transaction acquisition observation insert contract.
|
||||
pub use self::contracts::TransactionObservationInsert;
|
||||
/// Transaction acquisition observation origin.
|
||||
pub use self::contracts::TransactionObservationOrigin;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
pub use self::contracts::TransactionObservationRow;
|
||||
/// Transaction acquisition observation status.
|
||||
pub use self::contracts::TransactionObservationStatus;
|
||||
/// Storage error helper functions.
|
||||
pub use self::contracts::storage_contract_error;
|
||||
/// Allowed Solana table domains encoded in table names.
|
||||
pub use self::postgres::ALLOWED_SOLANA_TABLE_DOMAINS;
|
||||
/// Core account key table name.
|
||||
pub use self::postgres::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
/// Core balance change table name.
|
||||
pub use self::postgres::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
/// Core inner instruction table name.
|
||||
pub use self::postgres::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core instruction table name.
|
||||
pub use self::postgres::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core log table name.
|
||||
pub use self::postgres::CORE_LOGS_TABLE_NAME;
|
||||
/// Core Solana table names.
|
||||
pub use self::postgres::CORE_STORE_TABLE_NAMES;
|
||||
/// Core transaction table name.
|
||||
pub use self::postgres::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
/// Machine-readable decoder coverage declaration table name.
|
||||
pub use self::postgres::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
/// Observed decoder coverage classification table name.
|
||||
pub use self::postgres::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
/// Versioned decoded event table name.
|
||||
pub use self::postgres::DECODE_EVENTS_TABLE_NAME;
|
||||
/// Decode and materialization table names.
|
||||
pub use self::postgres::DECODE_STORE_TABLE_NAMES;
|
||||
/// Default PostgreSQL schema policy.
|
||||
pub use self::postgres::DEFAULT_SCHEMA_POLICY;
|
||||
/// Versioned materialized output table name.
|
||||
pub use self::postgres::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::postgres::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// PostgreSQL migration strategy used by this crate.
|
||||
pub use self::postgres::MIGRATION_STRATEGY;
|
||||
/// Migration table name used by sqlx.
|
||||
pub use self::postgres::MIGRATION_TABLE_NAME;
|
||||
/// Processing ledger table name.
|
||||
pub use self::postgres::PROCESSING_LEDGER_TABLE_NAME;
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
pub use self::postgres::PostgresBackendDiagnostics;
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
pub use self::postgres::PostgresReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::postgres::PostgresReplayEntityKind;
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
pub use self::postgres::PostgresReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::postgres::PostgresReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::postgres::PostgresReplayProgramScope;
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
pub use self::postgres::PostgresReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
pub use self::postgres::PostgresReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::postgres::PostgresReplayTransactionFilter;
|
||||
/// PostgreSQL store handle.
|
||||
pub use self::postgres::PostgresStore;
|
||||
/// PostgreSQL store connection options.
|
||||
pub use self::postgres::PostgresStoreOptions;
|
||||
/// PostgreSQL diagnostic table specification.
|
||||
pub use self::postgres::PostgresTableDiagnosticSpec;
|
||||
/// PostgreSQL table diagnostic snapshot.
|
||||
pub use self::postgres::PostgresTableDiagnostics;
|
||||
/// PostgreSQL table statistics snapshot.
|
||||
pub use self::postgres::PostgresTableStatistics;
|
||||
/// Canonical transaction acquisition table names.
|
||||
pub use self::postgres::RAW_STORE_TABLE_NAMES;
|
||||
/// Canonical raw transaction table name.
|
||||
pub use self::postgres::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
/// Solana application table prefix.
|
||||
pub use self::postgres::SOLANA_TABLE_PREFIX;
|
||||
/// Advisory lock id used while initializing store tables.
|
||||
pub use self::postgres::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
/// Transaction acquisition observation table name.
|
||||
pub use self::postgres::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
/// Core store SQL statements applied by the idempotent initializer.
|
||||
pub use self::postgres::core_store_schema_statements;
|
||||
/// Core table diagnostic specs.
|
||||
pub use self::postgres::core_store_table_diagnostic_specs;
|
||||
/// Decode and materialization SQL statements applied by the idempotent initializer.
|
||||
pub use self::postgres::decode_store_schema_statements;
|
||||
/// Decode and materialization table diagnostic specs.
|
||||
pub use self::postgres::decode_store_table_diagnostic_specs;
|
||||
/// Returns true when a Solana table name follows canonical rules.
|
||||
pub use self::postgres::is_valid_solana_table_name;
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub use self::postgres::mask_postgres_dsn;
|
||||
/// Canonical transaction acquisition SQL statements.
|
||||
pub use self::postgres::raw_store_schema_statements;
|
||||
/// Canonical transaction acquisition diagnostic specs.
|
||||
pub use self::postgres::raw_store_table_diagnostic_specs;
|
||||
/// Validates core store table names.
|
||||
pub use self::postgres::validate_core_store_table_names;
|
||||
/// Validates decode and materialization store table names.
|
||||
pub use self::postgres::validate_decode_store_table_names;
|
||||
/// Validates canonical transaction acquisition table names.
|
||||
pub use self::postgres::validate_raw_store_table_names;
|
||||
/// Validates a canonical Solana table name.
|
||||
pub use self::postgres::validate_solana_table_name;
|
||||
/// Current normalized core replay input contract version.
|
||||
pub use ks_lib::MD_CORE_REPLAY_INPUT_CONTRACT_VERSION;
|
||||
/// Decoder replay input containing one instruction plus extracted transaction context.
|
||||
pub use ks_lib::MdCoreInstructionReplayInput;
|
||||
111
ks-store/src/postgres.rs
Normal file
111
ks-store/src/postgres.rs
Normal file
@@ -0,0 +1,111 @@
|
||||
// file: ks-store/src/postgres.rs
|
||||
// version: 4
|
||||
|
||||
//! PostgreSQL storage implementation boundary.
|
||||
|
||||
mod migrations;
|
||||
mod query;
|
||||
mod replay_candidates;
|
||||
mod repository;
|
||||
mod store;
|
||||
#[cfg(test)]
|
||||
mod test_serial;
|
||||
|
||||
/// Allowed Solana table domains encoded in table names.
|
||||
pub use self::migrations::ALLOWED_SOLANA_TABLE_DOMAINS;
|
||||
/// Core account key table name.
|
||||
pub use self::migrations::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
/// Core balance change table name.
|
||||
pub use self::migrations::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
/// Core inner instruction table name.
|
||||
pub use self::migrations::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core instruction table name.
|
||||
pub use self::migrations::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core log table name.
|
||||
pub use self::migrations::CORE_LOGS_TABLE_NAME;
|
||||
/// Core Solana table names introduced by `0.2.4`.
|
||||
pub use self::migrations::CORE_STORE_TABLE_NAMES;
|
||||
/// Core transaction table name.
|
||||
pub use self::migrations::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
/// Machine-readable decoder coverage declaration table name.
|
||||
pub use self::migrations::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
/// Observed decoder coverage classification table name.
|
||||
pub use self::migrations::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
/// Versioned decoded event table name.
|
||||
pub use self::migrations::DECODE_EVENTS_TABLE_NAME;
|
||||
/// Decode and materialization table names introduced by `0.4.0`.
|
||||
pub use self::migrations::DECODE_STORE_TABLE_NAMES;
|
||||
/// Default PostgreSQL schema policy.
|
||||
pub use self::migrations::DEFAULT_SCHEMA_POLICY;
|
||||
/// Versioned materialized output table name.
|
||||
pub use self::migrations::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
/// PostgreSQL migration strategy used by this crate.
|
||||
pub use self::migrations::MIGRATION_STRATEGY;
|
||||
/// Migration table name used by sqlx when migrations are enabled later.
|
||||
pub use self::migrations::MIGRATION_TABLE_NAME;
|
||||
/// Processing ledger table name.
|
||||
pub use self::migrations::PROCESSING_LEDGER_TABLE_NAME;
|
||||
/// Table diagnostic metadata.
|
||||
pub use self::migrations::PostgresTableDiagnosticSpec;
|
||||
/// Canonical transaction acquisition table names active since `0.3.1`.
|
||||
pub use self::migrations::RAW_STORE_TABLE_NAMES;
|
||||
/// Canonical raw transaction table name.
|
||||
pub use self::migrations::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
/// Solana application table prefix.
|
||||
pub use self::migrations::SOLANA_TABLE_PREFIX;
|
||||
/// Advisory lock id used while initializing store schemas.
|
||||
pub use self::migrations::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
/// Transaction acquisition observation table name.
|
||||
pub use self::migrations::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
/// Core store SQL statements applied by the idempotent initializer.
|
||||
pub use self::migrations::core_store_schema_statements;
|
||||
/// Core table diagnostic specs.
|
||||
pub use self::migrations::core_store_table_diagnostic_specs;
|
||||
/// Decode and materialization SQL statements applied by the idempotent initializer.
|
||||
pub use self::migrations::decode_store_schema_statements;
|
||||
/// Decode and materialization table diagnostic specs.
|
||||
pub use self::migrations::decode_store_table_diagnostic_specs;
|
||||
/// Returns true when a Solana table name follows the canonical prefix and domain rules.
|
||||
pub use self::migrations::is_valid_solana_table_name;
|
||||
/// Canonical transaction acquisition SQL statements applied by the initializer.
|
||||
pub use self::migrations::raw_store_schema_statements;
|
||||
/// Canonical transaction acquisition diagnostic specs.
|
||||
pub use self::migrations::raw_store_table_diagnostic_specs;
|
||||
/// Validates core store table names.
|
||||
pub use self::migrations::validate_core_store_table_names;
|
||||
/// Validates decode and materialization store table names.
|
||||
pub use self::migrations::validate_decode_store_table_names;
|
||||
/// Validates canonical transaction acquisition table names.
|
||||
pub use self::migrations::validate_raw_store_table_names;
|
||||
/// Validates a canonical Solana table name.
|
||||
pub use self::migrations::validate_solana_table_name;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::replay_candidates::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
pub use self::replay_candidates::PostgresReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayEntityKind;
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
pub use self::replay_candidates::PostgresReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::replay_candidates::PostgresReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayProgramScope;
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
pub use self::replay_candidates::PostgresReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
pub use self::replay_candidates::PostgresReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayTransactionFilter;
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
pub use self::store::PostgresBackendDiagnostics;
|
||||
/// Minimal PostgreSQL store handle.
|
||||
pub use self::store::PostgresStore;
|
||||
/// PostgreSQL store connection options.
|
||||
pub use self::store::PostgresStoreOptions;
|
||||
/// PostgreSQL table diagnostic snapshot.
|
||||
pub use self::store::PostgresTableDiagnostics;
|
||||
/// PostgreSQL table statistics snapshot.
|
||||
pub use self::store::PostgresTableStatistics;
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub use self::store::mask_postgres_dsn;
|
||||
1145
ks-store/src/postgres/migrations.rs
Normal file
1145
ks-store/src/postgres/migrations.rs
Normal file
File diff suppressed because it is too large
Load Diff
52
ks-store/src/postgres/query.rs
Normal file
52
ks-store/src/postgres/query.rs
Normal file
@@ -0,0 +1,52 @@
|
||||
// file: ks-store/src/postgres/query.rs
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL query modules.
|
||||
|
||||
mod core_extraction_queries;
|
||||
mod core_queries;
|
||||
mod decode_pipeline_queries;
|
||||
mod health_queries;
|
||||
mod raw_queries;
|
||||
mod replay_candidate_queries;
|
||||
mod table_diagnostics_queries;
|
||||
|
||||
pub(in crate::postgres) use self::core_extraction_queries::is_core_extraction_current;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::list_raw_transactions_for_core_extraction;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::mark_core_extraction_failed;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::persist_core_extraction;
|
||||
pub(in crate::postgres) use self::core_queries::apply_core_store_schema;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_account_keys;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_balance_changes;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_inner_instructions;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_instructions;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_logs;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_transaction;
|
||||
pub(in crate::postgres) use self::core_queries::list_core_instruction_replay_inputs;
|
||||
pub(in crate::postgres) use self::core_queries::list_core_instructions_for_replay;
|
||||
pub(in crate::postgres) use self::core_queries::update_core_instruction_lifecycle;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::apply_decode_store_schema;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::is_decode_current;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_coverage_summary;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_inputs;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_materialized_events;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::mark_decode_failed;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_coverage_declarations;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_result;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_materialization_result;
|
||||
pub(in crate::postgres) use self::health_queries::load_current_schema;
|
||||
pub(in crate::postgres) use self::health_queries::load_latest_migration_version;
|
||||
pub(in crate::postgres) use self::health_queries::load_migration_table_name;
|
||||
pub(in crate::postgres) use self::health_queries::load_server_version;
|
||||
pub(in crate::postgres) use self::health_queries::run_health_check;
|
||||
pub(in crate::postgres) use self::raw_queries::apply_raw_store_schema;
|
||||
pub(in crate::postgres) use self::raw_queries::has_raw_transaction_signature;
|
||||
pub(in crate::postgres) use self::raw_queries::has_transaction_observation_key;
|
||||
pub(in crate::postgres) use self::raw_queries::insert_raw_transaction;
|
||||
pub(in crate::postgres) use self::raw_queries::insert_transaction_observation;
|
||||
pub(in crate::postgres) use self::raw_queries::update_raw_payload_lifecycle;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_entity_summaries;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_program_summaries;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_transaction_candidates;
|
||||
pub(in crate::postgres) use self::table_diagnostics_queries::load_table_statistics;
|
||||
pub(in crate::postgres) use self::table_diagnostics_queries::table_exists;
|
||||
1003
ks-store/src/postgres/query/core_extraction_queries.rs
Normal file
1003
ks-store/src/postgres/query/core_extraction_queries.rs
Normal file
File diff suppressed because it is too large
Load Diff
1219
ks-store/src/postgres/query/core_queries.rs
Normal file
1219
ks-store/src/postgres/query/core_queries.rs
Normal file
File diff suppressed because it is too large
Load Diff
1340
ks-store/src/postgres/query/decode_pipeline_queries.rs
Normal file
1340
ks-store/src/postgres/query/decode_pipeline_queries.rs
Normal file
File diff suppressed because it is too large
Load Diff
77
ks-store/src/postgres/query/health_queries.rs
Normal file
77
ks-store/src/postgres/query/health_queries.rs
Normal file
@@ -0,0 +1,77 @@
|
||||
// file: ks-store/src/postgres/query/health_queries.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL health and diagnostic SQL queries.
|
||||
|
||||
pub(in crate::postgres) async fn run_health_check(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, i32>("SELECT 1").fetch_one(pool).await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(_value) => std::result::Result::Ok(()),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres healthcheck failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_current_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::string::String> {
|
||||
let query_result =
|
||||
sqlx::query_scalar::<sqlx::Postgres, std::string::String>("SELECT current_schema()")
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(schema) => std::result::Result::Ok(schema),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres current schema query failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_server_version(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::string::String> {
|
||||
let query_result =
|
||||
sqlx::query_scalar::<sqlx::Postgres, std::string::String>("SELECT version()")
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(version) => std::result::Result::Ok(version),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres version query failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_migration_table_name(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::option::Option<std::string::String>> {
|
||||
let query_result =
|
||||
sqlx::query_scalar::<sqlx::Postgres, std::option::Option<std::string::String>>(
|
||||
"SELECT to_regclass('_sqlx_migrations')::text",
|
||||
)
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(table_name) => std::result::Result::Ok(table_name),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres migration table query failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_latest_migration_version(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::option::Option<std::string::String>> {
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, std::option::Option<std::string::String>>(
|
||||
"SELECT version::text FROM _sqlx_migrations WHERE success = true ORDER BY version DESC LIMIT 1",
|
||||
)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(version) => std::result::Result::Ok(version.flatten()),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres latest migration query failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
506
ks-store/src/postgres/query/raw_queries.rs
Normal file
506
ks-store/src/postgres/query/raw_queries.rs
Normal file
@@ -0,0 +1,506 @@
|
||||
// file: ks-store/src/postgres/query/raw_queries.rs
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL canonical transaction and acquisition observation SQL queries.
|
||||
|
||||
pub(in crate::postgres) async fn apply_raw_store_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<()> {
|
||||
let validation_result = crate::validate_raw_store_table_names();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let transaction_result = pool.begin().await;
|
||||
let mut transaction = match transaction_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical raw store schema transaction failed: {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let lock_result = sqlx::query("SELECT pg_advisory_xact_lock($1)")
|
||||
.bind(crate::STORE_SCHEMA_ADVISORY_LOCK_ID)
|
||||
.execute(&mut *transaction)
|
||||
.await;
|
||||
if let std::result::Result::Err(error) = lock_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical raw store schema advisory lock failed: {error}"
|
||||
)));
|
||||
}
|
||||
for statement in crate::raw_store_schema_statements() {
|
||||
let execution_result = sqlx::query(statement).execute(&mut *transaction).await;
|
||||
if let std::result::Result::Err(error) = execution_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical raw store schema statement failed: {error}"
|
||||
)));
|
||||
}
|
||||
}
|
||||
let commit_result = transaction.commit().await;
|
||||
if let std::result::Result::Err(error) = commit_result {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical raw store schema commit failed: {error}"
|
||||
)));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn has_raw_transaction_signature(
|
||||
pool: &sqlx::PgPool,
|
||||
signature: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
let validation_result = crate::postgres::query::raw_queries::validate_required_text(
|
||||
signature,
|
||||
"canonical raw transaction signature must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, bool>(
|
||||
"SELECT EXISTS(SELECT 1 FROM kb_sol_raw_transactions WHERE signature = $1)",
|
||||
)
|
||||
.bind(signature)
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical transaction signature lookup failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn has_transaction_observation_key(
|
||||
pool: &sqlx::PgPool,
|
||||
observation_key: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
let validation_result = crate::postgres::query::raw_queries::validate_required_text(
|
||||
observation_key,
|
||||
"transaction observation key must not be empty",
|
||||
);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, bool>(
|
||||
"SELECT EXISTS(SELECT 1 FROM kb_sol_obs_transaction_observations WHERE observation_key = $1)",
|
||||
)
|
||||
.bind(observation_key)
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres transaction observation lookup failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_raw_transaction(
|
||||
pool: &sqlx::PgPool,
|
||||
input: &crate::RawTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
let slot_result = crate::postgres::query::raw_queries::sql_slot_from_u64(input.slot);
|
||||
let slot = match slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let version_result = i32::try_from(input.canonical_format_version);
|
||||
let canonical_format_version = match version_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"canonical transaction format version does not fit into SQL INTEGER: {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, i64>(
|
||||
"INSERT INTO kb_sol_raw_transactions (signature, slot, canonical_json, canonical_json_hash, canonical_format_version, retention_state, processing_state) VALUES ($1, $2, $3, $4, $5, 'full', 'received') ON CONFLICT (signature) DO NOTHING RETURNING id",
|
||||
)
|
||||
.bind(input.signature.as_str())
|
||||
.bind(slot)
|
||||
.bind(&input.canonical_json)
|
||||
.bind(input.canonical_json_hash.as_deref())
|
||||
.bind(canonical_format_version)
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(std::option::Option::Some(_id)) => {
|
||||
std::result::Result::Ok(crate::InsertOutcome::new(1, 0, 0))
|
||||
},
|
||||
std::result::Result::Ok(std::option::Option::None) => {
|
||||
std::result::Result::Ok(crate::InsertOutcome::new(0, 0, 1))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres canonical transaction insert failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_transaction_observation(
|
||||
pool: &sqlx::PgPool,
|
||||
input: &crate::TransactionObservationInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
let slot_result = crate::postgres::query::raw_queries::optional_sql_slot_from_u64(input.slot);
|
||||
let slot = match slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let payload_size_result =
|
||||
crate::postgres::query::raw_queries::optional_sql_bigint_from_u64(input.payload_size_bytes);
|
||||
let payload_size_bytes = match payload_size_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let origin =
|
||||
crate::postgres::query::raw_queries::transaction_observation_origin_to_sql(input.origin);
|
||||
let status =
|
||||
crate::postgres::query::raw_queries::transaction_observation_status_to_sql(input.status);
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, i64>(
|
||||
"INSERT INTO kb_sol_obs_transaction_observations (raw_transaction_id, observation_key, signature, slot, provider, endpoint_code, protocol, acquisition_method, origin, commitment, capture_session_id, filter_code, detected_at, received_at, normalized_at, payload_size_bytes, source_payload_hash, status, error_code, error_message) VALUES (COALESCE($1, (SELECT id FROM kb_sol_raw_transactions WHERE signature = $3 LIMIT 1)), $2, $3, $4, $5, $6, $7, $8, $9, $10, $11, $12, $13, $14, $15, $16, $17, $18, $19, $20) ON CONFLICT (observation_key) DO NOTHING RETURNING id",
|
||||
)
|
||||
.bind(input.raw_transaction_id)
|
||||
.bind(input.observation_key.as_str())
|
||||
.bind(input.signature.as_deref())
|
||||
.bind(slot)
|
||||
.bind(input.provider.as_str())
|
||||
.bind(input.endpoint_code.as_deref())
|
||||
.bind(input.protocol.as_str())
|
||||
.bind(input.acquisition_method.as_str())
|
||||
.bind(origin)
|
||||
.bind(input.commitment.as_deref())
|
||||
.bind(input.capture_session_id.as_deref())
|
||||
.bind(input.filter_code.as_deref())
|
||||
.bind(input.detected_at.as_ref())
|
||||
.bind(input.received_at)
|
||||
.bind(input.normalized_at.as_ref())
|
||||
.bind(payload_size_bytes)
|
||||
.bind(input.source_payload_hash.as_deref())
|
||||
.bind(status)
|
||||
.bind(input.error_code.as_deref())
|
||||
.bind(input.error_message.as_deref())
|
||||
.fetch_optional(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(std::option::Option::Some(_id)) => {
|
||||
std::result::Result::Ok(crate::InsertOutcome::new(1, 0, 0))
|
||||
},
|
||||
std::result::Result::Ok(std::option::Option::None) => {
|
||||
std::result::Result::Ok(crate::InsertOutcome::new(0, 0, 1))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres transaction observation insert failed: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn update_raw_payload_lifecycle(
|
||||
pool: &sqlx::PgPool,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
if mark.raw_table_name != crate::RAW_TRANSACTIONS_TABLE_NAME {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"raw lifecycle table name is not supported by the PostgreSQL canonical raw store",
|
||||
));
|
||||
}
|
||||
return crate::postgres::query::raw_queries::update_raw_transaction_lifecycle(pool, mark).await;
|
||||
}
|
||||
|
||||
fn raw_retention_state_to_sql(state: crate::RawPayloadRetentionState) -> &'static str {
|
||||
return match state {
|
||||
crate::RawPayloadRetentionState::Full => "full",
|
||||
crate::RawPayloadRetentionState::Compacted => "compacted",
|
||||
crate::RawPayloadRetentionState::Archived => "archived",
|
||||
crate::RawPayloadRetentionState::Purged => "purged",
|
||||
};
|
||||
}
|
||||
|
||||
fn raw_processing_state_to_sql(state: crate::RawPayloadProcessingState) -> &'static str {
|
||||
return match state {
|
||||
crate::RawPayloadProcessingState::Received => "received",
|
||||
crate::RawPayloadProcessingState::CoreExtracted => "core_extracted",
|
||||
crate::RawPayloadProcessingState::Decoded => "decoded",
|
||||
crate::RawPayloadProcessingState::Materialized => "materialized",
|
||||
crate::RawPayloadProcessingState::Failed => "failed",
|
||||
};
|
||||
}
|
||||
|
||||
fn transaction_observation_origin_to_sql(
|
||||
origin: crate::TransactionObservationOrigin,
|
||||
) -> &'static str {
|
||||
return match origin {
|
||||
crate::TransactionObservationOrigin::Live => "live",
|
||||
crate::TransactionObservationOrigin::Backfill => "backfill",
|
||||
crate::TransactionObservationOrigin::Replay => "replay",
|
||||
crate::TransactionObservationOrigin::Repair => "repair",
|
||||
crate::TransactionObservationOrigin::Migration => "migration",
|
||||
};
|
||||
}
|
||||
|
||||
fn transaction_observation_status_to_sql(
|
||||
status: crate::TransactionObservationStatus,
|
||||
) -> &'static str {
|
||||
return match status {
|
||||
crate::TransactionObservationStatus::Detected => "detected",
|
||||
crate::TransactionObservationStatus::Received => "received",
|
||||
crate::TransactionObservationStatus::Normalized => "normalized",
|
||||
crate::TransactionObservationStatus::Persisted => "persisted",
|
||||
crate::TransactionObservationStatus::Failed => "failed",
|
||||
crate::TransactionObservationStatus::Missing => "missing",
|
||||
};
|
||||
}
|
||||
|
||||
fn sql_slot_from_u64(slot: u64) -> ks_core::Result<i64> {
|
||||
let conversion_result = i64::try_from(slot);
|
||||
return match conversion_result {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"Solana slot does not fit into PostgreSQL BIGINT: {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
fn optional_sql_slot_from_u64(
|
||||
slot: std::option::Option<u64>,
|
||||
) -> ks_core::Result<std::option::Option<i64>> {
|
||||
return crate::postgres::query::raw_queries::optional_sql_bigint_from_u64(slot);
|
||||
}
|
||||
|
||||
fn optional_sql_bigint_from_u64(
|
||||
value: std::option::Option<u64>,
|
||||
) -> ks_core::Result<std::option::Option<i64>> {
|
||||
return match value {
|
||||
std::option::Option::Some(raw_value) => {
|
||||
let conversion_result = i64::try_from(raw_value);
|
||||
match conversion_result {
|
||||
std::result::Result::Ok(converted) => {
|
||||
std::result::Result::Ok(std::option::Option::Some(converted))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(
|
||||
format!("unsigned value does not fit into PostgreSQL BIGINT: {error}"),
|
||||
)),
|
||||
}
|
||||
},
|
||||
std::option::Option::None => std::result::Result::Ok(std::option::Option::None),
|
||||
};
|
||||
}
|
||||
|
||||
fn validate_required_text(value: &str, message: &str) -> ks_core::Result<()> {
|
||||
if value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(message));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
async fn update_raw_transaction_lifecycle(
|
||||
pool: &sqlx::PgPool,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
let retention_state =
|
||||
crate::postgres::query::raw_queries::raw_retention_state_to_sql(mark.retention_state);
|
||||
let processing_state =
|
||||
crate::postgres::query::raw_queries::raw_processing_state_to_sql(mark.processing_state);
|
||||
let query_result = sqlx::query(
|
||||
"UPDATE kb_sol_raw_transactions SET retention_state = $1, processing_state = $2, lifecycle_reason = $3, updated_at = NOW() WHERE signature = $4",
|
||||
)
|
||||
.bind(retention_state)
|
||||
.bind(processing_state)
|
||||
.bind(mark.reason.as_deref())
|
||||
.bind(mark.raw_row_key.as_str())
|
||||
.execute(pool)
|
||||
.await;
|
||||
return crate::postgres::query::raw_queries::outcome_from_update_result(
|
||||
query_result,
|
||||
"postgres canonical raw transaction lifecycle update failed",
|
||||
);
|
||||
}
|
||||
|
||||
fn outcome_from_update_result(
|
||||
query_result: std::result::Result<sqlx::postgres::PgQueryResult, sqlx::Error>,
|
||||
error_prefix: &str,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return match query_result {
|
||||
std::result::Result::Ok(result) => {
|
||||
let rows_affected = result.rows_affected();
|
||||
if rows_affected == 0 {
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(0, 0, 1));
|
||||
}
|
||||
std::result::Result::Ok(crate::InsertOutcome::new(0, rows_affected, 0))
|
||||
},
|
||||
std::result::Result::Err(error) => {
|
||||
std::result::Result::Err(ks_core::Error::db(format!("{error_prefix}: {error}")))
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn retention_state_serializes_to_lower_snake_case() {
|
||||
let value = crate::postgres::query::raw_queries::raw_retention_state_to_sql(
|
||||
crate::RawPayloadRetentionState::Compacted,
|
||||
);
|
||||
assert_eq!(value, "compacted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn processing_state_serializes_to_lower_snake_case() {
|
||||
let value = crate::postgres::query::raw_queries::raw_processing_state_to_sql(
|
||||
crate::RawPayloadProcessingState::CoreExtracted,
|
||||
);
|
||||
assert_eq!(value, "core_extracted");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observation_origin_serializes_to_lower_snake_case() {
|
||||
let value = crate::postgres::query::raw_queries::transaction_observation_origin_to_sql(
|
||||
crate::TransactionObservationOrigin::Backfill,
|
||||
);
|
||||
assert_eq!(value, "backfill");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn observation_status_serializes_to_lower_snake_case() {
|
||||
let value = crate::postgres::query::raw_queries::transaction_observation_status_to_sql(
|
||||
crate::TransactionObservationStatus::Normalized,
|
||||
);
|
||||
assert_eq!(value, "normalized");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn sql_slot_rejects_values_above_bigint() {
|
||||
let result = crate::postgres::query::raw_queries::sql_slot_from_u64(u64::MAX);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_canonical_store_roundtrip_from_env() {
|
||||
let database_url = match std::env::var("KB_POSTGRES_TEST_URL") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected options error: {error}"),
|
||||
};
|
||||
let store_result = crate::PostgresStore::connect(options).await;
|
||||
let store = match store_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected connect error: {error}"),
|
||||
};
|
||||
let schema_result = store.initialize_raw_store_schema().await;
|
||||
if let std::result::Result::Err(error) = schema_result {
|
||||
panic!("unexpected schema error: {error}");
|
||||
}
|
||||
let signature = test_signature();
|
||||
let raw_input_result = crate::RawTransactionInsert::new(
|
||||
signature.clone(),
|
||||
1,
|
||||
serde_json::json!({"source": "test"}),
|
||||
1,
|
||||
);
|
||||
let raw_input = match raw_input_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected raw input error: {error}"),
|
||||
};
|
||||
let first_result =
|
||||
crate::RawTransactionStore::insert_raw_transaction(&store, &raw_input).await;
|
||||
let first_outcome = match first_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected first insert error: {error}"),
|
||||
};
|
||||
assert_eq!(first_outcome.inserted_count, 1);
|
||||
let duplicate_result =
|
||||
crate::RawTransactionStore::insert_raw_transaction(&store, &raw_input).await;
|
||||
let duplicate_outcome = match duplicate_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected duplicate insert error: {error}"),
|
||||
};
|
||||
assert_eq!(duplicate_outcome.skipped_count, 1);
|
||||
let has_result = crate::RawTransactionStore::has_raw_transaction_signature(
|
||||
&store,
|
||||
&ks_lib::MdSignature(signature.clone()),
|
||||
)
|
||||
.await;
|
||||
let has_signature = match has_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected signature lookup error: {error}"),
|
||||
};
|
||||
assert!(has_signature);
|
||||
let observation_key = std::string::String::from("test:http:") + signature.as_str();
|
||||
let observation_result = crate::TransactionObservationInsert::new(
|
||||
observation_key.clone(),
|
||||
"test_provider",
|
||||
"solana_http",
|
||||
"getTransaction",
|
||||
crate::TransactionObservationOrigin::Backfill,
|
||||
chrono::Utc::now(),
|
||||
);
|
||||
let observation = match observation_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected observation error: {error}"),
|
||||
};
|
||||
let identity_result =
|
||||
observation.with_transaction_identity(signature.clone(), std::option::Option::Some(1));
|
||||
let observation_with_identity = match identity_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected identity error: {error}"),
|
||||
};
|
||||
let observation_insert_result = crate::RawTransactionStore::insert_transaction_observation(
|
||||
&store,
|
||||
&observation_with_identity,
|
||||
)
|
||||
.await;
|
||||
let observation_outcome = match observation_insert_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
panic!("unexpected observation insert error: {error}")
|
||||
},
|
||||
};
|
||||
assert_eq!(observation_outcome.inserted_count, 1);
|
||||
let observation_lookup_result =
|
||||
crate::RawTransactionStore::has_transaction_observation_key(
|
||||
&store,
|
||||
observation_key.as_str(),
|
||||
)
|
||||
.await;
|
||||
let has_observation = match observation_lookup_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
panic!("unexpected observation lookup error: {error}")
|
||||
},
|
||||
};
|
||||
assert!(has_observation);
|
||||
let mark_result = crate::RawPayloadLifecycleMark::new(
|
||||
crate::RAW_TRANSACTIONS_TABLE_NAME,
|
||||
signature,
|
||||
crate::RawPayloadRetentionState::Full,
|
||||
crate::RawPayloadProcessingState::CoreExtracted,
|
||||
std::option::Option::Some(std::string::String::from("test extraction")),
|
||||
);
|
||||
let mark = match mark_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected mark error: {error}"),
|
||||
};
|
||||
let mark_outcome_result =
|
||||
crate::RawTransactionStore::mark_raw_payload_lifecycle(&store, &mark).await;
|
||||
let mark_outcome = match mark_outcome_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected lifecycle update error: {error}"),
|
||||
};
|
||||
assert_eq!(mark_outcome.updated_count, 1);
|
||||
return;
|
||||
}
|
||||
|
||||
fn test_signature() -> std::string::String {
|
||||
let now_result = std::time::SystemTime::now().duration_since(std::time::UNIX_EPOCH);
|
||||
let duration = match now_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => {
|
||||
return std::string::String::from("test_signature_fallback");
|
||||
},
|
||||
};
|
||||
return format!("test_signature_{}", duration.as_nanos());
|
||||
}
|
||||
}
|
||||
474
ks-store/src/postgres/query/replay_candidate_queries.rs
Normal file
474
ks-store/src/postgres/query/replay_candidate_queries.rs
Normal file
@@ -0,0 +1,474 @@
|
||||
// file: ks-store/src/postgres/query/replay_candidate_queries.rs
|
||||
// version: 3
|
||||
|
||||
//! Read-only PostgreSQL queries for replay candidate discovery.
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct ReplayTransactionCandidateRow {
|
||||
signature: std::string::String,
|
||||
slot: i64,
|
||||
raw_processing_state: std::string::String,
|
||||
retention_state: std::string::String,
|
||||
has_core_transaction: bool,
|
||||
transaction_failed: std::option::Option<bool>,
|
||||
ledger_status: std::string::String,
|
||||
processor_version: std::option::Option<std::string::String>,
|
||||
attempt_count: i32,
|
||||
outer_instruction_count: i64,
|
||||
inner_instruction_count: i64,
|
||||
outer_program_count: i64,
|
||||
inner_program_count: i64,
|
||||
updated_at: std::string::String,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct ReplayProgramSummaryRow {
|
||||
program_id: std::string::String,
|
||||
transaction_count: i64,
|
||||
outer_instruction_count: i64,
|
||||
inner_instruction_count: i64,
|
||||
log_count: i64,
|
||||
min_slot: i64,
|
||||
max_slot: i64,
|
||||
}
|
||||
|
||||
#[derive(sqlx::FromRow)]
|
||||
struct ReplayEntitySummaryRow {
|
||||
entity_kind: std::string::String,
|
||||
entity_value: std::string::String,
|
||||
transaction_count: i64,
|
||||
occurrence_count: i64,
|
||||
min_slot: i64,
|
||||
max_slot: i64,
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_transaction_candidates(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
|
||||
let min_slot_result =
|
||||
crate::postgres::query::replay_candidate_queries::optional_sql_bigint(filter.min_slot);
|
||||
let min_slot = match min_slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let max_slot_result =
|
||||
crate::postgres::query::replay_candidate_queries::optional_sql_bigint(filter.max_slot);
|
||||
let max_slot = match max_slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let entity_kind = match filter.entity_kind {
|
||||
std::option::Option::Some(value) => std::option::Option::Some(value.as_sql()),
|
||||
std::option::Option::None => std::option::Option::None,
|
||||
};
|
||||
let sql = if filter.newest_first {
|
||||
crate::postgres::query::replay_candidate_queries::transaction_candidate_sql_desc()
|
||||
} else {
|
||||
crate::postgres::query::replay_candidate_queries::transaction_candidate_sql_asc()
|
||||
};
|
||||
let query_result = sqlx::query_as::<
|
||||
sqlx::Postgres,
|
||||
crate::postgres::query::replay_candidate_queries::ReplayTransactionCandidateRow,
|
||||
>(sql)
|
||||
.bind(filter.signature_contains.as_deref())
|
||||
.bind(min_slot)
|
||||
.bind(max_slot)
|
||||
.bind(filter.raw_processing_state.as_deref())
|
||||
.bind(filter.ledger_status.as_deref())
|
||||
.bind(filter.program_id.as_deref())
|
||||
.bind(filter.program_scope.as_sql())
|
||||
.bind(entity_kind)
|
||||
.bind(filter.entity_value.as_deref())
|
||||
.bind(i64::from(filter.limit))
|
||||
.fetch_all(pool)
|
||||
.await;
|
||||
let rows = match query_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres replay transaction candidate query failed: {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayTransactionCandidate {
|
||||
signature: row.signature,
|
||||
slot: row.slot,
|
||||
raw_processing_state: row.raw_processing_state,
|
||||
retention_state: row.retention_state,
|
||||
has_core_transaction: row.has_core_transaction,
|
||||
transaction_failed: row.transaction_failed,
|
||||
ledger_status: row.ledger_status,
|
||||
processor_version: row.processor_version,
|
||||
attempt_count: row.attempt_count,
|
||||
outer_instruction_count: row.outer_instruction_count,
|
||||
inner_instruction_count: row.inner_instruction_count,
|
||||
outer_program_count: row.outer_program_count,
|
||||
inner_program_count: row.inner_program_count,
|
||||
updated_at: row.updated_at,
|
||||
});
|
||||
}
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_program_summaries(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
|
||||
let query_result = sqlx::query_as::<sqlx::Postgres, crate::postgres::query::replay_candidate_queries::ReplayProgramSummaryRow>(
|
||||
r#"WITH occurrences AS (
|
||||
SELECT program_id, signature, slot, 'outer'::TEXT AS scope FROM kb_sol_core_instructions
|
||||
UNION ALL
|
||||
SELECT program_id, signature, slot, 'inner'::TEXT AS scope FROM kb_sol_core_inner_instructions
|
||||
UNION ALL
|
||||
SELECT program_id, signature, slot, 'logs'::TEXT AS scope FROM kb_sol_core_logs WHERE program_id IS NOT NULL
|
||||
)
|
||||
SELECT program_id,
|
||||
COUNT(DISTINCT signature)::BIGINT AS transaction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'outer')::BIGINT AS outer_instruction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'inner')::BIGINT AS inner_instruction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'logs')::BIGINT AS log_count,
|
||||
MIN(slot)::BIGINT AS min_slot,
|
||||
MAX(slot)::BIGINT AS max_slot
|
||||
FROM occurrences
|
||||
WHERE ($1::TEXT IS NULL OR program_id ILIKE '%' || $1 || '%')
|
||||
GROUP BY program_id
|
||||
ORDER BY transaction_count DESC, program_id ASC
|
||||
LIMIT $2"#,
|
||||
)
|
||||
.bind(filter.program_id_contains.as_deref())
|
||||
.bind(i64::from(filter.limit))
|
||||
.fetch_all(pool)
|
||||
.await;
|
||||
let rows = match query_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres replay program summary query failed: {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayProgramSummary {
|
||||
program_id: row.program_id,
|
||||
transaction_count: row.transaction_count,
|
||||
outer_instruction_count: row.outer_instruction_count,
|
||||
inner_instruction_count: row.inner_instruction_count,
|
||||
log_count: row.log_count,
|
||||
min_slot: row.min_slot,
|
||||
max_slot: row.max_slot,
|
||||
});
|
||||
}
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_entity_summaries(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
|
||||
let entity_kind = filter.entity_kind.as_sql();
|
||||
let query_result = sqlx::query_as::<
|
||||
sqlx::Postgres,
|
||||
crate::postgres::query::replay_candidate_queries::ReplayEntitySummaryRow,
|
||||
>(
|
||||
r#"WITH entities AS (
|
||||
SELECT 'mint'::TEXT AS entity_kind, mint AS entity_value, signature, slot
|
||||
FROM kb_sol_core_balance_changes
|
||||
WHERE mint IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT 'owner'::TEXT AS entity_kind, owner AS entity_value, signature, slot
|
||||
FROM kb_sol_core_balance_changes
|
||||
WHERE owner IS NOT NULL
|
||||
UNION ALL
|
||||
SELECT 'account_key'::TEXT AS entity_kind, account_key AS entity_value, signature, slot
|
||||
FROM kb_sol_core_account_keys
|
||||
)
|
||||
SELECT entity_kind,
|
||||
entity_value,
|
||||
COUNT(DISTINCT signature)::BIGINT AS transaction_count,
|
||||
COUNT(*)::BIGINT AS occurrence_count,
|
||||
MIN(slot)::BIGINT AS min_slot,
|
||||
MAX(slot)::BIGINT AS max_slot
|
||||
FROM entities
|
||||
WHERE entity_kind = $1
|
||||
AND ($2::TEXT IS NULL OR entity_value ILIKE '%' || $2 || '%')
|
||||
GROUP BY entity_kind, entity_value
|
||||
ORDER BY transaction_count DESC, entity_value ASC
|
||||
LIMIT $3"#,
|
||||
)
|
||||
.bind(entity_kind)
|
||||
.bind(filter.entity_value_contains.as_deref())
|
||||
.bind(i64::from(filter.limit))
|
||||
.fetch_all(pool)
|
||||
.await;
|
||||
let rows = match query_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres replay entity summary query failed: {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayEntitySummary {
|
||||
entity_kind: row.entity_kind,
|
||||
entity_value: row.entity_value,
|
||||
transaction_count: row.transaction_count,
|
||||
occurrence_count: row.occurrence_count,
|
||||
min_slot: row.min_slot,
|
||||
max_slot: row.max_slot,
|
||||
});
|
||||
}
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
fn transaction_candidate_sql_desc() -> &'static str {
|
||||
return crate::postgres::query::replay_candidate_queries::transaction_candidate_sql("DESC");
|
||||
}
|
||||
|
||||
fn transaction_candidate_sql_asc() -> &'static str {
|
||||
return crate::postgres::query::replay_candidate_queries::transaction_candidate_sql("ASC");
|
||||
}
|
||||
|
||||
fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
if order == "DESC" {
|
||||
return r#"SELECT raw.signature,
|
||||
raw.slot,
|
||||
raw.processing_state AS raw_processing_state,
|
||||
raw.retention_state,
|
||||
(core.id IS NOT NULL) AS has_core_transaction,
|
||||
core.failed AS transaction_failed,
|
||||
COALESCE(ledger.status, 'not_started') AS ledger_status,
|
||||
ledger.processor_version,
|
||||
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
|
||||
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
|
||||
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
|
||||
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
|
||||
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
|
||||
raw.updated_at::TEXT AS updated_at
|
||||
FROM kb_sol_raw_transactions raw
|
||||
LEFT JOIN kb_sol_core_transactions core ON core.signature = raw.signature
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT status, processor_version, attempt_count
|
||||
FROM kb_sol_ops_processing_ledger
|
||||
WHERE stage = 'core_extraction'
|
||||
AND processor_name = 'canonical_to_core'
|
||||
AND input_key = raw.signature
|
||||
ORDER BY updated_at DESC, id DESC
|
||||
LIMIT 1
|
||||
) ledger ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_instructions
|
||||
WHERE signature = raw.signature
|
||||
) outer_stats ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_inner_instructions
|
||||
WHERE signature = raw.signature
|
||||
) inner_stats ON TRUE
|
||||
WHERE ($1::TEXT IS NULL OR raw.signature ILIKE '%' || $1 || '%')
|
||||
AND ($2::BIGINT IS NULL OR raw.slot >= $2)
|
||||
AND ($3::BIGINT IS NULL OR raw.slot <= $3)
|
||||
AND ($4::TEXT IS NULL OR raw.processing_state = $4)
|
||||
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
|
||||
AND ($6::TEXT IS NULL OR
|
||||
($7 = 'any' AND (
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
|
||||
)) OR
|
||||
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
|
||||
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
|
||||
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
|
||||
AND ($8::TEXT IS NULL OR
|
||||
($8 = 'mint' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_balance_changes candidate_balance
|
||||
WHERE candidate_balance.signature = raw.signature
|
||||
AND candidate_balance.mint = $9
|
||||
)) OR
|
||||
($8 = 'owner' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_balance_changes candidate_balance
|
||||
WHERE candidate_balance.signature = raw.signature
|
||||
AND candidate_balance.owner = $9
|
||||
)) OR
|
||||
($8 = 'account_key' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_account_keys candidate_account
|
||||
WHERE candidate_account.signature = raw.signature
|
||||
AND candidate_account.account_key = $9
|
||||
)))
|
||||
ORDER BY raw.slot DESC, raw.signature DESC
|
||||
LIMIT $10"#;
|
||||
}
|
||||
return r#"SELECT raw.signature,
|
||||
raw.slot,
|
||||
raw.processing_state AS raw_processing_state,
|
||||
raw.retention_state,
|
||||
(core.id IS NOT NULL) AS has_core_transaction,
|
||||
core.failed AS transaction_failed,
|
||||
COALESCE(ledger.status, 'not_started') AS ledger_status,
|
||||
ledger.processor_version,
|
||||
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
|
||||
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
|
||||
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
|
||||
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
|
||||
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
|
||||
raw.updated_at::TEXT AS updated_at
|
||||
FROM kb_sol_raw_transactions raw
|
||||
LEFT JOIN kb_sol_core_transactions core ON core.signature = raw.signature
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT status, processor_version, attempt_count
|
||||
FROM kb_sol_ops_processing_ledger
|
||||
WHERE stage = 'core_extraction'
|
||||
AND processor_name = 'canonical_to_core'
|
||||
AND input_key = raw.signature
|
||||
ORDER BY updated_at DESC, id DESC
|
||||
LIMIT 1
|
||||
) ledger ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_instructions
|
||||
WHERE signature = raw.signature
|
||||
) outer_stats ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_inner_instructions
|
||||
WHERE signature = raw.signature
|
||||
) inner_stats ON TRUE
|
||||
WHERE ($1::TEXT IS NULL OR raw.signature ILIKE '%' || $1 || '%')
|
||||
AND ($2::BIGINT IS NULL OR raw.slot >= $2)
|
||||
AND ($3::BIGINT IS NULL OR raw.slot <= $3)
|
||||
AND ($4::TEXT IS NULL OR raw.processing_state = $4)
|
||||
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
|
||||
AND ($6::TEXT IS NULL OR
|
||||
($7 = 'any' AND (
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
|
||||
)) OR
|
||||
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
|
||||
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
|
||||
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
|
||||
AND ($8::TEXT IS NULL OR
|
||||
($8 = 'mint' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_balance_changes candidate_balance
|
||||
WHERE candidate_balance.signature = raw.signature
|
||||
AND candidate_balance.mint = $9
|
||||
)) OR
|
||||
($8 = 'owner' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_balance_changes candidate_balance
|
||||
WHERE candidate_balance.signature = raw.signature
|
||||
AND candidate_balance.owner = $9
|
||||
)) OR
|
||||
($8 = 'account_key' AND EXISTS (
|
||||
SELECT 1 FROM kb_sol_core_account_keys candidate_account
|
||||
WHERE candidate_account.signature = raw.signature
|
||||
AND candidate_account.account_key = $9
|
||||
)))
|
||||
ORDER BY raw.slot ASC, raw.signature ASC
|
||||
LIMIT $10"#;
|
||||
}
|
||||
|
||||
fn optional_sql_bigint(
|
||||
value: std::option::Option<u64>,
|
||||
) -> ks_core::Result<std::option::Option<i64>> {
|
||||
return match value {
|
||||
std::option::Option::Some(raw_value) => {
|
||||
let conversion_result = i64::try_from(raw_value);
|
||||
match conversion_result {
|
||||
std::result::Result::Ok(converted) => {
|
||||
std::result::Result::Ok(std::option::Option::Some(converted))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(
|
||||
format!("replay candidate slot does not fit into PostgreSQL BIGINT: {error}"),
|
||||
)),
|
||||
}
|
||||
},
|
||||
std::option::Option::None => std::result::Result::Ok(std::option::Option::None),
|
||||
};
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_replay_candidate_queries_from_env() {
|
||||
let database_url = match std::env::var("KB_POSTGRES_TEST_URL") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected options error: {error}"),
|
||||
};
|
||||
let store_result = crate::PostgresStore::connect(options).await;
|
||||
let store = match store_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected connect error: {error}"),
|
||||
};
|
||||
let raw_schema_result = store.initialize_raw_store_schema().await;
|
||||
if let std::result::Result::Err(error) = raw_schema_result {
|
||||
panic!("unexpected raw schema error: {error}");
|
||||
}
|
||||
let core_schema_result = store.initialize_core_store_schema().await;
|
||||
if let std::result::Result::Err(error) = core_schema_result {
|
||||
panic!("unexpected core schema error: {error}");
|
||||
}
|
||||
let transaction_filter_result = crate::PostgresReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
10,
|
||||
true,
|
||||
);
|
||||
let transaction_filter = match transaction_filter_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
panic!("unexpected transaction filter error: {error}")
|
||||
},
|
||||
};
|
||||
let transaction_result = store.replay_transaction_candidates(&transaction_filter).await;
|
||||
if let std::result::Result::Err(error) = transaction_result {
|
||||
panic!("unexpected transaction candidate query error: {error}");
|
||||
}
|
||||
let program_filter_result =
|
||||
crate::PostgresReplayProgramFilter::new(std::option::Option::None, 10);
|
||||
let program_filter = match program_filter_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected program filter error: {error}"),
|
||||
};
|
||||
let program_result = store.replay_program_summaries(&program_filter).await;
|
||||
if let std::result::Result::Err(error) = program_result {
|
||||
panic!("unexpected program summary query error: {error}");
|
||||
}
|
||||
for entity_kind in [
|
||||
crate::PostgresReplayEntityKind::Mint,
|
||||
crate::PostgresReplayEntityKind::Owner,
|
||||
crate::PostgresReplayEntityKind::AccountKey,
|
||||
] {
|
||||
let entity_filter_result =
|
||||
crate::PostgresReplayEntityFilter::new(entity_kind, std::option::Option::None, 10);
|
||||
let entity_filter = match entity_filter_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
panic!("unexpected entity filter error: {error}")
|
||||
},
|
||||
};
|
||||
let entity_result = store.replay_entity_summaries(&entity_filter).await;
|
||||
if let std::result::Result::Err(error) = entity_result {
|
||||
panic!("unexpected entity summary query error: {error}");
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
197
ks-store/src/postgres/query/table_diagnostics_queries.rs
Normal file
197
ks-store/src/postgres/query/table_diagnostics_queries.rs
Normal file
@@ -0,0 +1,197 @@
|
||||
// file: ks-store/src/postgres/query/table_diagnostics_queries.rs
|
||||
// version: 3
|
||||
|
||||
//! Read-only PostgreSQL diagnostics for known Solana store tables.
|
||||
|
||||
use sqlx::Row; // rust-rules: trait-import
|
||||
|
||||
pub(in crate::postgres) async fn table_exists(
|
||||
pool: &sqlx::PgPool,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
let query_result =
|
||||
sqlx::query_scalar::<sqlx::Postgres, bool>("SELECT to_regclass($1)::text IS NOT NULL")
|
||||
.bind(table_name)
|
||||
.fetch_one(pool)
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(value) => std::result::Result::Ok(value),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table existence diagnostic failed for '{table_name}': {error}"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_table_statistics(
|
||||
pool: &sqlx::PgPool,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<crate::PostgresTableStatistics> {
|
||||
return match table_name {
|
||||
crate::RAW_TRANSACTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_raw_transactions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::TRANSACTION_OBSERVATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_obs_transaction_observations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_TRANSACTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_transactions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_ACCOUNT_KEYS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_account_keys_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_INSTRUCTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_instructions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_inner_instructions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_LOGS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_logs_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_BALANCE_CHANGES_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_balance_changes_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::PROCESSING_LEDGER_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_ops_processing_ledger_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_EVENTS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_events_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_declarations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_observations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::MATERIALIZED_EVENTS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_mat_events_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
_ => std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table statistics are not supported for '{table_name}'"
|
||||
))),
|
||||
};
|
||||
}
|
||||
|
||||
async fn load_table_statistics_from_sql(
|
||||
pool: &sqlx::PgPool,
|
||||
sql: &'static str,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<crate::PostgresTableStatistics> {
|
||||
let query_result = sqlx::query(sql).fetch_one(pool).await;
|
||||
let row = match query_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table statistics failed for '{table_name}': {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let row_count_result = row.try_get::<i64, _>("row_count");
|
||||
let row_count = match row_count_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table row_count mapping failed for '{table_name}': {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let min_slot_result = row.try_get::<std::option::Option<i64>, _>("min_slot");
|
||||
let min_slot = match min_slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table min_slot mapping failed for '{table_name}': {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let max_slot_result = row.try_get::<std::option::Option<i64>, _>("max_slot");
|
||||
let max_slot = match max_slot_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table max_slot mapping failed for '{table_name}': {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
let latest_created_at_result =
|
||||
row.try_get::<std::option::Option<std::string::String>, _>("latest_created_at");
|
||||
let latest_created_at = match latest_created_at_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres table latest_created_at mapping failed for '{table_name}': {error}"
|
||||
)));
|
||||
},
|
||||
};
|
||||
return std::result::Result::Ok(crate::PostgresTableStatistics {
|
||||
row_count,
|
||||
min_slot,
|
||||
max_slot,
|
||||
latest_created_at,
|
||||
});
|
||||
}
|
||||
380
ks-store/src/postgres/replay_candidates.rs
Normal file
380
ks-store/src/postgres/replay_candidates.rs
Normal file
@@ -0,0 +1,380 @@
|
||||
// file: ks-store/src/postgres/replay_candidates.rs
|
||||
// version: 3
|
||||
|
||||
//! Read-only replay candidate filters and PostgreSQL result rows.
|
||||
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub const MAX_REPLAY_CANDIDATE_ROWS: u32 = 100_000;
|
||||
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum PostgresReplayProgramScope {
|
||||
/// Match outer instructions, inner instructions or reliably linked logs.
|
||||
Any,
|
||||
/// Match only top-level instructions.
|
||||
Outer,
|
||||
/// Match only inner instructions.
|
||||
Inner,
|
||||
/// Match only logs with a reliably linked program id.
|
||||
Logs,
|
||||
}
|
||||
|
||||
impl PostgresReplayProgramScope {
|
||||
/// Returns the stable SQL code for this scope.
|
||||
pub fn as_sql(self) -> &'static str {
|
||||
return match self {
|
||||
Self::Any => "any",
|
||||
Self::Outer => "outer",
|
||||
Self::Inner => "inner",
|
||||
Self::Logs => "logs",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum PostgresReplayEntityKind {
|
||||
/// SPL or Token-2022 mint address.
|
||||
Mint,
|
||||
/// Token account owner address.
|
||||
Owner,
|
||||
/// Native or token account address.
|
||||
AccountKey,
|
||||
}
|
||||
|
||||
impl PostgresReplayEntityKind {
|
||||
/// Returns the stable SQL code for this entity kind.
|
||||
pub fn as_sql(self) -> &'static str {
|
||||
return match self {
|
||||
Self::Mint => "mint",
|
||||
Self::Owner => "owner",
|
||||
Self::AccountKey => "account_key",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayTransactionFilter {
|
||||
/// Optional partial signature search.
|
||||
pub signature_contains: std::option::Option<std::string::String>,
|
||||
/// Optional inclusive minimum slot.
|
||||
pub min_slot: std::option::Option<u64>,
|
||||
/// Optional inclusive maximum slot.
|
||||
pub max_slot: std::option::Option<u64>,
|
||||
/// Optional raw processing state.
|
||||
pub raw_processing_state: std::option::Option<std::string::String>,
|
||||
/// Optional latest core extraction ledger status, including `not_started`.
|
||||
pub ledger_status: std::option::Option<std::string::String>,
|
||||
/// Optional exact program id.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Program occurrence scope.
|
||||
pub program_scope: crate::PostgresReplayProgramScope,
|
||||
/// Optional core entity kind.
|
||||
pub entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
|
||||
/// Optional exact entity value.
|
||||
pub entity_value: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
/// Orders newest slots first when true.
|
||||
pub newest_first: bool,
|
||||
}
|
||||
|
||||
impl PostgresReplayTransactionFilter {
|
||||
/// Creates and validates a bounded transaction candidate filter.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new(
|
||||
signature_contains: std::option::Option<std::string::String>,
|
||||
min_slot: std::option::Option<u64>,
|
||||
max_slot: std::option::Option<u64>,
|
||||
raw_processing_state: std::option::Option<std::string::String>,
|
||||
ledger_status: std::option::Option<std::string::String>,
|
||||
program_id: std::option::Option<std::string::String>,
|
||||
program_scope: crate::PostgresReplayProgramScope,
|
||||
entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
|
||||
entity_value: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
newest_first: bool,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_contains_value = trim_optional_text(signature_contains);
|
||||
let raw_processing_state_value = trim_optional_text(raw_processing_state);
|
||||
let ledger_status_value = trim_optional_text(ledger_status);
|
||||
let program_id_value = trim_optional_text(program_id);
|
||||
let entity_value_value = trim_optional_text(entity_value);
|
||||
let slot_result = validate_slot_range(min_slot, max_slot);
|
||||
if let std::result::Result::Err(error) = slot_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let limit_result = validate_limit(limit);
|
||||
if let std::result::Result::Err(error) = limit_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let raw_state_result = validate_optional_code(
|
||||
raw_processing_state_value.as_deref(),
|
||||
&["received", "core_extracted", "decoded", "materialized", "failed"],
|
||||
"raw processing state",
|
||||
);
|
||||
if let std::result::Result::Err(error) = raw_state_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let ledger_status_result = validate_optional_code(
|
||||
ledger_status_value.as_deref(),
|
||||
&["not_started", "running", "succeeded", "failed"],
|
||||
"ledger status",
|
||||
);
|
||||
if let std::result::Result::Err(error) = ledger_status_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
if entity_kind.is_some() != entity_value_value.is_some() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"replay entity kind and value must either both be present or both be absent",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature_contains: signature_contains_value,
|
||||
min_slot,
|
||||
max_slot,
|
||||
raw_processing_state: raw_processing_state_value,
|
||||
ledger_status: ledger_status_value,
|
||||
program_id: program_id_value,
|
||||
program_scope,
|
||||
entity_kind,
|
||||
entity_value: entity_value_value,
|
||||
limit,
|
||||
newest_first,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayTransactionCandidate {
|
||||
/// Canonical transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot.
|
||||
pub slot: i64,
|
||||
/// Current raw processing state.
|
||||
pub raw_processing_state: std::string::String,
|
||||
/// Current raw retention state.
|
||||
pub retention_state: std::string::String,
|
||||
/// Whether a core transaction row exists.
|
||||
pub has_core_transaction: bool,
|
||||
/// Core transaction failure flag when a core row exists.
|
||||
pub transaction_failed: std::option::Option<bool>,
|
||||
/// Latest core extraction ledger status or `not_started`.
|
||||
pub ledger_status: std::string::String,
|
||||
/// Latest core extraction processor version when available.
|
||||
pub processor_version: std::option::Option<std::string::String>,
|
||||
/// Latest core extraction attempt count.
|
||||
pub attempt_count: i32,
|
||||
/// Number of top-level instructions.
|
||||
pub outer_instruction_count: i64,
|
||||
/// Number of inner instructions.
|
||||
pub inner_instruction_count: i64,
|
||||
/// Number of distinct top-level programs.
|
||||
pub outer_program_count: i64,
|
||||
/// Number of distinct inner programs.
|
||||
pub inner_program_count: i64,
|
||||
/// Raw row update timestamp rendered by PostgreSQL.
|
||||
pub updated_at: std::string::String,
|
||||
}
|
||||
|
||||
/// Bounded read-only filter for program summaries.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayProgramFilter {
|
||||
/// Optional partial program id search.
|
||||
pub program_id_contains: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl PostgresReplayProgramFilter {
|
||||
/// Creates and validates a bounded program summary filter.
|
||||
pub fn new(
|
||||
program_id_contains: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
let limit_result = validate_limit(limit);
|
||||
if let std::result::Result::Err(error) = limit_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
program_id_contains: trim_optional_text(program_id_contains),
|
||||
limit,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayProgramSummary {
|
||||
/// Program id.
|
||||
pub program_id: std::string::String,
|
||||
/// Number of distinct transactions containing the program.
|
||||
pub transaction_count: i64,
|
||||
/// Number of top-level instruction occurrences.
|
||||
pub outer_instruction_count: i64,
|
||||
/// Number of inner instruction occurrences.
|
||||
pub inner_instruction_count: i64,
|
||||
/// Number of reliably linked log occurrences.
|
||||
pub log_count: i64,
|
||||
/// Lowest observed slot.
|
||||
pub min_slot: i64,
|
||||
/// Highest observed slot.
|
||||
pub max_slot: i64,
|
||||
}
|
||||
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayEntityFilter {
|
||||
/// Entity kind to aggregate.
|
||||
pub entity_kind: crate::PostgresReplayEntityKind,
|
||||
/// Optional partial entity value search.
|
||||
pub entity_value_contains: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl PostgresReplayEntityFilter {
|
||||
/// Creates and validates a bounded entity summary filter.
|
||||
pub fn new(
|
||||
entity_kind: crate::PostgresReplayEntityKind,
|
||||
entity_value_contains: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
let limit_result = validate_limit(limit);
|
||||
if let std::result::Result::Err(error) = limit_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
entity_kind,
|
||||
entity_value_contains: trim_optional_text(entity_value_contains),
|
||||
limit,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayEntitySummary {
|
||||
/// Stable entity kind code.
|
||||
pub entity_kind: std::string::String,
|
||||
/// Mint, owner or account-key address.
|
||||
pub entity_value: std::string::String,
|
||||
/// Number of distinct transactions containing the entity.
|
||||
pub transaction_count: i64,
|
||||
/// Total number of core-table occurrences.
|
||||
pub occurrence_count: i64,
|
||||
/// Lowest observed slot.
|
||||
pub min_slot: i64,
|
||||
/// Highest observed slot.
|
||||
pub max_slot: i64,
|
||||
}
|
||||
|
||||
fn trim_optional_text(
|
||||
value: std::option::Option<std::string::String>,
|
||||
) -> std::option::Option<std::string::String> {
|
||||
return match value {
|
||||
std::option::Option::Some(text) => {
|
||||
let trimmed = text.trim();
|
||||
if trimmed.is_empty() {
|
||||
return std::option::Option::None;
|
||||
}
|
||||
std::option::Option::Some(trimmed.to_string())
|
||||
},
|
||||
std::option::Option::None => std::option::Option::None,
|
||||
};
|
||||
}
|
||||
|
||||
fn validate_slot_range(
|
||||
min_slot: std::option::Option<u64>,
|
||||
max_slot: std::option::Option<u64>,
|
||||
) -> ks_core::Result<()> {
|
||||
if let (std::option::Option::Some(minimum), std::option::Option::Some(maximum)) =
|
||||
(min_slot, max_slot)
|
||||
{
|
||||
if minimum > maximum {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"replay candidate minimum slot must not exceed maximum slot",
|
||||
));
|
||||
}
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_limit(limit: u32) -> ks_core::Result<()> {
|
||||
if limit == 0 || limit > crate::MAX_REPLAY_CANDIDATE_ROWS {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"replay candidate limit must be between 1 and {}",
|
||||
crate::MAX_REPLAY_CANDIDATE_ROWS
|
||||
)));
|
||||
}
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
fn validate_optional_code(
|
||||
value: std::option::Option<&str>,
|
||||
allowed: &[&str],
|
||||
label: &str,
|
||||
) -> ks_core::Result<()> {
|
||||
let selected = match value {
|
||||
std::option::Option::Some(code) => code,
|
||||
std::option::Option::None => return std::result::Result::Ok(()),
|
||||
};
|
||||
for allowed_code in allowed {
|
||||
if selected == *allowed_code {
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
}
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"unsupported {label}: {selected}"
|
||||
)));
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn transaction_filter_rejects_inverted_slots() {
|
||||
let result = crate::PostgresReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::Some(20),
|
||||
std::option::Option::Some(10),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
100,
|
||||
true,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn transaction_filter_requires_complete_entity_pair() {
|
||||
let result = crate::PostgresReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
std::option::Option::Some(crate::PostgresReplayEntityKind::Mint),
|
||||
std::option::Option::None,
|
||||
100,
|
||||
true,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn program_filter_rejects_limit_above_maximum() {
|
||||
let result = crate::PostgresReplayProgramFilter::new(
|
||||
std::option::Option::None,
|
||||
crate::MAX_REPLAY_CANDIDATE_ROWS + 1,
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
10
ks-store/src/postgres/repository.rs
Normal file
10
ks-store/src/postgres/repository.rs
Normal file
@@ -0,0 +1,10 @@
|
||||
// file: ks-store/src/postgres/repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL repository implementations.
|
||||
|
||||
mod core_extraction_repository;
|
||||
mod core_transaction_repository;
|
||||
mod decode_pipeline_repository;
|
||||
mod raw_transaction_repository;
|
||||
mod store_health_repository;
|
||||
@@ -0,0 +1,57 @@
|
||||
// file: ks-store/src/postgres/repository/core_extraction_repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL atomic canonical transaction to core extraction repository.
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl crate::CoreExtractionStore for crate::PostgresStore {
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_raw_transactions_for_core_extraction(
|
||||
&self,
|
||||
filter: &crate::CoreExtractionSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>> {
|
||||
return crate::postgres::query::list_raw_transactions_for_core_extraction(
|
||||
self.pool(),
|
||||
filter,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn is_core_extraction_current(
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::is_core_extraction_current(self.pool(), identity).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn persist_core_extraction(
|
||||
&self,
|
||||
bundle: &crate::CoreExtractionBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_core_extraction(self.pool(), bundle, force_replay)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn mark_core_extraction_failed(
|
||||
&self,
|
||||
failure: &crate::CoreExtractionFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::mark_core_extraction_failed(self.pool(), failure).await;
|
||||
}
|
||||
}
|
||||
118
ks-store/src/postgres/repository/core_transaction_repository.rs
Normal file
118
ks-store/src/postgres/repository/core_transaction_repository.rs
Normal file
@@ -0,0 +1,118 @@
|
||||
// file: ks-store/src/postgres/repository/core_transaction_repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL core Solana repository implementation.
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_transaction(
|
||||
&self,
|
||||
input: &crate::CoreTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_transaction(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_account_keys(
|
||||
&self,
|
||||
inputs: &[crate::CoreAccountKeyInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_account_keys(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_instructions(
|
||||
&self,
|
||||
inputs: &[crate::CoreInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_instructions(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_inner_instructions(
|
||||
&self,
|
||||
inputs: &[crate::CoreInnerInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_inner_instructions(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_logs(
|
||||
&self,
|
||||
inputs: &[crate::CoreLogInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_logs(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_core_balance_changes(
|
||||
&self,
|
||||
inputs: &[crate::CoreBalanceChangeInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_balance_changes(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_core_instructions_for_replay(
|
||||
&self,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::CoreInstructionRow>> {
|
||||
return crate::postgres::query::list_core_instructions_for_replay(
|
||||
self.pool(),
|
||||
filter,
|
||||
page_request,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_core_instruction_replay_inputs(
|
||||
&self,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
return crate::postgres::query::list_core_instruction_replay_inputs(
|
||||
self.pool(),
|
||||
filter,
|
||||
page_request,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn mark_core_instruction_lifecycle(
|
||||
&self,
|
||||
mark: &crate::CoreInstructionLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::update_core_instruction_lifecycle(self.pool(), mark).await;
|
||||
}
|
||||
}
|
||||
115
ks-store/src/postgres/repository/decode_pipeline_repository.rs
Normal file
115
ks-store/src/postgres/repository/decode_pipeline_repository.rs
Normal file
@@ -0,0 +1,115 @@
|
||||
// file: ks-store/src/postgres/repository/decode_pipeline_repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL contextual decode and materialization repository.
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_decode_inputs(
|
||||
&self,
|
||||
filter: &crate::DecodeSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
return crate::postgres::query::list_decode_inputs(self.pool(), filter).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn is_decode_current(
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::is_decode_current(self.pool(), identity).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn persist_decode_coverage_declarations(
|
||||
&self,
|
||||
declarations: &[crate::DecodeCoverageDeclarationInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_decode_coverage_declarations(
|
||||
self.pool(),
|
||||
declarations,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn persist_decode_result(
|
||||
&self,
|
||||
bundle: &crate::DecodePersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_decode_result(self.pool(), bundle, force_replay)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn mark_decode_failed(
|
||||
&self,
|
||||
failure: &crate::DecodeFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::mark_decode_failed(self.pool(), failure).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn persist_materialization_result(
|
||||
&self,
|
||||
bundle: &crate::MaterializationPersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_materialization_result(
|
||||
self.pool(),
|
||||
bundle,
|
||||
force_replay,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_decode_coverage_summary(
|
||||
&self,
|
||||
processor_name: std::option::Option<&str>,
|
||||
processor_version: std::option::Option<&str>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>> {
|
||||
return crate::postgres::query::list_decode_coverage_summary(
|
||||
self.pool(),
|
||||
processor_name,
|
||||
processor_version,
|
||||
limit,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn list_materialized_events(
|
||||
&self,
|
||||
filter: &crate::MaterializedEventFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>> {
|
||||
return crate::postgres::query::list_materialized_events(self.pool(), filter).await;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,70 @@
|
||||
// file: ks-store/src/postgres/repository/raw_transaction_repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL canonical transaction and acquisition observation repository implementation.
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn has_raw_transaction_signature(
|
||||
&self,
|
||||
signature: &ks_lib::MdSignature,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::has_raw_transaction_signature(
|
||||
self.pool(),
|
||||
signature.0.as_str(),
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn has_transaction_observation_key(
|
||||
&self,
|
||||
observation_key: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::has_transaction_observation_key(
|
||||
self.pool(),
|
||||
observation_key,
|
||||
)
|
||||
.await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_raw_transaction(
|
||||
&self,
|
||||
input: &crate::RawTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_raw_transaction(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn insert_transaction_observation(
|
||||
&self,
|
||||
input: &crate::TransactionObservationInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_transaction_observation(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn mark_raw_payload_lifecycle(
|
||||
&self,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::update_raw_payload_lifecycle(self.pool(), mark).await;
|
||||
}
|
||||
}
|
||||
31
ks-store/src/postgres/repository/store_health_repository.rs
Normal file
31
ks-store/src/postgres/repository/store_health_repository.rs
Normal file
@@ -0,0 +1,31 @@
|
||||
// file: ks-store/src/postgres/repository/store_health_repository.rs
|
||||
// version: 2
|
||||
|
||||
//! PostgreSQL store health repository implementation.
|
||||
|
||||
#[async_trait::async_trait]
|
||||
impl crate::StoreHealthStore for crate::PostgresStore {
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn backend_descriptor(&self) -> ks_core::Result<crate::StoreBackendDescriptor> {
|
||||
return crate::PostgresStore::backend_descriptor(self).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
|
||||
return crate::PostgresStore::health_snapshot(self).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
clippy::implicit_return,
|
||||
reason = "async_trait expansion triggers implicit_return on generated async trait methods."
|
||||
)]
|
||||
async fn migration_snapshot(&self) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
return crate::PostgresStore::migration_snapshot(self).await;
|
||||
}
|
||||
}
|
||||
512
ks-store/src/postgres/store.rs
Normal file
512
ks-store/src/postgres/store.rs
Normal file
@@ -0,0 +1,512 @@
|
||||
// file: ks-store/src/postgres/store.rs
|
||||
// version: 3
|
||||
|
||||
//! Store implementation scaffold for the `ks-store` crate.
|
||||
|
||||
/// PostgreSQL store connection options.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresStoreOptions {
|
||||
/// Database URL or DSN.
|
||||
pub database_url: std::string::String,
|
||||
/// Maximum connection count.
|
||||
pub max_connections: u32,
|
||||
/// Connection timeout in milliseconds.
|
||||
pub connect_timeout_ms: u64,
|
||||
/// Enables idempotent raw schema initialization at startup.
|
||||
pub auto_initialize_schema: bool,
|
||||
}
|
||||
|
||||
impl PostgresStoreOptions {
|
||||
/// Creates validated PostgreSQL store options.
|
||||
pub fn new(
|
||||
database_url: impl std::convert::Into<std::string::String>,
|
||||
max_connections: u32,
|
||||
connect_timeout_ms: u64,
|
||||
auto_initialize_schema: bool,
|
||||
) -> ks_core::Result<Self> {
|
||||
let database_url_value = database_url.into();
|
||||
if database_url_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"postgres database url must not be empty",
|
||||
));
|
||||
}
|
||||
if max_connections == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"postgres max_connections must be greater than zero",
|
||||
));
|
||||
}
|
||||
if connect_timeout_ms == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"postgres connect_timeout_ms must be greater than zero",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
database_url: database_url_value,
|
||||
max_connections,
|
||||
connect_timeout_ms,
|
||||
auto_initialize_schema,
|
||||
});
|
||||
}
|
||||
|
||||
/// Returns a DSN masked for diagnostics.
|
||||
pub fn masked_dsn(&self) -> std::string::String {
|
||||
return crate::mask_postgres_dsn(self.database_url.as_str());
|
||||
}
|
||||
}
|
||||
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresBackendDiagnostics {
|
||||
/// Backend descriptor safe for UI display.
|
||||
pub descriptor: crate::StoreBackendDescriptor,
|
||||
/// Backend health snapshot.
|
||||
pub health: crate::StoreHealthSnapshot,
|
||||
/// Migration status snapshot.
|
||||
pub migrations: crate::StoreMigrationSnapshot,
|
||||
/// Full PostgreSQL server version string when available.
|
||||
pub server_version: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only statistics for one PostgreSQL table.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresTableStatistics {
|
||||
/// Number of rows currently stored in the table.
|
||||
pub row_count: i64,
|
||||
/// Lowest observed Solana slot when the table contains a slot column and rows.
|
||||
pub min_slot: std::option::Option<i64>,
|
||||
/// Highest observed Solana slot when the table contains a slot column and rows.
|
||||
pub max_slot: std::option::Option<i64>,
|
||||
/// Latest insertion timestamp rendered by PostgreSQL for UI diagnostics.
|
||||
pub latest_created_at: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only diagnostics for one expected PostgreSQL table.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresTableDiagnostics {
|
||||
/// Expected table name.
|
||||
pub table_name: std::string::String,
|
||||
/// Logical Solana domain encoded in the table name.
|
||||
pub domain: std::string::String,
|
||||
/// Human-readable role of the table.
|
||||
pub role: std::string::String,
|
||||
/// Whether the table exists in the current PostgreSQL search path.
|
||||
pub exists: bool,
|
||||
/// Table statistics when the table exists.
|
||||
pub statistics: std::option::Option<crate::PostgresTableStatistics>,
|
||||
}
|
||||
|
||||
/// PostgreSQL store handle.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct PostgresStore {
|
||||
options: crate::PostgresStoreOptions,
|
||||
pool: sqlx::PgPool,
|
||||
}
|
||||
|
||||
impl PostgresStore {
|
||||
/// Connects to PostgreSQL from typed store options.
|
||||
pub async fn connect(options: crate::PostgresStoreOptions) -> ks_core::Result<Self> {
|
||||
let pool_options = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(options.max_connections)
|
||||
.acquire_timeout(std::time::Duration::from_millis(options.connect_timeout_ms));
|
||||
let connect_result = pool_options.connect(options.database_url.as_str()).await;
|
||||
return match connect_result {
|
||||
std::result::Result::Ok(pool) => {
|
||||
let store = Self { options, pool };
|
||||
if store.options.auto_initialize_schema {
|
||||
let schema_result = store.initialize_store_schema().await;
|
||||
if let std::result::Result::Err(error) = schema_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
}
|
||||
std::result::Result::Ok(store)
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(
|
||||
format!("postgres connection failed: {error}"),
|
||||
)),
|
||||
};
|
||||
}
|
||||
|
||||
/// Creates a store handle from an existing PostgreSQL pool.
|
||||
pub fn from_pool(options: crate::PostgresStoreOptions, pool: sqlx::PgPool) -> Self {
|
||||
return Self { options, pool };
|
||||
}
|
||||
|
||||
/// Returns the underlying PostgreSQL pool.
|
||||
pub fn pool(&self) -> &sqlx::PgPool {
|
||||
return &self.pool;
|
||||
}
|
||||
|
||||
/// Returns the connection options used to create this store.
|
||||
pub fn options(&self) -> &crate::PostgresStoreOptions {
|
||||
return &self.options;
|
||||
}
|
||||
|
||||
/// Applies each idempotent store schema once per invocation in dependency order.
|
||||
pub async fn initialize_store_schema(&self) -> ks_core::Result<()> {
|
||||
let raw_result = crate::postgres::query::apply_raw_store_schema(&self.pool).await;
|
||||
if let std::result::Result::Err(error) = raw_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let core_result = crate::postgres::query::apply_core_store_schema(&self.pool).await;
|
||||
if let std::result::Result::Err(error) = core_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Applies the idempotent minimal raw Solana store schema.
|
||||
pub async fn initialize_raw_store_schema(&self) -> ks_core::Result<()> {
|
||||
return crate::postgres::query::apply_raw_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Applies the idempotent minimal core Solana store schema.
|
||||
pub async fn initialize_core_store_schema(&self) -> ks_core::Result<()> {
|
||||
let raw_result = self.initialize_raw_store_schema().await;
|
||||
if let std::result::Result::Err(error) = raw_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_core_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Applies the idempotent common decode and materialization store schema.
|
||||
pub async fn initialize_decode_store_schema(&self) -> ks_core::Result<()> {
|
||||
let core_result = self.initialize_core_store_schema().await;
|
||||
if let std::result::Result::Err(error) = core_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Reads a UI-safe backend descriptor.
|
||||
pub async fn backend_descriptor(&self) -> ks_core::Result<crate::StoreBackendDescriptor> {
|
||||
let schema_result = crate::postgres::query::load_current_schema(&self.pool).await;
|
||||
return match schema_result {
|
||||
std::result::Result::Ok(schema) => crate::StoreBackendDescriptor::new(
|
||||
crate::StoreBackendKind::Postgres,
|
||||
"postgres",
|
||||
std::option::Option::Some(self.options.masked_dsn()),
|
||||
std::option::Option::Some(schema),
|
||||
),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a PostgreSQL health snapshot.
|
||||
pub async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
|
||||
let health_result = crate::postgres::query::run_health_check(&self.pool).await;
|
||||
return match health_result {
|
||||
std::result::Result::Ok(()) => crate::StoreHealthSnapshot::new(
|
||||
"postgres",
|
||||
crate::StoreHealthStatus::Healthy,
|
||||
std::option::Option::Some(std::string::String::from("SELECT 1 succeeded")),
|
||||
),
|
||||
std::result::Result::Err(error) => crate::StoreHealthSnapshot::new(
|
||||
"postgres",
|
||||
crate::StoreHealthStatus::Unhealthy,
|
||||
std::option::Option::Some(error.to_string()),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a non-destructive migration snapshot.
|
||||
pub async fn migration_snapshot(&self) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
let migration_table_result =
|
||||
crate::postgres::query::load_migration_table_name(&self.pool).await;
|
||||
return match migration_table_result {
|
||||
std::result::Result::Ok(std::option::Option::None) => {
|
||||
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
|
||||
crate::StoreMigrationStatus::NotInitialized,
|
||||
std::option::Option::None,
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(std::string::String::from(
|
||||
"no sqlx migration table detected; 0.3.1 canonical acquisition/core schemas use idempotent crate-managed DDL",
|
||||
)),
|
||||
))
|
||||
},
|
||||
std::result::Result::Ok(std::option::Option::Some(_table_name)) => {
|
||||
self.migration_snapshot_from_existing_table().await
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a complete PostgreSQL diagnostic snapshot.
|
||||
pub async fn backend_diagnostics(&self) -> ks_core::Result<crate::PostgresBackendDiagnostics> {
|
||||
let descriptor_result = self.backend_descriptor().await;
|
||||
let descriptor = match descriptor_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let health_result = self.health_snapshot().await;
|
||||
let health = match health_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let migrations_result = self.migration_snapshot().await;
|
||||
let migrations = match migrations_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let server_version = match crate::postgres::query::load_server_version(&self.pool).await {
|
||||
std::result::Result::Ok(value) => std::option::Option::Some(value),
|
||||
std::result::Result::Err(_error) => std::option::Option::None,
|
||||
};
|
||||
return std::result::Result::Ok(crate::PostgresBackendDiagnostics {
|
||||
descriptor,
|
||||
health,
|
||||
migrations,
|
||||
server_version,
|
||||
});
|
||||
}
|
||||
|
||||
/// Lists bounded raw transaction candidates enriched with core and ledger diagnostics.
|
||||
pub async fn replay_transaction_candidates(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
|
||||
return crate::postgres::query::list_replay_transaction_candidates(&self.pool, filter)
|
||||
.await;
|
||||
}
|
||||
|
||||
/// Lists bounded program summaries across outer, inner and reliably linked logs.
|
||||
pub async fn replay_program_summaries(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
|
||||
return crate::postgres::query::list_replay_program_summaries(&self.pool, filter).await;
|
||||
}
|
||||
|
||||
/// Lists bounded mint, owner or account-key summaries from core tables.
|
||||
pub async fn replay_entity_summaries(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
|
||||
return crate::postgres::query::list_replay_entity_summaries(&self.pool, filter).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for raw Solana store tables without changing the schema.
|
||||
pub async fn raw_table_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
let specs = crate::raw_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for core Solana store tables without changing the schema.
|
||||
pub async fn core_table_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
let specs = crate::core_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for decode and materialization store tables without changing the schema.
|
||||
pub async fn decode_table_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
let specs = crate::decode_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for every known raw/core/decode Solana store table.
|
||||
pub async fn known_table_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
let mut diagnostics = std::vec::Vec::new();
|
||||
let raw_result = self.raw_table_diagnostics().await;
|
||||
let raw_tables = match raw_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in raw_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
let core_result = self.core_table_diagnostics().await;
|
||||
let core_tables = match core_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in core_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
let decode_result = self.decode_table_diagnostics().await;
|
||||
let decode_tables = match decode_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in decode_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
return std::result::Result::Ok(diagnostics);
|
||||
}
|
||||
|
||||
async fn table_diagnostics(
|
||||
&self,
|
||||
specs: &[crate::PostgresTableDiagnosticSpec],
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
let mut diagnostics = std::vec::Vec::new();
|
||||
for spec in specs {
|
||||
let exists_result =
|
||||
crate::postgres::query::table_exists(&self.pool, spec.table_name).await;
|
||||
let exists = match exists_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let statistics = if exists {
|
||||
let statistics_result =
|
||||
crate::postgres::query::load_table_statistics(&self.pool, spec.table_name)
|
||||
.await;
|
||||
match statistics_result {
|
||||
std::result::Result::Ok(value) => std::option::Option::Some(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
}
|
||||
} else {
|
||||
std::option::Option::None
|
||||
};
|
||||
diagnostics.push(crate::PostgresTableDiagnostics {
|
||||
table_name: spec.table_name.to_string(),
|
||||
domain: spec.domain.to_string(),
|
||||
role: spec.role.to_string(),
|
||||
exists,
|
||||
statistics,
|
||||
});
|
||||
}
|
||||
return std::result::Result::Ok(diagnostics);
|
||||
}
|
||||
|
||||
async fn migration_snapshot_from_existing_table(
|
||||
&self,
|
||||
) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
let version_result =
|
||||
crate::postgres::query::load_latest_migration_version(&self.pool).await;
|
||||
return match version_result {
|
||||
std::result::Result::Ok(current_version) => {
|
||||
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
|
||||
crate::StoreMigrationStatus::Current,
|
||||
current_version,
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(std::string::String::from(
|
||||
"sqlx migration table detected; canonical acquisition/core schema remains idempotent and crate-managed in 0.3.1",
|
||||
)),
|
||||
))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub fn mask_postgres_dsn(dsn: &str) -> std::string::String {
|
||||
let trimmed_dsn = dsn.trim();
|
||||
if trimmed_dsn.is_empty() {
|
||||
return std::string::String::from("");
|
||||
}
|
||||
let queryless = crate::postgres::store::strip_query(trimmed_dsn);
|
||||
return match queryless.split_once("://") {
|
||||
std::option::Option::Some((scheme, remainder)) => {
|
||||
crate::postgres::store::mask_scheme_remainder(
|
||||
scheme,
|
||||
remainder,
|
||||
trimmed_dsn.contains('?'),
|
||||
)
|
||||
},
|
||||
std::option::Option::None => crate::postgres::store::mask_plain_dsn(queryless.as_str()),
|
||||
};
|
||||
}
|
||||
|
||||
fn strip_query(dsn: &str) -> std::string::String {
|
||||
return match dsn.split_once('?') {
|
||||
std::option::Option::Some((prefix, _query)) => std::string::String::from(prefix),
|
||||
std::option::Option::None => std::string::String::from(dsn),
|
||||
};
|
||||
}
|
||||
|
||||
fn mask_scheme_remainder(scheme: &str, remainder: &str, had_query: bool) -> std::string::String {
|
||||
let suffix = crate::postgres::store::query_suffix(had_query);
|
||||
return match remainder.rsplit_once('@') {
|
||||
std::option::Option::Some((_userinfo, host_path)) => {
|
||||
format!("{scheme}://***:***@{host_path}{suffix}")
|
||||
},
|
||||
std::option::Option::None => format!("{scheme}://{remainder}{suffix}"),
|
||||
};
|
||||
}
|
||||
|
||||
fn mask_plain_dsn(dsn: &str) -> std::string::String {
|
||||
if dsn.contains("password=") {
|
||||
return std::string::String::from("<postgres-dsn-redacted>");
|
||||
}
|
||||
return std::string::String::from(dsn);
|
||||
}
|
||||
|
||||
fn query_suffix(had_query: bool) -> std::string::String {
|
||||
if had_query {
|
||||
return std::string::String::from("?<redacted>");
|
||||
}
|
||||
return std::string::String::from("");
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn options_reject_empty_database_url() {
|
||||
let result = crate::PostgresStoreOptions::new(" ", 1, 1000, false);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn options_reject_zero_max_connections() {
|
||||
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 0, 1000, false);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn options_reject_zero_connect_timeout() {
|
||||
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 1, 0, false);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_healthcheck_from_env() {
|
||||
let database_url = match std::env::var("KB_POSTGRES_TEST_URL") {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected options error: {error}"),
|
||||
};
|
||||
let store_result = crate::PostgresStore::connect(options).await;
|
||||
let store = match store_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected connect error: {error}"),
|
||||
};
|
||||
let health_result = store.health_snapshot().await;
|
||||
let health = match health_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected health error: {error}"),
|
||||
};
|
||||
assert_eq!(health.status, crate::StoreHealthStatus::Healthy);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_userinfo() {
|
||||
let masked = crate::mask_postgres_dsn("postgres://user:secret@localhost:5432/db");
|
||||
assert_eq!(masked, "postgres://***:***@localhost:5432/db");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_query_string() {
|
||||
let masked =
|
||||
crate::mask_postgres_dsn("postgres://localhost/db?sslmode=require&password=secret");
|
||||
assert_eq!(masked, "postgres://localhost/db?<redacted>");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_plain_password_dsn() {
|
||||
let masked = crate::mask_postgres_dsn("host=localhost password=secret dbname=khadhroony");
|
||||
assert_eq!(masked, "<postgres-dsn-redacted>");
|
||||
}
|
||||
}
|
||||
15
ks-store/src/postgres/test_serial.rs
Normal file
15
ks-store/src/postgres/test_serial.rs
Normal file
@@ -0,0 +1,15 @@
|
||||
// file: ks-store/src/postgres/test_serial.rs
|
||||
// version: 2
|
||||
|
||||
//! Test-only serialization helpers for optional real PostgreSQL tests.
|
||||
|
||||
static POSTGRES_TEST_MUTEX: std::sync::OnceLock<std::sync::Arc<tokio::sync::Mutex<()>>> =
|
||||
std::sync::OnceLock::new();
|
||||
|
||||
/// Acquires the process-local guard shared by optional real PostgreSQL tests.
|
||||
pub(in crate::postgres) async fn postgres_test_guard() -> tokio::sync::OwnedMutexGuard<()> {
|
||||
let mutex = POSTGRES_TEST_MUTEX
|
||||
.get_or_init(|| return std::sync::Arc::new(tokio::sync::Mutex::new(())))
|
||||
.clone();
|
||||
return mutex.lock_owned().await;
|
||||
}
|
||||
Reference in New Issue
Block a user