v0.5.3-pre.002

This commit is contained in:
2026-08-11 22:22:40 +02:00
parent 01d78b5845
commit 8448ad1079
134 changed files with 4518 additions and 3595 deletions

View File

@@ -1,8 +1,48 @@
<!-- file: ks-store/CHANGELOG.md -->
<!-- version: 19 -->
<!-- version: 23 -->
# CHANGELOG — ks-store
## `0.5.3-pre.002-delta-fix-003`
- corrige deux appels de helpers strictement privés PostgreSQL qui avaient été qualifiés à tort via `crate::`; ils restent locaux à leur module et ne sont pas réexportés au crate-root ;
- ne modifie aucun contrat public ni schéma SQL.
## `0.5.3-pre.002-delta-fix-002`
- remplace les visibilités `pub(in crate::postgres)` par la convention workspace : privé ou `pub(crate)` + réexport crate-root et consommation via `crate::Item` ;
- ajoute le canari Rust interdisant `pub(in ...)`/`pub(super)` et corrige les reexports `#[cfg(test)]` de `ks-pipeline-demo-scenarios` ;
- supprime les appels obsolètes à `Store::initialize_store_schema()` : linitialisation reste possédée par `Store::open()` via la configuration backend ;
- renomme structurellement les modules, fenêtres, commandes, HTML/TypeScript et DTO `demo_sql_*` en `demo_store_*`, sans attendre la refonte fonctionnelle finale de `pre.005` ;
- réconcilie le tracing PostgreSQL avec la règle dun target unique : `target: crate::TRACING_TARGET`, `backend="postgres"`, `domain="ks-store.pg"`, `action=...` ;
- supprime les helpers/constantes transitoires morts signalés par `cargo check`/Clippy et lattente Clippy devenue inutile.
## `0.5.3-pre.002-delta-fix-001`
- corrige les helpers PostgreSQL de validation de noms de tables rendus inaccessibles par la nouvelle frontière privée et aligne leurs appels/tests sur la façade interne `crate::postgres::*` ;
- supprime les réexports PostgreSQL internes devenus inutiles qui produisaient des warnings et conserve le target générique `ks-store` en l'utilisant réellement à l'ouverture de la façade `Store` ;
- aligne les nouveaux types publics/crate-publics introduits par `pre.002` sur la règle de qualification `crate::Item` dans leur crate propriétaire.
## `0.5.3-pre.002`
### Façade et configuration
- introduit `Store` et `StoreOpenOptions` comme frontière publique backend-agnostique et confine `PostgresStore`, `PostgresStoreOptions`, `PgPool`, les constantes physiques et les requêtes SQL à l'implémentation privée ;
- remplace les sections runtime PostgreSQL/SQLite imposées par un code `backend` et un objet `backend_options` opaque jusqu'à son interprétation par `ks-store`, sans créer de dépendance `ks-store -> ks-config` ;
- retire les quatre anciens traits/DTO historiques sans implémentation réelle et renomme les contrats publics de replay/diagnostic sans préfixe PostgreSQL ;
- conserve `Store` dans l'état applicatif desktop via une initialisation unique et réutilisable par les commandes/scénarios.
### Diagnostics et sécurité
- ajoute les résumés génériques de configuration, backend, initialisation et vérification par modèle, sans exposition des noms physiques de tables/index ni des options backend brutes ;
- masque le descripteur PostgreSQL et sanitise l'erreur de connexion afin qu'aucun DSN/password ne traverse les diagnostics normaux ;
- introduit le domaine structuré PostgreSQL `ks-store.pg` sous lunique `target: crate::TRACING_TARGET`, ainsi qu'un canari workspace empêchant les types PostgreSQL/SQLx de sortir de `ks-store` ;
- adapte les consommateurs Rust et les payloads desktop aux ressources/diagnostics génériques ; le renommage structurel `demo_sql_* -> demo_store_*` est finalisé par `delta-fix-002`, tandis que la refonte fonctionnelle finale reste à la tranche consommateurs.
### Portée
- ne modifie aucune table ni migration SQL de production : le namespace `k_sol_*`, `block_time`, la symétrie top-level/CPI et les ressources SQL atomiques restent réservés à `pre.003`.
## `0.5.3-pre.001`
### Audit / plan
@@ -16,7 +56,7 @@
- normalise le vocabulaire d'instruction vers `top-level`, conserve les CPI séparées et exige `stack_height`/parent CPI immédiat lorsque reconstructible ;
- ajoute un canari de gel future-decoder/Anchor-readiness : N1/N2/N3 doivent conserver les données Solana génériques nécessaires aux instructions/CPI, événements/logs, `returnData`, états/observations de comptes et provenance de schéma, sans ajouter de contrat `anchor_*` ; les 13 tables existantes restent une baseline et non un quota final si un manque générique est découvert avant gel ;
- formalise la reconstruction `kb_sol_* -> k_sol_*`, l'arborescence `migrations/postgres/`, une instruction SQL par fichier exécutée via `include_str!`, l'audit des index/paginations/BIGINT et les règles de replay/invalidation ;
- définit la façade de stockage backend-agnostique, l'encapsulation de PostgreSQL, la stratégie de configuration backend opaque jusqu'à `ks-store`, le résumé runtime/init sans secrets, la future réutilisation d'un `Store` persistant dans `AppState`, la refonte backend-agnostique de `demo_sql`/`demo_config`/splash et les targets `ks-store.pg.*` ;
- définit la façade de stockage backend-agnostique, l'encapsulation de PostgreSQL, la stratégie de configuration backend opaque jusqu'à `ks-store`, le résumé runtime/init sans secrets, la future réutilisation d'un `Store` persistant dans `AppState`, le renommage structurel `demo_sql_* -> demo_store_*`, la refonte backend-agnostique de `demo_config`/splash et le domaine de tracing PostgreSQL `ks-store.pg` sous lunique target `ks-store` ;
- ajoute [`docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md`](../docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md) ;
- ne crée aucune table DEX/trading ni projection metadata N4 dans `0.5.3` ;
- considère `pre.007` comme premier candidat de clôture et autorise explicitement son décalage si des manques apparaissent ;

View File

@@ -1,48 +1,56 @@
<!-- file: ks-store/README.md -->
<!-- version: 4 -->
<!-- version: 6 -->
# ks-store
`ks-store` consolide les contrats de stockage et limplémentation PostgreSQL de `khadhroony-bot3`.
`ks-store` porte la frontière de persistance généraliste de Khadhroony Solana. Les consommateurs utilisent des contrats fonctionnels et une façade `Store` indépendante du moteur concret ; PostgreSQL reste l'implémentation opérationnelle de `0.5.3`, confinée à l'intérieur de la crate.
## Périmètre
La crate expose :
- DTO et entités raw, observations, Core, décodage, matérialisation et ledger ;
- traits async de repositories indépendants du backend ;
- pagination et filtres bornés ;
- `PostgresStore` et ses options de connexion ;
- initialisation idempotente du schéma et migrations SQL ;
- diagnostics read-only, healthchecks et résumés de replay ;
- transactions atomiques pour extraction, décodage et matérialisation.
- `Store` et `StoreOpenOptions` comme façade d'ouverture et handle persistant backend-agnostique ;
- les DTO et entités raw, observations, Core, décodage, matérialisation et ledger ;
- les traits async `StoreHealthStore`, `RawTransactionStore`, `CoreTransactionStore`, `CoreExtractionStore` et `DecodePipelineStore` ;
- les filtres et résultats de replay sans préfixe backend ;
- des diagnostics et résumés runtime/init sanitisés ne révélant ni secret ni nom physique d'objet ;
- des transactions atomiques pour extraction, décodage et matérialisation.
## Frontière backend
`PostgresStore`, `PostgresStoreOptions`, `sqlx::PgPool`, les requêtes SQL et les noms physiques de tables sont privés à `ks-store`. `ks-pipeline`, les scénarios et les applications ne doivent connaître que `Store`, les repositories et les DTO fonctionnels.
`ks-store` ne dépend pas de `ks-config`. Un consommateur lui fournit un backend sélectionné et un objet `backend_options` opaque ; seul `ks-store` interprète et valide les paramètres PostgreSQL puis crée la connexion.
## Configuration et sécurité
`StoreOpenOptions` ne dérive pas `Debug` et ne rend pas les options backend accessibles après construction. Les résumés publics exposent uniquement des informations explicitement sanitisées : code backend, présence d'une connexion, état d'initialisation, health, modèle logique et compteurs.
Le backend PostgreSQL masque le descripteur de connexion et utilise lunique target de crate `ks-store`, complété par `backend="postgres"`, `domain="ks-store.pg"` et un champ `action` structuré. Aucun DSN, password ou valeur bindée n'est destiné aux diagnostics publics.
## Schéma pendant `0.5.3`
`0.5.3-pre.002` ne modifie pas encore les tables héritées `kb_sol_*`. Leur reconstruction sous `k_sol_*`, les ressources SQL atomiques et le gel N1/N2/N3 commencent en `pre.003` conformément au plan actif.
## Responsabilités
- garantir les invariants des données persistées ;
- séparer contrats et implémentation PostgreSQL à lintérieur de la crate consolidée ;
- maintenir les tables canoniques `kb_sol_*` ;
- fournir au pipeline des frontières async typées ;
- posséder la connexion et l'initialisation du backend actif ;
- fournir des frontières async typées au pipeline ;
- protéger les opérations multi-tables par transactions ;
- borner toutes les requêtes de diagnostic et de replay exposées.
- borner les diagnostics et replay exposés ;
- permettre le remplacement futur de PostgreSQL sans réécriture des consommateurs.
## Hors périmètre
La crate ne décode pas les instructions, nacquiert pas les transactions et ne décide pas quelle matérialisation exécuter. Elle persiste les résultats produits par `ks-lib` et orchestrés par `ks-pipeline`.
## API publique
Les consommateurs utilisent principalement `PostgresStore`, `PostgresStoreOptions`, les traits `*Store`, les DTO `*Insert`/`*Row`, les filtres de replay et les diagnostics. Les constantes de tables et fonctions de validation sont publiques pour les audits et outils dadministration.
La crate ne décode pas les instructions, n'acquiert pas les transactions et ne décide pas quelle matérialisation ou projection N4 exécuter. Elle persiste les faits produits par `ks-lib` et orchestrés par `ks-pipeline`.
## Relations
- dépend de `ks-core` pour les erreurs ;
- utilise les contrats de replay de `ks-lib` ;
- est consommée principalement par `ks-pipeline`, les scénarios de démonstration et le desktop.
## Statut
Le stockage consolidé dispose de tests unitaires étendus et de tests PostgreSQL optionnels pilotés par environnement. Les opérations réelles exigent un serveur PostgreSQL et lapplication préalable des migrations.
- est consommée principalement par `ks-pipeline`, `ks-pipeline-demo-scenarios` et `kb-app-demo-desktop` ;
- ne dépend pas de `ks-config`.
## Documents
@@ -50,3 +58,4 @@ Le stockage consolidé dispose de tests unitaires étendus et de tests PostgreSQ
- [Travaux restants](TODO.md)
- [Historique](CHANGELOG.md)
- [Architecture du stockage](../docs/architecture/STORAGE_ARCHITECTURE.md)
- [Plan actif `0.5.3`](../docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md)

View File

@@ -1,18 +1,16 @@
<!-- file: ks-store/TODO.md -->
<!-- version: 11 -->
<!-- version: 13 -->
# TODO — ks-store
## Série `0.5.x`
- [ ] `0.5.3` — introduire une façade `Store` backend-agnostique, rendre PostgreSQL/SQLx privés et supprimer les contrats publics historiques non implémentés.
- [ ] `0.5.3` — remodeler la configuration store afin que le backend sélectionné et ses options restent opaques jusqu'à leur validation/interprétation par `ks-store`.
- [ ] `0.5.3` — reconstruire `kb_sol_* -> k_sol_*`, ajouter `block_time` aux ancres raw/Core, conserver top-level/CPI dans deux tables Core avec lifecycle/replay équivalents, `stack_height`/parent CPI audités et renommer le journal N3 en `k_sol_mat_outputs`.
- [ ] `0.5.3` — déplacer les ressources sous `migrations/postgres/`, imposer une instruction SQL par fichier avec exécution privée via `include_str!`, séparer tables/keys/index/drop/truncate et supprimer le DDL dupliqué en Rust.
- [ ] `0.5.3` — corriger l'invalidation descendante, formaliser les replays N1 -> N2 / N2 -> N3 / frontière N3 -> N4, les paginations/limites, les conversions BIGINT et les index justifiés par les requêtes réelles.
- [ ] `0.5.3`migrer `ks-pipeline-demo-scenarios` et `kb-app-demo-desktop` vers les API génériques sans symboles `Postgres*`, conserver un `Store` générique dans létat applicatif, fournir un résumé runtime/init sans secrets et réconcilier `demo_sql`/`demo_config`/splash sans noms d'objets physiques.
- [ ] `0.5.3`finaliser en `pre.005` la présentation fonctionnelle des surfaces `demo_store_*`, `demo_config` et le splash autour des modèles/compteurs du nouveau baseline sans noms d'objets physiques ; le renommage technique `demo_sql_* -> demo_store_*` est réalisé en `pre.002`.
- [ ] `0.5.3` — exécuter avant gel le canari future-decoder/Anchor-readiness sur N1/N2/N3 : bytes d'instruction, comptes ordonnés/flags, hiérarchie CPI, logs, `returnData`, observations/états de comptes et provenance decoder/schema ; ajouter seulement les contrats Solana génériques manquants, sans table `anchor_*`.
- [ ] `0.5.3` — produire le snapshot machine-readable des tables N1/N2/N3 réellement stabilisées après ce canari (13 tables existantes de baseline, plus éventuels contrats génériques nécessaires), documenter la politique N4 plus souple et les futures projections metadata asset/token + program metadata, puis ajouter les canaris anti-`kb_sol_*`, anti-backend leak, anti-secret et logging `ks-store.pg.*`.
- [ ] `0.5.3` — produire le snapshot machine-readable des tables N1/N2/N3 réellement stabilisées après ce canari (13 tables existantes de baseline, plus éventuels contrats génériques nécessaires), documenter la politique N4 plus souple et les futures projections metadata asset/token + program metadata, puis ajouter les canaris anti-`kb_sol_*`, anti-backend leak, anti-secret et logging sous le target canonique `ks-store` avec domaine structuré `ks-store.pg`.
- [ ] PostgreSQL - auditer le dimensionnement du pool selon les charges réelles.
- [ ] Résilience - décider et implémenter, si justifié, un retry borné pour les timeouts transitoires.
- [ ] Administration - définir les outils supplémentaires prévus par le ROADMAP avant leur implémentation.

View File

@@ -1,90 +1,79 @@
<!-- file: ks-store/USAGE.md -->
<!-- version: 4 -->
<!-- version: 5 -->
# Utilisation de ks-store
## Connexion PostgreSQL
## Ouvrir le store
Le consommateur transmet uniquement le backend sélectionné et son objet d'options déjà résolu. Il ne construit jamais un pool PostgreSQL.
```rust
let options = match ks_store::PostgresStoreOptions::new(
database_url,
10,
10_000,
let options = match ks_store::StoreOpenOptions::new(
true,
"postgres",
serde_json::json!({
"url": database_url,
"max_connections": 10,
"connect_timeout_ms": 10_000,
"auto_initialize_schema": true
}),
) {
Ok(value) => value,
Err(error) => return Err(error),
};
println!("postgres endpoint: {}", options.masked_dsn());
let store = match ks_store::PostgresStore::connect(options).await {
let store = match ks_store::Store::open(options).await {
Ok(value) => value,
Err(error) => return Err(error),
};
```
`PostgresStoreOptions::new` valide le DSN, le nombre de connexions et le timeout. Lorsque `auto_initialize_schema` vaut `true`, la connexion applique les schémas idempotents. Utiliser `masked_dsn` ou `mask_postgres_dsn` dans les logs ; ne jamais journaliser le DSN brut.
`Store::open` interprète les options du backend actif, crée la connexion, exécute l'auto-initialisation lorsqu'elle est configurée et capture un résumé d'initialisation sanitisé. PostgreSQL reste le seul backend opérationnel exigé en `0.5.3-pre.002`; un code backend inconnu est refusé proprement.
## Diagnostics
## Résumés sûrs
```rust
let health = store.health_snapshot().await;
let migrations = store.migration_snapshot().await;
let backend = store.backend_diagnostics().await;
println!("health={:?}", health.status);
println!("migration={:?}", migrations.status);
println!("backend={:?}", backend.descriptor.backend);
```
Les diagnostics de tables raw, Core et decode/materialization sont également disponibles par les méthodes `*_table_diagnostics`.
```rust
let raw_tables = match store.raw_table_diagnostics().await {
let configuration = store.configuration_summary();
let initialization = store.initialization_summary();
let runtime = match store.runtime_summary().await {
Ok(value) => value,
Err(error) => return Err(error),
};
```
for table in raw_tables {
Ces contrats ne contiennent pas les options backend brutes. Le descripteur de connexion éventuellement exposé par `runtime.backend` est masqué par le backend propriétaire.
Le résumé d'initialisation classe les ressources par modèles logiques et fournit des compteurs. Il expose aussi `objects`, une liste de catégories physiques sans noms d'objets : PostgreSQL publie déjà le total `table`, tandis que `pre.003` ajoutera le total `index` avec le nouveau baseline. Les noms physiques de tables, index et contraintes restent internes au backend.
## Diagnostics de ressources
```rust
let raw_resources = match store.raw_resource_diagnostics().await {
Ok(value) => value,
Err(error) => return Err(error),
};
for resource in raw_resources {
println!(
"table={} domain={} exists={}",
table.table_name,
table.domain,
table.exists
"resource={} model={} available={}",
resource.resource_code,
resource.model_code,
resource.available
);
}
```
## Pagination bornée
```rust
let first_page = ks_store::PageRequest::first_page();
let next_page = match ks_store::PageRequest::new(250, 250) {
Ok(value) => value,
Err(error) => return Err(error),
};
assert_eq!(first_page.limit, ks_store::DEFAULT_PAGE_SIZE);
assert!(next_page.limit <= ks_store::MAX_PAGE_SIZE);
```
Les méthodes `core_resource_diagnostics`, `decode_resource_diagnostics` et `known_resource_diagnostics` suivent le même contrat backend-agnostique.
## Contrats de repositories
Les traits publics principaux sont :
Les traits publics effectivement implémentés sont :
- `StoreHealthStore` ;
- `RawTransactionStore` ;
- `CoreTransactionStore` ;
- `CoreExtractionStore` ;
- `DecodePipelineStore` ;
- `ProgramObservationStore` ;
- `DecodedEventStore` ;
- `MaterializedEventStore` ;
- `ProcessingLedgerStore`.
- `DecodePipelineStore`.
Ils permettent au pipeline de dépendre dun contrat async plutôt que dune requête SQL directe.
`Store` implémente ces traits et délègue au backend privé. Les anciens traits historiques sans implémentation réelle ont été retirés avant gel de l'API.
## Replay et requêtes bornées
@@ -93,71 +82,43 @@ let candidates = match store.replay_transaction_candidates(filter).await {
Ok(value) => value,
Err(error) => return Err(error),
};
for candidate in candidates {
println!(
"signature={} slot={}",
candidate.signature,
candidate.slot
);
}
```
```rust
let programs = match store.replay_program_summaries(program_filter).await {
Ok(value) => value,
Err(error) => return Err(error),
};
let entities = match store.replay_entity_summaries(entity_filter).await {
Ok(value) => value,
Err(error) => return Err(error),
};
println!("programs={}, entities={}", programs.len(), entities.len());
```
Les filtres refusent les limites nulles ou supérieures aux bornes publiques. `PageRequest` impose également `DEFAULT_PAGE_SIZE` et `MAX_PAGE_SIZE`.
Les contrats publics utilisent `ReplayTransaction*`, `ReplayProgram*` et `ReplayEntity*`. Le scope d'instruction utilise désormais `TopLevel`, `Inner` et `Logs`; le terme historique `outer` n'est pas introduit dans la nouvelle API.
## Pagination bornée
```rust
let first_page = ks_store::PageRequest::first_page();
let next_page = match ks_store::PageRequest::new(250, 250) {
Ok(value) => value,
Err(error) => return Err(error),
};
assert_eq!(first_page.limit, ks_store::DEFAULT_PAGE_SIZE);
assert!(next_page.limit <= ks_store::MAX_PAGE_SIZE);
```
La refonte cursorisée prévue par le plan `0.5.3` reste dans la tranche repositories/replay ultérieure.
## Transactions atomiques
Les bundles `CoreExtractionBundle`, `DecodePersistenceBundle` et `MaterializationPersistenceBundle` regroupent les écritures qui doivent réussir ou être annulées ensemble. Les consommateurs ne doivent pas reproduire manuellement ces transactions avec des écritures isolées.
Les bundles `CoreExtractionBundle`, `DecodePersistenceBundle` et `MaterializationPersistenceBundle` regroupent les écritures qui doivent réussir ou être annulées ensemble. Les consommateurs ne doivent pas reproduire manuellement ces transactions avec des écritures SQL isolées.
## Validation des noms de tables
## Tests PostgreSQL réels
```rust
if let Err(error) = ks_store::validate_raw_store_table_names() {
return Err(error);
}
if let Err(error) = ks_store::validate_core_store_table_names() {
return Err(error);
}
if let Err(error) = ks_store::validate_decode_store_table_names() {
return Err(error);
}
Les tests opt-in utilisent `KS_SECRET_POSTGRES_TEST_URL`. Aucun credential réel ne doit être enregistré dans les fixtures ou logs. Les erreurs d'ouverture de connexion sont volontairement sanitisées.
assert!(ks_store::is_valid_solana_table_name(
ks_store::CORE_TRANSACTIONS_TABLE_NAME
));
```
## Limites de `pre.002`
## Schéma et tables
Les constantes `RAW_STORE_TABLE_NAMES`, `CORE_STORE_TABLE_NAMES` et `DECODE_STORE_TABLE_NAMES` exposent les noms canoniques. Les fonctions `validate_*_table_names` et `is_valid_solana_table_name` servent aux audits et outils dadministration.
## Erreurs et invariants
Toutes les APIs utilisent `ks_core::Result`. Les DTO valident notamment les signatures, chemins, Program IDs, clés, montants, états de traitement et identités de ledger. Les erreurs de contrat peuvent être construites avec `storage_contract_error`.
## Tests instructifs
- les tests `*_rejects_*` documentent les invariants des DTO et filtres ;
- `decoded_events_and_ledger_roll_back_together` vérifie latomicité ;
- `optional_postgres_*_from_env` couvre les parcours réels lorsque lenvironnement PostgreSQL est configuré ;
- les tests de schéma vérifient les noms, index, contraintes et migrations.
## Limites
- backend actif : PostgreSQL ;
- les tests réels sont optionnels sans DSN de test ;
- la crate ne prend aucune décision de décodage ou de matérialisation.
- les tables restent temporairement sous leur namespace historique jusqu'à `pre.003` ;
- les migrations PostgreSQL ne sont pas encore déplacées sous `migrations/postgres/` ;
- la pagination cursorisée et l'invalidation descendante sont traitées dans la tranche repositories/replay ;
- la refonte complète des fenêtres desktop `demo_store_*` est prévue dans la tranche consommateurs.

View File

@@ -1,7 +1,9 @@
// file: ks-store/src/constants.rs
// version: 2
// version: 5
//! Local constants for the `ks-store` crate.
/// Canonical tracing target for this crate.
/// Canonical tracing target for backend-independent storage operations.
pub(crate) const TRACING_TARGET: &str = "ks-store";
/// Transitional schema contract identifier used until the `pre.003` baseline rebuild.
pub const STORE_SCHEMA_CONTRACT_VERSION: &str = "0.5.3-pre.2-legacy-schema";

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/contracts.rs
// version: 3
// version: 6
//! Backend-neutral storage contracts used by pipeline crates.
@@ -8,6 +8,7 @@ mod entity;
mod error;
mod health;
mod pagination;
mod replay;
mod repository;
/// Core account key insert contract.
@@ -52,8 +53,6 @@ pub use self::dto::DecodeObservationInsert;
pub use self::dto::DecodePersistenceBundle;
/// Bounded contextual instruction selection filter for decode campaigns.
pub use self::dto::DecodeSelectionFilter;
/// Decoded event insert contract.
pub use self::dto::DecodedEventInsert;
/// Insert or upsert result contract returned by repositories.
pub use self::dto::InsertOutcome;
/// Maximum number of materialized rows returned by one bounded query.
@@ -62,16 +61,12 @@ pub use self::dto::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
pub use self::dto::MaterializationPersistenceBundle;
/// Bounded read-only materialized event selection.
pub use self::dto::MaterializedEventFilter;
/// Materialized event insert contract.
pub use self::dto::MaterializedEventInsert;
/// One materialized output returned by a bounded query.
pub use self::dto::MaterializedEventQueryRow;
/// One processor-owned materialized output row.
pub use self::dto::MaterializedOutputInsert;
/// Stable processing ledger identity.
pub use self::dto::ProcessingLedgerIdentity;
/// Processing ledger mark request contract.
pub use self::dto::ProcessingLedgerMark;
/// Stable processing ledger status.
pub use self::dto::ProcessingLedgerStatus;
/// Raw payload lifecycle mark request.
@@ -84,12 +79,30 @@ pub use self::dto::RawPayloadRetentionState;
pub use self::dto::RawTransactionInsert;
/// Store backend diagnostic contract.
pub use self::dto::StoreBackendDescriptor;
/// Store backend kind contract.
pub use self::dto::StoreBackendKind;
/// Backend-neutral diagnostic snapshot.
pub use self::dto::StoreBackendDiagnostics;
/// Store configuration summary safe for UI and logs.
pub use self::dto::StoreConfigurationSummary;
/// Store initialization status.
pub use self::dto::StoreInitializationStatus;
/// Initialization and model-verification summary captured while opening a store.
pub use self::dto::StoreInitializationSummary;
/// Store migration diagnostic snapshot contract.
pub use self::dto::StoreMigrationSnapshot;
/// Store migration status contract.
pub use self::dto::StoreMigrationStatus;
/// Store model verification status.
pub use self::dto::StoreModelVerificationStatus;
/// Verification summary for one logical store model.
pub use self::dto::StoreModelVerificationSummary;
/// Count summary for one backend object category without exposing physical names.
pub use self::dto::StoreObjectVerificationSummary;
/// Read-only diagnostics for one logical store resource.
pub use self::dto::StoreResourceDiagnostics;
/// Read-only statistics for one logical store resource.
pub use self::dto::StoreResourceStatistics;
/// Complete runtime summary safe for application diagnostics.
pub use self::dto::StoreRuntimeSummary;
/// Transaction acquisition observation insert contract.
pub use self::dto::TransactionObservationInsert;
/// Transaction acquisition observation origin.
@@ -108,12 +121,6 @@ pub use self::entity::CoreInstructionRow;
pub use self::entity::CoreLogRow;
/// Core transaction SQL-like row contract.
pub use self::entity::CoreTransactionRow;
/// Decoded event SQL-like row contract.
pub use self::entity::DecodedEventRow;
/// Materialized event SQL-like row contract.
pub use self::entity::MaterializedEventRow;
/// Processing ledger SQL-like row contract.
pub use self::entity::ProcessingLedgerRow;
/// Canonical raw Solana transaction SQL-like row contract.
pub use self::entity::RawTransactionRow;
/// Transaction acquisition observation SQL-like row contract.
@@ -132,20 +139,30 @@ pub use self::pagination::MAX_PAGE_SIZE;
pub use self::pagination::PageRequest;
/// Sort direction for repository list operations.
pub use self::pagination::SortDirection;
/// Maximum number of rows returned by one replay candidate query.
pub use self::replay::MAX_REPLAY_CANDIDATE_ROWS;
/// Bounded read-only filter for core entity summaries.
pub use self::replay::ReplayEntityFilter;
/// Core entity kind used while filtering replay candidates.
pub use self::replay::ReplayEntityKind;
/// Aggregated mint, owner or account-key occurrences from Core facts.
pub use self::replay::ReplayEntitySummary;
/// Bounded read-only filter for program summaries.
pub use self::replay::ReplayProgramFilter;
/// Program occurrence scope used while filtering replay candidates.
pub use self::replay::ReplayProgramScope;
/// Aggregated program occurrences across top-level instructions, inner instructions and linked logs.
pub use self::replay::ReplayProgramSummary;
/// One raw transaction candidate enriched with Core and processing-ledger diagnostics.
pub use self::replay::ReplayTransactionCandidate;
/// Bounded read-only filter for transaction replay candidates.
pub use self::replay::ReplayTransactionFilter;
/// Canonical transaction to core extraction storage behavior.
pub use self::repository::CoreExtractionStore;
/// Core Solana storage behavior.
pub use self::repository::CoreTransactionStore;
/// Contextual instruction decode and materialization storage behavior.
pub use self::repository::DecodePipelineStore;
/// Decoded event storage behavior.
pub use self::repository::DecodedEventStore;
/// Materialized event storage behavior.
pub use self::repository::MaterializedEventStore;
/// Processing ledger storage behavior.
pub use self::repository::ProcessingLedgerStore;
/// Program observation storage behavior.
pub use self::repository::ProgramObservationStore;
/// Raw transaction storage behavior.
pub use self::repository::RawTransactionStore;
/// Store health storage behavior.

View File

@@ -1,13 +1,12 @@
// file: ks-store/src/contracts/dto.rs
// version: 3
// version: 6
//! Backend-neutral DTO exports for storage repository contracts.
mod core;
mod core_extraction;
mod decode;
mod event;
mod ledger;
mod outcome;
mod raw;
mod store;
@@ -67,14 +66,8 @@ pub use self::decode::MaterializedEventFilter;
pub use self::decode::MaterializedEventQueryRow;
/// One processor-owned materialized output row.
pub use self::decode::MaterializedOutputInsert;
/// Decoded event insert contract.
pub use self::event::DecodedEventInsert;
/// Insert or upsert result contract returned by repositories.
pub use self::event::InsertOutcome;
/// Materialized event insert contract.
pub use self::event::MaterializedEventInsert;
/// Processing ledger mark request contract.
pub use self::ledger::ProcessingLedgerMark;
pub use self::outcome::InsertOutcome;
/// Raw payload lifecycle mark request.
pub use self::raw::RawPayloadLifecycleMark;
/// Raw payload processing state.
@@ -91,9 +84,27 @@ pub use self::raw::TransactionObservationOrigin;
pub use self::raw::TransactionObservationStatus;
/// Store backend diagnostic contract.
pub use self::store::StoreBackendDescriptor;
/// Store backend kind contract.
pub use self::store::StoreBackendKind;
/// Backend-neutral diagnostic snapshot.
pub use self::store::StoreBackendDiagnostics;
/// Store configuration summary safe for UI and logs.
pub use self::store::StoreConfigurationSummary;
/// Store initialization status.
pub use self::store::StoreInitializationStatus;
/// Initialization and model-verification summary captured while opening a store.
pub use self::store::StoreInitializationSummary;
/// Store migration diagnostic snapshot contract.
pub use self::store::StoreMigrationSnapshot;
/// Store migration status contract.
pub use self::store::StoreMigrationStatus;
/// Store model verification status.
pub use self::store::StoreModelVerificationStatus;
/// Verification summary for one logical store model.
pub use self::store::StoreModelVerificationSummary;
/// Count summary for one backend object category without exposing physical names.
pub use self::store::StoreObjectVerificationSummary;
/// Read-only diagnostics for one logical store resource.
pub use self::store::StoreResourceDiagnostics;
/// Read-only statistics for one logical store resource.
pub use self::store::StoreResourceStatistics;
/// Complete runtime summary safe for application diagnostics.
pub use self::store::StoreRuntimeSummary;

View File

@@ -1,143 +0,0 @@
// file: ks-store/src/contracts/dto/event.rs
// version: 2
//! Decoded and materialized event storage DTOs.
/// Insert or upsert result contract returned by repositories.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct InsertOutcome {
/// Number of rows inserted by the repository call.
pub inserted_count: u64,
/// Number of rows updated by the repository call.
pub updated_count: u64,
/// Number of rows skipped by the repository call.
pub skipped_count: u64,
}
impl InsertOutcome {
/// Builds an insert outcome from explicit counters.
pub fn new(inserted_count: u64, updated_count: u64, skipped_count: u64) -> Self {
return Self {
inserted_count,
updated_count,
skipped_count,
};
}
/// Returns the sum of inserted, updated and skipped rows.
pub fn total_count(&self) -> u64 {
return self.inserted_count + self.updated_count + self.skipped_count;
}
}
/// Decoded event insert contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct DecodedEventInsert {
/// Transaction signature as non-empty base58 text.
pub signature: std::string::String,
/// Transaction slot in Solana unsigned representation.
pub slot: u64,
/// Stable instruction path, for example `0` or `2/1`.
pub instruction_path: std::string::String,
/// Program id as non-empty base58 text.
pub program_id: std::string::String,
/// Protocol family code.
pub protocol_code: std::string::String,
/// Protocol surface code.
pub surface_code: std::string::String,
/// Canonical event code.
pub event_code: std::string::String,
/// Decoded payload JSON.
pub payload_json: serde_json::Value,
}
impl DecodedEventInsert {
/// Builds a decoded event insert contract from the shared decoded model.
pub fn from_model(
event: &ks_lib::MdDecodedProtocolEvent,
payload_json: serde_json::Value,
) -> ks_core::Result<Self> {
if event.signature.0.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"decoded event signature must not be empty",
));
}
if event.program_id.0.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"decoded event program id must not be empty",
));
}
return std::result::Result::Ok(Self {
signature: event.signature.0.clone(),
slot: event.slot.0,
instruction_path: event.instruction_path.0.clone(),
program_id: event.program_id.0.clone(),
protocol_code: event.protocol_code.0.clone(),
surface_code: event.surface_code.0.clone(),
event_code: event.event_code.0.clone(),
payload_json,
});
}
}
/// Materialized event insert contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct MaterializedEventInsert {
/// Source transaction signature as non-empty base58 text.
pub signature: std::string::String,
/// Source transaction slot in Solana unsigned representation.
pub slot: u64,
/// Materialized event family code.
pub materialized_family: std::string::String,
/// Materialized payload JSON.
pub payload_json: serde_json::Value,
}
impl MaterializedEventInsert {
/// Builds a materialized event insert contract after minimal validation.
pub fn new(
signature: impl std::convert::Into<std::string::String>,
slot: u64,
materialized_family: impl std::convert::Into<std::string::String>,
payload_json: serde_json::Value,
) -> ks_core::Result<Self> {
let signature_value = signature.into();
let materialized_family_value = materialized_family.into();
if signature_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"materialized event signature must not be empty",
));
}
if materialized_family_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"materialized event family must not be empty",
));
}
return std::result::Result::Ok(Self {
signature: signature_value,
slot,
materialized_family: materialized_family_value,
payload_json,
});
}
}
#[cfg(test)]
mod tests {
#[test]
fn insert_outcome_total_counts_all_buckets() {
let outcome = crate::InsertOutcome::new(1, 2, 3);
assert_eq!(outcome.total_count(), 6);
}
#[test]
fn materialized_event_rejects_empty_family() {
let result = crate::MaterializedEventInsert::new(
"abc",
1,
" ",
serde_json::json!({"kind": "trade"}),
);
assert!(result.is_err());
}
}

View File

@@ -1,73 +0,0 @@
// file: ks-store/src/contracts/dto/ledger.rs
// version: 2
//! Processing ledger storage DTOs.
/// Processing ledger mark request contract.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct ProcessingLedgerMark {
/// Processing stage name, for example `raw_ingest`, `core_extract` or `decode`.
pub stage: std::string::String,
/// Processing module name.
pub module_name: std::string::String,
/// Processing module version.
pub module_version: std::string::String,
/// Stable input key, usually a signature or notification id.
pub input_key: std::string::String,
}
impl ProcessingLedgerMark {
/// Builds a processing ledger mark request after minimal validation.
pub fn new(
stage: impl std::convert::Into<std::string::String>,
module_name: impl std::convert::Into<std::string::String>,
module_version: impl std::convert::Into<std::string::String>,
input_key: impl std::convert::Into<std::string::String>,
) -> ks_core::Result<Self> {
let stage_value = stage.into();
let module_name_value = module_name.into();
let module_version_value = module_version.into();
let input_key_value = input_key.into();
if stage_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"processing ledger stage must not be empty",
));
}
if module_name_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"processing ledger module name must not be empty",
));
}
if module_version_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"processing ledger module version must not be empty",
));
}
if input_key_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"processing ledger input key must not be empty",
));
}
return std::result::Result::Ok(Self {
stage: stage_value,
module_name: module_name_value,
module_version: module_version_value,
input_key: input_key_value,
});
}
}
#[cfg(test)]
mod tests {
#[test]
fn ledger_mark_rejects_empty_input_key() {
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", " ");
assert!(result.is_err());
}
#[test]
fn ledger_mark_accepts_minimal_values() {
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", "signature");
assert!(result.is_ok());
}
}

View File

@@ -0,0 +1,40 @@
// file: ks-store/src/contracts/dto/outcome.rs
// version: 4
//! Generic repository operation outcome DTOs.
/// Insert or upsert result contract returned by repositories.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct InsertOutcome {
/// Number of rows inserted by the repository call.
pub inserted_count: u64,
/// Number of rows updated by the repository call.
pub updated_count: u64,
/// Number of rows skipped by the repository call.
pub skipped_count: u64,
}
impl crate::InsertOutcome {
/// Builds an insert outcome from explicit counters.
pub fn new(inserted_count: u64, updated_count: u64, skipped_count: u64) -> Self {
return Self {
inserted_count,
updated_count,
skipped_count,
};
}
/// Returns the sum of inserted, updated and skipped rows.
pub fn total_count(&self) -> u64 {
return self.inserted_count + self.updated_count + self.skipped_count;
}
}
#[cfg(test)]
mod tests {
#[test]
fn insert_outcome_total_counts_all_buckets() {
let outcome = crate::InsertOutcome::new(1, 2, 3);
assert_eq!(outcome.total_count(), 6);
}
}

View File

@@ -1,20 +1,7 @@
// file: ks-store/src/contracts/dto/store.rs
// version: 2
// version: 5
//! Store backend diagnostic DTOs.
/// Store backend kind contract.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub enum StoreBackendKind {
/// PostgreSQL backend.
Postgres,
/// SQLite backend retained for tests and legacy imports.
Sqlite,
/// In-memory backend used by offline tests.
Memory,
/// Backend is not known.
Unknown,
}
//! Backend-agnostic store configuration and diagnostic DTOs.
/// Store migration status contract.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
@@ -33,38 +20,81 @@ pub enum StoreMigrationStatus {
Failed,
}
/// Store backend diagnostic contract.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreBackendDescriptor {
/// Store backend kind.
pub backend_kind: StoreBackendKind,
/// Human-readable backend label.
pub backend_label: std::string::String,
/// Masked DSN or connection descriptor safe for diagnostics.
pub masked_dsn: std::option::Option<std::string::String>,
/// Current PostgreSQL schema or equivalent namespace when known.
pub current_schema: std::option::Option<std::string::String>,
/// Store initialization status exposed without backend-specific details.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub enum StoreInitializationStatus {
/// Persistence is disabled for the selected profile.
Disabled,
/// Every expected logical model is available.
Ready,
/// The backend is reachable but at least one expected logical model is incomplete.
Partial,
/// Initialization or verification failed.
Failed,
}
impl StoreBackendDescriptor {
/// Store model verification status.
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub enum StoreModelVerificationStatus {
/// Every expected resource for the logical model is available.
Ready,
/// At least one expected resource is missing.
Incomplete,
}
/// Store configuration summary safe for UI and logs.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreConfigurationSummary {
/// Whether persistence is enabled for the selected profile.
pub enabled: bool,
/// Stable backend code interpreted by `ks-store`.
pub backend_code: std::string::String,
/// Whether the selected backend supplied a connection/location value.
pub connection_configured: bool,
/// Whether schema auto-initialization is enabled for this backend.
pub auto_initialize_schema: bool,
/// Number of backend options supplied after configuration composition.
pub backend_option_count: u32,
}
/// Store backend diagnostic contract safe for UI display.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreBackendDescriptor {
/// Stable backend code such as `postgres`.
pub backend_code: std::string::String,
/// Human-readable backend label.
pub backend_label: std::string::String,
/// Masked connection descriptor safe for diagnostics.
pub masked_connection_descriptor: std::option::Option<std::string::String>,
/// Backend namespace, schema or equivalent logical location when known.
pub namespace: std::option::Option<std::string::String>,
}
impl crate::StoreBackendDescriptor {
/// Builds a store backend descriptor after minimal validation.
pub fn new(
backend_kind: StoreBackendKind,
backend_code: impl std::convert::Into<std::string::String>,
backend_label: impl std::convert::Into<std::string::String>,
masked_dsn: std::option::Option<std::string::String>,
current_schema: std::option::Option<std::string::String>,
masked_connection_descriptor: std::option::Option<std::string::String>,
namespace: std::option::Option<std::string::String>,
) -> ks_core::Result<Self> {
let backend_code_value = backend_code.into();
let backend_label_value = backend_label.into();
if backend_code_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"store backend code must not be empty",
));
}
if backend_label_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"store backend label must not be empty",
));
}
return std::result::Result::Ok(Self {
backend_kind,
backend_code: backend_code_value,
backend_label: backend_label_value,
masked_dsn,
current_schema,
masked_connection_descriptor,
namespace,
});
}
}
@@ -73,7 +103,7 @@ impl StoreBackendDescriptor {
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreMigrationSnapshot {
/// Current migration status.
pub status: StoreMigrationStatus,
pub status: crate::StoreMigrationStatus,
/// Last applied migration identifier when known.
pub current_version: std::option::Option<std::string::String>,
/// Pending migration identifiers when known.
@@ -82,10 +112,10 @@ pub struct StoreMigrationSnapshot {
pub message: std::option::Option<std::string::String>,
}
impl StoreMigrationSnapshot {
impl crate::StoreMigrationSnapshot {
/// Builds a migration snapshot from explicit values.
pub fn new(
status: StoreMigrationStatus,
status: crate::StoreMigrationStatus,
current_version: std::option::Option<std::string::String>,
pending_versions: std::vec::Vec<std::string::String>,
message: std::option::Option<std::string::String>,
@@ -99,13 +129,114 @@ impl StoreMigrationSnapshot {
}
}
/// Backend-neutral diagnostic snapshot.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreBackendDiagnostics {
/// Backend descriptor safe for UI display.
pub descriptor: crate::StoreBackendDescriptor,
/// Backend health snapshot.
pub health: crate::StoreHealthSnapshot,
/// Migration status snapshot.
pub migrations: crate::StoreMigrationSnapshot,
/// Backend version string when the backend exposes one.
pub backend_version: std::option::Option<std::string::String>,
}
/// Read-only statistics for one logical store resource.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreResourceStatistics {
/// Number of records currently stored in the resource.
pub record_count: i64,
/// Lowest observed Solana slot when meaningful for the resource.
pub min_slot: std::option::Option<i64>,
/// Highest observed Solana slot when meaningful for the resource.
pub max_slot: std::option::Option<i64>,
/// Latest creation timestamp rendered by the backend for diagnostics.
pub latest_created_at: std::option::Option<std::string::String>,
}
/// Read-only diagnostics for one logical store resource.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreResourceDiagnostics {
/// Stable backend-independent resource code.
pub resource_code: std::string::String,
/// Logical model containing this resource.
pub model_code: std::string::String,
/// Human-readable resource role.
pub role: std::string::String,
/// Whether the resource is available in the active backend.
pub available: bool,
/// Resource statistics when available and supported.
pub statistics: std::option::Option<crate::StoreResourceStatistics>,
}
/// Verification summary for one logical store model.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreModelVerificationSummary {
/// Stable logical model code such as `raw`, `core`, `processing`, `decode` or `materialization`.
pub model_code: std::string::String,
/// Verification status for the model.
pub status: crate::StoreModelVerificationStatus,
/// Number of resources expected by the current store contract.
pub expected_resource_count: u32,
/// Number of expected resources available after verification.
pub available_resource_count: u32,
/// Number of resources created during this initialization pass.
pub created_resource_count: u32,
/// Number of expected resources still missing.
pub missing_resource_count: u32,
}
/// Count summary for one backend object category without exposing physical object names.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreObjectVerificationSummary {
/// Stable object category such as `table`, `index` or a backend-specific equivalent.
pub object_kind: std::string::String,
/// Number of objects expected by the active backend contract.
pub expected_count: u32,
/// Number of expected objects available after verification.
pub available_count: u32,
/// Number of expected objects created during this initialization pass.
pub created_count: u32,
}
/// Initialization and model-verification summary captured while opening a store.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreInitializationSummary {
/// Overall initialization status.
pub status: crate::StoreInitializationStatus,
/// Stable schema/model contract version.
pub schema_contract_version: std::string::String,
/// Per-model verification summaries.
pub models: std::vec::Vec<crate::StoreModelVerificationSummary>,
/// Total number of expected logical resources.
pub expected_resource_count: u32,
/// Total number of available logical resources.
pub available_resource_count: u32,
/// Total number of logical resources created during opening.
pub created_resource_count: u32,
/// Backend object counts grouped by category without physical object names.
pub objects: std::vec::Vec<crate::StoreObjectVerificationSummary>,
}
/// Complete runtime summary safe for desktop diagnostics.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct StoreRuntimeSummary {
/// Sanitized selected configuration.
pub configuration: crate::StoreConfigurationSummary,
/// Backend diagnostics when persistence is enabled and opened.
pub backend: std::option::Option<crate::StoreBackendDiagnostics>,
/// Initialization/model verification result captured during opening.
pub initialization: crate::StoreInitializationSummary,
}
#[cfg(test)]
mod tests {
#[test]
fn store_descriptor_rejects_empty_label() {
fn store_descriptor_rejects_empty_code() {
let result = crate::StoreBackendDescriptor::new(
crate::StoreBackendKind::Postgres,
" ",
"PostgreSQL",
std::option::Option::None,
std::option::Option::None,
);

View File

@@ -1,32 +1,24 @@
// file: ks-store/src/contracts/entity.rs
// version: 3
// version: 5
//! Backend-neutral SQL-like entity exports for storage adapters.
//! Backend-neutral persisted entity exports for storage adapters.
mod core;
mod event;
mod ledger;
mod raw;
/// Core account key SQL-like row contract.
/// Core account key persisted row contract.
pub use self::core::CoreAccountKeyRow;
/// Core balance change SQL-like row contract.
/// Core balance change persisted row contract.
pub use self::core::CoreBalanceChangeRow;
/// Core inner instruction SQL-like row contract.
/// Core inner instruction persisted row contract.
pub use self::core::CoreInnerInstructionRow;
/// Core instruction SQL-like row contract.
/// Core instruction persisted row contract.
pub use self::core::CoreInstructionRow;
/// Core log SQL-like row contract.
/// Core log persisted row contract.
pub use self::core::CoreLogRow;
/// Core transaction SQL-like row contract.
/// Core transaction persisted row contract.
pub use self::core::CoreTransactionRow;
/// Decoded event SQL-like row contract.
pub use self::event::DecodedEventRow;
/// Materialized event SQL-like row contract.
pub use self::event::MaterializedEventRow;
/// Processing ledger SQL-like row contract.
pub use self::ledger::ProcessingLedgerRow;
/// Canonical raw Solana transaction SQL-like row contract.
/// Canonical raw Solana transaction persisted row contract.
pub use self::raw::RawTransactionRow;
/// Transaction acquisition observation SQL-like row contract.
/// Transaction acquisition observation persisted row contract.
pub use self::raw::TransactionObservationRow;

View File

@@ -1,9 +1,9 @@
// file: ks-store/src/contracts/entity/core.rs
// version: 2
// version: 3
//! Core Solana SQL-like entities.
//! Core Solana persisted entities.
/// Core transaction SQL-like row contract.
/// Core transaction persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreTransactionRow {
/// Technical primary key.
@@ -24,7 +24,7 @@ pub struct CoreTransactionRow {
pub updated_at: chrono::DateTime<chrono::Utc>,
}
/// Core account key SQL-like row contract.
/// Core account key persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreAccountKeyRow {
/// Technical primary key.
@@ -51,7 +51,7 @@ pub struct CoreAccountKeyRow {
pub created_at: chrono::DateTime<chrono::Utc>,
}
/// Core instruction SQL-like row contract.
/// Core instruction persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreInstructionRow {
/// Technical primary key.
@@ -80,7 +80,7 @@ pub struct CoreInstructionRow {
pub updated_at: chrono::DateTime<chrono::Utc>,
}
/// Core inner instruction SQL-like row contract.
/// Core inner instruction persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreInnerInstructionRow {
/// Technical primary key.
@@ -107,7 +107,7 @@ pub struct CoreInnerInstructionRow {
pub created_at: chrono::DateTime<chrono::Utc>,
}
/// Core log SQL-like row contract.
/// Core log persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreLogRow {
/// Technical primary key.
@@ -132,7 +132,7 @@ pub struct CoreLogRow {
pub created_at: chrono::DateTime<chrono::Utc>,
}
/// Core balance change SQL-like row contract.
/// Core balance change persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct CoreBalanceChangeRow {
/// Technical primary key.

View File

@@ -1,46 +0,0 @@
// file: ks-store/src/contracts/entity/event.rs
// version: 2
//! Decoded and materialized event SQL-like entities.
/// Decoded event SQL-like row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct DecodedEventRow {
/// Technical primary key.
pub id: i64,
/// Transaction signature as non-empty base58 text.
pub signature: std::string::String,
/// Transaction slot stored as SQL `BIGINT`.
pub slot: i64,
/// Stable instruction path, for example `0` or `2/1`.
pub instruction_path: std::string::String,
/// Program id as non-empty base58 text.
pub program_id: std::string::String,
/// Protocol family code.
pub protocol_code: std::string::String,
/// Protocol surface code.
pub surface_code: std::string::String,
/// Canonical event code.
pub event_code: std::string::String,
/// Decoded payload JSON.
pub payload_json: serde_json::Value,
/// Insert timestamp.
pub created_at: chrono::DateTime<chrono::Utc>,
}
/// Materialized event SQL-like row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct MaterializedEventRow {
/// Technical primary key.
pub id: i64,
/// Source transaction signature as non-empty base58 text.
pub signature: std::string::String,
/// Source transaction slot stored as SQL `BIGINT`.
pub slot: i64,
/// Materialized event family code.
pub materialized_family: std::string::String,
/// Materialized payload JSON.
pub payload_json: serde_json::Value,
/// Insert timestamp.
pub created_at: chrono::DateTime<chrono::Utc>,
}

View File

@@ -1,37 +0,0 @@
// file: ks-store/src/contracts/entity/ledger.rs
// version: 2
//! Processing ledger SQL-like entities.
/// Processing ledger SQL-like row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct ProcessingLedgerRow {
/// Technical primary key.
pub id: i64,
/// Processing stage name.
pub stage: std::string::String,
/// Processor implementation name.
pub processor_name: std::string::String,
/// Processor semantic version.
pub processor_version: std::string::String,
/// Stable input key, usually a transaction signature.
pub input_key: std::string::String,
/// Deterministic input hash.
pub input_hash: std::string::String,
/// Current processing status.
pub status: crate::ProcessingLedgerStatus,
/// Number of started attempts for this processor identity.
pub attempt_count: i32,
/// Optional start timestamp for the latest attempt.
pub started_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
/// Optional finish timestamp for the latest terminal attempt.
pub finished_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
/// Optional machine-readable error code.
pub error_code: std::option::Option<std::string::String>,
/// Optional diagnostic error message.
pub error_message: std::option::Option<std::string::String>,
/// Insert timestamp.
pub created_at: chrono::DateTime<chrono::Utc>,
/// Last update timestamp.
pub updated_at: chrono::DateTime<chrono::Utc>,
}

View File

@@ -1,9 +1,9 @@
// file: ks-store/src/contracts/entity/raw.rs
// version: 2
// version: 3
//! Canonical Solana transaction and acquisition observation SQL-like entities.
//! Canonical Solana transaction and acquisition observation persisted entities.
/// Canonical raw Solana transaction SQL-like row contract.
/// Canonical raw Solana transaction persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct RawTransactionRow {
/// Technical primary key.
@@ -28,7 +28,7 @@ pub struct RawTransactionRow {
pub updated_at: chrono::DateTime<chrono::Utc>,
}
/// Transaction acquisition observation SQL-like row contract.
/// Transaction acquisition observation persisted row contract.
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct TransactionObservationRow {
/// Technical primary key.

View File

@@ -1,30 +1,30 @@
// file: ks-store/src/postgres/replay_candidates.rs
// version: 3
// file: ks-store/src/contracts/replay.rs
// version: 6
//! Read-only replay candidate filters and PostgreSQL result rows.
//! Backend-agnostic replay candidate filters and read models.
/// Maximum number of rows returned by one replay candidate query.
pub const MAX_REPLAY_CANDIDATE_ROWS: u32 = 100_000;
/// Program occurrence scope used while filtering replay candidates.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum PostgresReplayProgramScope {
/// Match outer instructions, inner instructions or reliably linked logs.
pub enum ReplayProgramScope {
/// Match top-level instructions, inner instructions or reliably linked logs.
Any,
/// Match only top-level instructions.
Outer,
TopLevel,
/// Match only inner instructions.
Inner,
/// Match only logs with a reliably linked program id.
Logs,
}
impl PostgresReplayProgramScope {
impl crate::ReplayProgramScope {
/// Returns the stable SQL code for this scope.
pub fn as_sql(self) -> &'static str {
return match self {
Self::Any => "any",
Self::Outer => "outer",
Self::TopLevel => "top_level",
Self::Inner => "inner",
Self::Logs => "logs",
};
@@ -33,7 +33,7 @@ impl PostgresReplayProgramScope {
/// Core entity kind used while filtering replay candidates.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub enum PostgresReplayEntityKind {
pub enum ReplayEntityKind {
/// SPL or Token-2022 mint address.
Mint,
/// Token account owner address.
@@ -42,7 +42,7 @@ pub enum PostgresReplayEntityKind {
AccountKey,
}
impl PostgresReplayEntityKind {
impl crate::ReplayEntityKind {
/// Returns the stable SQL code for this entity kind.
pub fn as_sql(self) -> &'static str {
return match self {
@@ -55,7 +55,7 @@ impl PostgresReplayEntityKind {
/// Bounded read-only filter for transaction replay candidates.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PostgresReplayTransactionFilter {
pub struct ReplayTransactionFilter {
/// Optional partial signature search.
pub signature_contains: std::option::Option<std::string::String>,
/// Optional inclusive minimum slot.
@@ -69,9 +69,9 @@ pub struct PostgresReplayTransactionFilter {
/// Optional exact program id.
pub program_id: std::option::Option<std::string::String>,
/// Program occurrence scope.
pub program_scope: crate::PostgresReplayProgramScope,
pub program_scope: crate::ReplayProgramScope,
/// Optional core entity kind.
pub entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
pub entity_kind: std::option::Option<crate::ReplayEntityKind>,
/// Optional exact entity value.
pub entity_value: std::option::Option<std::string::String>,
/// Maximum returned rows.
@@ -80,9 +80,8 @@ pub struct PostgresReplayTransactionFilter {
pub newest_first: bool,
}
impl PostgresReplayTransactionFilter {
impl crate::ReplayTransactionFilter {
/// Creates and validates a bounded transaction candidate filter.
#[allow(clippy::too_many_arguments)]
pub fn new(
signature_contains: std::option::Option<std::string::String>,
min_slot: std::option::Option<u64>,
@@ -90,8 +89,8 @@ impl PostgresReplayTransactionFilter {
raw_processing_state: std::option::Option<std::string::String>,
ledger_status: std::option::Option<std::string::String>,
program_id: std::option::Option<std::string::String>,
program_scope: crate::PostgresReplayProgramScope,
entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
program_scope: crate::ReplayProgramScope,
entity_kind: std::option::Option<crate::ReplayEntityKind>,
entity_value: std::option::Option<std::string::String>,
limit: u32,
newest_first: bool,
@@ -148,7 +147,7 @@ impl PostgresReplayTransactionFilter {
/// One raw transaction candidate enriched with core and ledger diagnostics.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresReplayTransactionCandidate {
pub struct ReplayTransactionCandidate {
/// Canonical transaction signature.
pub signature: std::string::String,
/// Transaction slot.
@@ -168,27 +167,27 @@ pub struct PostgresReplayTransactionCandidate {
/// Latest core extraction attempt count.
pub attempt_count: i32,
/// Number of top-level instructions.
pub outer_instruction_count: i64,
pub top_level_instruction_count: i64,
/// Number of inner instructions.
pub inner_instruction_count: i64,
/// Number of distinct top-level programs.
pub outer_program_count: i64,
pub top_level_program_count: i64,
/// Number of distinct inner programs.
pub inner_program_count: i64,
/// Raw row update timestamp rendered by PostgreSQL.
/// Raw row update timestamp rendered by the active backend.
pub updated_at: std::string::String,
}
/// Bounded read-only filter for program summaries.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PostgresReplayProgramFilter {
pub struct ReplayProgramFilter {
/// Optional partial program id search.
pub program_id_contains: std::option::Option<std::string::String>,
/// Maximum returned rows.
pub limit: u32,
}
impl PostgresReplayProgramFilter {
impl crate::ReplayProgramFilter {
/// Creates and validates a bounded program summary filter.
pub fn new(
program_id_contains: std::option::Option<std::string::String>,
@@ -205,15 +204,15 @@ impl PostgresReplayProgramFilter {
}
}
/// Aggregated program occurrences across outer, inner and linked logs.
/// Aggregated program occurrences across top-level, inner and linked logs.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresReplayProgramSummary {
pub struct ReplayProgramSummary {
/// Program id.
pub program_id: std::string::String,
/// Number of distinct transactions containing the program.
pub transaction_count: i64,
/// Number of top-level instruction occurrences.
pub outer_instruction_count: i64,
pub top_level_instruction_count: i64,
/// Number of inner instruction occurrences.
pub inner_instruction_count: i64,
/// Number of reliably linked log occurrences.
@@ -226,19 +225,19 @@ pub struct PostgresReplayProgramSummary {
/// Bounded read-only filter for core entity summaries.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PostgresReplayEntityFilter {
pub struct ReplayEntityFilter {
/// Entity kind to aggregate.
pub entity_kind: crate::PostgresReplayEntityKind,
pub entity_kind: crate::ReplayEntityKind,
/// Optional partial entity value search.
pub entity_value_contains: std::option::Option<std::string::String>,
/// Maximum returned rows.
pub limit: u32,
}
impl PostgresReplayEntityFilter {
impl crate::ReplayEntityFilter {
/// Creates and validates a bounded entity summary filter.
pub fn new(
entity_kind: crate::PostgresReplayEntityKind,
entity_kind: crate::ReplayEntityKind,
entity_value_contains: std::option::Option<std::string::String>,
limit: u32,
) -> ks_core::Result<Self> {
@@ -256,7 +255,7 @@ impl PostgresReplayEntityFilter {
/// Aggregated mint, owner or account-key occurrences from core tables.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresReplayEntitySummary {
pub struct ReplayEntitySummary {
/// Stable entity kind code.
pub entity_kind: std::string::String,
/// Mint, owner or account-key address.
@@ -335,14 +334,14 @@ fn validate_optional_code(
mod tests {
#[test]
fn transaction_filter_rejects_inverted_slots() {
let result = crate::PostgresReplayTransactionFilter::new(
let result = crate::ReplayTransactionFilter::new(
std::option::Option::None,
std::option::Option::Some(20),
std::option::Option::Some(10),
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
crate::PostgresReplayProgramScope::Any,
crate::ReplayProgramScope::Any,
std::option::Option::None,
std::option::Option::None,
100,
@@ -353,15 +352,15 @@ mod tests {
#[test]
fn transaction_filter_requires_complete_entity_pair() {
let result = crate::PostgresReplayTransactionFilter::new(
let result = crate::ReplayTransactionFilter::new(
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
crate::PostgresReplayProgramScope::Any,
std::option::Option::Some(crate::PostgresReplayEntityKind::Mint),
crate::ReplayProgramScope::Any,
std::option::Option::Some(crate::ReplayEntityKind::Mint),
std::option::Option::None,
100,
true,
@@ -371,7 +370,7 @@ mod tests {
#[test]
fn program_filter_rejects_limit_above_maximum() {
let result = crate::PostgresReplayProgramFilter::new(
let result = crate::ReplayProgramFilter::new(
std::option::Option::None,
crate::MAX_REPLAY_CANDIDATE_ROWS + 1,
);

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/contracts/repository.rs
// version: 2
// version: 3
//! Storage trait definitions shared by concrete stores.
@@ -192,46 +192,3 @@ pub trait DecodePipelineStore {
filter: &crate::MaterializedEventFilter,
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>>;
}
/// Program observation storage behavior.
#[async_trait::async_trait]
pub trait ProgramObservationStore {
/// Stores program observations.
async fn store_observations(
&self,
observations: &[ks_lib::MdProgramObservation],
) -> ks_core::Result<crate::InsertOutcome>;
}
/// Decoded event storage behavior.
#[async_trait::async_trait]
pub trait DecodedEventStore {
/// Stores decoded protocol events.
async fn store_decoded_events(
&self,
events: &[crate::DecodedEventInsert],
) -> ks_core::Result<crate::InsertOutcome>;
}
/// Materialized event storage behavior.
#[async_trait::async_trait]
pub trait MaterializedEventStore {
/// Stores materialized business events.
async fn store_materialized_events(
&self,
events: &[crate::MaterializedEventInsert],
) -> ks_core::Result<crate::InsertOutcome>;
}
/// Processing ledger storage behavior.
#[async_trait::async_trait]
pub trait ProcessingLedgerStore {
/// Marks an input as processed for a module version.
async fn mark_processed(
&self,
mark: &crate::ProcessingLedgerMark,
) -> ks_core::Result<crate::InsertOutcome>;
/// Returns true when an input was already processed for a module version.
async fn is_processed(&self, mark: &crate::ProcessingLedgerMark) -> ks_core::Result<bool>;
}

View File

@@ -1,7 +1,7 @@
// file: ks-store/src/lib.rs
// version: 5
// version: 10
//! Backend-neutral storage contracts and PostgreSQL implementation.
//! Backend-agnostic Solana storage contracts and store facade.
#![warn(missing_docs)]
#![deny(unreachable_pub)]
#![forbid(unsafe_code)]
@@ -9,13 +9,177 @@
mod constants;
mod contracts;
mod postgres;
mod store;
/// Canonical tracing target for storage operations.
/// Canonical crate-internal tracing target shared through the crate-root facade.
pub(crate) use self::constants::TRACING_TARGET;
/// Crate-internal storage symbol `CORE_ACCOUNT_KEYS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_ACCOUNT_KEYS_TABLE_NAME;
/// Crate-internal storage symbol `CORE_BALANCE_CHANGES_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_BALANCE_CHANGES_TABLE_NAME;
/// Crate-internal storage symbol `CORE_INNER_INSTRUCTIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
/// Crate-internal storage symbol `CORE_INSTRUCTIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_INSTRUCTIONS_TABLE_NAME;
/// Crate-internal storage symbol `CORE_LOGS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_LOGS_TABLE_NAME;
/// Crate-internal storage symbol `CORE_TRANSACTIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::CORE_TRANSACTIONS_TABLE_NAME;
/// Crate-internal storage symbol `DECODE_COVERAGE_DECLARATIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
/// Crate-internal storage symbol `DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
/// Crate-internal storage symbol `DECODE_EVENTS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::DECODE_EVENTS_TABLE_NAME;
/// Crate-internal storage symbol `MATERIALIZED_EVENTS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::MATERIALIZED_EVENTS_TABLE_NAME;
/// Crate-internal storage symbol `PROCESSING_LEDGER_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::PROCESSING_LEDGER_TABLE_NAME;
/// Crate-internal storage symbol `PostgresStore` shared through the crate-root facade.
pub(crate) use self::postgres::PostgresStore;
/// Crate-internal storage symbol `PostgresStoreOptions` shared through the crate-root facade.
pub(crate) use self::postgres::PostgresStoreOptions;
/// Crate-internal storage symbol `PostgresTableDiagnosticSpec` shared through the crate-root facade.
pub(crate) use self::postgres::PostgresTableDiagnosticSpec;
/// Crate-internal storage symbol `RAW_TRANSACTIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::RAW_TRANSACTIONS_TABLE_NAME;
/// Crate-internal storage symbol `STORE_SCHEMA_ADVISORY_LOCK_ID` shared through the crate-root facade.
pub(crate) use self::postgres::STORE_SCHEMA_ADVISORY_LOCK_ID;
/// Crate-internal storage symbol `TRANSACTION_OBSERVATIONS_TABLE_NAME` shared through the crate-root facade.
pub(crate) use self::postgres::TRANSACTION_OBSERVATIONS_TABLE_NAME;
/// Crate-internal storage symbol `apply_core_store_schema` shared through the crate-root facade.
pub(crate) use self::postgres::apply_core_store_schema;
/// Crate-internal storage symbol `apply_decode_store_schema` shared through the crate-root facade.
pub(crate) use self::postgres::apply_decode_store_schema;
/// Crate-internal storage symbol `apply_raw_store_schema` shared through the crate-root facade.
pub(crate) use self::postgres::apply_raw_store_schema;
/// Crate-internal storage symbol `core_store_schema_statements` shared through the crate-root facade.
pub(crate) use self::postgres::core_store_schema_statements;
/// Crate-internal storage symbol `core_store_table_diagnostic_specs` shared through the crate-root facade.
pub(crate) use self::postgres::core_store_table_diagnostic_specs;
/// Crate-internal storage symbol `decode_store_schema_statements` shared through the crate-root facade.
pub(crate) use self::postgres::decode_store_schema_statements;
/// Crate-internal storage symbol `decode_store_table_diagnostic_specs` shared through the crate-root facade.
pub(crate) use self::postgres::decode_store_table_diagnostic_specs;
/// Crate-internal storage symbol `has_raw_transaction_signature` shared through the crate-root facade.
pub(crate) use self::postgres::has_raw_transaction_signature;
/// Crate-internal storage symbol `has_transaction_observation_key` shared through the crate-root facade.
pub(crate) use self::postgres::has_transaction_observation_key;
/// Crate-internal storage symbol `insert_core_account_keys` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_account_keys;
/// Crate-internal storage symbol `insert_core_balance_changes` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_balance_changes;
/// Crate-internal storage symbol `insert_core_inner_instructions` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_inner_instructions;
/// Crate-internal storage symbol `insert_core_instructions` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_instructions;
/// Crate-internal storage symbol `insert_core_logs` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_logs;
/// Crate-internal storage symbol `insert_core_transaction` shared through the crate-root facade.
pub(crate) use self::postgres::insert_core_transaction;
/// Crate-internal storage symbol `insert_raw_transaction` shared through the crate-root facade.
pub(crate) use self::postgres::insert_raw_transaction;
/// Crate-internal storage symbol `insert_transaction_observation` shared through the crate-root facade.
pub(crate) use self::postgres::insert_transaction_observation;
/// Crate-internal storage symbol `is_core_extraction_current` shared through the crate-root facade.
pub(crate) use self::postgres::is_core_extraction_current;
/// Crate-internal storage symbol `is_decode_current` shared through the crate-root facade.
pub(crate) use self::postgres::is_decode_current;
/// Crate-internal storage symbol `list_core_instruction_replay_inputs` shared through the crate-root facade.
pub(crate) use self::postgres::list_core_instruction_replay_inputs;
/// Crate-internal storage symbol `list_core_instructions_for_replay` shared through the crate-root facade.
pub(crate) use self::postgres::list_core_instructions_for_replay;
/// Crate-internal storage symbol `list_decode_coverage_summary` shared through the crate-root facade.
pub(crate) use self::postgres::list_decode_coverage_summary;
/// Crate-internal storage symbol `list_decode_inputs` shared through the crate-root facade.
pub(crate) use self::postgres::list_decode_inputs;
/// Crate-internal storage symbol `list_decode_replay_inputs` shared through the crate-root facade.
pub(crate) use self::postgres::list_decode_replay_inputs;
/// Crate-internal storage symbol `list_materialized_events` shared through the crate-root facade.
pub(crate) use self::postgres::list_materialized_events;
/// Crate-internal storage symbol `list_raw_transactions_for_core_extraction` shared through the crate-root facade.
pub(crate) use self::postgres::list_raw_transactions_for_core_extraction;
/// Crate-internal storage symbol `list_replay_entity_summaries` shared through the crate-root facade.
pub(crate) use self::postgres::list_replay_entity_summaries;
/// Crate-internal storage symbol `list_replay_program_summaries` shared through the crate-root facade.
pub(crate) use self::postgres::list_replay_program_summaries;
/// Crate-internal storage symbol `list_replay_transaction_candidates` shared through the crate-root facade.
pub(crate) use self::postgres::list_replay_transaction_candidates;
/// Crate-internal storage symbol `load_current_schema` shared through the crate-root facade.
pub(crate) use self::postgres::load_current_schema;
/// Crate-internal storage symbol `load_latest_migration_version` shared through the crate-root facade.
pub(crate) use self::postgres::load_latest_migration_version;
/// Crate-internal storage symbol `load_migration_table_name` shared through the crate-root facade.
pub(crate) use self::postgres::load_migration_table_name;
/// Crate-internal storage symbol `load_server_version` shared through the crate-root facade.
pub(crate) use self::postgres::load_server_version;
/// Crate-internal storage symbol `load_table_statistics` shared through the crate-root facade.
pub(crate) use self::postgres::load_table_statistics;
/// Crate-internal storage symbol `mark_core_extraction_failed` shared through the crate-root facade.
pub(crate) use self::postgres::mark_core_extraction_failed;
/// Crate-internal storage symbol `mark_decode_failed` shared through the crate-root facade.
pub(crate) use self::postgres::mark_decode_failed;
/// Crate-internal storage symbol `persist_core_extraction` shared through the crate-root facade.
pub(crate) use self::postgres::persist_core_extraction;
/// Crate-internal storage symbol `persist_decode_coverage_declarations` shared through the crate-root facade.
pub(crate) use self::postgres::persist_decode_coverage_declarations;
/// Crate-internal storage symbol `persist_decode_result` shared through the crate-root facade.
pub(crate) use self::postgres::persist_decode_result;
/// Crate-internal storage symbol `persist_materialization_result` shared through the crate-root facade.
pub(crate) use self::postgres::persist_materialization_result;
#[cfg(test)]
/// Crate-internal serialized PostgreSQL test guard shared through the crate-root facade.
pub(crate) use self::postgres::postgres_test_guard;
/// Crate-internal storage symbol `raw_store_schema_statements` shared through the crate-root facade.
pub(crate) use self::postgres::raw_store_schema_statements;
/// Crate-internal storage symbol `raw_store_table_diagnostic_specs` shared through the crate-root facade.
pub(crate) use self::postgres::raw_store_table_diagnostic_specs;
/// Crate-internal storage symbol `run_health_check` shared through the crate-root facade.
pub(crate) use self::postgres::run_health_check;
/// Crate-internal storage symbol `table_exists` shared through the crate-root facade.
pub(crate) use self::postgres::table_exists;
/// Crate-internal storage symbol `table_stats_kb_sol_core_account_keys_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_account_keys_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_core_balance_changes_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_balance_changes_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_core_inner_instructions_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_inner_instructions_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_core_instructions_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_instructions_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_core_logs_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_logs_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_core_transactions_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_core_transactions_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_decode_coverage_declarations_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_decode_coverage_declarations_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_decode_coverage_observations_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_decode_coverage_observations_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_decode_events_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_decode_events_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_mat_events_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_mat_events_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_obs_transaction_observations_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_obs_transaction_observations_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_ops_processing_ledger_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_ops_processing_ledger_sql;
/// Crate-internal storage symbol `table_stats_kb_sol_raw_transactions_sql` shared through the crate-root facade.
pub(crate) use self::postgres::table_stats_kb_sol_raw_transactions_sql;
/// Crate-internal storage symbol `update_core_instruction_lifecycle` shared through the crate-root facade.
pub(crate) use self::postgres::update_core_instruction_lifecycle;
/// Crate-internal storage symbol `update_raw_payload_lifecycle` shared through the crate-root facade.
pub(crate) use self::postgres::update_raw_payload_lifecycle;
/// Crate-internal storage symbol `validate_core_store_table_names` shared through the crate-root facade.
pub(crate) use self::postgres::validate_core_store_table_names;
/// Crate-internal storage symbol `validate_decode_store_table_names` shared through the crate-root facade.
pub(crate) use self::postgres::validate_decode_store_table_names;
/// Crate-internal storage symbol `validate_raw_store_table_names` shared through the crate-root facade.
pub(crate) use self::postgres::validate_raw_store_table_names;
/// Transitional logical schema contract identifier used until the `0.5.3-pre.3` rebuild.
pub use self::constants::STORE_SCHEMA_CONTRACT_VERSION;
/// Core account key insert contract.
pub use self::contracts::CoreAccountKeyInsert;
/// Core account key SQL-like row contract.
/// Core account key persisted row contract.
pub use self::contracts::CoreAccountKeyRow;
/// Core account key source category.
pub use self::contracts::CoreAccountKeySource;
@@ -23,19 +187,19 @@ pub use self::contracts::CoreAccountKeySource;
pub use self::contracts::CoreBalanceChangeInsert;
/// Core balance change kind.
pub use self::contracts::CoreBalanceChangeKind;
/// Core balance change SQL-like row contract.
/// Core balance change persisted row contract.
pub use self::contracts::CoreBalanceChangeRow;
/// Complete normalized core extraction write bundle.
/// Complete normalized Core extraction write bundle.
pub use self::contracts::CoreExtractionBundle;
/// Failure details persisted for one canonical to core extraction attempt.
/// Failure details persisted for one canonical-to-Core extraction attempt.
pub use self::contracts::CoreExtractionFailure;
/// Bounded canonical transaction selection filter for core extraction.
/// Bounded canonical transaction selection filter for Core extraction.
pub use self::contracts::CoreExtractionSelectionFilter;
/// Canonical transaction to core extraction storage behavior.
/// Canonical transaction-to-Core extraction storage behavior.
pub use self::contracts::CoreExtractionStore;
/// Core inner instruction insert contract.
pub use self::contracts::CoreInnerInstructionInsert;
/// Core inner instruction SQL-like row contract.
/// Core inner instruction persisted row contract.
pub use self::contracts::CoreInnerInstructionRow;
/// Core instruction insert contract.
pub use self::contracts::CoreInstructionInsert;
@@ -45,15 +209,15 @@ pub use self::contracts::CoreInstructionLifecycleMark;
pub use self::contracts::CoreInstructionProcessingState;
/// Core instruction replay filter contract.
pub use self::contracts::CoreInstructionReplayFilter;
/// Core instruction SQL-like row contract.
/// Core instruction persisted row contract.
pub use self::contracts::CoreInstructionRow;
/// Core log insert contract.
pub use self::contracts::CoreLogInsert;
/// Core log SQL-like row contract.
/// Core log persisted row contract.
pub use self::contracts::CoreLogRow;
/// Core transaction insert contract.
pub use self::contracts::CoreTransactionInsert;
/// Core transaction SQL-like row contract.
/// Core transaction persisted row contract.
pub use self::contracts::CoreTransactionRow;
/// Core Solana storage behavior.
pub use self::contracts::CoreTransactionStore;
@@ -65,7 +229,7 @@ pub use self::contracts::DecodeCoverageDeclarationInsert;
pub use self::contracts::DecodeCoverageObservationInsert;
/// One row of aggregated decoder coverage diagnostics.
pub use self::contracts::DecodeCoverageSummaryRow;
/// Failed decode attempt persisted in the common ledger.
/// Failed decode attempt persisted in the common processing ledger.
pub use self::contracts::DecodeFailure;
/// One processor-owned decoded observation row.
pub use self::contracts::DecodeObservationInsert;
@@ -75,46 +239,28 @@ pub use self::contracts::DecodePersistenceBundle;
pub use self::contracts::DecodePipelineStore;
/// Bounded contextual instruction selection filter for decode campaigns.
pub use self::contracts::DecodeSelectionFilter;
/// Decoded event insert contract.
pub use self::contracts::DecodedEventInsert;
/// Decoded event SQL-like row contract.
pub use self::contracts::DecodedEventRow;
/// Decoded event storage behavior.
pub use self::contracts::DecodedEventStore;
/// Insert or upsert result contract returned by repositories.
pub use self::contracts::InsertOutcome;
/// Maximum number of materialized rows returned by one bounded query.
pub use self::contracts::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
/// Maximum repository page size.
pub use self::contracts::MAX_PAGE_SIZE;
/// Maximum number of rows returned by one replay candidate query.
pub use self::contracts::MAX_REPLAY_CANDIDATE_ROWS;
/// Atomic persistence bundle for one materializer and one decoded observation.
pub use self::contracts::MaterializationPersistenceBundle;
/// Bounded read-only materialized event selection.
pub use self::contracts::MaterializedEventFilter;
/// Materialized event insert contract.
pub use self::contracts::MaterializedEventInsert;
/// One materialized output returned by a bounded query.
pub use self::contracts::MaterializedEventQueryRow;
/// Materialized event SQL-like row contract.
pub use self::contracts::MaterializedEventRow;
/// Materialized event storage behavior.
pub use self::contracts::MaterializedEventStore;
/// One processor-owned materialized output row.
/// One processor-owned generic materialized output row.
pub use self::contracts::MaterializedOutputInsert;
/// Page request contract for repository list operations.
pub use self::contracts::PageRequest;
/// Stable processing ledger identity.
pub use self::contracts::ProcessingLedgerIdentity;
/// Processing ledger mark request contract.
pub use self::contracts::ProcessingLedgerMark;
/// Processing ledger SQL-like row contract.
pub use self::contracts::ProcessingLedgerRow;
/// Stable processing ledger status.
pub use self::contracts::ProcessingLedgerStatus;
/// Processing ledger storage behavior.
pub use self::contracts::ProcessingLedgerStore;
/// Program observation storage behavior.
pub use self::contracts::ProgramObservationStore;
/// Raw payload lifecycle mark request.
pub use self::contracts::RawPayloadLifecycleMark;
/// Raw payload processing state.
@@ -123,135 +269,75 @@ pub use self::contracts::RawPayloadProcessingState;
pub use self::contracts::RawPayloadRetentionState;
/// Canonical raw Solana transaction insert contract.
pub use self::contracts::RawTransactionInsert;
/// Canonical raw Solana transaction SQL-like row contract.
/// Canonical raw Solana transaction persisted row contract.
pub use self::contracts::RawTransactionRow;
/// Raw transaction storage behavior.
pub use self::contracts::RawTransactionStore;
/// Bounded read-only filter for Core entity summaries.
pub use self::contracts::ReplayEntityFilter;
/// Core entity kind used while filtering replay candidates.
pub use self::contracts::ReplayEntityKind;
/// Aggregated mint, owner, or account-key occurrences from Core facts.
pub use self::contracts::ReplayEntitySummary;
/// Bounded read-only filter for program summaries.
pub use self::contracts::ReplayProgramFilter;
/// Program occurrence scope used while filtering replay candidates.
pub use self::contracts::ReplayProgramScope;
/// Aggregated program occurrences across top-level instructions, inner instructions, and logs.
pub use self::contracts::ReplayProgramSummary;
/// One raw transaction candidate enriched with Core and processing diagnostics.
pub use self::contracts::ReplayTransactionCandidate;
/// Bounded read-only filter for transaction replay candidates.
pub use self::contracts::ReplayTransactionFilter;
/// Sort direction for repository list operations.
pub use self::contracts::SortDirection;
/// Store backend diagnostic contract.
/// Store backend diagnostic descriptor.
pub use self::contracts::StoreBackendDescriptor;
/// Store backend kind contract.
pub use self::contracts::StoreBackendKind;
/// Backend-neutral diagnostic snapshot.
pub use self::contracts::StoreBackendDiagnostics;
/// Store configuration summary safe for UI and logs.
pub use self::contracts::StoreConfigurationSummary;
/// Store backend health snapshot.
pub use self::contracts::StoreHealthSnapshot;
/// Store backend health status.
pub use self::contracts::StoreHealthStatus;
/// Store health storage behavior.
pub use self::contracts::StoreHealthStore;
/// Store initialization status.
pub use self::contracts::StoreInitializationStatus;
/// Initialization and model-verification summary captured while opening a store.
pub use self::contracts::StoreInitializationSummary;
/// Store migration diagnostic snapshot contract.
pub use self::contracts::StoreMigrationSnapshot;
/// Store migration status contract.
pub use self::contracts::StoreMigrationStatus;
/// Store model verification status.
pub use self::contracts::StoreModelVerificationStatus;
/// Verification summary for one logical store model.
pub use self::contracts::StoreModelVerificationSummary;
/// Count summary for one backend object category without exposing physical names.
pub use self::contracts::StoreObjectVerificationSummary;
/// Read-only diagnostics for one logical store resource.
pub use self::contracts::StoreResourceDiagnostics;
/// Read-only statistics for one logical store resource.
pub use self::contracts::StoreResourceStatistics;
/// Complete runtime summary safe for application diagnostics.
pub use self::contracts::StoreRuntimeSummary;
/// Transaction acquisition observation insert contract.
pub use self::contracts::TransactionObservationInsert;
/// Transaction acquisition observation origin.
pub use self::contracts::TransactionObservationOrigin;
/// Transaction acquisition observation SQL-like row contract.
/// Transaction acquisition observation persisted row contract.
pub use self::contracts::TransactionObservationRow;
/// Transaction acquisition observation status.
pub use self::contracts::TransactionObservationStatus;
/// Storage error helper functions.
pub use self::contracts::storage_contract_error;
/// Allowed Solana table domains encoded in table names.
pub use self::postgres::ALLOWED_SOLANA_TABLE_DOMAINS;
/// Core account key table name.
pub use self::postgres::CORE_ACCOUNT_KEYS_TABLE_NAME;
/// Core balance change table name.
pub use self::postgres::CORE_BALANCE_CHANGES_TABLE_NAME;
/// Core inner instruction table name.
pub use self::postgres::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
/// Core instruction table name.
pub use self::postgres::CORE_INSTRUCTIONS_TABLE_NAME;
/// Core log table name.
pub use self::postgres::CORE_LOGS_TABLE_NAME;
/// Core Solana table names.
pub use self::postgres::CORE_STORE_TABLE_NAMES;
/// Core transaction table name.
pub use self::postgres::CORE_TRANSACTIONS_TABLE_NAME;
/// Machine-readable decoder coverage declaration table name.
pub use self::postgres::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
/// Observed decoder coverage classification table name.
pub use self::postgres::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
/// Versioned decoded event table name.
pub use self::postgres::DECODE_EVENTS_TABLE_NAME;
/// Decode and materialization table names.
pub use self::postgres::DECODE_STORE_TABLE_NAMES;
/// Default PostgreSQL schema policy.
pub use self::postgres::DEFAULT_SCHEMA_POLICY;
/// Versioned materialized output table name.
pub use self::postgres::MATERIALIZED_EVENTS_TABLE_NAME;
/// Maximum number of rows returned by one replay candidate query.
pub use self::postgres::MAX_REPLAY_CANDIDATE_ROWS;
/// PostgreSQL migration strategy used by this crate.
pub use self::postgres::MIGRATION_STRATEGY;
/// Migration table name used by sqlx.
pub use self::postgres::MIGRATION_TABLE_NAME;
/// Processing ledger table name.
pub use self::postgres::PROCESSING_LEDGER_TABLE_NAME;
/// PostgreSQL diagnostic snapshot.
pub use self::postgres::PostgresBackendDiagnostics;
/// Bounded read-only filter for core entity summaries.
pub use self::postgres::PostgresReplayEntityFilter;
/// Core entity kind used while filtering replay candidates.
pub use self::postgres::PostgresReplayEntityKind;
/// Aggregated mint, owner or account-key occurrences from core tables.
pub use self::postgres::PostgresReplayEntitySummary;
/// Bounded read-only filter for program summaries.
pub use self::postgres::PostgresReplayProgramFilter;
/// Program occurrence scope used while filtering replay candidates.
pub use self::postgres::PostgresReplayProgramScope;
/// Aggregated program occurrences across outer, inner and linked logs.
pub use self::postgres::PostgresReplayProgramSummary;
/// One raw transaction candidate enriched with core and ledger diagnostics.
pub use self::postgres::PostgresReplayTransactionCandidate;
/// Bounded read-only filter for transaction replay candidates.
pub use self::postgres::PostgresReplayTransactionFilter;
/// PostgreSQL store handle.
pub use self::postgres::PostgresStore;
/// PostgreSQL store connection options.
pub use self::postgres::PostgresStoreOptions;
/// PostgreSQL diagnostic table specification.
pub use self::postgres::PostgresTableDiagnosticSpec;
/// PostgreSQL table diagnostic snapshot.
pub use self::postgres::PostgresTableDiagnostics;
/// PostgreSQL table statistics snapshot.
pub use self::postgres::PostgresTableStatistics;
/// Canonical transaction acquisition table names.
pub use self::postgres::RAW_STORE_TABLE_NAMES;
/// Canonical raw transaction table name.
pub use self::postgres::RAW_TRANSACTIONS_TABLE_NAME;
/// Solana application table prefix.
pub use self::postgres::SOLANA_TABLE_PREFIX;
/// Advisory lock id used while initializing store tables.
pub use self::postgres::STORE_SCHEMA_ADVISORY_LOCK_ID;
/// Transaction acquisition observation table name.
pub use self::postgres::TRANSACTION_OBSERVATIONS_TABLE_NAME;
/// Core store SQL statements applied by the idempotent initializer.
pub use self::postgres::core_store_schema_statements;
/// Core table diagnostic specs.
pub use self::postgres::core_store_table_diagnostic_specs;
/// Decode and materialization SQL statements applied by the idempotent initializer.
pub use self::postgres::decode_store_schema_statements;
/// Decode and materialization table diagnostic specs.
pub use self::postgres::decode_store_table_diagnostic_specs;
/// Returns true when a Solana table name follows canonical rules.
pub use self::postgres::is_valid_solana_table_name;
/// Returns a DSN masked for logs and UI diagnostics.
pub use self::postgres::mask_postgres_dsn;
/// Canonical transaction acquisition SQL statements.
pub use self::postgres::raw_store_schema_statements;
/// Canonical transaction acquisition diagnostic specs.
pub use self::postgres::raw_store_table_diagnostic_specs;
/// Validates core store table names.
pub use self::postgres::validate_core_store_table_names;
/// Validates decode and materialization store table names.
pub use self::postgres::validate_decode_store_table_names;
/// Validates canonical transaction acquisition table names.
pub use self::postgres::validate_raw_store_table_names;
/// Validates a canonical Solana table name.
pub use self::postgres::validate_solana_table_name;
/// Current normalized core replay input contract version.
/// Backend-agnostic persistent store facade.
pub use self::store::Store;
/// Backend-agnostic options used to open one persistent store.
pub use self::store::StoreOpenOptions;
/// Current normalized Core replay input contract version.
pub use ks_lib::MD_CORE_REPLAY_INPUT_CONTRACT_VERSION;
/// Decoder replay input containing one instruction plus extracted transaction context.
pub use ks_lib::MdCoreInstructionReplayInput;

View File

@@ -1,111 +1,94 @@
// file: ks-store/src/postgres.rs
// version: 4
// version: 11
//! PostgreSQL storage implementation boundary.
//! Private PostgreSQL implementation of the backend-agnostic store contracts.
mod migrations;
mod query;
mod replay_candidates;
mod repository;
mod store;
#[cfg(test)]
mod test_serial;
/// Allowed Solana table domains encoded in table names.
pub use self::migrations::ALLOWED_SOLANA_TABLE_DOMAINS;
/// Core account key table name.
pub use self::migrations::CORE_ACCOUNT_KEYS_TABLE_NAME;
/// Core balance change table name.
pub use self::migrations::CORE_BALANCE_CHANGES_TABLE_NAME;
/// Core inner instruction table name.
pub use self::migrations::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
/// Core instruction table name.
pub use self::migrations::CORE_INSTRUCTIONS_TABLE_NAME;
/// Core log table name.
pub use self::migrations::CORE_LOGS_TABLE_NAME;
/// Core Solana table names introduced by `0.2.4`.
pub use self::migrations::CORE_STORE_TABLE_NAMES;
/// Core transaction table name.
pub use self::migrations::CORE_TRANSACTIONS_TABLE_NAME;
/// Machine-readable decoder coverage declaration table name.
pub use self::migrations::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
/// Observed decoder coverage classification table name.
pub use self::migrations::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
/// Versioned decoded event table name.
pub use self::migrations::DECODE_EVENTS_TABLE_NAME;
/// Decode and materialization table names introduced by `0.4.0`.
pub use self::migrations::DECODE_STORE_TABLE_NAMES;
/// Default PostgreSQL schema policy.
pub use self::migrations::DEFAULT_SCHEMA_POLICY;
/// Versioned materialized output table name.
pub use self::migrations::MATERIALIZED_EVENTS_TABLE_NAME;
/// PostgreSQL migration strategy used by this crate.
pub use self::migrations::MIGRATION_STRATEGY;
/// Migration table name used by sqlx when migrations are enabled later.
pub use self::migrations::MIGRATION_TABLE_NAME;
/// Processing ledger table name.
pub use self::migrations::PROCESSING_LEDGER_TABLE_NAME;
/// Table diagnostic metadata.
pub use self::migrations::PostgresTableDiagnosticSpec;
/// Canonical transaction acquisition table names active since `0.3.1`.
pub use self::migrations::RAW_STORE_TABLE_NAMES;
/// Canonical raw transaction table name.
pub use self::migrations::RAW_TRANSACTIONS_TABLE_NAME;
/// Solana application table prefix.
pub use self::migrations::SOLANA_TABLE_PREFIX;
/// Advisory lock id used while initializing store schemas.
pub use self::migrations::STORE_SCHEMA_ADVISORY_LOCK_ID;
/// Transaction acquisition observation table name.
pub use self::migrations::TRANSACTION_OBSERVATIONS_TABLE_NAME;
/// Core store SQL statements applied by the idempotent initializer.
pub use self::migrations::core_store_schema_statements;
/// Core table diagnostic specs.
pub use self::migrations::core_store_table_diagnostic_specs;
/// Decode and materialization SQL statements applied by the idempotent initializer.
pub use self::migrations::decode_store_schema_statements;
/// Decode and materialization table diagnostic specs.
pub use self::migrations::decode_store_table_diagnostic_specs;
/// Returns true when a Solana table name follows the canonical prefix and domain rules.
pub use self::migrations::is_valid_solana_table_name;
/// Canonical transaction acquisition SQL statements applied by the initializer.
pub use self::migrations::raw_store_schema_statements;
/// Canonical transaction acquisition diagnostic specs.
pub use self::migrations::raw_store_table_diagnostic_specs;
/// Validates core store table names.
pub use self::migrations::validate_core_store_table_names;
/// Validates decode and materialization store table names.
pub use self::migrations::validate_decode_store_table_names;
/// Validates canonical transaction acquisition table names.
pub use self::migrations::validate_raw_store_table_names;
/// Validates a canonical Solana table name.
pub use self::migrations::validate_solana_table_name;
/// Maximum number of rows returned by one replay candidate query.
pub use self::replay_candidates::MAX_REPLAY_CANDIDATE_ROWS;
/// Bounded read-only filter for core entity summaries.
pub use self::replay_candidates::PostgresReplayEntityFilter;
/// Core entity kind used while filtering replay candidates.
pub use self::replay_candidates::PostgresReplayEntityKind;
/// Aggregated mint, owner or account-key occurrences from core tables.
pub use self::replay_candidates::PostgresReplayEntitySummary;
/// Bounded read-only filter for program summaries.
pub use self::replay_candidates::PostgresReplayProgramFilter;
/// Program occurrence scope used while filtering replay candidates.
pub use self::replay_candidates::PostgresReplayProgramScope;
/// Aggregated program occurrences across outer, inner and linked logs.
pub use self::replay_candidates::PostgresReplayProgramSummary;
/// One raw transaction candidate enriched with core and ledger diagnostics.
pub use self::replay_candidates::PostgresReplayTransactionCandidate;
/// Bounded read-only filter for transaction replay candidates.
pub use self::replay_candidates::PostgresReplayTransactionFilter;
/// PostgreSQL diagnostic snapshot.
pub use self::store::PostgresBackendDiagnostics;
/// Minimal PostgreSQL store handle.
pub use self::store::PostgresStore;
/// PostgreSQL store connection options.
pub use self::store::PostgresStoreOptions;
/// PostgreSQL table diagnostic snapshot.
pub use self::store::PostgresTableDiagnostics;
/// PostgreSQL table statistics snapshot.
pub use self::store::PostgresTableStatistics;
/// Returns a DSN masked for logs and UI diagnostics.
pub use self::store::mask_postgres_dsn;
pub(crate) use self::migrations::CORE_ACCOUNT_KEYS_TABLE_NAME;
pub(crate) use self::migrations::CORE_BALANCE_CHANGES_TABLE_NAME;
pub(crate) use self::migrations::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
pub(crate) use self::migrations::CORE_INSTRUCTIONS_TABLE_NAME;
pub(crate) use self::migrations::CORE_LOGS_TABLE_NAME;
pub(crate) use self::migrations::CORE_TRANSACTIONS_TABLE_NAME;
pub(crate) use self::migrations::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
pub(crate) use self::migrations::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
pub(crate) use self::migrations::DECODE_EVENTS_TABLE_NAME;
pub(crate) use self::migrations::MATERIALIZED_EVENTS_TABLE_NAME;
pub(crate) use self::migrations::PROCESSING_LEDGER_TABLE_NAME;
pub(crate) use self::migrations::PostgresTableDiagnosticSpec;
pub(crate) use self::migrations::RAW_TRANSACTIONS_TABLE_NAME;
pub(crate) use self::migrations::STORE_SCHEMA_ADVISORY_LOCK_ID;
pub(crate) use self::migrations::TRANSACTION_OBSERVATIONS_TABLE_NAME;
pub(crate) use self::migrations::core_store_schema_statements;
pub(crate) use self::migrations::core_store_table_diagnostic_specs;
pub(crate) use self::migrations::decode_store_schema_statements;
pub(crate) use self::migrations::decode_store_table_diagnostic_specs;
pub(crate) use self::migrations::raw_store_schema_statements;
pub(crate) use self::migrations::raw_store_table_diagnostic_specs;
pub(crate) use self::migrations::table_stats_kb_sol_core_account_keys_sql;
pub(crate) use self::migrations::table_stats_kb_sol_core_balance_changes_sql;
pub(crate) use self::migrations::table_stats_kb_sol_core_inner_instructions_sql;
pub(crate) use self::migrations::table_stats_kb_sol_core_instructions_sql;
pub(crate) use self::migrations::table_stats_kb_sol_core_logs_sql;
pub(crate) use self::migrations::table_stats_kb_sol_core_transactions_sql;
pub(crate) use self::migrations::table_stats_kb_sol_decode_coverage_declarations_sql;
pub(crate) use self::migrations::table_stats_kb_sol_decode_coverage_observations_sql;
pub(crate) use self::migrations::table_stats_kb_sol_decode_events_sql;
pub(crate) use self::migrations::table_stats_kb_sol_mat_events_sql;
pub(crate) use self::migrations::table_stats_kb_sol_obs_transaction_observations_sql;
pub(crate) use self::migrations::table_stats_kb_sol_ops_processing_ledger_sql;
pub(crate) use self::migrations::table_stats_kb_sol_raw_transactions_sql;
pub(crate) use self::migrations::validate_core_store_table_names;
pub(crate) use self::migrations::validate_decode_store_table_names;
pub(crate) use self::migrations::validate_raw_store_table_names;
pub(crate) use self::query::apply_core_store_schema;
pub(crate) use self::query::apply_decode_store_schema;
pub(crate) use self::query::apply_raw_store_schema;
pub(crate) use self::query::has_raw_transaction_signature;
pub(crate) use self::query::has_transaction_observation_key;
pub(crate) use self::query::insert_core_account_keys;
pub(crate) use self::query::insert_core_balance_changes;
pub(crate) use self::query::insert_core_inner_instructions;
pub(crate) use self::query::insert_core_instructions;
pub(crate) use self::query::insert_core_logs;
pub(crate) use self::query::insert_core_transaction;
pub(crate) use self::query::insert_raw_transaction;
pub(crate) use self::query::insert_transaction_observation;
pub(crate) use self::query::is_core_extraction_current;
pub(crate) use self::query::is_decode_current;
pub(crate) use self::query::list_core_instruction_replay_inputs;
pub(crate) use self::query::list_core_instructions_for_replay;
pub(crate) use self::query::list_decode_coverage_summary;
pub(crate) use self::query::list_decode_inputs;
pub(crate) use self::query::list_decode_replay_inputs;
pub(crate) use self::query::list_materialized_events;
pub(crate) use self::query::list_raw_transactions_for_core_extraction;
pub(crate) use self::query::list_replay_entity_summaries;
pub(crate) use self::query::list_replay_program_summaries;
pub(crate) use self::query::list_replay_transaction_candidates;
pub(crate) use self::query::load_current_schema;
pub(crate) use self::query::load_latest_migration_version;
pub(crate) use self::query::load_migration_table_name;
pub(crate) use self::query::load_server_version;
pub(crate) use self::query::load_table_statistics;
pub(crate) use self::query::mark_core_extraction_failed;
pub(crate) use self::query::mark_decode_failed;
pub(crate) use self::query::persist_core_extraction;
pub(crate) use self::query::persist_decode_coverage_declarations;
pub(crate) use self::query::persist_decode_result;
pub(crate) use self::query::persist_materialization_result;
pub(crate) use self::query::run_health_check;
pub(crate) use self::query::table_exists;
pub(crate) use self::query::update_core_instruction_lifecycle;
pub(crate) use self::query::update_raw_payload_lifecycle;
pub(crate) use self::store::PostgresStore;
pub(crate) use self::store::PostgresStoreOptions;
#[cfg(test)]
pub(crate) use self::test_serial::postgres_test_guard;

View File

@@ -1,165 +1,175 @@
// file: ks-store/src/postgres/migrations.rs
// version: 3
// version: 7
//! PostgreSQL migration conventions for the storage backend.
/// PostgreSQL schema policy used by application migrations.
pub const DEFAULT_SCHEMA_POLICY: &str = "current_profile_schema";
/// Canonical Solana table prefix.
pub const SOLANA_TABLE_PREFIX: &str = "kb_sol_";
/// Migration table name used by sqlx when migrations are enabled later.
pub const MIGRATION_TABLE_NAME: &str = "_sqlx_migrations";
/// Migration strategy used by the canonical transaction and core stores.
pub const MIGRATION_STRATEGY: &str =
"idempotent_canonical_transaction_observation_and_core_tables_without_application_schemas";
const SOLANA_TABLE_PREFIX: &str = "kb_sol_";
/// Advisory lock id used while applying idempotent store schema statements.
pub const STORE_SCHEMA_ADVISORY_LOCK_ID: i64 = 2_024_000_204;
pub(crate) const STORE_SCHEMA_ADVISORY_LOCK_ID: i64 = 2_024_000_204;
/// Canonical raw transaction table name.
pub const RAW_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_raw_transactions";
pub(crate) const RAW_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_raw_transactions";
/// Lightweight transaction acquisition observation table name.
pub const TRANSACTION_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_obs_transaction_observations";
pub(crate) const TRANSACTION_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_obs_transaction_observations";
/// Core transaction table name.
pub const CORE_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_core_transactions";
pub(crate) const CORE_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_core_transactions";
/// Core account key table name.
pub const CORE_ACCOUNT_KEYS_TABLE_NAME: &str = "kb_sol_core_account_keys";
pub(crate) const CORE_ACCOUNT_KEYS_TABLE_NAME: &str = "kb_sol_core_account_keys";
/// Core instruction table name.
pub const CORE_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_instructions";
pub(crate) const CORE_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_instructions";
/// Core inner instruction table name.
pub const CORE_INNER_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_inner_instructions";
pub(crate) const CORE_INNER_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_inner_instructions";
/// Core log table name.
pub const CORE_LOGS_TABLE_NAME: &str = "kb_sol_core_logs";
pub(crate) const CORE_LOGS_TABLE_NAME: &str = "kb_sol_core_logs";
/// Core balance change table name.
pub const CORE_BALANCE_CHANGES_TABLE_NAME: &str = "kb_sol_core_balance_changes";
pub(crate) const CORE_BALANCE_CHANGES_TABLE_NAME: &str = "kb_sol_core_balance_changes";
/// Processing ledger table name.
pub const PROCESSING_LEDGER_TABLE_NAME: &str = "kb_sol_ops_processing_ledger";
pub(crate) const PROCESSING_LEDGER_TABLE_NAME: &str = "kb_sol_ops_processing_ledger";
/// Versioned decoded event table name.
pub const DECODE_EVENTS_TABLE_NAME: &str = "kb_sol_decode_events";
pub(crate) const DECODE_EVENTS_TABLE_NAME: &str = "kb_sol_decode_events";
/// Machine-readable decoder coverage declaration table name.
pub const DECODE_COVERAGE_DECLARATIONS_TABLE_NAME: &str = "kb_sol_decode_coverage_declarations";
pub(crate) const DECODE_COVERAGE_DECLARATIONS_TABLE_NAME: &str =
"kb_sol_decode_coverage_declarations";
/// Observed decoder coverage classification table name.
pub const DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_decode_coverage_observations";
pub(crate) const DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME: &str =
"kb_sol_decode_coverage_observations";
/// Versioned materialized output table name.
pub const MATERIALIZED_EVENTS_TABLE_NAME: &str = "kb_sol_mat_events";
pub(crate) const MATERIALIZED_EVENTS_TABLE_NAME: &str = "kb_sol_mat_events";
/// Canonical transaction acquisition table names active since `0.3.1`.
pub const RAW_STORE_TABLE_NAMES: &[&str] =
&[crate::RAW_TRANSACTIONS_TABLE_NAME, crate::TRANSACTION_OBSERVATIONS_TABLE_NAME];
const RAW_STORE_TABLE_NAMES: &[&str] =
&[RAW_TRANSACTIONS_TABLE_NAME, TRANSACTION_OBSERVATIONS_TABLE_NAME];
/// Core Solana table names introduced by `0.2.4`.
pub const CORE_STORE_TABLE_NAMES: &[&str] = &[
crate::CORE_TRANSACTIONS_TABLE_NAME,
crate::CORE_ACCOUNT_KEYS_TABLE_NAME,
crate::CORE_INSTRUCTIONS_TABLE_NAME,
crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME,
crate::CORE_LOGS_TABLE_NAME,
crate::CORE_BALANCE_CHANGES_TABLE_NAME,
crate::PROCESSING_LEDGER_TABLE_NAME,
const CORE_STORE_TABLE_NAMES: &[&str] = &[
CORE_TRANSACTIONS_TABLE_NAME,
CORE_ACCOUNT_KEYS_TABLE_NAME,
CORE_INSTRUCTIONS_TABLE_NAME,
CORE_INNER_INSTRUCTIONS_TABLE_NAME,
CORE_LOGS_TABLE_NAME,
CORE_BALANCE_CHANGES_TABLE_NAME,
PROCESSING_LEDGER_TABLE_NAME,
];
/// Decode and materialization table names introduced by `0.4.0`.
pub const DECODE_STORE_TABLE_NAMES: &[&str] = &[
crate::DECODE_EVENTS_TABLE_NAME,
crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
crate::MATERIALIZED_EVENTS_TABLE_NAME,
const DECODE_STORE_TABLE_NAMES: &[&str] = &[
DECODE_EVENTS_TABLE_NAME,
DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
MATERIALIZED_EVENTS_TABLE_NAME,
];
/// Allowed Solana table domains encoded after the `kb_sol_` prefix.
pub const ALLOWED_SOLANA_TABLE_DOMAINS: &[&str] =
const ALLOWED_SOLANA_TABLE_DOMAINS: &[&str] =
&["raw", "core", "obs", "decode", "mat", "catalog", "agg", "ops", "wallet"];
/// Diagnostic metadata for one expected PostgreSQL table.
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
pub struct PostgresTableDiagnosticSpec {
/// Expected table name.
pub(crate) struct PostgresTableDiagnosticSpec {
/// Physical PostgreSQL table name kept backend-private.
pub table_name: &'static str,
/// Logical Solana domain encoded in the table name.
pub domain: &'static str,
/// Human-readable table role.
/// Stable backend-independent logical resource code.
pub resource_code: &'static str,
/// Logical store model code.
pub model_code: &'static str,
/// Human-readable resource role.
pub role: &'static str,
}
/// Diagnostic metadata for canonical transaction acquisition tables.
pub fn raw_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 2] {
pub(crate) fn raw_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 2] {
return [
crate::PostgresTableDiagnosticSpec {
table_name: crate::RAW_TRANSACTIONS_TABLE_NAME,
domain: "raw",
table_name: RAW_TRANSACTIONS_TABLE_NAME,
resource_code: "raw_transactions",
model_code: "raw",
role: "Canonical transactions",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::TRANSACTION_OBSERVATIONS_TABLE_NAME,
domain: "obs",
table_name: TRANSACTION_OBSERVATIONS_TABLE_NAME,
resource_code: "transaction_observations",
model_code: "raw",
role: "Transaction acquisition observations",
},
];
}
/// Diagnostic metadata for core Solana store tables.
pub fn core_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 7] {
pub(crate) fn core_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 7] {
return [
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_TRANSACTIONS_TABLE_NAME,
domain: "core",
table_name: CORE_TRANSACTIONS_TABLE_NAME,
resource_code: "transactions",
model_code: "core",
role: "Transactions",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_ACCOUNT_KEYS_TABLE_NAME,
domain: "core",
table_name: CORE_ACCOUNT_KEYS_TABLE_NAME,
resource_code: "account_keys",
model_code: "core",
role: "Resolved account keys",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_INSTRUCTIONS_TABLE_NAME,
domain: "core",
table_name: CORE_INSTRUCTIONS_TABLE_NAME,
resource_code: "top_level_instructions",
model_code: "core",
role: "Top-level instructions",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME,
domain: "core",
table_name: CORE_INNER_INSTRUCTIONS_TABLE_NAME,
resource_code: "inner_instructions",
model_code: "core",
role: "Inner instruction tree",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_LOGS_TABLE_NAME,
domain: "core",
table_name: CORE_LOGS_TABLE_NAME,
resource_code: "logs",
model_code: "core",
role: "Transaction logs",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::CORE_BALANCE_CHANGES_TABLE_NAME,
domain: "core",
table_name: CORE_BALANCE_CHANGES_TABLE_NAME,
resource_code: "balance_changes",
model_code: "core",
role: "Balance changes",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::PROCESSING_LEDGER_TABLE_NAME,
domain: "ops",
table_name: PROCESSING_LEDGER_TABLE_NAME,
resource_code: "processing_ledger",
model_code: "processing",
role: "Processing ledger",
},
];
}
/// Diagnostic metadata for decode and materialization store tables.
pub fn decode_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 4] {
pub(crate) fn decode_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 4] {
return [
crate::PostgresTableDiagnosticSpec {
table_name: crate::DECODE_EVENTS_TABLE_NAME,
domain: "decode",
table_name: DECODE_EVENTS_TABLE_NAME,
resource_code: "decoded_observations",
model_code: "decode",
role: "Versioned decoded observations",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
domain: "decode",
table_name: DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
resource_code: "decode_coverage_declarations",
model_code: "decode",
role: "Declared decoder coverage",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
domain: "decode",
table_name: DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
resource_code: "decode_coverage_observations",
model_code: "decode",
role: "Observed decoder coverage",
},
crate::PostgresTableDiagnosticSpec {
table_name: crate::MATERIALIZED_EVENTS_TABLE_NAME,
domain: "mat",
table_name: MATERIALIZED_EVENTS_TABLE_NAME,
resource_code: "materialized_outputs",
model_code: "materialization",
role: "Versioned materialized outputs",
},
];
}
/// Idempotent SQL statements creating or upgrading the canonical transaction acquisition store.
pub fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
pub(crate) fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
return vec![
crate::postgres::migrations::migrate_legacy_raw_table_name_sql(),
crate::postgres::migrations::migrate_legacy_raw_columns_sql(),
@@ -185,7 +195,7 @@ pub fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
}
/// Idempotent SQL statements creating the minimal core Solana store.
pub fn core_store_schema_statements() -> [&'static str; 28] {
pub(crate) fn core_store_schema_statements() -> [&'static str; 28] {
return [
crate::postgres::migrations::create_table_kb_sol_core_transactions_sql(),
crate::postgres::migrations::create_ux_kb_sol_core_transactions_signature_sql(),
@@ -219,7 +229,7 @@ pub fn core_store_schema_statements() -> [&'static str; 28] {
}
/// Idempotent SQL statements creating the common decode and materialization store.
pub fn decode_store_schema_statements() -> [&'static str; 15] {
pub(crate) fn decode_store_schema_statements() -> [&'static str; 15] {
return [
crate::postgres::migrations::create_table_kb_sol_decode_events_sql(),
crate::postgres::migrations::create_ux_kb_sol_decode_events_identity_sql(),
@@ -240,7 +250,7 @@ pub fn decode_store_schema_statements() -> [&'static str; 15] {
}
/// SQL conditionally renaming the historical raw RPC transaction table.
pub(in crate::postgres) fn migrate_legacy_raw_table_name_sql() -> &'static str {
fn migrate_legacy_raw_table_name_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('kb_sol_raw_rpc_transactions') IS NOT NULL
@@ -255,7 +265,7 @@ $$"#;
}
/// SQL conditionally renaming historical raw payload columns and adding the format version.
pub(in crate::postgres) fn migrate_legacy_raw_columns_sql() -> &'static str {
fn migrate_legacy_raw_columns_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('kb_sol_raw_transactions') IS NOT NULL THEN
@@ -274,7 +284,7 @@ $$"#;
}
/// SQL conditionally renaming historical canonical transaction constraints.
pub(in crate::postgres) fn migrate_legacy_raw_constraints_sql() -> &'static str {
fn migrate_legacy_raw_constraints_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('kb_sol_raw_transactions') IS NOT NULL THEN
@@ -291,7 +301,7 @@ $$"#;
}
/// SQL conditionally renaming historical canonical transaction indexes.
pub(in crate::postgres) fn migrate_legacy_raw_indexes_sql() -> &'static str {
fn migrate_legacy_raw_indexes_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('ux_kb_sol_raw_rpc_transactions_signature') IS NOT NULL AND to_regclass('ux_kb_sol_raw_transactions_signature') IS NULL THEN ALTER INDEX ux_kb_sol_raw_rpc_transactions_signature RENAME TO ux_kb_sol_raw_transactions_signature; END IF;
@@ -303,7 +313,7 @@ $$"#;
}
/// SQL conditionally renaming historical core-to-raw lineage.
pub(in crate::postgres) fn migrate_legacy_core_lineage_sql() -> &'static str {
fn migrate_legacy_core_lineage_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('kb_sol_core_transactions') IS NOT NULL THEN
@@ -320,7 +330,7 @@ $$"#;
}
/// SQL creating `kb_sol_raw_transactions`.
pub(in crate::postgres) fn create_table_kb_sol_raw_transactions_sql() -> &'static str {
fn create_table_kb_sol_raw_transactions_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_raw_transactions (
id BIGSERIAL,
signature TEXT NOT NULL,
@@ -344,32 +354,32 @@ pub(in crate::postgres) fn create_table_kb_sol_raw_transactions_sql() -> &'stati
}
/// SQL creating the unique signature index for canonical raw transactions.
pub(in crate::postgres) fn create_ux_kb_sol_raw_transactions_signature_sql() -> &'static str {
fn create_ux_kb_sol_raw_transactions_signature_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_raw_transactions_signature ON kb_sol_raw_transactions (signature)";
}
/// SQL creating the slot index for canonical raw transactions.
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_slot_sql() -> &'static str {
fn create_ix_kb_sol_raw_transactions_slot_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_slot ON kb_sol_raw_transactions (slot)";
}
/// SQL creating the created timestamp index for canonical raw transactions.
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_created_at_sql() -> &'static str {
fn create_ix_kb_sol_raw_transactions_created_at_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_created_at ON kb_sol_raw_transactions (created_at)";
}
/// SQL creating the processing state index for canonical raw transactions.
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_processing_sql() -> &'static str {
fn create_ix_kb_sol_raw_transactions_processing_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_processing ON kb_sol_raw_transactions (processing_state)";
}
/// SQL creating the optional canonical document hash index.
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_canonical_hash_sql() -> &'static str {
fn create_ix_kb_sol_raw_transactions_canonical_hash_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_canonical_hash ON kb_sol_raw_transactions (canonical_json_hash) WHERE canonical_json_hash IS NOT NULL";
}
/// SQL creating `kb_sol_obs_transaction_observations`.
pub(in crate::postgres) fn create_table_kb_sol_obs_transaction_observations_sql() -> &'static str {
fn create_table_kb_sol_obs_transaction_observations_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_obs_transaction_observations (
id BIGSERIAL,
raw_transaction_id BIGINT,
@@ -415,48 +425,42 @@ pub(in crate::postgres) fn create_table_kb_sol_obs_transaction_observations_sql(
}
/// SQL creating the unique observation key index.
pub(in crate::postgres) fn create_ux_kb_sol_obs_transaction_observations_key_sql() -> &'static str {
fn create_ux_kb_sol_obs_transaction_observations_key_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_obs_transaction_observations_key ON kb_sol_obs_transaction_observations (observation_key)";
}
/// SQL creating the optional signature index for transaction observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_signature_sql()
-> &'static str {
fn create_ix_kb_sol_obs_transaction_observations_signature_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_signature ON kb_sol_obs_transaction_observations (signature) WHERE signature IS NOT NULL";
}
/// SQL creating the optional slot index for transaction observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_slot_sql() -> &'static str
{
fn create_ix_kb_sol_obs_transaction_observations_slot_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_slot ON kb_sol_obs_transaction_observations (slot) WHERE slot IS NOT NULL";
}
/// SQL creating the provider and acquisition method index for transaction observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_provider_method_sql()
-> &'static str {
fn create_ix_kb_sol_obs_transaction_observations_provider_method_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_provider_method ON kb_sol_obs_transaction_observations (provider, acquisition_method)";
}
/// SQL creating the received timestamp index for transaction observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_received_at_sql()
-> &'static str {
fn create_ix_kb_sol_obs_transaction_observations_received_at_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_received_at ON kb_sol_obs_transaction_observations (received_at)";
}
/// SQL creating the optional canonical transaction lineage index for observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_raw_transaction_sql()
-> &'static str {
fn create_ix_kb_sol_obs_transaction_observations_raw_transaction_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_raw_transaction ON kb_sol_obs_transaction_observations (raw_transaction_id) WHERE raw_transaction_id IS NOT NULL";
}
/// SQL creating the status index for transaction observations.
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_status_sql() -> &'static str
{
fn create_ix_kb_sol_obs_transaction_observations_status_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_status ON kb_sol_obs_transaction_observations (status)";
}
/// SQL migrating lightweight metadata and dropping the historical WebSocket payload table.
pub(in crate::postgres) fn migrate_and_drop_legacy_ws_notifications_sql() -> &'static str {
fn migrate_and_drop_legacy_ws_notifications_sql() -> &'static str {
return r#"DO $$
BEGIN
IF to_regclass('kb_sol_raw_ws_notifications') IS NOT NULL THEN
@@ -471,7 +475,7 @@ $$"#;
}
/// SQL creating `kb_sol_core_transactions`.
pub(in crate::postgres) fn create_table_kb_sol_core_transactions_sql() -> &'static str {
fn create_table_kb_sol_core_transactions_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_transactions (
id BIGSERIAL,
raw_transaction_id BIGINT,
@@ -489,22 +493,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_transactions_sql() -> &'stat
}
/// SQL creating the unique signature index for core transactions.
pub(in crate::postgres) fn create_ux_kb_sol_core_transactions_signature_sql() -> &'static str {
fn create_ux_kb_sol_core_transactions_signature_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_transactions_signature ON kb_sol_core_transactions (signature)";
}
/// SQL creating the slot index for core transactions.
pub(in crate::postgres) fn create_ix_kb_sol_core_transactions_slot_sql() -> &'static str {
fn create_ix_kb_sol_core_transactions_slot_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_slot ON kb_sol_core_transactions (slot)";
}
/// SQL creating the created_at index for core transactions.
pub(in crate::postgres) fn create_ix_kb_sol_core_transactions_created_at_sql() -> &'static str {
fn create_ix_kb_sol_core_transactions_created_at_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_created_at ON kb_sol_core_transactions (created_at)";
}
/// SQL creating `kb_sol_core_account_keys`.
pub(in crate::postgres) fn create_table_kb_sol_core_account_keys_sql() -> &'static str {
fn create_table_kb_sol_core_account_keys_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_account_keys (
id BIGSERIAL,
transaction_id BIGINT NOT NULL,
@@ -528,17 +532,17 @@ pub(in crate::postgres) fn create_table_kb_sol_core_account_keys_sql() -> &'stat
}
/// SQL creating the unique signature/account index for core account keys.
pub(in crate::postgres) fn create_ux_kb_sol_core_account_keys_sig_index_sql() -> &'static str {
fn create_ux_kb_sol_core_account_keys_sig_index_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_account_keys_sig_index ON kb_sol_core_account_keys (signature, account_index)";
}
/// SQL creating the account key index for core account keys.
pub(in crate::postgres) fn create_ix_kb_sol_core_account_keys_key_sql() -> &'static str {
fn create_ix_kb_sol_core_account_keys_key_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_account_keys_key ON kb_sol_core_account_keys (account_key)";
}
/// SQL creating `kb_sol_core_instructions`.
pub(in crate::postgres) fn create_table_kb_sol_core_instructions_sql() -> &'static str {
fn create_table_kb_sol_core_instructions_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_instructions (
id BIGSERIAL,
transaction_id BIGINT NOT NULL,
@@ -566,27 +570,27 @@ pub(in crate::postgres) fn create_table_kb_sol_core_instructions_sql() -> &'stat
}
/// SQL creating the unique signature/path index for core instructions.
pub(in crate::postgres) fn create_ux_kb_sol_core_instructions_sig_path_sql() -> &'static str {
fn create_ux_kb_sol_core_instructions_sig_path_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_instructions_sig_path ON kb_sol_core_instructions (signature, instruction_path)";
}
/// SQL creating the program index for core instructions.
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_program_sql() -> &'static str {
fn create_ix_kb_sol_core_instructions_program_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_program ON kb_sol_core_instructions (program_id)";
}
/// SQL creating the slot index for core instructions.
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_slot_sql() -> &'static str {
fn create_ix_kb_sol_core_instructions_slot_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_slot ON kb_sol_core_instructions (slot)";
}
/// SQL creating the processing state index for core instructions.
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_processing_sql() -> &'static str {
fn create_ix_kb_sol_core_instructions_processing_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_processing ON kb_sol_core_instructions (processing_state)";
}
/// SQL creating `kb_sol_core_inner_instructions`.
pub(in crate::postgres) fn create_table_kb_sol_core_inner_instructions_sql() -> &'static str {
fn create_table_kb_sol_core_inner_instructions_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_inner_instructions (
id BIGSERIAL,
transaction_id BIGINT NOT NULL,
@@ -610,22 +614,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_inner_instructions_sql() ->
}
/// SQL creating the unique signature/path index for core inner instructions.
pub(in crate::postgres) fn create_ux_kb_sol_core_inner_instructions_sig_path_sql() -> &'static str {
fn create_ux_kb_sol_core_inner_instructions_sig_path_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_inner_instructions_sig_path ON kb_sol_core_inner_instructions (signature, instruction_path)";
}
/// SQL creating the parent instruction index for core inner instructions.
pub(in crate::postgres) fn create_ix_kb_sol_core_inner_instructions_parent_sql() -> &'static str {
fn create_ix_kb_sol_core_inner_instructions_parent_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_parent ON kb_sol_core_inner_instructions (signature, parent_instruction_path)";
}
/// SQL creating the program index for core inner instructions.
pub(in crate::postgres) fn create_ix_kb_sol_core_inner_instructions_program_sql() -> &'static str {
fn create_ix_kb_sol_core_inner_instructions_program_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_program ON kb_sol_core_inner_instructions (program_id)";
}
/// SQL creating `kb_sol_core_logs`.
pub(in crate::postgres) fn create_table_kb_sol_core_logs_sql() -> &'static str {
fn create_table_kb_sol_core_logs_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_logs (
id BIGSERIAL,
transaction_id BIGINT NOT NULL,
@@ -649,22 +653,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_logs_sql() -> &'static str {
}
/// SQL creating the unique signature/log index for core logs.
pub(in crate::postgres) fn create_ux_kb_sol_core_logs_sig_index_sql() -> &'static str {
fn create_ux_kb_sol_core_logs_sig_index_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_logs_sig_index ON kb_sol_core_logs (signature, log_index)";
}
/// SQL creating the program index for core logs.
pub(in crate::postgres) fn create_ix_kb_sol_core_logs_program_sql() -> &'static str {
fn create_ix_kb_sol_core_logs_program_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_program ON kb_sol_core_logs (program_id) WHERE program_id IS NOT NULL";
}
/// SQL creating the instruction path index for core logs.
pub(in crate::postgres) fn create_ix_kb_sol_core_logs_path_sql() -> &'static str {
fn create_ix_kb_sol_core_logs_path_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_path ON kb_sol_core_logs (signature, instruction_path) WHERE instruction_path IS NOT NULL";
}
/// SQL creating `kb_sol_core_balance_changes`.
pub(in crate::postgres) fn create_table_kb_sol_core_balance_changes_sql() -> &'static str {
fn create_table_kb_sol_core_balance_changes_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_balance_changes (
id BIGSERIAL,
transaction_id BIGINT NOT NULL,
@@ -694,27 +698,28 @@ pub(in crate::postgres) fn create_table_kb_sol_core_balance_changes_sql() -> &'s
}
/// SQL creating the unique signature/balance index for core balance changes.
pub(in crate::postgres) fn create_ux_kb_sol_core_balance_changes_sig_index_sql() -> &'static str {
fn create_ux_kb_sol_core_balance_changes_sig_index_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_balance_changes_sig_index ON kb_sol_core_balance_changes (signature, balance_change_index)";
}
/// SQL creating the account key index for core balance changes.
pub(in crate::postgres) fn create_ix_kb_sol_core_balance_changes_account_sql() -> &'static str {
fn create_ix_kb_sol_core_balance_changes_account_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_account ON kb_sol_core_balance_changes (account_key) WHERE account_key IS NOT NULL";
}
/// SQL creating the mint index for core balance changes.
pub(in crate::postgres) fn create_ix_kb_sol_core_balance_changes_mint_sql() -> &'static str {
fn create_ix_kb_sol_core_balance_changes_mint_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_mint ON kb_sol_core_balance_changes (mint) WHERE mint IS NOT NULL";
}
/// Returns true when a Solana table name follows the canonical prefix and domain rules.
pub fn is_valid_solana_table_name(table_name: &str) -> bool {
return crate::validate_solana_table_name(table_name).is_ok();
#[cfg(test)]
fn is_valid_solana_table_name(table_name: &str) -> bool {
return validate_solana_table_name(table_name).is_ok();
}
/// Validates a canonical Solana table name.
pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
let trimmed_table_name = table_name.trim();
if trimmed_table_name.is_empty() {
return std::result::Result::Err(ks_core::Error::db("table name must not be empty"));
@@ -724,12 +729,12 @@ pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
"solana table names must not contain an explicit PostgreSQL schema",
));
}
if !trimmed_table_name.starts_with(crate::SOLANA_TABLE_PREFIX) {
if !trimmed_table_name.starts_with(SOLANA_TABLE_PREFIX) {
return std::result::Result::Err(ks_core::Error::db(
"solana table name must start with kb_sol_",
));
}
let suffix = &trimmed_table_name[crate::SOLANA_TABLE_PREFIX.len()..];
let suffix = &trimmed_table_name[SOLANA_TABLE_PREFIX.len()..];
let domain = crate::postgres::migrations::first_segment(suffix);
if domain.is_empty() {
return std::result::Result::Err(ks_core::Error::db(
@@ -755,23 +760,23 @@ pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
}
/// Validates the raw store table names introduced by `0.2.3`.
pub fn validate_raw_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(crate::RAW_STORE_TABLE_NAMES);
pub(crate) fn validate_raw_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(RAW_STORE_TABLE_NAMES);
}
/// Validates the core store table names introduced by `0.2.4`.
pub fn validate_core_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(crate::CORE_STORE_TABLE_NAMES);
pub(crate) fn validate_core_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(CORE_STORE_TABLE_NAMES);
}
/// Validates the decode and materialization table names introduced by `0.4.0`.
pub fn validate_decode_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(crate::DECODE_STORE_TABLE_NAMES);
pub(crate) fn validate_decode_store_table_names() -> ks_core::Result<()> {
return crate::postgres::migrations::validate_table_names(DECODE_STORE_TABLE_NAMES);
}
fn validate_table_names(table_names: &[&str]) -> ks_core::Result<()> {
for table_name in table_names {
let validation_result = crate::validate_solana_table_name(table_name);
let validation_result = validate_solana_table_name(table_name);
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
}
@@ -787,7 +792,7 @@ fn first_segment(value: &str) -> &str {
}
fn is_allowed_domain(domain: &str) -> bool {
for allowed_domain in crate::ALLOWED_SOLANA_TABLE_DOMAINS {
for allowed_domain in ALLOWED_SOLANA_TABLE_DOMAINS {
if domain == *allowed_domain {
return true;
}
@@ -812,7 +817,7 @@ fn contains_only_table_name_chars(table_name: &str) -> bool {
}
/// SQL creating `kb_sol_ops_processing_ledger`.
pub(in crate::postgres) fn create_table_kb_sol_ops_processing_ledger_sql() -> &'static str {
fn create_table_kb_sol_ops_processing_ledger_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_ops_processing_ledger (
id BIGSERIAL,
stage TEXT NOT NULL,
@@ -842,87 +847,87 @@ pub(in crate::postgres) fn create_table_kb_sol_ops_processing_ledger_sql() -> &'
}
/// SQL creating the unique processing ledger identity index.
pub(in crate::postgres) fn create_ux_kb_sol_ops_processing_ledger_identity_sql() -> &'static str {
fn create_ux_kb_sol_ops_processing_ledger_identity_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_ops_processing_ledger_identity ON kb_sol_ops_processing_ledger (stage, processor_name, processor_version, input_key)";
}
/// SQL creating the processing ledger status index.
pub(in crate::postgres) fn create_ix_kb_sol_ops_processing_ledger_status_sql() -> &'static str {
fn create_ix_kb_sol_ops_processing_ledger_status_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_status ON kb_sol_ops_processing_ledger (stage, processor_name, status, updated_at)";
}
/// SQL creating the processing ledger input hash index.
pub(in crate::postgres) fn create_ix_kb_sol_ops_processing_ledger_input_hash_sql() -> &'static str {
fn create_ix_kb_sol_ops_processing_ledger_input_hash_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_input_hash ON kb_sol_ops_processing_ledger (input_hash)";
}
/// SQL statistics query for `kb_sol_raw_transactions`.
pub(in crate::postgres) fn table_stats_kb_sol_raw_transactions_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_raw_transactions_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_raw_transactions";
}
/// SQL statistics query for `kb_sol_obs_transaction_observations`.
pub(in crate::postgres) fn table_stats_kb_sol_obs_transaction_observations_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_obs_transaction_observations_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(persisted_at)::text AS latest_created_at FROM kb_sol_obs_transaction_observations";
}
/// SQL statistics query for `kb_sol_core_transactions`.
pub(in crate::postgres) fn table_stats_kb_sol_core_transactions_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_transactions_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_transactions";
}
/// SQL statistics query for `kb_sol_core_account_keys`.
pub(in crate::postgres) fn table_stats_kb_sol_core_account_keys_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_account_keys_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_account_keys";
}
/// SQL statistics query for `kb_sol_core_instructions`.
pub(in crate::postgres) fn table_stats_kb_sol_core_instructions_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_instructions_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_instructions";
}
/// SQL statistics query for `kb_sol_core_inner_instructions`.
pub(in crate::postgres) fn table_stats_kb_sol_core_inner_instructions_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_inner_instructions_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_inner_instructions";
}
/// SQL statistics query for `kb_sol_core_logs`.
pub(in crate::postgres) fn table_stats_kb_sol_core_logs_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_logs_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_logs";
}
/// SQL statistics query for `kb_sol_core_balance_changes`.
pub(in crate::postgres) fn table_stats_kb_sol_core_balance_changes_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_core_balance_changes_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_balance_changes";
}
/// SQL statistics query for `kb_sol_ops_processing_ledger`.
pub(in crate::postgres) fn table_stats_kb_sol_ops_processing_ledger_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_ops_processing_ledger_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, NULL::BIGINT AS min_slot, NULL::BIGINT AS max_slot, MAX(updated_at)::text AS latest_created_at FROM kb_sol_ops_processing_ledger";
}
/// SQL statistics query for `kb_sol_decode_events`.
pub(in crate::postgres) fn table_stats_kb_sol_decode_events_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_decode_events_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_events";
}
/// SQL statistics query for `kb_sol_decode_coverage_declarations`.
pub(in crate::postgres) fn table_stats_kb_sol_decode_coverage_declarations_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_decode_coverage_declarations_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, NULL::BIGINT AS min_slot, NULL::BIGINT AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_coverage_declarations";
}
/// SQL statistics query for `kb_sol_decode_coverage_observations`.
pub(in crate::postgres) fn table_stats_kb_sol_decode_coverage_observations_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_decode_coverage_observations_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_coverage_observations";
}
/// SQL statistics query for `kb_sol_mat_events`.
pub(in crate::postgres) fn table_stats_kb_sol_mat_events_sql() -> &'static str {
pub(crate) fn table_stats_kb_sol_mat_events_sql() -> &'static str {
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_mat_events";
}
/// SQL creating `kb_sol_decode_events`.
pub(in crate::postgres) fn create_table_kb_sol_decode_events_sql() -> &'static str {
fn create_table_kb_sol_decode_events_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_events (
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, event_key TEXT NOT NULL,
@@ -940,27 +945,27 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_events_sql() -> &'static s
}
/// SQL creating the decoded observation stable identity index.
pub(in crate::postgres) fn create_ux_kb_sol_decode_events_identity_sql() -> &'static str {
fn create_ux_kb_sol_decode_events_identity_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_events_processor_input_event ON kb_sol_decode_events (processor_name, processor_version, input_key, event_key)";
}
/// SQL creating the decoded observation signature and path index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_signature_path_sql() -> &'static str {
fn create_ix_kb_sol_decode_events_signature_path_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_signature_path ON kb_sol_decode_events (signature, instruction_path)";
}
/// SQL creating the decoded observation program and surface index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_program_surface_sql() -> &'static str {
fn create_ix_kb_sol_decode_events_program_surface_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_program_surface ON kb_sol_decode_events (program_id, surface_code, event_code)";
}
/// SQL creating the decoded observation family and commit index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_family_commit_sql() -> &'static str {
fn create_ix_kb_sol_decode_events_family_commit_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_family_commit ON kb_sol_decode_events (event_family, transaction_failed, observation_committed)";
}
/// SQL creating `kb_sol_decode_coverage_declarations`.
pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_declarations_sql() -> &'static str {
fn create_table_kb_sol_decode_coverage_declarations_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_declarations (
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
program_id TEXT NOT NULL, surface_code TEXT, entry_kind TEXT NOT NULL,
@@ -971,19 +976,17 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_declarations_sql(
}
/// SQL creating the declared coverage stable identity index.
pub(in crate::postgres) fn create_ux_kb_sol_decode_coverage_declarations_identity_sql()
-> &'static str {
fn create_ux_kb_sol_decode_coverage_declarations_identity_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_declarations_identity ON kb_sol_decode_coverage_declarations (processor_name, processor_version, program_id, COALESCE(surface_code, ''), entry_kind, entry_code, COALESCE(discriminator_hex, ''))";
}
/// SQL creating the declared coverage program index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_declarations_program_sql()
-> &'static str {
fn create_ix_kb_sol_decode_coverage_declarations_program_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_declarations_program ON kb_sol_decode_coverage_declarations (program_id, processor_name, processor_version)";
}
/// SQL creating `kb_sol_decode_coverage_observations`.
pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_observations_sql() -> &'static str {
fn create_table_kb_sol_decode_coverage_observations_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_observations (
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, signature TEXT NOT NULL,
@@ -1000,25 +1003,22 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_observations_sql(
}
/// SQL creating the observed coverage stable identity index.
pub(in crate::postgres) fn create_ux_kb_sol_decode_coverage_observations_identity_sql()
-> &'static str {
fn create_ux_kb_sol_decode_coverage_observations_identity_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_observations_identity ON kb_sol_decode_coverage_observations (processor_name, processor_version, input_key)";
}
/// SQL creating the observed coverage program and status index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_observations_program_status_sql()
-> &'static str {
fn create_ix_kb_sol_decode_coverage_observations_program_status_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_program_status ON kb_sol_decode_coverage_observations (program_id, status, transaction_failed)";
}
/// SQL creating the observed coverage entry index.
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_observations_entry_sql() -> &'static str
{
fn create_ix_kb_sol_decode_coverage_observations_entry_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_entry ON kb_sol_decode_coverage_observations (processor_name, processor_version, surface_code, entry_code)";
}
/// SQL creating `kb_sol_mat_events`.
pub(in crate::postgres) fn create_table_kb_sol_mat_events_sql() -> &'static str {
fn create_table_kb_sol_mat_events_sql() -> &'static str {
return r#"CREATE TABLE IF NOT EXISTS kb_sol_mat_events (
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, output_key TEXT NOT NULL,
@@ -1033,12 +1033,12 @@ pub(in crate::postgres) fn create_table_kb_sol_mat_events_sql() -> &'static str
}
/// SQL creating the materialized output stable identity index.
pub(in crate::postgres) fn create_ux_kb_sol_mat_events_identity_sql() -> &'static str {
fn create_ux_kb_sol_mat_events_identity_sql() -> &'static str {
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_mat_events_processor_input_output ON kb_sol_mat_events (processor_name, processor_version, input_key, output_key)";
}
/// SQL creating materialized output signature and family indexes.
pub(in crate::postgres) fn create_ix_kb_sol_mat_events_signature_family_sql() -> &'static str {
fn create_ix_kb_sol_mat_events_signature_family_sql() -> &'static str {
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_mat_events_signature_family ON kb_sol_mat_events (source_decoder_name, source_decoder_version, source_decode_input_key, signature, materialized_family)";
}
@@ -1046,28 +1046,28 @@ pub(in crate::postgres) fn create_ix_kb_sol_mat_events_signature_family_sql() ->
mod tests {
#[test]
fn valid_table_name_accepts_domain_prefix() {
assert!(crate::is_valid_solana_table_name("kb_sol_raw_transactions"));
assert!(super::is_valid_solana_table_name("kb_sol_raw_transactions"));
}
#[test]
fn table_name_rejects_explicit_schema() {
let invalid_name = std::string::String::from("raw") + "." + "kb_sol_rpc_transactions";
assert!(!crate::is_valid_solana_table_name(invalid_name.as_str()));
assert!(!super::is_valid_solana_table_name(invalid_name.as_str()));
}
#[test]
fn table_name_rejects_unknown_domain() {
assert!(!crate::is_valid_solana_table_name("kb_sol_unknown_rows"));
assert!(!super::is_valid_solana_table_name("kb_sol_unknown_rows"));
}
#[test]
fn table_name_rejects_missing_table_suffix() {
assert!(!crate::is_valid_solana_table_name("kb_sol_raw"));
assert!(!super::is_valid_solana_table_name("kb_sol_raw"));
}
#[test]
fn table_name_rejects_uppercase() {
assert!(!crate::is_valid_solana_table_name("kb_sol_raw_RPC_transactions"));
assert!(!super::is_valid_solana_table_name("kb_sol_raw_RPC_transactions"));
}
#[test]

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/query.rs
// version: 3
// version: 4
//! PostgreSQL query modules.
@@ -11,42 +11,43 @@ mod raw_queries;
mod replay_candidate_queries;
mod table_diagnostics_queries;
pub(in crate::postgres) use self::core_extraction_queries::is_core_extraction_current;
pub(in crate::postgres) use self::core_extraction_queries::list_raw_transactions_for_core_extraction;
pub(in crate::postgres) use self::core_extraction_queries::mark_core_extraction_failed;
pub(in crate::postgres) use self::core_extraction_queries::persist_core_extraction;
pub(in crate::postgres) use self::core_queries::apply_core_store_schema;
pub(in crate::postgres) use self::core_queries::insert_core_account_keys;
pub(in crate::postgres) use self::core_queries::insert_core_balance_changes;
pub(in crate::postgres) use self::core_queries::insert_core_inner_instructions;
pub(in crate::postgres) use self::core_queries::insert_core_instructions;
pub(in crate::postgres) use self::core_queries::insert_core_logs;
pub(in crate::postgres) use self::core_queries::insert_core_transaction;
pub(in crate::postgres) use self::core_queries::list_core_instruction_replay_inputs;
pub(in crate::postgres) use self::core_queries::list_core_instructions_for_replay;
pub(in crate::postgres) use self::core_queries::update_core_instruction_lifecycle;
pub(in crate::postgres) use self::decode_pipeline_queries::apply_decode_store_schema;
pub(in crate::postgres) use self::decode_pipeline_queries::is_decode_current;
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_coverage_summary;
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_inputs;
pub(in crate::postgres) use self::decode_pipeline_queries::list_materialized_events;
pub(in crate::postgres) use self::decode_pipeline_queries::mark_decode_failed;
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_coverage_declarations;
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_result;
pub(in crate::postgres) use self::decode_pipeline_queries::persist_materialization_result;
pub(in crate::postgres) use self::health_queries::load_current_schema;
pub(in crate::postgres) use self::health_queries::load_latest_migration_version;
pub(in crate::postgres) use self::health_queries::load_migration_table_name;
pub(in crate::postgres) use self::health_queries::load_server_version;
pub(in crate::postgres) use self::health_queries::run_health_check;
pub(in crate::postgres) use self::raw_queries::apply_raw_store_schema;
pub(in crate::postgres) use self::raw_queries::has_raw_transaction_signature;
pub(in crate::postgres) use self::raw_queries::has_transaction_observation_key;
pub(in crate::postgres) use self::raw_queries::insert_raw_transaction;
pub(in crate::postgres) use self::raw_queries::insert_transaction_observation;
pub(in crate::postgres) use self::raw_queries::update_raw_payload_lifecycle;
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_entity_summaries;
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_program_summaries;
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_transaction_candidates;
pub(in crate::postgres) use self::table_diagnostics_queries::load_table_statistics;
pub(in crate::postgres) use self::table_diagnostics_queries::table_exists;
pub(crate) use self::core_extraction_queries::is_core_extraction_current;
pub(crate) use self::core_extraction_queries::list_raw_transactions_for_core_extraction;
pub(crate) use self::core_extraction_queries::mark_core_extraction_failed;
pub(crate) use self::core_extraction_queries::persist_core_extraction;
pub(crate) use self::core_queries::apply_core_store_schema;
pub(crate) use self::core_queries::insert_core_account_keys;
pub(crate) use self::core_queries::insert_core_balance_changes;
pub(crate) use self::core_queries::insert_core_inner_instructions;
pub(crate) use self::core_queries::insert_core_instructions;
pub(crate) use self::core_queries::insert_core_logs;
pub(crate) use self::core_queries::insert_core_transaction;
pub(crate) use self::core_queries::list_core_instruction_replay_inputs;
pub(crate) use self::core_queries::list_core_instructions_for_replay;
pub(crate) use self::core_queries::list_decode_replay_inputs;
pub(crate) use self::core_queries::update_core_instruction_lifecycle;
pub(crate) use self::decode_pipeline_queries::apply_decode_store_schema;
pub(crate) use self::decode_pipeline_queries::is_decode_current;
pub(crate) use self::decode_pipeline_queries::list_decode_coverage_summary;
pub(crate) use self::decode_pipeline_queries::list_decode_inputs;
pub(crate) use self::decode_pipeline_queries::list_materialized_events;
pub(crate) use self::decode_pipeline_queries::mark_decode_failed;
pub(crate) use self::decode_pipeline_queries::persist_decode_coverage_declarations;
pub(crate) use self::decode_pipeline_queries::persist_decode_result;
pub(crate) use self::decode_pipeline_queries::persist_materialization_result;
pub(crate) use self::health_queries::load_current_schema;
pub(crate) use self::health_queries::load_latest_migration_version;
pub(crate) use self::health_queries::load_migration_table_name;
pub(crate) use self::health_queries::load_server_version;
pub(crate) use self::health_queries::run_health_check;
pub(crate) use self::raw_queries::apply_raw_store_schema;
pub(crate) use self::raw_queries::has_raw_transaction_signature;
pub(crate) use self::raw_queries::has_transaction_observation_key;
pub(crate) use self::raw_queries::insert_raw_transaction;
pub(crate) use self::raw_queries::insert_transaction_observation;
pub(crate) use self::raw_queries::update_raw_payload_lifecycle;
pub(crate) use self::replay_candidate_queries::list_replay_entity_summaries;
pub(crate) use self::replay_candidate_queries::list_replay_program_summaries;
pub(crate) use self::replay_candidate_queries::list_replay_transaction_candidates;
pub(crate) use self::table_diagnostics_queries::load_table_statistics;
pub(crate) use self::table_diagnostics_queries::table_exists;

View File

@@ -1,11 +1,11 @@
// file: ks-store/src/postgres/query/core_extraction_queries.rs
// version: 4
// version: 6
//! PostgreSQL queries for atomic canonical transaction to core extraction.
use sqlx::Row; // rust-rules: trait-import
pub(in crate::postgres) async fn list_raw_transactions_for_core_extraction(
pub(crate) async fn list_raw_transactions_for_core_extraction(
pool: &sqlx::PgPool,
filter: &crate::CoreExtractionSelectionFilter,
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>> {
@@ -59,7 +59,7 @@ pub(in crate::postgres) async fn list_raw_transactions_for_core_extraction(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn is_core_extraction_current(
pub(crate) async fn is_core_extraction_current(
pool: &sqlx::PgPool,
identity: &crate::ProcessingLedgerIdentity,
) -> ks_core::Result<bool> {
@@ -81,7 +81,7 @@ pub(in crate::postgres) async fn is_core_extraction_current(
};
}
pub(in crate::postgres) async fn persist_core_extraction(
pub(crate) async fn persist_core_extraction(
pool: &sqlx::PgPool,
bundle: &crate::CoreExtractionBundle,
_force_replay: bool,
@@ -126,11 +126,7 @@ pub(in crate::postgres) async fn persist_core_extraction(
}
}
let transaction_id_result =
crate::postgres::query::core_extraction_queries::insert_core_transaction_in_transaction(
&mut transaction,
&bundle.transaction,
)
.await;
insert_core_transaction_in_transaction(&mut transaction, &bundle.transaction).await;
let transaction_id = match transaction_id_result {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
@@ -226,7 +222,7 @@ pub(in crate::postgres) async fn persist_core_extraction(
));
}
pub(in crate::postgres) async fn mark_core_extraction_failed(
pub(crate) async fn mark_core_extraction_failed(
pool: &sqlx::PgPool,
failure: &crate::CoreExtractionFailure,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -783,7 +779,7 @@ mod tests {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_error) => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
let options = match options_result {
std::result::Result::Ok(value) => value,

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/query/core_queries.rs
// version: 4
// version: 6
//! PostgreSQL queries for normalized Solana core storage.
@@ -7,9 +7,7 @@ use sqlx::Row; // rust-rules: trait-import
const OUTER_INSTRUCTIONS_CONTEXT_SQL: &str = "SELECT COALESCE(jsonb_agg(jsonb_build_object('instructionIndex', instruction_path::BIGINT, 'instructionPath', instruction_path, 'programId', program_id, 'payloadJson', payload_json, 'payloadHash', payload_json_hash) ORDER BY instruction_path::BIGINT, instruction_path ASC), '[]'::jsonb) FROM kb_sol_core_instructions WHERE signature = $1 AND instruction_path ~ '^[0-9]+$'";
pub(in crate::postgres) async fn apply_core_store_schema(
pool: &sqlx::PgPool,
) -> ks_core::Result<()> {
pub(crate) async fn apply_core_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
let validation_result = crate::validate_core_store_table_names();
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
@@ -49,7 +47,7 @@ pub(in crate::postgres) async fn apply_core_store_schema(
return std::result::Result::Ok(());
}
pub(in crate::postgres) async fn insert_core_transaction(
pub(crate) async fn insert_core_transaction(
pool: &sqlx::PgPool,
input: &crate::CoreTransactionInsert,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -74,7 +72,7 @@ pub(in crate::postgres) async fn insert_core_transaction(
);
}
pub(in crate::postgres) async fn insert_core_account_keys(
pub(crate) async fn insert_core_account_keys(
pool: &sqlx::PgPool,
inputs: &[crate::CoreAccountKeyInsert],
) -> ks_core::Result<crate::InsertOutcome> {
@@ -122,7 +120,7 @@ pub(in crate::postgres) async fn insert_core_account_keys(
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
}
pub(in crate::postgres) async fn insert_core_instructions(
pub(crate) async fn insert_core_instructions(
pool: &sqlx::PgPool,
inputs: &[crate::CoreInstructionInsert],
) -> ks_core::Result<crate::InsertOutcome> {
@@ -165,7 +163,7 @@ pub(in crate::postgres) async fn insert_core_instructions(
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
}
pub(in crate::postgres) async fn insert_core_inner_instructions(
pub(crate) async fn insert_core_inner_instructions(
pool: &sqlx::PgPool,
inputs: &[crate::CoreInnerInstructionInsert],
) -> ks_core::Result<crate::InsertOutcome> {
@@ -204,7 +202,7 @@ pub(in crate::postgres) async fn insert_core_inner_instructions(
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
}
pub(in crate::postgres) async fn insert_core_logs(
pub(crate) async fn insert_core_logs(
pool: &sqlx::PgPool,
inputs: &[crate::CoreLogInsert],
) -> ks_core::Result<crate::InsertOutcome> {
@@ -250,7 +248,7 @@ pub(in crate::postgres) async fn insert_core_logs(
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
}
pub(in crate::postgres) async fn insert_core_balance_changes(
pub(crate) async fn insert_core_balance_changes(
pool: &sqlx::PgPool,
inputs: &[crate::CoreBalanceChangeInsert],
) -> ks_core::Result<crate::InsertOutcome> {
@@ -310,7 +308,7 @@ pub(in crate::postgres) async fn insert_core_balance_changes(
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
}
pub(in crate::postgres) async fn list_core_instructions_for_replay(
pub(crate) async fn list_core_instructions_for_replay(
pool: &sqlx::PgPool,
filter: &crate::CoreInstructionReplayFilter,
page_request: &crate::PageRequest,
@@ -368,18 +366,13 @@ pub(in crate::postgres) async fn list_core_instructions_for_replay(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn list_core_instruction_replay_inputs(
pub(crate) async fn list_core_instruction_replay_inputs(
pool: &sqlx::PgPool,
filter: &crate::CoreInstructionReplayFilter,
page_request: &crate::PageRequest,
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
let instructions_result =
crate::postgres::query::core_queries::list_core_instructions_for_replay(
pool,
filter,
page_request,
)
.await;
crate::list_core_instructions_for_replay(pool, filter, page_request).await;
let instructions = match instructions_result {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
@@ -399,7 +392,7 @@ pub(in crate::postgres) async fn list_core_instruction_replay_inputs(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn list_decode_replay_inputs(
pub(crate) async fn list_decode_replay_inputs(
pool: &sqlx::PgPool,
filter: &crate::DecodeSelectionFilter,
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
@@ -482,7 +475,7 @@ pub(in crate::postgres) async fn list_decode_replay_inputs(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn update_core_instruction_lifecycle(
pub(crate) async fn update_core_instruction_lifecycle(
pool: &sqlx::PgPool,
mark: &crate::CoreInstructionLifecycleMark,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -507,7 +500,7 @@ pub(in crate::postgres) async fn update_core_instruction_lifecycle(
);
}
pub(in crate::postgres) fn instruction_processing_state_to_sql(
fn instruction_processing_state_to_sql(
state: crate::CoreInstructionProcessingState,
) -> &'static str {
return match state {
@@ -755,7 +748,7 @@ where
};
}
pub(in crate::postgres) async fn load_replay_input_for_instruction(
async fn load_replay_input_for_instruction(
pool: &sqlx::PgPool,
instruction: &crate::CoreInstructionRow,
) -> ks_core::Result<crate::MdCoreInstructionReplayInput> {
@@ -990,7 +983,7 @@ mod tests {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_error) => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
let options = match options_result {
std::result::Result::Ok(value) => value,

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/query/decode_pipeline_queries.rs
// version: 7
// version: 10
//! PostgreSQL queries for contextual decode, coverage and materialization persistence.
@@ -22,10 +22,8 @@ struct MaterializedEventDatabaseRow {
updated_at: std::string::String,
}
pub(in crate::postgres) async fn apply_decode_store_schema(
pool: &sqlx::PgPool,
) -> ks_core::Result<()> {
tracing::debug!(target: crate::TRACING_TARGET, action = "apply_decode_store_schema", statement_count = crate::decode_store_schema_statements().len(), "apply PostgreSQL decode store schema");
pub(crate) async fn apply_decode_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "apply_decode_store_schema", statement_count = crate::decode_store_schema_statements().len(), "apply PostgreSQL decode store schema");
let validation_result = crate::validate_decode_store_table_names();
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
@@ -62,17 +60,16 @@ pub(in crate::postgres) async fn apply_decode_store_schema(
"postgres decode store schema commit failed: {error}"
)));
}
tracing::debug!(target: crate::TRACING_TARGET, action = "apply_decode_store_schema", committed = true, "PostgreSQL decode store schema applied");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "apply_decode_store_schema", committed = true, "PostgreSQL decode store schema applied");
return std::result::Result::Ok(());
}
pub(in crate::postgres) async fn list_decode_inputs(
pub(crate) async fn list_decode_inputs(
pool: &sqlx::PgPool,
filter: &crate::DecodeSelectionFilter,
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_inputs", signature_count = filter.signatures.len(), signature_sample = ?filter.signatures.iter().take(5).map(std::string::String::as_str).collect::<std::vec::Vec<_>>(), processing_states = ?filter.processing_states, min_slot = ?filter.min_slot, max_slot = ?filter.max_slot, program_ids = ?filter.program_ids, instruction_paths = ?filter.instruction_paths, limit = filter.limit, "query PostgreSQL contextual decode inputs");
let result =
crate::postgres::query::core_queries::list_decode_replay_inputs(pool, filter).await;
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", signature_count = filter.signatures.len(), signature_sample = ?filter.signatures.iter().take(5).map(std::string::String::as_str).collect::<std::vec::Vec<_>>(), processing_states = ?filter.processing_states, min_slot = ?filter.min_slot, max_slot = ?filter.max_slot, program_ids = ?filter.program_ids, instruction_paths = ?filter.instruction_paths, limit = filter.limit, "query PostgreSQL contextual decode inputs");
let result = crate::list_decode_replay_inputs(pool, filter).await;
return match result {
std::result::Result::Ok(inputs) => {
let selected_input_keys = inputs
@@ -80,21 +77,21 @@ pub(in crate::postgres) async fn list_decode_inputs(
.take(10)
.map(|input| return input.replay_input_key.as_str())
.collect::<std::vec::Vec<_>>();
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_inputs", selected_count = inputs.len(), input_key_sample = ?selected_input_keys, "PostgreSQL contextual decode inputs selected");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", selected_count = inputs.len(), input_key_sample = ?selected_input_keys, "PostgreSQL contextual decode inputs selected");
std::result::Result::Ok(inputs)
},
std::result::Result::Err(error) => {
tracing::error!(target: crate::TRACING_TARGET, action = "list_decode_inputs", error = %error, "PostgreSQL contextual decode input query failed");
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", error = %error, "PostgreSQL contextual decode input query failed");
std::result::Result::Err(error)
},
};
}
pub(in crate::postgres) async fn list_materialized_events(
pub(crate) async fn list_materialized_events(
pool: &sqlx::PgPool,
filter: &crate::MaterializedEventFilter,
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>> {
tracing::debug!(target: crate::TRACING_TARGET, action = "list_materialized_events", processor_name = ?filter.processor_name, materialized_family = ?filter.materialized_family, signature_contains = ?filter.signature_contains, limit = filter.limit, "query bounded PostgreSQL materialized events");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_materialized_events", processor_name = ?filter.processor_name, materialized_family = ?filter.materialized_family, signature_contains = ?filter.signature_contains, limit = filter.limit, "query bounded PostgreSQL materialized events");
if filter.limit == 0 || filter.limit > crate::MAX_MATERIALIZED_EVENT_QUERY_ROWS {
return std::result::Result::Err(ks_core::Error::db(format!(
"materialized event query limit must be between 1 and {}",
@@ -145,15 +142,15 @@ pub(in crate::postgres) async fn list_materialized_events(
updated_at: row.updated_at,
});
}
tracing::debug!(target: crate::TRACING_TARGET, action = "list_materialized_events", row_count = output.len(), "bounded PostgreSQL materialized events loaded");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_materialized_events", row_count = output.len(), "bounded PostgreSQL materialized events loaded");
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn is_decode_current(
pub(crate) async fn is_decode_current(
pool: &sqlx::PgPool,
identity: &crate::ProcessingLedgerIdentity,
) -> ks_core::Result<bool> {
tracing::debug!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, "query PostgreSQL processing ledger current state");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, "query PostgreSQL processing ledger current state");
let query_result = sqlx::query_scalar::<sqlx::Postgres, bool>(
"SELECT EXISTS(SELECT 1 FROM kb_sol_ops_processing_ledger WHERE stage = $1 AND processor_name = $2 AND processor_version = $3 AND input_key = $4 AND input_hash = $5 AND status = 'succeeded')",
)
@@ -166,11 +163,11 @@ pub(in crate::postgres) async fn is_decode_current(
.await;
return match query_result {
std::result::Result::Ok(value) => {
tracing::debug!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, current = value, "PostgreSQL processing ledger current state loaded");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, current = value, "PostgreSQL processing ledger current state loaded");
std::result::Result::Ok(value)
},
std::result::Result::Err(error) => {
tracing::error!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, error = %error, "PostgreSQL processing ledger current check failed");
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, error = %error, "PostgreSQL processing ledger current check failed");
std::result::Result::Err(ks_core::Error::db(format!(
"postgres decode ledger current check failed: {error}"
)))
@@ -178,12 +175,12 @@ pub(in crate::postgres) async fn is_decode_current(
};
}
pub(in crate::postgres) async fn persist_decode_coverage_declarations(
pub(crate) async fn persist_decode_coverage_declarations(
pool: &sqlx::PgPool,
declarations: &[crate::DecodeCoverageDeclarationInsert],
) -> ks_core::Result<crate::InsertOutcome> {
if declarations.is_empty() {
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", declaration_count = 0_usize, "skip empty PostgreSQL decode coverage declarations");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", declaration_count = 0_usize, "skip empty PostgreSQL decode coverage declarations");
return std::result::Result::Ok(crate::InsertOutcome::new(0, 0, 0));
}
let processor_name = declarations[0].processor_name.as_str();
@@ -192,7 +189,7 @@ pub(in crate::postgres) async fn persist_decode_coverage_declarations(
.iter()
.map(|entry| return entry.program_id.as_str())
.collect::<std::vec::Vec<_>>();
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, declaration_count = declarations.len(), program_ids = ?program_ids, "persist PostgreSQL decode coverage declarations");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, declaration_count = declarations.len(), program_ids = ?program_ids, "persist PostgreSQL decode coverage declarations");
if declarations.iter().any(|entry| {
return entry.processor_name != processor_name
|| entry.processor_version != processor_version
@@ -351,16 +348,16 @@ pub(in crate::postgres) async fn persist_decode_coverage_declarations(
)));
}
let outcome = crate::InsertOutcome::new(inserted_count, updated_count, skipped_count);
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, outcome = ?outcome, "PostgreSQL decode coverage declarations persisted");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, outcome = ?outcome, "PostgreSQL decode coverage declarations persisted");
return std::result::Result::Ok(outcome);
}
pub(in crate::postgres) async fn persist_decode_result(
pub(crate) async fn persist_decode_result(
pool: &sqlx::PgPool,
bundle: &crate::DecodePersistenceBundle,
force_replay: bool,
) -> ks_core::Result<crate::InsertOutcome> {
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, status = %bundle.status, observation_count = bundle.observations.len(), coverage_program_id = %bundle.coverage.program_id, force_replay, "persist PostgreSQL contextual decode result");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, status = %bundle.status, observation_count = bundle.observations.len(), coverage_program_id = %bundle.coverage.program_id, force_replay, "persist PostgreSQL contextual decode result");
let validation_result = bundle.validate();
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
@@ -513,15 +510,15 @@ pub(in crate::postgres) async fn persist_decode_result(
},
};
let outcome = crate::InsertOutcome::new(count, 1, 0);
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL contextual decode result persisted");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL contextual decode result persisted");
return std::result::Result::Ok(outcome);
}
pub(in crate::postgres) async fn mark_decode_failed(
pub(crate) async fn mark_decode_failed(
pool: &sqlx::PgPool,
failure: &crate::DecodeFailure,
) -> ks_core::Result<crate::InsertOutcome> {
tracing::error!(target: crate::TRACING_TARGET, action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, input_key = %failure.ledger_identity.input_key, input_hash = %failure.ledger_identity.input_hash, error_code = %failure.error_code, error_message = %failure.error_message, "persist PostgreSQL contextual decode failure");
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, input_key = %failure.ledger_identity.input_key, input_hash = %failure.ledger_identity.input_hash, error_code = %failure.error_code, error_message = %failure.error_message, "persist PostgreSQL contextual decode failure");
if failure.ledger_identity.stage != "instruction_decode"
|| failure.signature.trim().is_empty()
|| failure.instruction_path.trim().is_empty()
@@ -572,18 +569,18 @@ pub(in crate::postgres) async fn mark_decode_failed(
)));
}
let outcome = crate::InsertOutcome::new(0, 1, 0);
tracing::debug!(target: crate::TRACING_TARGET, action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, outcome = ?outcome, committed = true, "PostgreSQL contextual decode failure persisted");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, outcome = ?outcome, committed = true, "PostgreSQL contextual decode failure persisted");
return std::result::Result::Ok(outcome);
}
pub(in crate::postgres) async fn persist_materialization_result(
pub(crate) async fn persist_materialization_result(
pool: &sqlx::PgPool,
bundle: &crate::MaterializationPersistenceBundle,
force_replay: bool,
) -> ks_core::Result<crate::InsertOutcome> {
if bundle.status == "failed" {
tracing::error!(
target: crate::TRACING_TARGET,
target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg",
action = "persist_materialization_failure",
signature = %bundle.signature,
instruction_path = %bundle.instruction_path,
@@ -598,7 +595,7 @@ pub(in crate::postgres) async fn persist_materialization_result(
"persist PostgreSQL materialization failure"
);
}
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, source_decoder_name = %bundle.source_decoder_name, source_decoder_version = %bundle.source_decoder_version, source_decode_input_key = %bundle.source_decode_input_key, status = %bundle.status, output_count = bundle.outputs.len(), force_replay, "persist PostgreSQL materialization result");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, source_decoder_name = %bundle.source_decoder_name, source_decoder_version = %bundle.source_decoder_version, source_decode_input_key = %bundle.source_decode_input_key, status = %bundle.status, output_count = bundle.outputs.len(), force_replay, "persist PostgreSQL materialization result");
let validation_result = bundle.validate();
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
@@ -715,17 +712,17 @@ pub(in crate::postgres) async fn persist_materialization_result(
},
};
let outcome = crate::InsertOutcome::new(count, 1, 0);
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL materialization result persisted");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL materialization result persisted");
return std::result::Result::Ok(outcome);
}
pub(in crate::postgres) async fn list_decode_coverage_summary(
pub(crate) async fn list_decode_coverage_summary(
pool: &sqlx::PgPool,
processor_name: std::option::Option<&str>,
processor_version: std::option::Option<&str>,
limit: u32,
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>> {
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, limit, "query PostgreSQL decode coverage summary");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, limit, "query PostgreSQL decode coverage summary");
if limit == 0 {
return std::result::Result::Err(ks_core::Error::db(
"decode coverage summary limit must be greater than zero",
@@ -756,7 +753,7 @@ pub(in crate::postgres) async fn list_decode_coverage_summary(
std::result::Result::Err(error) => return std::result::Result::Err(error),
}
}
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, row_count = output.len(), "PostgreSQL decode coverage summary loaded");
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, row_count = output.len(), "PostgreSQL decode coverage summary loaded");
return std::result::Result::Ok(output);
}
@@ -1035,17 +1032,15 @@ mod tests {
};
let pool_result = sqlx::PgPool::connect(url.as_str()).await;
let pool = result_or_panic(pool_result);
result_or_panic(crate::postgres::query::raw_queries::apply_raw_store_schema(&pool).await);
result_or_panic(crate::postgres::query::core_queries::apply_core_store_schema(&pool).await);
result_or_panic(
crate::postgres::query::decode_pipeline_queries::apply_decode_store_schema(&pool).await,
);
result_or_panic(crate::apply_raw_store_schema(&pool).await);
result_or_panic(crate::apply_core_store_schema(&pool).await);
result_or_panic(crate::apply_decode_store_schema(&pool).await);
return std::option::Option::Some(pool);
}
#[tokio::test]
async fn optional_postgres_coverage_declarations_report_insert_skip_and_update_from_env() {
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let pool = match test_pool_from_env().await {
std::option::Option::Some(value) => value,
std::option::Option::None => return,
@@ -1062,28 +1057,16 @@ mod tests {
historical: false,
};
let first = result_or_panic(
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
&pool,
&[declaration.clone()],
)
.await,
crate::persist_decode_coverage_declarations(&pool, &[declaration.clone()]).await,
);
assert_eq!(first, crate::InsertOutcome::new(1, 0, 0));
let second = result_or_panic(
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
&pool,
&[declaration.clone()],
)
.await,
crate::persist_decode_coverage_declarations(&pool, &[declaration.clone()]).await,
);
assert_eq!(second, crate::InsertOutcome::new(0, 0, 1));
declaration.historical = true;
let third = result_or_panic(
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
&pool,
&[declaration],
)
.await,
crate::persist_decode_coverage_declarations(&pool, &[declaration]).await,
);
assert_eq!(third, crate::InsertOutcome::new(0, 1, 0));
let cleanup_result = sqlx::query(
@@ -1097,7 +1080,7 @@ mod tests {
#[tokio::test]
async fn optional_postgres_same_version_and_hash_is_current_from_env() {
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let pool = match test_pool_from_env().await {
std::option::Option::Some(value) => value,
std::option::Option::None => return,
@@ -1140,16 +1123,8 @@ mod tests {
observations: std::vec::Vec::new(),
coverage,
};
result_or_panic(
crate::postgres::query::decode_pipeline_queries::persist_decode_result(
&pool, &bundle, true,
)
.await,
);
let current = result_or_panic(
crate::postgres::query::decode_pipeline_queries::is_decode_current(&pool, &identity)
.await,
);
result_or_panic(crate::persist_decode_result(&pool, &bundle, true).await);
let current = result_or_panic(crate::is_decode_current(&pool, &identity).await);
assert!(current);
let cleanup_coverage = sqlx::query(
"DELETE FROM kb_sol_decode_coverage_observations WHERE processor_name = $1",
@@ -1168,7 +1143,7 @@ mod tests {
#[tokio::test]
async fn optional_postgres_materialized_event_query_is_bounded_and_typed_from_env() {
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let pool = match test_pool_from_env().await {
std::option::Option::Some(value) => value,
std::option::Option::None => return,
@@ -1189,12 +1164,7 @@ mod tests {
std::option::Option::Some(input_key.clone()),
1,
));
let rows = result_or_panic(
crate::postgres::query::decode_pipeline_queries::list_materialized_events(
&pool, &filter,
)
.await,
);
let rows = result_or_panic(crate::list_materialized_events(&pool, &filter).await);
assert_eq!(rows.len(), 1);
assert_eq!(rows[0].slot, 42);
assert_eq!(rows[0].payload_json["text"], "postgres annotation");
@@ -1211,14 +1181,12 @@ mod tests {
std::result::Result::Ok(value) if !value.trim().is_empty() => value,
_ => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let pool_result = sqlx::PgPool::connect(url.as_str()).await;
let pool = result_or_panic(pool_result);
result_or_panic(crate::postgres::query::raw_queries::apply_raw_store_schema(&pool).await);
result_or_panic(crate::postgres::query::core_queries::apply_core_store_schema(&pool).await);
result_or_panic(
crate::postgres::query::decode_pipeline_queries::apply_decode_store_schema(&pool).await,
);
result_or_panic(crate::apply_raw_store_schema(&pool).await);
result_or_panic(crate::apply_core_store_schema(&pool).await);
result_or_panic(crate::apply_decode_store_schema(&pool).await);
execute_sql(
&pool,
"DELETE FROM kb_sol_decode_events WHERE processor_name = 'decode_atomic_rollback_test'",
@@ -1310,11 +1278,7 @@ mod tests {
observations: std::vec![observation],
coverage,
};
let persistence_result =
crate::postgres::query::decode_pipeline_queries::persist_decode_result(
&pool, &bundle, true,
)
.await;
let persistence_result = crate::persist_decode_result(&pool, &bundle, true).await;
assert!(persistence_result.is_err());
let event_count_result = sqlx::query_scalar::<sqlx::Postgres, i64>(
"SELECT COUNT(*) FROM kb_sol_decode_events WHERE processor_name = 'decode_atomic_rollback_test'",

View File

@@ -1,9 +1,9 @@
// file: ks-store/src/postgres/query/health_queries.rs
// version: 2
// version: 3
//! PostgreSQL health and diagnostic SQL queries.
pub(in crate::postgres) async fn run_health_check(pool: &sqlx::PgPool) -> ks_core::Result<()> {
pub(crate) async fn run_health_check(pool: &sqlx::PgPool) -> ks_core::Result<()> {
let query_result = sqlx::query_scalar::<sqlx::Postgres, i32>("SELECT 1").fetch_one(pool).await;
return match query_result {
std::result::Result::Ok(_value) => std::result::Result::Ok(()),
@@ -13,7 +13,7 @@ pub(in crate::postgres) async fn run_health_check(pool: &sqlx::PgPool) -> ks_cor
};
}
pub(in crate::postgres) async fn load_current_schema(
pub(crate) async fn load_current_schema(
pool: &sqlx::PgPool,
) -> ks_core::Result<std::string::String> {
let query_result =
@@ -28,7 +28,7 @@ pub(in crate::postgres) async fn load_current_schema(
};
}
pub(in crate::postgres) async fn load_server_version(
pub(crate) async fn load_server_version(
pool: &sqlx::PgPool,
) -> ks_core::Result<std::string::String> {
let query_result =
@@ -43,7 +43,7 @@ pub(in crate::postgres) async fn load_server_version(
};
}
pub(in crate::postgres) async fn load_migration_table_name(
pub(crate) async fn load_migration_table_name(
pool: &sqlx::PgPool,
) -> ks_core::Result<std::option::Option<std::string::String>> {
let query_result =
@@ -60,7 +60,7 @@ pub(in crate::postgres) async fn load_migration_table_name(
};
}
pub(in crate::postgres) async fn load_latest_migration_version(
pub(crate) async fn load_latest_migration_version(
pool: &sqlx::PgPool,
) -> ks_core::Result<std::option::Option<std::string::String>> {
let query_result = sqlx::query_scalar::<sqlx::Postgres, std::option::Option<std::string::String>>(

View File

@@ -1,11 +1,9 @@
// file: ks-store/src/postgres/query/raw_queries.rs
// version: 4
// version: 6
//! PostgreSQL canonical transaction and acquisition observation SQL queries.
pub(in crate::postgres) async fn apply_raw_store_schema(
pool: &sqlx::PgPool,
) -> ks_core::Result<()> {
pub(crate) async fn apply_raw_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
let validation_result = crate::validate_raw_store_table_names();
if let std::result::Result::Err(error) = validation_result {
return std::result::Result::Err(error);
@@ -45,7 +43,7 @@ pub(in crate::postgres) async fn apply_raw_store_schema(
return std::result::Result::Ok(());
}
pub(in crate::postgres) async fn has_raw_transaction_signature(
pub(crate) async fn has_raw_transaction_signature(
pool: &sqlx::PgPool,
signature: &str,
) -> ks_core::Result<bool> {
@@ -70,7 +68,7 @@ pub(in crate::postgres) async fn has_raw_transaction_signature(
};
}
pub(in crate::postgres) async fn has_transaction_observation_key(
pub(crate) async fn has_transaction_observation_key(
pool: &sqlx::PgPool,
observation_key: &str,
) -> ks_core::Result<bool> {
@@ -95,7 +93,7 @@ pub(in crate::postgres) async fn has_transaction_observation_key(
};
}
pub(in crate::postgres) async fn insert_raw_transaction(
pub(crate) async fn insert_raw_transaction(
pool: &sqlx::PgPool,
input: &crate::RawTransactionInsert,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -136,7 +134,7 @@ pub(in crate::postgres) async fn insert_raw_transaction(
};
}
pub(in crate::postgres) async fn insert_transaction_observation(
pub(crate) async fn insert_transaction_observation(
pool: &sqlx::PgPool,
input: &crate::TransactionObservationInsert,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -193,7 +191,7 @@ pub(in crate::postgres) async fn insert_transaction_observation(
};
}
pub(in crate::postgres) async fn update_raw_payload_lifecycle(
pub(crate) async fn update_raw_payload_lifecycle(
pool: &sqlx::PgPool,
mark: &crate::RawPayloadLifecycleMark,
) -> ks_core::Result<crate::InsertOutcome> {
@@ -378,7 +376,7 @@ mod tests {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_error) => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
let options = match options_result {
std::result::Result::Ok(value) => value,

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/query/replay_candidate_queries.rs
// version: 4
// version: 6
//! Read-only PostgreSQL queries for replay candidate discovery.
@@ -14,9 +14,9 @@ struct ReplayTransactionCandidateRow {
ledger_status: std::string::String,
processor_version: std::option::Option<std::string::String>,
attempt_count: i32,
outer_instruction_count: i64,
top_level_instruction_count: i64,
inner_instruction_count: i64,
outer_program_count: i64,
top_level_program_count: i64,
inner_program_count: i64,
updated_at: std::string::String,
}
@@ -25,7 +25,7 @@ struct ReplayTransactionCandidateRow {
struct ReplayProgramSummaryRow {
program_id: std::string::String,
transaction_count: i64,
outer_instruction_count: i64,
top_level_instruction_count: i64,
inner_instruction_count: i64,
log_count: i64,
min_slot: i64,
@@ -42,10 +42,10 @@ struct ReplayEntitySummaryRow {
max_slot: i64,
}
pub(in crate::postgres) async fn list_replay_transaction_candidates(
pub(crate) async fn list_replay_transaction_candidates(
pool: &sqlx::PgPool,
filter: &crate::PostgresReplayTransactionFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
filter: &crate::ReplayTransactionFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayTransactionCandidate>> {
let min_slot_result =
crate::postgres::query::replay_candidate_queries::optional_sql_bigint(filter.min_slot);
let min_slot = match min_slot_result {
@@ -93,7 +93,7 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
};
let mut output = std::vec::Vec::with_capacity(rows.len());
for row in rows {
output.push(crate::PostgresReplayTransactionCandidate {
output.push(crate::ReplayTransactionCandidate {
signature: row.signature,
slot: row.slot,
raw_processing_state: row.raw_processing_state,
@@ -103,9 +103,9 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
ledger_status: row.ledger_status,
processor_version: row.processor_version,
attempt_count: row.attempt_count,
outer_instruction_count: row.outer_instruction_count,
top_level_instruction_count: row.top_level_instruction_count,
inner_instruction_count: row.inner_instruction_count,
outer_program_count: row.outer_program_count,
top_level_program_count: row.top_level_program_count,
inner_program_count: row.inner_program_count,
updated_at: row.updated_at,
});
@@ -113,13 +113,13 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn list_replay_program_summaries(
pub(crate) async fn list_replay_program_summaries(
pool: &sqlx::PgPool,
filter: &crate::PostgresReplayProgramFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
filter: &crate::ReplayProgramFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayProgramSummary>> {
let query_result = sqlx::query_as::<sqlx::Postgres, crate::postgres::query::replay_candidate_queries::ReplayProgramSummaryRow>(
r#"WITH occurrences AS (
SELECT program_id, signature, slot, 'outer'::TEXT AS scope FROM kb_sol_core_instructions
SELECT program_id, signature, slot, 'top_level'::TEXT AS scope FROM kb_sol_core_instructions
UNION ALL
SELECT program_id, signature, slot, 'inner'::TEXT AS scope FROM kb_sol_core_inner_instructions
UNION ALL
@@ -127,7 +127,7 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
)
SELECT program_id,
COUNT(DISTINCT signature)::BIGINT AS transaction_count,
COUNT(*) FILTER (WHERE scope = 'outer')::BIGINT AS outer_instruction_count,
COUNT(*) FILTER (WHERE scope = 'top_level')::BIGINT AS top_level_instruction_count,
COUNT(*) FILTER (WHERE scope = 'inner')::BIGINT AS inner_instruction_count,
COUNT(*) FILTER (WHERE scope = 'logs')::BIGINT AS log_count,
MIN(slot)::BIGINT AS min_slot,
@@ -152,10 +152,10 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
};
let mut output = std::vec::Vec::with_capacity(rows.len());
for row in rows {
output.push(crate::PostgresReplayProgramSummary {
output.push(crate::ReplayProgramSummary {
program_id: row.program_id,
transaction_count: row.transaction_count,
outer_instruction_count: row.outer_instruction_count,
top_level_instruction_count: row.top_level_instruction_count,
inner_instruction_count: row.inner_instruction_count,
log_count: row.log_count,
min_slot: row.min_slot,
@@ -165,10 +165,10 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
return std::result::Result::Ok(output);
}
pub(in crate::postgres) async fn list_replay_entity_summaries(
pub(crate) async fn list_replay_entity_summaries(
pool: &sqlx::PgPool,
filter: &crate::PostgresReplayEntityFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
filter: &crate::ReplayEntityFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayEntitySummary>> {
let entity_kind = filter.entity_kind.as_sql();
let query_result = sqlx::query_as::<
sqlx::Postgres,
@@ -214,7 +214,7 @@ pub(in crate::postgres) async fn list_replay_entity_summaries(
};
let mut output = std::vec::Vec::with_capacity(rows.len());
for row in rows {
output.push(crate::PostgresReplayEntitySummary {
output.push(crate::ReplayEntitySummary {
entity_kind: row.entity_kind,
entity_value: row.entity_value,
transaction_count: row.transaction_count,
@@ -245,9 +245,9 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
COALESCE(ledger.status, 'not_started') AS ledger_status,
ledger.processor_version,
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
COALESCE(top_level_stats.instruction_count, 0)::BIGINT AS top_level_instruction_count,
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
COALESCE(top_level_stats.program_count, 0)::BIGINT AS top_level_program_count,
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
raw.updated_at::TEXT AS updated_at
FROM kb_sol_raw_transactions raw
@@ -265,7 +265,7 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
FROM kb_sol_core_instructions
WHERE signature = raw.signature
) outer_stats ON TRUE
) top_level_stats ON TRUE
LEFT JOIN LATERAL (
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
FROM kb_sol_core_inner_instructions
@@ -278,11 +278,11 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
AND ($6::TEXT IS NULL OR
($7 = 'any' AND (
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
)) OR
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
($7 = 'top_level' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6)) OR
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
AND ($8::TEXT IS NULL OR
@@ -313,9 +313,9 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
COALESCE(ledger.status, 'not_started') AS ledger_status,
ledger.processor_version,
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
COALESCE(top_level_stats.instruction_count, 0)::BIGINT AS top_level_instruction_count,
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
COALESCE(top_level_stats.program_count, 0)::BIGINT AS top_level_program_count,
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
raw.updated_at::TEXT AS updated_at
FROM kb_sol_raw_transactions raw
@@ -333,7 +333,7 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
FROM kb_sol_core_instructions
WHERE signature = raw.signature
) outer_stats ON TRUE
) top_level_stats ON TRUE
LEFT JOIN LATERAL (
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
FROM kb_sol_core_inner_instructions
@@ -346,11 +346,11 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
AND ($6::TEXT IS NULL OR
($7 = 'any' AND (
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
)) OR
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
($7 = 'top_level' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6)) OR
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
AND ($8::TEXT IS NULL OR
@@ -400,7 +400,7 @@ mod tests {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_error) => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
let options = match options_result {
std::result::Result::Ok(value) => value,
@@ -419,14 +419,14 @@ mod tests {
if let std::result::Result::Err(error) = core_schema_result {
panic!("unexpected core schema error: {error}");
}
let transaction_filter_result = crate::PostgresReplayTransactionFilter::new(
let transaction_filter_result = crate::ReplayTransactionFilter::new(
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
std::option::Option::None,
crate::PostgresReplayProgramScope::Any,
crate::ReplayProgramScope::Any,
std::option::Option::None,
std::option::Option::None,
10,
@@ -442,8 +442,7 @@ mod tests {
if let std::result::Result::Err(error) = transaction_result {
panic!("unexpected transaction candidate query error: {error}");
}
let program_filter_result =
crate::PostgresReplayProgramFilter::new(std::option::Option::None, 10);
let program_filter_result = crate::ReplayProgramFilter::new(std::option::Option::None, 10);
let program_filter = match program_filter_result {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => panic!("unexpected program filter error: {error}"),
@@ -453,12 +452,12 @@ mod tests {
panic!("unexpected program summary query error: {error}");
}
for entity_kind in [
crate::PostgresReplayEntityKind::Mint,
crate::PostgresReplayEntityKind::Owner,
crate::PostgresReplayEntityKind::AccountKey,
crate::ReplayEntityKind::Mint,
crate::ReplayEntityKind::Owner,
crate::ReplayEntityKind::AccountKey,
] {
let entity_filter_result =
crate::PostgresReplayEntityFilter::new(entity_kind, std::option::Option::None, 10);
crate::ReplayEntityFilter::new(entity_kind, std::option::Option::None, 10);
let entity_filter = match entity_filter_result {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => {

View File

@@ -1,14 +1,11 @@
// file: ks-store/src/postgres/query/table_diagnostics_queries.rs
// version: 3
// version: 6
//! Read-only PostgreSQL diagnostics for known Solana store tables.
use sqlx::Row; // rust-rules: trait-import
pub(in crate::postgres) async fn table_exists(
pool: &sqlx::PgPool,
table_name: &str,
) -> ks_core::Result<bool> {
pub(crate) async fn table_exists(pool: &sqlx::PgPool, table_name: &str) -> ks_core::Result<bool> {
let query_result =
sqlx::query_scalar::<sqlx::Postgres, bool>("SELECT to_regclass($1)::text IS NOT NULL")
.bind(table_name)
@@ -22,111 +19,111 @@ pub(in crate::postgres) async fn table_exists(
};
}
pub(in crate::postgres) async fn load_table_statistics(
pub(crate) async fn load_table_statistics(
pool: &sqlx::PgPool,
table_name: &str,
) -> ks_core::Result<crate::PostgresTableStatistics> {
) -> ks_core::Result<crate::StoreResourceStatistics> {
return match table_name {
crate::RAW_TRANSACTIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_raw_transactions_sql(),
crate::table_stats_kb_sol_raw_transactions_sql(),
table_name,
)
.await
},
crate::TRANSACTION_OBSERVATIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_obs_transaction_observations_sql(),
crate::table_stats_kb_sol_obs_transaction_observations_sql(),
table_name,
)
.await
},
crate::CORE_TRANSACTIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_transactions_sql(),
crate::table_stats_kb_sol_core_transactions_sql(),
table_name,
)
.await
},
crate::CORE_ACCOUNT_KEYS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_account_keys_sql(),
crate::table_stats_kb_sol_core_account_keys_sql(),
table_name,
)
.await
},
crate::CORE_INSTRUCTIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_instructions_sql(),
crate::table_stats_kb_sol_core_instructions_sql(),
table_name,
)
.await
},
crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_inner_instructions_sql(),
crate::table_stats_kb_sol_core_inner_instructions_sql(),
table_name,
)
.await
},
crate::CORE_LOGS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_logs_sql(),
crate::table_stats_kb_sol_core_logs_sql(),
table_name,
)
.await
},
crate::CORE_BALANCE_CHANGES_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_core_balance_changes_sql(),
crate::table_stats_kb_sol_core_balance_changes_sql(),
table_name,
)
.await
},
crate::PROCESSING_LEDGER_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_ops_processing_ledger_sql(),
crate::table_stats_kb_sol_ops_processing_ledger_sql(),
table_name,
)
.await
},
crate::DECODE_EVENTS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_decode_events_sql(),
crate::table_stats_kb_sol_decode_events_sql(),
table_name,
)
.await
},
crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_declarations_sql(),
crate::table_stats_kb_sol_decode_coverage_declarations_sql(),
table_name,
)
.await
},
crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_observations_sql(),
crate::table_stats_kb_sol_decode_coverage_observations_sql(),
table_name,
)
.await
},
crate::MATERIALIZED_EVENTS_TABLE_NAME => {
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
load_table_statistics_from_sql(
pool,
crate::postgres::migrations::table_stats_kb_sol_mat_events_sql(),
crate::table_stats_kb_sol_mat_events_sql(),
table_name,
)
.await
@@ -141,7 +138,7 @@ async fn load_table_statistics_from_sql(
pool: &sqlx::PgPool,
sql: &'static str,
table_name: &str,
) -> ks_core::Result<crate::PostgresTableStatistics> {
) -> ks_core::Result<crate::StoreResourceStatistics> {
let query_result = sqlx::query(sql).fetch_one(pool).await;
let row = match query_result {
std::result::Result::Ok(value) => value,
@@ -188,8 +185,8 @@ async fn load_table_statistics_from_sql(
)));
},
};
return std::result::Result::Ok(crate::PostgresTableStatistics {
row_count,
return std::result::Result::Ok(crate::StoreResourceStatistics {
record_count: row_count,
min_slot,
max_slot,
latest_created_at,

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/repository/core_extraction_repository.rs
// version: 2
// version: 3
//! PostgreSQL atomic canonical transaction to core extraction repository.
@@ -13,11 +13,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
&self,
filter: &crate::CoreExtractionSelectionFilter,
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>> {
return crate::postgres::query::list_raw_transactions_for_core_extraction(
self.pool(),
filter,
)
.await;
return crate::list_raw_transactions_for_core_extraction(self.pool(), filter).await;
}
#[expect(
@@ -28,7 +24,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
&self,
identity: &crate::ProcessingLedgerIdentity,
) -> ks_core::Result<bool> {
return crate::postgres::query::is_core_extraction_current(self.pool(), identity).await;
return crate::is_core_extraction_current(self.pool(), identity).await;
}
#[expect(
@@ -40,8 +36,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
bundle: &crate::CoreExtractionBundle,
force_replay: bool,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::persist_core_extraction(self.pool(), bundle, force_replay)
.await;
return crate::persist_core_extraction(self.pool(), bundle, force_replay).await;
}
#[expect(
@@ -52,6 +47,6 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
&self,
failure: &crate::CoreExtractionFailure,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::mark_core_extraction_failed(self.pool(), failure).await;
return crate::mark_core_extraction_failed(self.pool(), failure).await;
}
}

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/repository/core_transaction_repository.rs
// version: 2
// version: 3
//! PostgreSQL core Solana repository implementation.
@@ -13,7 +13,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
input: &crate::CoreTransactionInsert,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_transaction(self.pool(), input).await;
return crate::insert_core_transaction(self.pool(), input).await;
}
#[expect(
@@ -24,7 +24,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
inputs: &[crate::CoreAccountKeyInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_account_keys(self.pool(), inputs).await;
return crate::insert_core_account_keys(self.pool(), inputs).await;
}
#[expect(
@@ -35,7 +35,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
inputs: &[crate::CoreInstructionInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_instructions(self.pool(), inputs).await;
return crate::insert_core_instructions(self.pool(), inputs).await;
}
#[expect(
@@ -46,7 +46,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
inputs: &[crate::CoreInnerInstructionInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_inner_instructions(self.pool(), inputs).await;
return crate::insert_core_inner_instructions(self.pool(), inputs).await;
}
#[expect(
@@ -57,7 +57,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
inputs: &[crate::CoreLogInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_logs(self.pool(), inputs).await;
return crate::insert_core_logs(self.pool(), inputs).await;
}
#[expect(
@@ -68,7 +68,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
inputs: &[crate::CoreBalanceChangeInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_core_balance_changes(self.pool(), inputs).await;
return crate::insert_core_balance_changes(self.pool(), inputs).await;
}
#[expect(
@@ -80,12 +80,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
filter: &crate::CoreInstructionReplayFilter,
page_request: &crate::PageRequest,
) -> ks_core::Result<std::vec::Vec<crate::CoreInstructionRow>> {
return crate::postgres::query::list_core_instructions_for_replay(
self.pool(),
filter,
page_request,
)
.await;
return crate::list_core_instructions_for_replay(self.pool(), filter, page_request).await;
}
#[expect(
@@ -97,12 +92,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
filter: &crate::CoreInstructionReplayFilter,
page_request: &crate::PageRequest,
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
return crate::postgres::query::list_core_instruction_replay_inputs(
self.pool(),
filter,
page_request,
)
.await;
return crate::list_core_instruction_replay_inputs(self.pool(), filter, page_request).await;
}
#[expect(
@@ -113,6 +103,6 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
&self,
mark: &crate::CoreInstructionLifecycleMark,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::update_core_instruction_lifecycle(self.pool(), mark).await;
return crate::update_core_instruction_lifecycle(self.pool(), mark).await;
}
}

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/repository/decode_pipeline_repository.rs
// version: 2
// version: 3
//! PostgreSQL contextual decode and materialization repository.
@@ -13,7 +13,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
&self,
filter: &crate::DecodeSelectionFilter,
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
return crate::postgres::query::list_decode_inputs(self.pool(), filter).await;
return crate::list_decode_inputs(self.pool(), filter).await;
}
#[expect(
@@ -24,7 +24,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
&self,
identity: &crate::ProcessingLedgerIdentity,
) -> ks_core::Result<bool> {
return crate::postgres::query::is_decode_current(self.pool(), identity).await;
return crate::is_decode_current(self.pool(), identity).await;
}
#[expect(
@@ -35,11 +35,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
&self,
declarations: &[crate::DecodeCoverageDeclarationInsert],
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::persist_decode_coverage_declarations(
self.pool(),
declarations,
)
.await;
return crate::persist_decode_coverage_declarations(self.pool(), declarations).await;
}
#[expect(
@@ -51,8 +47,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
bundle: &crate::DecodePersistenceBundle,
force_replay: bool,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::persist_decode_result(self.pool(), bundle, force_replay)
.await;
return crate::persist_decode_result(self.pool(), bundle, force_replay).await;
}
#[expect(
@@ -63,7 +58,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
&self,
failure: &crate::DecodeFailure,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::mark_decode_failed(self.pool(), failure).await;
return crate::mark_decode_failed(self.pool(), failure).await;
}
#[expect(
@@ -75,12 +70,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
bundle: &crate::MaterializationPersistenceBundle,
force_replay: bool,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::persist_materialization_result(
self.pool(),
bundle,
force_replay,
)
.await;
return crate::persist_materialization_result(self.pool(), bundle, force_replay).await;
}
#[expect(
@@ -93,7 +83,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
processor_version: std::option::Option<&str>,
limit: u32,
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>> {
return crate::postgres::query::list_decode_coverage_summary(
return crate::list_decode_coverage_summary(
self.pool(),
processor_name,
processor_version,
@@ -110,6 +100,6 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
&self,
filter: &crate::MaterializedEventFilter,
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>> {
return crate::postgres::query::list_materialized_events(self.pool(), filter).await;
return crate::list_materialized_events(self.pool(), filter).await;
}
}

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/repository/raw_transaction_repository.rs
// version: 2
// version: 3
//! PostgreSQL canonical transaction and acquisition observation repository implementation.
@@ -13,11 +13,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
&self,
signature: &ks_lib::MdSignature,
) -> ks_core::Result<bool> {
return crate::postgres::query::has_raw_transaction_signature(
self.pool(),
signature.0.as_str(),
)
.await;
return crate::has_raw_transaction_signature(self.pool(), signature.0.as_str()).await;
}
#[expect(
@@ -28,11 +24,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
&self,
observation_key: &str,
) -> ks_core::Result<bool> {
return crate::postgres::query::has_transaction_observation_key(
self.pool(),
observation_key,
)
.await;
return crate::has_transaction_observation_key(self.pool(), observation_key).await;
}
#[expect(
@@ -43,7 +35,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
&self,
input: &crate::RawTransactionInsert,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_raw_transaction(self.pool(), input).await;
return crate::insert_raw_transaction(self.pool(), input).await;
}
#[expect(
@@ -54,7 +46,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
&self,
input: &crate::TransactionObservationInsert,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::insert_transaction_observation(self.pool(), input).await;
return crate::insert_transaction_observation(self.pool(), input).await;
}
#[expect(
@@ -65,6 +57,6 @@ impl crate::RawTransactionStore for crate::PostgresStore {
&self,
mark: &crate::RawPayloadLifecycleMark,
) -> ks_core::Result<crate::InsertOutcome> {
return crate::postgres::query::update_raw_payload_lifecycle(self.pool(), mark).await;
return crate::update_raw_payload_lifecycle(self.pool(), mark).await;
}
}

View File

@@ -1,42 +1,131 @@
// file: ks-store/src/postgres/store.rs
// version: 4
// version: 9
//! Store implementation scaffold for the `ks-store` crate.
//! PostgreSQL store implementation kept behind the backend-agnostic `Store` facade.
/// PostgreSQL store connection options.
#[derive(Clone, Debug, Eq, PartialEq)]
pub struct PostgresStoreOptions {
/// Database URL or DSN.
pub database_url: std::string::String,
/// Maximum connection count.
pub max_connections: u32,
/// Connection timeout in milliseconds.
pub connect_timeout_ms: u64,
/// Enables idempotent raw schema initialization at startup.
pub auto_initialize_schema: bool,
/// PostgreSQL store connection options interpreted only inside `ks-store`.
#[derive(Clone, Eq, PartialEq)]
pub(crate) struct PostgresStoreOptions {
database_url: std::string::String,
max_connections: u32,
connect_timeout_ms: u64,
auto_initialize_schema: bool,
option_count: u32,
}
impl PostgresStoreOptions {
/// Creates validated PostgreSQL store options.
pub fn new(
impl crate::PostgresStoreOptions {
/// Creates validated PostgreSQL store options for crate-internal tests and adapters.
#[cfg(test)]
pub(crate) fn new(
database_url: impl std::convert::Into<std::string::String>,
max_connections: u32,
connect_timeout_ms: u64,
auto_initialize_schema: bool,
) -> ks_core::Result<Self> {
return Self::new_with_option_count(
database_url,
max_connections,
connect_timeout_ms,
auto_initialize_schema,
4,
);
}
/// Interprets the selected opaque backend options supplied to `Store::open`.
pub(crate) fn from_backend_options(options: &serde_json::Value) -> ks_core::Result<Self> {
let object = match options.as_object() {
std::option::Option::Some(value) => value,
std::option::Option::None => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_invalid",
"postgres backend options must be a JSON object",
));
},
};
let database_url = match object.get("url").and_then(serde_json::Value::as_str) {
std::option::Option::Some(value) => value.to_string(),
std::option::Option::None => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_incomplete",
"postgres backend requires a non-empty connection URL",
));
},
};
let max_connections_u64 =
match object.get("max_connections").and_then(serde_json::Value::as_u64) {
std::option::Option::Some(value) => value,
std::option::Option::None => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_incomplete",
"postgres backend requires max_connections greater than zero",
));
},
};
let max_connections = match u32::try_from(max_connections_u64) {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_) => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_invalid",
"postgres max_connections does not fit into u32",
));
},
};
let connect_timeout_ms =
match object.get("connect_timeout_ms").and_then(serde_json::Value::as_u64) {
std::option::Option::Some(value) => value,
std::option::Option::None => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_incomplete",
"postgres backend requires connect_timeout_ms greater than zero",
));
},
};
let auto_initialize_schema =
match object.get("auto_initialize_schema").and_then(serde_json::Value::as_bool) {
std::option::Option::Some(value) => value,
std::option::Option::None => {
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_incomplete",
"postgres backend requires auto_initialize_schema",
));
},
};
let option_count = match u32::try_from(object.len()) {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_) => u32::MAX,
};
return Self::new_with_option_count(
database_url,
max_connections,
connect_timeout_ms,
auto_initialize_schema,
option_count,
);
}
fn new_with_option_count(
database_url: impl std::convert::Into<std::string::String>,
max_connections: u32,
connect_timeout_ms: u64,
auto_initialize_schema: bool,
option_count: u32,
) -> ks_core::Result<Self> {
let database_url_value = database_url.into();
if database_url_value.trim().is_empty() {
return std::result::Result::Err(ks_core::Error::db(
"postgres database url must not be empty",
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_incomplete",
"postgres backend requires a non-empty connection URL",
));
}
if max_connections == 0 {
return std::result::Result::Err(ks_core::Error::db(
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_invalid",
"postgres max_connections must be greater than zero",
));
}
if connect_timeout_ms == 0 {
return std::result::Result::Err(ks_core::Error::db(
return std::result::Result::Err(crate::storage_contract_error(
"store_backend_options_invalid",
"postgres connect_timeout_ms must be greater than zero",
));
}
@@ -45,173 +134,142 @@ impl PostgresStoreOptions {
max_connections,
connect_timeout_ms,
auto_initialize_schema,
option_count,
});
}
/// Returns whether automatic schema initialization is enabled.
pub(crate) fn auto_initialize_schema(&self) -> bool {
return self.auto_initialize_schema;
}
/// Returns a backend-neutral safe configuration summary.
pub(crate) fn configuration_summary(&self) -> crate::StoreConfigurationSummary {
return crate::StoreConfigurationSummary {
enabled: true,
backend_code: "postgres".to_string(),
connection_configured: !self.database_url.trim().is_empty(),
auto_initialize_schema: self.auto_initialize_schema,
backend_option_count: self.option_count,
};
}
/// Returns a DSN masked for diagnostics.
pub fn masked_dsn(&self) -> std::string::String {
return crate::mask_postgres_dsn(self.database_url.as_str());
pub(crate) fn masked_connection_descriptor(&self) -> std::string::String {
return mask_postgres_dsn(self.database_url.as_str());
}
}
/// PostgreSQL diagnostic snapshot.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresBackendDiagnostics {
/// Backend descriptor safe for UI display.
pub descriptor: crate::StoreBackendDescriptor,
/// Backend health snapshot.
pub health: crate::StoreHealthSnapshot,
/// Migration status snapshot.
pub migrations: crate::StoreMigrationSnapshot,
/// Full PostgreSQL server version string when available.
pub server_version: std::option::Option<std::string::String>,
}
/// Read-only statistics for one PostgreSQL table.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresTableStatistics {
/// Number of rows currently stored in the table.
pub row_count: i64,
/// Lowest observed Solana slot when the table contains a slot column and rows.
pub min_slot: std::option::Option<i64>,
/// Highest observed Solana slot when the table contains a slot column and rows.
pub max_slot: std::option::Option<i64>,
/// Latest insertion timestamp rendered by PostgreSQL for UI diagnostics.
pub latest_created_at: std::option::Option<std::string::String>,
}
/// Read-only diagnostics for one expected PostgreSQL table.
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
pub struct PostgresTableDiagnostics {
/// Expected table name.
pub table_name: std::string::String,
/// Logical Solana domain encoded in the table name.
pub domain: std::string::String,
/// Human-readable role of the table.
pub role: std::string::String,
/// Whether the table exists in the current PostgreSQL search path.
pub exists: bool,
/// Table statistics when the table exists.
pub statistics: std::option::Option<crate::PostgresTableStatistics>,
}
/// PostgreSQL store handle.
#[derive(Clone, Debug)]
pub struct PostgresStore {
/// PostgreSQL store handle kept private to `ks-store`.
#[derive(Clone)]
pub(crate) struct PostgresStore {
options: crate::PostgresStoreOptions,
pool: sqlx::PgPool,
}
impl PostgresStore {
/// Connects to PostgreSQL from typed store options.
pub async fn connect(options: crate::PostgresStoreOptions) -> ks_core::Result<Self> {
impl crate::PostgresStore {
/// Connects to PostgreSQL from validated backend options.
pub(crate) async fn connect(options: crate::PostgresStoreOptions) -> ks_core::Result<Self> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", "open PostgreSQL store connection");
let pool_options = sqlx::postgres::PgPoolOptions::new()
.max_connections(options.max_connections)
.acquire_timeout(std::time::Duration::from_millis(options.connect_timeout_ms));
let connect_result = pool_options.connect(options.database_url.as_str()).await;
return match connect_result {
std::result::Result::Ok(pool) => {
let store = Self { options, pool };
if store.options.auto_initialize_schema {
let schema_result = store.initialize_store_schema().await;
if let std::result::Result::Err(error) = schema_result {
return std::result::Result::Err(error);
}
}
std::result::Result::Ok(store)
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", connected = true, "PostgreSQL store connection opened");
std::result::Result::Ok(Self { options, pool })
},
std::result::Result::Err(_error) => {
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", connected = false, "PostgreSQL store connection failed");
std::result::Result::Err(crate::storage_contract_error(
"store_backend_connection_failed",
"postgres connection failed",
))
},
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(
format!("postgres connection failed: {error}"),
)),
};
}
/// Creates a store handle from an existing PostgreSQL pool.
pub fn from_pool(options: crate::PostgresStoreOptions, pool: sqlx::PgPool) -> Self {
return Self { options, pool };
}
/// Returns the underlying PostgreSQL pool.
pub fn pool(&self) -> &sqlx::PgPool {
/// Returns the underlying PostgreSQL pool to backend-private repositories.
pub(crate) fn pool(&self) -> &sqlx::PgPool {
return &self.pool;
}
/// Returns the connection options used to create this store.
pub fn options(&self) -> &crate::PostgresStoreOptions {
return &self.options;
}
/// Applies each idempotent store schema once per invocation in dependency order.
pub async fn initialize_store_schema(&self) -> ks_core::Result<()> {
let raw_result = crate::postgres::query::apply_raw_store_schema(&self.pool).await;
pub(crate) async fn initialize_store_schema(&self) -> ks_core::Result<()> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "schema_initialize", "initialize PostgreSQL store schema");
let raw_result = crate::apply_raw_store_schema(&self.pool).await;
if let std::result::Result::Err(error) = raw_result {
return std::result::Result::Err(error);
}
let core_result = crate::postgres::query::apply_core_store_schema(&self.pool).await;
let core_result = crate::apply_core_store_schema(&self.pool).await;
if let std::result::Result::Err(error) = core_result {
return std::result::Result::Err(error);
}
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
let result = crate::apply_decode_store_schema(&self.pool).await;
if result.is_ok() {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "schema_initialize", initialized = true, "PostgreSQL store schema initialized");
}
return result;
}
/// Applies the idempotent minimal raw Solana store schema.
pub async fn initialize_raw_store_schema(&self) -> ks_core::Result<()> {
return crate::postgres::query::apply_raw_store_schema(&self.pool).await;
#[cfg(test)]
pub(crate) async fn initialize_raw_store_schema(&self) -> ks_core::Result<()> {
return crate::apply_raw_store_schema(&self.pool).await;
}
/// Applies the idempotent minimal core Solana store schema.
pub async fn initialize_core_store_schema(&self) -> ks_core::Result<()> {
/// Applies the idempotent minimal Core Solana store schema.
#[cfg(test)]
pub(crate) async fn initialize_core_store_schema(&self) -> ks_core::Result<()> {
let raw_result = self.initialize_raw_store_schema().await;
if let std::result::Result::Err(error) = raw_result {
return std::result::Result::Err(error);
}
return crate::postgres::query::apply_core_store_schema(&self.pool).await;
}
/// Applies the idempotent common decode and materialization store schema.
pub async fn initialize_decode_store_schema(&self) -> ks_core::Result<()> {
let core_result = self.initialize_core_store_schema().await;
if let std::result::Result::Err(error) = core_result {
return std::result::Result::Err(error);
}
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
return crate::apply_core_store_schema(&self.pool).await;
}
/// Reads a UI-safe backend descriptor.
pub async fn backend_descriptor(&self) -> ks_core::Result<crate::StoreBackendDescriptor> {
let schema_result = crate::postgres::query::load_current_schema(&self.pool).await;
pub(crate) async fn backend_descriptor(
&self,
) -> ks_core::Result<crate::StoreBackendDescriptor> {
let schema_result = crate::load_current_schema(&self.pool).await;
return match schema_result {
std::result::Result::Ok(schema) => crate::StoreBackendDescriptor::new(
crate::StoreBackendKind::Postgres,
"postgres",
std::option::Option::Some(self.options.masked_dsn()),
"PostgreSQL",
std::option::Option::Some(self.options.masked_connection_descriptor()),
std::option::Option::Some(schema),
),
std::result::Result::Err(error) => std::result::Result::Err(error),
};
}
/// Reads a PostgreSQL health snapshot.
pub async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
let health_result = crate::postgres::query::run_health_check(&self.pool).await;
/// Reads a PostgreSQL health snapshot without exposing backend error details.
pub(crate) async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "health_check", "run PostgreSQL store health check");
let health_result = crate::run_health_check(&self.pool).await;
return match health_result {
std::result::Result::Ok(()) => crate::StoreHealthSnapshot::new(
"postgres",
crate::StoreHealthStatus::Healthy,
std::option::Option::Some(std::string::String::from("SELECT 1 succeeded")),
std::option::Option::Some(std::string::String::from(
"backend health check succeeded",
)),
),
std::result::Result::Err(error) => crate::StoreHealthSnapshot::new(
std::result::Result::Err(_error) => crate::StoreHealthSnapshot::new(
"postgres",
crate::StoreHealthStatus::Unhealthy,
std::option::Option::Some(error.to_string()),
std::option::Option::Some(std::string::String::from("backend health check failed")),
),
};
}
/// Reads a non-destructive migration snapshot.
pub async fn migration_snapshot(&self) -> ks_core::Result<crate::StoreMigrationSnapshot> {
let migration_table_result =
crate::postgres::query::load_migration_table_name(&self.pool).await;
pub(crate) async fn migration_snapshot(
&self,
) -> ks_core::Result<crate::StoreMigrationSnapshot> {
let migration_table_result = crate::load_migration_table_name(&self.pool).await;
return match migration_table_result {
std::result::Result::Ok(std::option::Option::None) => {
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
@@ -219,7 +277,7 @@ impl PostgresStore {
std::option::Option::None,
std::vec::Vec::new(),
std::option::Option::Some(std::string::String::from(
"no sqlx migration table detected; 0.3.1 canonical acquisition/core schemas use idempotent crate-managed DDL",
"no migration history table detected; crate-managed schema initialization is active",
)),
))
},
@@ -230,132 +288,121 @@ impl PostgresStore {
};
}
/// Reads a complete PostgreSQL diagnostic snapshot.
pub async fn backend_diagnostics(&self) -> ks_core::Result<crate::PostgresBackendDiagnostics> {
let descriptor_result = self.backend_descriptor().await;
let descriptor = match descriptor_result {
/// Reads a complete backend-neutral diagnostic snapshot.
pub(crate) async fn backend_diagnostics(
&self,
) -> ks_core::Result<crate::StoreBackendDiagnostics> {
let descriptor = match self.backend_descriptor().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
let health_result = self.health_snapshot().await;
let health = match health_result {
let health = match self.health_snapshot().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
let migrations_result = self.migration_snapshot().await;
let migrations = match migrations_result {
let migrations = match self.migration_snapshot().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
let server_version = match crate::postgres::query::load_server_version(&self.pool).await {
let backend_version = match crate::load_server_version(&self.pool).await {
std::result::Result::Ok(value) => std::option::Option::Some(value),
std::result::Result::Err(_error) => std::option::Option::None,
};
return std::result::Result::Ok(crate::PostgresBackendDiagnostics {
return std::result::Result::Ok(crate::StoreBackendDiagnostics {
descriptor,
health,
migrations,
server_version,
backend_version,
});
}
/// Lists bounded raw transaction candidates enriched with core and ledger diagnostics.
pub async fn replay_transaction_candidates(
/// Lists bounded raw transaction candidates enriched with Core and ledger diagnostics.
pub(crate) async fn replay_transaction_candidates(
&self,
filter: &crate::PostgresReplayTransactionFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
return crate::postgres::query::list_replay_transaction_candidates(&self.pool, filter)
.await;
filter: &crate::ReplayTransactionFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayTransactionCandidate>> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_transaction_candidates", "query PostgreSQL replay transaction candidates");
return crate::list_replay_transaction_candidates(&self.pool, filter).await;
}
/// Lists bounded program summaries across outer, inner and reliably linked logs.
pub async fn replay_program_summaries(
/// Lists bounded program summaries across top-level, inner and reliably linked logs.
pub(crate) async fn replay_program_summaries(
&self,
filter: &crate::PostgresReplayProgramFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
return crate::postgres::query::list_replay_program_summaries(&self.pool, filter).await;
filter: &crate::ReplayProgramFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayProgramSummary>> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_program_summaries", "query PostgreSQL replay program summaries");
return crate::list_replay_program_summaries(&self.pool, filter).await;
}
/// Lists bounded mint, owner or account-key summaries from core tables.
pub async fn replay_entity_summaries(
/// Lists bounded mint, owner or account-key summaries from Core facts.
pub(crate) async fn replay_entity_summaries(
&self,
filter: &crate::PostgresReplayEntityFilter,
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
return crate::postgres::query::list_replay_entity_summaries(&self.pool, filter).await;
filter: &crate::ReplayEntityFilter,
) -> ks_core::Result<std::vec::Vec<crate::ReplayEntitySummary>> {
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_entity_summaries", "query PostgreSQL replay entity summaries");
return crate::list_replay_entity_summaries(&self.pool, filter).await;
}
/// Reads diagnostics for raw Solana store tables without changing the schema.
pub async fn raw_table_diagnostics(
/// Reads diagnostics for raw store resources without changing the schema.
pub(crate) async fn raw_resource_diagnostics(
&self,
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
let specs = crate::raw_store_table_diagnostic_specs();
return self.table_diagnostics(&specs).await;
return self.resource_diagnostics(&specs).await;
}
/// Reads diagnostics for core Solana store tables without changing the schema.
pub async fn core_table_diagnostics(
/// Reads diagnostics for Core and processing store resources without changing the schema.
pub(crate) async fn core_resource_diagnostics(
&self,
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
let specs = crate::core_store_table_diagnostic_specs();
return self.table_diagnostics(&specs).await;
return self.resource_diagnostics(&specs).await;
}
/// Reads diagnostics for decode and materialization store tables without changing the schema.
pub async fn decode_table_diagnostics(
/// Reads diagnostics for decode and materialization resources without changing the schema.
pub(crate) async fn decode_resource_diagnostics(
&self,
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
let specs = crate::decode_store_table_diagnostic_specs();
return self.table_diagnostics(&specs).await;
return self.resource_diagnostics(&specs).await;
}
/// Reads diagnostics for every known raw/core/decode Solana store table.
pub async fn known_table_diagnostics(
/// Reads diagnostics for every known logical store resource.
pub(crate) async fn known_resource_diagnostics(
&self,
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
let mut diagnostics = std::vec::Vec::new();
let raw_result = self.raw_table_diagnostics().await;
let raw_tables = match raw_result {
let raw_resources = match self.raw_resource_diagnostics().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
for table in raw_tables {
diagnostics.push(table);
}
let core_result = self.core_table_diagnostics().await;
let core_tables = match core_result {
diagnostics.extend(raw_resources);
let core_resources = match self.core_resource_diagnostics().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
for table in core_tables {
diagnostics.push(table);
}
let decode_result = self.decode_table_diagnostics().await;
let decode_tables = match decode_result {
diagnostics.extend(core_resources);
let decode_resources = match self.decode_resource_diagnostics().await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
for table in decode_tables {
diagnostics.push(table);
}
diagnostics.extend(decode_resources);
return std::result::Result::Ok(diagnostics);
}
async fn table_diagnostics(
async fn resource_diagnostics(
&self,
specs: &[crate::PostgresTableDiagnosticSpec],
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
let mut diagnostics = std::vec::Vec::new();
for spec in specs {
let exists_result =
crate::postgres::query::table_exists(&self.pool, spec.table_name).await;
let exists = match exists_result {
let available = match crate::table_exists(&self.pool, spec.table_name).await {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => return std::result::Result::Err(error),
};
let statistics = if exists {
let statistics = if available {
let statistics_result =
crate::postgres::query::load_table_statistics(&self.pool, spec.table_name)
.await;
crate::load_table_statistics(&self.pool, spec.table_name).await;
match statistics_result {
std::result::Result::Ok(value) => std::option::Option::Some(value),
std::result::Result::Err(error) => return std::result::Result::Err(error),
@@ -363,11 +410,11 @@ impl PostgresStore {
} else {
std::option::Option::None
};
diagnostics.push(crate::PostgresTableDiagnostics {
table_name: spec.table_name.to_string(),
domain: spec.domain.to_string(),
diagnostics.push(crate::StoreResourceDiagnostics {
resource_code: spec.resource_code.to_string(),
model_code: spec.model_code.to_string(),
role: spec.role.to_string(),
exists,
available,
statistics,
});
}
@@ -377,8 +424,7 @@ impl PostgresStore {
async fn migration_snapshot_from_existing_table(
&self,
) -> ks_core::Result<crate::StoreMigrationSnapshot> {
let version_result =
crate::postgres::query::load_latest_migration_version(&self.pool).await;
let version_result = crate::load_latest_migration_version(&self.pool).await;
return match version_result {
std::result::Result::Ok(current_version) => {
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
@@ -386,7 +432,7 @@ impl PostgresStore {
current_version,
std::vec::Vec::new(),
std::option::Option::Some(std::string::String::from(
"sqlx migration table detected; canonical acquisition/core schema remains idempotent and crate-managed in 0.3.1",
"migration history table detected",
)),
))
},
@@ -395,22 +441,18 @@ impl PostgresStore {
}
}
/// Returns a DSN masked for logs and UI diagnostics.
pub fn mask_postgres_dsn(dsn: &str) -> std::string::String {
/// Returns a PostgreSQL connection descriptor masked for logs and diagnostics.
fn mask_postgres_dsn(dsn: &str) -> std::string::String {
let trimmed_dsn = dsn.trim();
if trimmed_dsn.is_empty() {
return std::string::String::from("");
}
let queryless = crate::postgres::store::strip_query(trimmed_dsn);
let queryless = strip_query(trimmed_dsn);
return match queryless.split_once("://") {
std::option::Option::Some((scheme, remainder)) => {
crate::postgres::store::mask_scheme_remainder(
scheme,
remainder,
trimmed_dsn.contains('?'),
)
mask_scheme_remainder(scheme, remainder, trimmed_dsn.contains('?'))
},
std::option::Option::None => crate::postgres::store::mask_plain_dsn(queryless.as_str()),
std::option::Option::None => mask_plain_dsn(queryless.as_str()),
};
}
@@ -422,7 +464,7 @@ fn strip_query(dsn: &str) -> std::string::String {
}
fn mask_scheme_remainder(scheme: &str, remainder: &str, had_query: bool) -> std::string::String {
let suffix = crate::postgres::store::query_suffix(had_query);
let suffix = query_suffix(had_query);
return match remainder.rsplit_once('@') {
std::option::Option::Some((_userinfo, host_path)) => {
format!("{scheme}://***:***@{host_path}{suffix}")
@@ -448,21 +490,51 @@ fn query_suffix(had_query: bool) -> std::string::String {
#[cfg(test)]
mod tests {
#[test]
fn options_reject_empty_database_url() {
fn backend_options_reject_empty_database_url() {
let result = crate::PostgresStoreOptions::new(" ", 1, 1000, false);
assert!(result.is_err());
}
#[test]
fn options_reject_zero_max_connections() {
fn backend_options_reject_zero_max_connections() {
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 0, 1000, false);
assert!(result.is_err());
}
#[test]
fn options_reject_zero_connect_timeout() {
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 1, 0, false);
assert!(result.is_err());
fn incomplete_backend_options_do_not_echo_secret_values() {
let options = serde_json::json!({
"url": "postgres://operator:STORE-SECRET-CANARY@localhost/db",
"connect_timeout_ms": 5000,
"auto_initialize_schema": true
});
let error = match crate::PostgresStoreOptions::from_backend_options(&options) {
std::result::Result::Ok(_) => panic!("incomplete backend options must be rejected"),
std::result::Result::Err(error) => error,
};
assert!(!error.to_string().contains("STORE-SECRET-CANARY"));
assert!(!error.to_string().contains("postgres://"));
}
#[test]
fn opaque_backend_options_are_sanitized_in_summary() {
let options = serde_json::json!({
"url": "postgres://operator:STORE-SECRET-CANARY@localhost/db",
"max_connections": 4,
"connect_timeout_ms": 5000,
"auto_initialize_schema": true
});
let parsed = crate::PostgresStoreOptions::from_backend_options(&options);
let value = match parsed {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => panic!("backend options must parse: {error}"),
};
let serialized = match serde_json::to_string(&value.configuration_summary()) {
std::result::Result::Ok(value) => value,
std::result::Result::Err(error) => panic!("summary must serialize: {error}"),
};
assert!(!serialized.contains("STORE-SECRET-CANARY"));
assert!(!serialized.contains("postgres://"));
}
#[tokio::test]
@@ -471,7 +543,7 @@ mod tests {
std::result::Result::Ok(value) => value,
std::result::Result::Err(_error) => return,
};
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
let _postgres_guard = crate::postgres_test_guard().await;
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
let options = match options_result {
std::result::Result::Ok(value) => value,
@@ -488,25 +560,24 @@ mod tests {
std::result::Result::Err(error) => panic!("unexpected health error: {error}"),
};
assert_eq!(health.status, crate::StoreHealthStatus::Healthy);
return;
}
#[test]
fn mask_postgres_dsn_masks_userinfo() {
let masked = crate::mask_postgres_dsn("postgres://user:secret@localhost:5432/db");
let masked = super::mask_postgres_dsn("postgres://user:secret@localhost:5432/db");
assert_eq!(masked, "postgres://***:***@localhost:5432/db");
}
#[test]
fn mask_postgres_dsn_masks_query_string() {
let masked =
crate::mask_postgres_dsn("postgres://localhost/db?sslmode=require&password=secret");
super::mask_postgres_dsn("postgres://localhost/db?sslmode=require&password=secret");
assert_eq!(masked, "postgres://localhost/db?<redacted>");
}
#[test]
fn mask_postgres_dsn_masks_plain_password_dsn() {
let masked = crate::mask_postgres_dsn("host=localhost password=secret dbname=khadhroony");
let masked = super::mask_postgres_dsn("host=localhost password=secret dbname=khadhroony");
assert_eq!(masked, "<postgres-dsn-redacted>");
}
}

View File

@@ -1,5 +1,5 @@
// file: ks-store/src/postgres/test_serial.rs
// version: 2
// version: 3
//! Test-only serialization helpers for optional real PostgreSQL tests.
@@ -7,7 +7,7 @@ static POSTGRES_TEST_MUTEX: std::sync::OnceLock<std::sync::Arc<tokio::sync::Mute
std::sync::OnceLock::new();
/// Acquires the process-local guard shared by optional real PostgreSQL tests.
pub(in crate::postgres) async fn postgres_test_guard() -> tokio::sync::OwnedMutexGuard<()> {
pub(crate) async fn postgres_test_guard() -> tokio::sync::OwnedMutexGuard<()> {
let mutex = POSTGRES_TEST_MUTEX
.get_or_init(|| return std::sync::Arc::new(tokio::sync::Mutex::new(())))
.clone();

1023
ks-store/src/store.rs Normal file

File diff suppressed because it is too large Load Diff