v0.5.3-pre.002
This commit is contained in:
@@ -1,8 +1,48 @@
|
||||
<!-- file: ks-store/CHANGELOG.md -->
|
||||
<!-- version: 19 -->
|
||||
<!-- version: 23 -->
|
||||
|
||||
# CHANGELOG — ks-store
|
||||
|
||||
## `0.5.3-pre.002-delta-fix-003`
|
||||
|
||||
- corrige deux appels de helpers strictement privés PostgreSQL qui avaient été qualifiés à tort via `crate::`; ils restent locaux à leur module et ne sont pas réexportés au crate-root ;
|
||||
- ne modifie aucun contrat public ni schéma SQL.
|
||||
|
||||
## `0.5.3-pre.002-delta-fix-002`
|
||||
|
||||
- remplace les visibilités `pub(in crate::postgres)` par la convention workspace : privé ou `pub(crate)` + réexport crate-root et consommation via `crate::Item` ;
|
||||
- ajoute le canari Rust interdisant `pub(in ...)`/`pub(super)` et corrige les reexports `#[cfg(test)]` de `ks-pipeline-demo-scenarios` ;
|
||||
- supprime les appels obsolètes à `Store::initialize_store_schema()` : l’initialisation reste possédée par `Store::open()` via la configuration backend ;
|
||||
- renomme structurellement les modules, fenêtres, commandes, HTML/TypeScript et DTO `demo_sql_*` en `demo_store_*`, sans attendre la refonte fonctionnelle finale de `pre.005` ;
|
||||
- réconcilie le tracing PostgreSQL avec la règle d’un target unique : `target: crate::TRACING_TARGET`, `backend="postgres"`, `domain="ks-store.pg"`, `action=...` ;
|
||||
- supprime les helpers/constantes transitoires morts signalés par `cargo check`/Clippy et l’attente Clippy devenue inutile.
|
||||
|
||||
## `0.5.3-pre.002-delta-fix-001`
|
||||
|
||||
- corrige les helpers PostgreSQL de validation de noms de tables rendus inaccessibles par la nouvelle frontière privée et aligne leurs appels/tests sur la façade interne `crate::postgres::*` ;
|
||||
- supprime les réexports PostgreSQL internes devenus inutiles qui produisaient des warnings et conserve le target générique `ks-store` en l'utilisant réellement à l'ouverture de la façade `Store` ;
|
||||
- aligne les nouveaux types publics/crate-publics introduits par `pre.002` sur la règle de qualification `crate::Item` dans leur crate propriétaire.
|
||||
|
||||
## `0.5.3-pre.002`
|
||||
|
||||
### Façade et configuration
|
||||
|
||||
- introduit `Store` et `StoreOpenOptions` comme frontière publique backend-agnostique et confine `PostgresStore`, `PostgresStoreOptions`, `PgPool`, les constantes physiques et les requêtes SQL à l'implémentation privée ;
|
||||
- remplace les sections runtime PostgreSQL/SQLite imposées par un code `backend` et un objet `backend_options` opaque jusqu'à son interprétation par `ks-store`, sans créer de dépendance `ks-store -> ks-config` ;
|
||||
- retire les quatre anciens traits/DTO historiques sans implémentation réelle et renomme les contrats publics de replay/diagnostic sans préfixe PostgreSQL ;
|
||||
- conserve `Store` dans l'état applicatif desktop via une initialisation unique et réutilisable par les commandes/scénarios.
|
||||
|
||||
### Diagnostics et sécurité
|
||||
|
||||
- ajoute les résumés génériques de configuration, backend, initialisation et vérification par modèle, sans exposition des noms physiques de tables/index ni des options backend brutes ;
|
||||
- masque le descripteur PostgreSQL et sanitise l'erreur de connexion afin qu'aucun DSN/password ne traverse les diagnostics normaux ;
|
||||
- introduit le domaine structuré PostgreSQL `ks-store.pg` sous l’unique `target: crate::TRACING_TARGET`, ainsi qu'un canari workspace empêchant les types PostgreSQL/SQLx de sortir de `ks-store` ;
|
||||
- adapte les consommateurs Rust et les payloads desktop aux ressources/diagnostics génériques ; le renommage structurel `demo_sql_* -> demo_store_*` est finalisé par `delta-fix-002`, tandis que la refonte fonctionnelle finale reste à la tranche consommateurs.
|
||||
|
||||
### Portée
|
||||
|
||||
- ne modifie aucune table ni migration SQL de production : le namespace `k_sol_*`, `block_time`, la symétrie top-level/CPI et les ressources SQL atomiques restent réservés à `pre.003`.
|
||||
|
||||
## `0.5.3-pre.001`
|
||||
|
||||
### Audit / plan
|
||||
@@ -16,7 +56,7 @@
|
||||
- normalise le vocabulaire d'instruction vers `top-level`, conserve les CPI séparées et exige `stack_height`/parent CPI immédiat lorsque reconstructible ;
|
||||
- ajoute un canari de gel future-decoder/Anchor-readiness : N1/N2/N3 doivent conserver les données Solana génériques nécessaires aux instructions/CPI, événements/logs, `returnData`, états/observations de comptes et provenance de schéma, sans ajouter de contrat `anchor_*` ; les 13 tables existantes restent une baseline et non un quota final si un manque générique est découvert avant gel ;
|
||||
- formalise la reconstruction `kb_sol_* -> k_sol_*`, l'arborescence `migrations/postgres/`, une instruction SQL par fichier exécutée via `include_str!`, l'audit des index/paginations/BIGINT et les règles de replay/invalidation ;
|
||||
- définit la façade de stockage backend-agnostique, l'encapsulation de PostgreSQL, la stratégie de configuration backend opaque jusqu'à `ks-store`, le résumé runtime/init sans secrets, la future réutilisation d'un `Store` persistant dans `AppState`, la refonte backend-agnostique de `demo_sql`/`demo_config`/splash et les targets `ks-store.pg.*` ;
|
||||
- définit la façade de stockage backend-agnostique, l'encapsulation de PostgreSQL, la stratégie de configuration backend opaque jusqu'à `ks-store`, le résumé runtime/init sans secrets, la future réutilisation d'un `Store` persistant dans `AppState`, le renommage structurel `demo_sql_* -> demo_store_*`, la refonte backend-agnostique de `demo_config`/splash et le domaine de tracing PostgreSQL `ks-store.pg` sous l’unique target `ks-store` ;
|
||||
- ajoute [`docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md`](../docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md) ;
|
||||
- ne crée aucune table DEX/trading ni projection metadata N4 dans `0.5.3` ;
|
||||
- considère `pre.007` comme premier candidat de clôture et autorise explicitement son décalage si des manques apparaissent ;
|
||||
|
||||
@@ -1,48 +1,56 @@
|
||||
<!-- file: ks-store/README.md -->
|
||||
<!-- version: 4 -->
|
||||
<!-- version: 6 -->
|
||||
|
||||
# ks-store
|
||||
|
||||
`ks-store` consolide les contrats de stockage et l’implémentation PostgreSQL de `khadhroony-bot3`.
|
||||
`ks-store` porte la frontière de persistance généraliste de Khadhroony Solana. Les consommateurs utilisent des contrats fonctionnels et une façade `Store` indépendante du moteur concret ; PostgreSQL reste l'implémentation opérationnelle de `0.5.3`, confinée à l'intérieur de la crate.
|
||||
|
||||
## Périmètre
|
||||
|
||||
La crate expose :
|
||||
|
||||
- DTO et entités raw, observations, Core, décodage, matérialisation et ledger ;
|
||||
- traits async de repositories indépendants du backend ;
|
||||
- pagination et filtres bornés ;
|
||||
- `PostgresStore` et ses options de connexion ;
|
||||
- initialisation idempotente du schéma et migrations SQL ;
|
||||
- diagnostics read-only, healthchecks et résumés de replay ;
|
||||
- transactions atomiques pour extraction, décodage et matérialisation.
|
||||
- `Store` et `StoreOpenOptions` comme façade d'ouverture et handle persistant backend-agnostique ;
|
||||
- les DTO et entités raw, observations, Core, décodage, matérialisation et ledger ;
|
||||
- les traits async `StoreHealthStore`, `RawTransactionStore`, `CoreTransactionStore`, `CoreExtractionStore` et `DecodePipelineStore` ;
|
||||
- les filtres et résultats de replay sans préfixe backend ;
|
||||
- des diagnostics et résumés runtime/init sanitisés ne révélant ni secret ni nom physique d'objet ;
|
||||
- des transactions atomiques pour extraction, décodage et matérialisation.
|
||||
|
||||
## Frontière backend
|
||||
|
||||
`PostgresStore`, `PostgresStoreOptions`, `sqlx::PgPool`, les requêtes SQL et les noms physiques de tables sont privés à `ks-store`. `ks-pipeline`, les scénarios et les applications ne doivent connaître que `Store`, les repositories et les DTO fonctionnels.
|
||||
|
||||
`ks-store` ne dépend pas de `ks-config`. Un consommateur lui fournit un backend sélectionné et un objet `backend_options` opaque ; seul `ks-store` interprète et valide les paramètres PostgreSQL puis crée la connexion.
|
||||
|
||||
## Configuration et sécurité
|
||||
|
||||
`StoreOpenOptions` ne dérive pas `Debug` et ne rend pas les options backend accessibles après construction. Les résumés publics exposent uniquement des informations explicitement sanitisées : code backend, présence d'une connexion, état d'initialisation, health, modèle logique et compteurs.
|
||||
|
||||
Le backend PostgreSQL masque le descripteur de connexion et utilise l’unique target de crate `ks-store`, complété par `backend="postgres"`, `domain="ks-store.pg"` et un champ `action` structuré. Aucun DSN, password ou valeur bindée n'est destiné aux diagnostics publics.
|
||||
|
||||
## Schéma pendant `0.5.3`
|
||||
|
||||
`0.5.3-pre.002` ne modifie pas encore les tables héritées `kb_sol_*`. Leur reconstruction sous `k_sol_*`, les ressources SQL atomiques et le gel N1/N2/N3 commencent en `pre.003` conformément au plan actif.
|
||||
|
||||
## Responsabilités
|
||||
|
||||
- garantir les invariants des données persistées ;
|
||||
- séparer contrats et implémentation PostgreSQL à l’intérieur de la crate consolidée ;
|
||||
- maintenir les tables canoniques `kb_sol_*` ;
|
||||
- fournir au pipeline des frontières async typées ;
|
||||
- posséder la connexion et l'initialisation du backend actif ;
|
||||
- fournir des frontières async typées au pipeline ;
|
||||
- protéger les opérations multi-tables par transactions ;
|
||||
- borner toutes les requêtes de diagnostic et de replay exposées.
|
||||
- borner les diagnostics et replay exposés ;
|
||||
- permettre le remplacement futur de PostgreSQL sans réécriture des consommateurs.
|
||||
|
||||
## Hors périmètre
|
||||
|
||||
La crate ne décode pas les instructions, n’acquiert pas les transactions et ne décide pas quelle matérialisation exécuter. Elle persiste les résultats produits par `ks-lib` et orchestrés par `ks-pipeline`.
|
||||
|
||||
## API publique
|
||||
|
||||
Les consommateurs utilisent principalement `PostgresStore`, `PostgresStoreOptions`, les traits `*Store`, les DTO `*Insert`/`*Row`, les filtres de replay et les diagnostics. Les constantes de tables et fonctions de validation sont publiques pour les audits et outils d’administration.
|
||||
La crate ne décode pas les instructions, n'acquiert pas les transactions et ne décide pas quelle matérialisation ou projection N4 exécuter. Elle persiste les faits produits par `ks-lib` et orchestrés par `ks-pipeline`.
|
||||
|
||||
## Relations
|
||||
|
||||
- dépend de `ks-core` pour les erreurs ;
|
||||
- utilise les contrats de replay de `ks-lib` ;
|
||||
- est consommée principalement par `ks-pipeline`, les scénarios de démonstration et le desktop.
|
||||
|
||||
## Statut
|
||||
|
||||
Le stockage consolidé dispose de tests unitaires étendus et de tests PostgreSQL optionnels pilotés par environnement. Les opérations réelles exigent un serveur PostgreSQL et l’application préalable des migrations.
|
||||
- est consommée principalement par `ks-pipeline`, `ks-pipeline-demo-scenarios` et `kb-app-demo-desktop` ;
|
||||
- ne dépend pas de `ks-config`.
|
||||
|
||||
## Documents
|
||||
|
||||
@@ -50,3 +58,4 @@ Le stockage consolidé dispose de tests unitaires étendus et de tests PostgreSQ
|
||||
- [Travaux restants](TODO.md)
|
||||
- [Historique](CHANGELOG.md)
|
||||
- [Architecture du stockage](../docs/architecture/STORAGE_ARCHITECTURE.md)
|
||||
- [Plan actif `0.5.3`](../docs/plans/V0_5_3_KS_STORE_NORMALIZATION_PLAN.md)
|
||||
|
||||
@@ -1,18 +1,16 @@
|
||||
<!-- file: ks-store/TODO.md -->
|
||||
<!-- version: 11 -->
|
||||
<!-- version: 13 -->
|
||||
|
||||
# TODO — ks-store
|
||||
|
||||
## Série `0.5.x`
|
||||
|
||||
- [ ] `0.5.3` — introduire une façade `Store` backend-agnostique, rendre PostgreSQL/SQLx privés et supprimer les contrats publics historiques non implémentés.
|
||||
- [ ] `0.5.3` — remodeler la configuration store afin que le backend sélectionné et ses options restent opaques jusqu'à leur validation/interprétation par `ks-store`.
|
||||
- [ ] `0.5.3` — reconstruire `kb_sol_* -> k_sol_*`, ajouter `block_time` aux ancres raw/Core, conserver top-level/CPI dans deux tables Core avec lifecycle/replay équivalents, `stack_height`/parent CPI audités et renommer le journal N3 en `k_sol_mat_outputs`.
|
||||
- [ ] `0.5.3` — déplacer les ressources sous `migrations/postgres/`, imposer une instruction SQL par fichier avec exécution privée via `include_str!`, séparer tables/keys/index/drop/truncate et supprimer le DDL dupliqué en Rust.
|
||||
- [ ] `0.5.3` — corriger l'invalidation descendante, formaliser les replays N1 -> N2 / N2 -> N3 / frontière N3 -> N4, les paginations/limites, les conversions BIGINT et les index justifiés par les requêtes réelles.
|
||||
- [ ] `0.5.3` — migrer `ks-pipeline-demo-scenarios` et `kb-app-demo-desktop` vers les API génériques sans symboles `Postgres*`, conserver un `Store` générique dans l’état applicatif, fournir un résumé runtime/init sans secrets et réconcilier `demo_sql`/`demo_config`/splash sans noms d'objets physiques.
|
||||
- [ ] `0.5.3` — finaliser en `pre.005` la présentation fonctionnelle des surfaces `demo_store_*`, `demo_config` et le splash autour des modèles/compteurs du nouveau baseline sans noms d'objets physiques ; le renommage technique `demo_sql_* -> demo_store_*` est réalisé en `pre.002`.
|
||||
- [ ] `0.5.3` — exécuter avant gel le canari future-decoder/Anchor-readiness sur N1/N2/N3 : bytes d'instruction, comptes ordonnés/flags, hiérarchie CPI, logs, `returnData`, observations/états de comptes et provenance decoder/schema ; ajouter seulement les contrats Solana génériques manquants, sans table `anchor_*`.
|
||||
- [ ] `0.5.3` — produire le snapshot machine-readable des tables N1/N2/N3 réellement stabilisées après ce canari (13 tables existantes de baseline, plus éventuels contrats génériques nécessaires), documenter la politique N4 plus souple et les futures projections metadata asset/token + program metadata, puis ajouter les canaris anti-`kb_sol_*`, anti-backend leak, anti-secret et logging `ks-store.pg.*`.
|
||||
- [ ] `0.5.3` — produire le snapshot machine-readable des tables N1/N2/N3 réellement stabilisées après ce canari (13 tables existantes de baseline, plus éventuels contrats génériques nécessaires), documenter la politique N4 plus souple et les futures projections metadata asset/token + program metadata, puis ajouter les canaris anti-`kb_sol_*`, anti-backend leak, anti-secret et logging sous le target canonique `ks-store` avec domaine structuré `ks-store.pg`.
|
||||
- [ ] PostgreSQL - auditer le dimensionnement du pool selon les charges réelles.
|
||||
- [ ] Résilience - décider et implémenter, si justifié, un retry borné pour les timeouts transitoires.
|
||||
- [ ] Administration - définir les outils supplémentaires prévus par le ROADMAP avant leur implémentation.
|
||||
|
||||
@@ -1,90 +1,79 @@
|
||||
<!-- file: ks-store/USAGE.md -->
|
||||
<!-- version: 4 -->
|
||||
<!-- version: 5 -->
|
||||
|
||||
# Utilisation de ks-store
|
||||
|
||||
## Connexion PostgreSQL
|
||||
## Ouvrir le store
|
||||
|
||||
Le consommateur transmet uniquement le backend sélectionné et son objet d'options déjà résolu. Il ne construit jamais un pool PostgreSQL.
|
||||
|
||||
```rust
|
||||
let options = match ks_store::PostgresStoreOptions::new(
|
||||
database_url,
|
||||
10,
|
||||
10_000,
|
||||
let options = match ks_store::StoreOpenOptions::new(
|
||||
true,
|
||||
"postgres",
|
||||
serde_json::json!({
|
||||
"url": database_url,
|
||||
"max_connections": 10,
|
||||
"connect_timeout_ms": 10_000,
|
||||
"auto_initialize_schema": true
|
||||
}),
|
||||
) {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
println!("postgres endpoint: {}", options.masked_dsn());
|
||||
|
||||
let store = match ks_store::PostgresStore::connect(options).await {
|
||||
let store = match ks_store::Store::open(options).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
```
|
||||
|
||||
`PostgresStoreOptions::new` valide le DSN, le nombre de connexions et le timeout. Lorsque `auto_initialize_schema` vaut `true`, la connexion applique les schémas idempotents. Utiliser `masked_dsn` ou `mask_postgres_dsn` dans les logs ; ne jamais journaliser le DSN brut.
|
||||
`Store::open` interprète les options du backend actif, crée la connexion, exécute l'auto-initialisation lorsqu'elle est configurée et capture un résumé d'initialisation sanitisé. PostgreSQL reste le seul backend opérationnel exigé en `0.5.3-pre.002`; un code backend inconnu est refusé proprement.
|
||||
|
||||
## Diagnostics
|
||||
## Résumés sûrs
|
||||
|
||||
```rust
|
||||
let health = store.health_snapshot().await;
|
||||
let migrations = store.migration_snapshot().await;
|
||||
let backend = store.backend_diagnostics().await;
|
||||
|
||||
println!("health={:?}", health.status);
|
||||
println!("migration={:?}", migrations.status);
|
||||
println!("backend={:?}", backend.descriptor.backend);
|
||||
```
|
||||
|
||||
Les diagnostics de tables raw, Core et decode/materialization sont également disponibles par les méthodes `*_table_diagnostics`.
|
||||
|
||||
```rust
|
||||
let raw_tables = match store.raw_table_diagnostics().await {
|
||||
let configuration = store.configuration_summary();
|
||||
let initialization = store.initialization_summary();
|
||||
let runtime = match store.runtime_summary().await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
```
|
||||
|
||||
for table in raw_tables {
|
||||
Ces contrats ne contiennent pas les options backend brutes. Le descripteur de connexion éventuellement exposé par `runtime.backend` est masqué par le backend propriétaire.
|
||||
|
||||
Le résumé d'initialisation classe les ressources par modèles logiques et fournit des compteurs. Il expose aussi `objects`, une liste de catégories physiques sans noms d'objets : PostgreSQL publie déjà le total `table`, tandis que `pre.003` ajoutera le total `index` avec le nouveau baseline. Les noms physiques de tables, index et contraintes restent internes au backend.
|
||||
|
||||
## Diagnostics de ressources
|
||||
|
||||
```rust
|
||||
let raw_resources = match store.raw_resource_diagnostics().await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
for resource in raw_resources {
|
||||
println!(
|
||||
"table={} domain={} exists={}",
|
||||
table.table_name,
|
||||
table.domain,
|
||||
table.exists
|
||||
"resource={} model={} available={}",
|
||||
resource.resource_code,
|
||||
resource.model_code,
|
||||
resource.available
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
## Pagination bornée
|
||||
|
||||
```rust
|
||||
let first_page = ks_store::PageRequest::first_page();
|
||||
|
||||
let next_page = match ks_store::PageRequest::new(250, 250) {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
assert_eq!(first_page.limit, ks_store::DEFAULT_PAGE_SIZE);
|
||||
assert!(next_page.limit <= ks_store::MAX_PAGE_SIZE);
|
||||
```
|
||||
Les méthodes `core_resource_diagnostics`, `decode_resource_diagnostics` et `known_resource_diagnostics` suivent le même contrat backend-agnostique.
|
||||
|
||||
## Contrats de repositories
|
||||
|
||||
Les traits publics principaux sont :
|
||||
Les traits publics effectivement implémentés sont :
|
||||
|
||||
- `StoreHealthStore` ;
|
||||
- `RawTransactionStore` ;
|
||||
- `CoreTransactionStore` ;
|
||||
- `CoreExtractionStore` ;
|
||||
- `DecodePipelineStore` ;
|
||||
- `ProgramObservationStore` ;
|
||||
- `DecodedEventStore` ;
|
||||
- `MaterializedEventStore` ;
|
||||
- `ProcessingLedgerStore`.
|
||||
- `DecodePipelineStore`.
|
||||
|
||||
Ils permettent au pipeline de dépendre d’un contrat async plutôt que d’une requête SQL directe.
|
||||
`Store` implémente ces traits et délègue au backend privé. Les anciens traits historiques sans implémentation réelle ont été retirés avant gel de l'API.
|
||||
|
||||
## Replay et requêtes bornées
|
||||
|
||||
@@ -93,71 +82,43 @@ let candidates = match store.replay_transaction_candidates(filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
for candidate in candidates {
|
||||
println!(
|
||||
"signature={} slot={}",
|
||||
candidate.signature,
|
||||
candidate.slot
|
||||
);
|
||||
}
|
||||
```
|
||||
|
||||
```rust
|
||||
let programs = match store.replay_program_summaries(program_filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
let entities = match store.replay_entity_summaries(entity_filter).await {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
|
||||
println!("programs={}, entities={}", programs.len(), entities.len());
|
||||
```
|
||||
|
||||
Les filtres refusent les limites nulles ou supérieures aux bornes publiques. `PageRequest` impose également `DEFAULT_PAGE_SIZE` et `MAX_PAGE_SIZE`.
|
||||
Les contrats publics utilisent `ReplayTransaction*`, `ReplayProgram*` et `ReplayEntity*`. Le scope d'instruction utilise désormais `TopLevel`, `Inner` et `Logs`; le terme historique `outer` n'est pas introduit dans la nouvelle API.
|
||||
|
||||
## Pagination bornée
|
||||
|
||||
```rust
|
||||
let first_page = ks_store::PageRequest::first_page();
|
||||
let next_page = match ks_store::PageRequest::new(250, 250) {
|
||||
Ok(value) => value,
|
||||
Err(error) => return Err(error),
|
||||
};
|
||||
assert_eq!(first_page.limit, ks_store::DEFAULT_PAGE_SIZE);
|
||||
assert!(next_page.limit <= ks_store::MAX_PAGE_SIZE);
|
||||
```
|
||||
|
||||
La refonte cursorisée prévue par le plan `0.5.3` reste dans la tranche repositories/replay ultérieure.
|
||||
|
||||
## Transactions atomiques
|
||||
|
||||
Les bundles `CoreExtractionBundle`, `DecodePersistenceBundle` et `MaterializationPersistenceBundle` regroupent les écritures qui doivent réussir ou être annulées ensemble. Les consommateurs ne doivent pas reproduire manuellement ces transactions avec des écritures isolées.
|
||||
Les bundles `CoreExtractionBundle`, `DecodePersistenceBundle` et `MaterializationPersistenceBundle` regroupent les écritures qui doivent réussir ou être annulées ensemble. Les consommateurs ne doivent pas reproduire manuellement ces transactions avec des écritures SQL isolées.
|
||||
|
||||
## Validation des noms de tables
|
||||
## Tests PostgreSQL réels
|
||||
|
||||
```rust
|
||||
if let Err(error) = ks_store::validate_raw_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = ks_store::validate_core_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
if let Err(error) = ks_store::validate_decode_store_table_names() {
|
||||
return Err(error);
|
||||
}
|
||||
Les tests opt-in utilisent `KS_SECRET_POSTGRES_TEST_URL`. Aucun credential réel ne doit être enregistré dans les fixtures ou logs. Les erreurs d'ouverture de connexion sont volontairement sanitisées.
|
||||
|
||||
assert!(ks_store::is_valid_solana_table_name(
|
||||
ks_store::CORE_TRANSACTIONS_TABLE_NAME
|
||||
));
|
||||
```
|
||||
## Limites de `pre.002`
|
||||
|
||||
## Schéma et tables
|
||||
|
||||
Les constantes `RAW_STORE_TABLE_NAMES`, `CORE_STORE_TABLE_NAMES` et `DECODE_STORE_TABLE_NAMES` exposent les noms canoniques. Les fonctions `validate_*_table_names` et `is_valid_solana_table_name` servent aux audits et outils d’administration.
|
||||
|
||||
## Erreurs et invariants
|
||||
|
||||
Toutes les APIs utilisent `ks_core::Result`. Les DTO valident notamment les signatures, chemins, Program IDs, clés, montants, états de traitement et identités de ledger. Les erreurs de contrat peuvent être construites avec `storage_contract_error`.
|
||||
|
||||
## Tests instructifs
|
||||
|
||||
- les tests `*_rejects_*` documentent les invariants des DTO et filtres ;
|
||||
- `decoded_events_and_ledger_roll_back_together` vérifie l’atomicité ;
|
||||
- `optional_postgres_*_from_env` couvre les parcours réels lorsque l’environnement PostgreSQL est configuré ;
|
||||
- les tests de schéma vérifient les noms, index, contraintes et migrations.
|
||||
|
||||
## Limites
|
||||
|
||||
- backend actif : PostgreSQL ;
|
||||
- les tests réels sont optionnels sans DSN de test ;
|
||||
- la crate ne prend aucune décision de décodage ou de matérialisation.
|
||||
- les tables restent temporairement sous leur namespace historique jusqu'à `pre.003` ;
|
||||
- les migrations PostgreSQL ne sont pas encore déplacées sous `migrations/postgres/` ;
|
||||
- la pagination cursorisée et l'invalidation descendante sont traitées dans la tranche repositories/replay ;
|
||||
- la refonte complète des fenêtres desktop `demo_store_*` est prévue dans la tranche consommateurs.
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
// file: ks-store/src/constants.rs
|
||||
// version: 2
|
||||
// version: 5
|
||||
|
||||
//! Local constants for the `ks-store` crate.
|
||||
|
||||
/// Canonical tracing target for this crate.
|
||||
/// Canonical tracing target for backend-independent storage operations.
|
||||
pub(crate) const TRACING_TARGET: &str = "ks-store";
|
||||
/// Transitional schema contract identifier used until the `pre.003` baseline rebuild.
|
||||
pub const STORE_SCHEMA_CONTRACT_VERSION: &str = "0.5.3-pre.2-legacy-schema";
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/contracts.rs
|
||||
// version: 3
|
||||
// version: 6
|
||||
|
||||
//! Backend-neutral storage contracts used by pipeline crates.
|
||||
|
||||
@@ -8,6 +8,7 @@ mod entity;
|
||||
mod error;
|
||||
mod health;
|
||||
mod pagination;
|
||||
mod replay;
|
||||
mod repository;
|
||||
|
||||
/// Core account key insert contract.
|
||||
@@ -52,8 +53,6 @@ pub use self::dto::DecodeObservationInsert;
|
||||
pub use self::dto::DecodePersistenceBundle;
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
pub use self::dto::DecodeSelectionFilter;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::dto::DecodedEventInsert;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::dto::InsertOutcome;
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
@@ -62,16 +61,12 @@ pub use self::dto::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
|
||||
pub use self::dto::MaterializationPersistenceBundle;
|
||||
/// Bounded read-only materialized event selection.
|
||||
pub use self::dto::MaterializedEventFilter;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::dto::MaterializedEventInsert;
|
||||
/// One materialized output returned by a bounded query.
|
||||
pub use self::dto::MaterializedEventQueryRow;
|
||||
/// One processor-owned materialized output row.
|
||||
pub use self::dto::MaterializedOutputInsert;
|
||||
/// Stable processing ledger identity.
|
||||
pub use self::dto::ProcessingLedgerIdentity;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::dto::ProcessingLedgerMark;
|
||||
/// Stable processing ledger status.
|
||||
pub use self::dto::ProcessingLedgerStatus;
|
||||
/// Raw payload lifecycle mark request.
|
||||
@@ -84,12 +79,30 @@ pub use self::dto::RawPayloadRetentionState;
|
||||
pub use self::dto::RawTransactionInsert;
|
||||
/// Store backend diagnostic contract.
|
||||
pub use self::dto::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::dto::StoreBackendKind;
|
||||
/// Backend-neutral diagnostic snapshot.
|
||||
pub use self::dto::StoreBackendDiagnostics;
|
||||
/// Store configuration summary safe for UI and logs.
|
||||
pub use self::dto::StoreConfigurationSummary;
|
||||
/// Store initialization status.
|
||||
pub use self::dto::StoreInitializationStatus;
|
||||
/// Initialization and model-verification summary captured while opening a store.
|
||||
pub use self::dto::StoreInitializationSummary;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::dto::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::dto::StoreMigrationStatus;
|
||||
/// Store model verification status.
|
||||
pub use self::dto::StoreModelVerificationStatus;
|
||||
/// Verification summary for one logical store model.
|
||||
pub use self::dto::StoreModelVerificationSummary;
|
||||
/// Count summary for one backend object category without exposing physical names.
|
||||
pub use self::dto::StoreObjectVerificationSummary;
|
||||
/// Read-only diagnostics for one logical store resource.
|
||||
pub use self::dto::StoreResourceDiagnostics;
|
||||
/// Read-only statistics for one logical store resource.
|
||||
pub use self::dto::StoreResourceStatistics;
|
||||
/// Complete runtime summary safe for application diagnostics.
|
||||
pub use self::dto::StoreRuntimeSummary;
|
||||
/// Transaction acquisition observation insert contract.
|
||||
pub use self::dto::TransactionObservationInsert;
|
||||
/// Transaction acquisition observation origin.
|
||||
@@ -108,12 +121,6 @@ pub use self::entity::CoreInstructionRow;
|
||||
pub use self::entity::CoreLogRow;
|
||||
/// Core transaction SQL-like row contract.
|
||||
pub use self::entity::CoreTransactionRow;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::entity::DecodedEventRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::entity::MaterializedEventRow;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::entity::ProcessingLedgerRow;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
pub use self::entity::RawTransactionRow;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
@@ -132,20 +139,30 @@ pub use self::pagination::MAX_PAGE_SIZE;
|
||||
pub use self::pagination::PageRequest;
|
||||
/// Sort direction for repository list operations.
|
||||
pub use self::pagination::SortDirection;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::replay::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
pub use self::replay::ReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::replay::ReplayEntityKind;
|
||||
/// Aggregated mint, owner or account-key occurrences from Core facts.
|
||||
pub use self::replay::ReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::replay::ReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::replay::ReplayProgramScope;
|
||||
/// Aggregated program occurrences across top-level instructions, inner instructions and linked logs.
|
||||
pub use self::replay::ReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with Core and processing-ledger diagnostics.
|
||||
pub use self::replay::ReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::replay::ReplayTransactionFilter;
|
||||
/// Canonical transaction to core extraction storage behavior.
|
||||
pub use self::repository::CoreExtractionStore;
|
||||
/// Core Solana storage behavior.
|
||||
pub use self::repository::CoreTransactionStore;
|
||||
/// Contextual instruction decode and materialization storage behavior.
|
||||
pub use self::repository::DecodePipelineStore;
|
||||
/// Decoded event storage behavior.
|
||||
pub use self::repository::DecodedEventStore;
|
||||
/// Materialized event storage behavior.
|
||||
pub use self::repository::MaterializedEventStore;
|
||||
/// Processing ledger storage behavior.
|
||||
pub use self::repository::ProcessingLedgerStore;
|
||||
/// Program observation storage behavior.
|
||||
pub use self::repository::ProgramObservationStore;
|
||||
/// Raw transaction storage behavior.
|
||||
pub use self::repository::RawTransactionStore;
|
||||
/// Store health storage behavior.
|
||||
|
||||
@@ -1,13 +1,12 @@
|
||||
// file: ks-store/src/contracts/dto.rs
|
||||
// version: 3
|
||||
// version: 6
|
||||
|
||||
//! Backend-neutral DTO exports for storage repository contracts.
|
||||
|
||||
mod core;
|
||||
mod core_extraction;
|
||||
mod decode;
|
||||
mod event;
|
||||
mod ledger;
|
||||
mod outcome;
|
||||
mod raw;
|
||||
mod store;
|
||||
|
||||
@@ -67,14 +66,8 @@ pub use self::decode::MaterializedEventFilter;
|
||||
pub use self::decode::MaterializedEventQueryRow;
|
||||
/// One processor-owned materialized output row.
|
||||
pub use self::decode::MaterializedOutputInsert;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::event::DecodedEventInsert;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::event::InsertOutcome;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::event::MaterializedEventInsert;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::ledger::ProcessingLedgerMark;
|
||||
pub use self::outcome::InsertOutcome;
|
||||
/// Raw payload lifecycle mark request.
|
||||
pub use self::raw::RawPayloadLifecycleMark;
|
||||
/// Raw payload processing state.
|
||||
@@ -91,9 +84,27 @@ pub use self::raw::TransactionObservationOrigin;
|
||||
pub use self::raw::TransactionObservationStatus;
|
||||
/// Store backend diagnostic contract.
|
||||
pub use self::store::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::store::StoreBackendKind;
|
||||
/// Backend-neutral diagnostic snapshot.
|
||||
pub use self::store::StoreBackendDiagnostics;
|
||||
/// Store configuration summary safe for UI and logs.
|
||||
pub use self::store::StoreConfigurationSummary;
|
||||
/// Store initialization status.
|
||||
pub use self::store::StoreInitializationStatus;
|
||||
/// Initialization and model-verification summary captured while opening a store.
|
||||
pub use self::store::StoreInitializationSummary;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::store::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::store::StoreMigrationStatus;
|
||||
/// Store model verification status.
|
||||
pub use self::store::StoreModelVerificationStatus;
|
||||
/// Verification summary for one logical store model.
|
||||
pub use self::store::StoreModelVerificationSummary;
|
||||
/// Count summary for one backend object category without exposing physical names.
|
||||
pub use self::store::StoreObjectVerificationSummary;
|
||||
/// Read-only diagnostics for one logical store resource.
|
||||
pub use self::store::StoreResourceDiagnostics;
|
||||
/// Read-only statistics for one logical store resource.
|
||||
pub use self::store::StoreResourceStatistics;
|
||||
/// Complete runtime summary safe for application diagnostics.
|
||||
pub use self::store::StoreRuntimeSummary;
|
||||
|
||||
@@ -1,143 +0,0 @@
|
||||
// file: ks-store/src/contracts/dto/event.rs
|
||||
// version: 2
|
||||
|
||||
//! Decoded and materialized event storage DTOs.
|
||||
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct InsertOutcome {
|
||||
/// Number of rows inserted by the repository call.
|
||||
pub inserted_count: u64,
|
||||
/// Number of rows updated by the repository call.
|
||||
pub updated_count: u64,
|
||||
/// Number of rows skipped by the repository call.
|
||||
pub skipped_count: u64,
|
||||
}
|
||||
|
||||
impl InsertOutcome {
|
||||
/// Builds an insert outcome from explicit counters.
|
||||
pub fn new(inserted_count: u64, updated_count: u64, skipped_count: u64) -> Self {
|
||||
return Self {
|
||||
inserted_count,
|
||||
updated_count,
|
||||
skipped_count,
|
||||
};
|
||||
}
|
||||
|
||||
/// Returns the sum of inserted, updated and skipped rows.
|
||||
pub fn total_count(&self) -> u64 {
|
||||
return self.inserted_count + self.updated_count + self.skipped_count;
|
||||
}
|
||||
}
|
||||
|
||||
/// Decoded event insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodedEventInsert {
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Protocol family code.
|
||||
pub protocol_code: std::string::String,
|
||||
/// Protocol surface code.
|
||||
pub surface_code: std::string::String,
|
||||
/// Canonical event code.
|
||||
pub event_code: std::string::String,
|
||||
/// Decoded payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
}
|
||||
|
||||
impl DecodedEventInsert {
|
||||
/// Builds a decoded event insert contract from the shared decoded model.
|
||||
pub fn from_model(
|
||||
event: &ks_lib::MdDecodedProtocolEvent,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
if event.signature.0.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded event signature must not be empty",
|
||||
));
|
||||
}
|
||||
if event.program_id.0.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decoded event program id must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: event.signature.0.clone(),
|
||||
slot: event.slot.0,
|
||||
instruction_path: event.instruction_path.0.clone(),
|
||||
program_id: event.program_id.0.clone(),
|
||||
protocol_code: event.protocol_code.0.clone(),
|
||||
surface_code: event.surface_code.0.clone(),
|
||||
event_code: event.event_code.0.clone(),
|
||||
payload_json,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
/// Materialized event insert contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventInsert {
|
||||
/// Source transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot in Solana unsigned representation.
|
||||
pub slot: u64,
|
||||
/// Materialized event family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Materialized payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
}
|
||||
|
||||
impl MaterializedEventInsert {
|
||||
/// Builds a materialized event insert contract after minimal validation.
|
||||
pub fn new(
|
||||
signature: impl std::convert::Into<std::string::String>,
|
||||
slot: u64,
|
||||
materialized_family: impl std::convert::Into<std::string::String>,
|
||||
payload_json: serde_json::Value,
|
||||
) -> ks_core::Result<Self> {
|
||||
let signature_value = signature.into();
|
||||
let materialized_family_value = materialized_family.into();
|
||||
if signature_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialized event signature must not be empty",
|
||||
));
|
||||
}
|
||||
if materialized_family_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"materialized event family must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
signature: signature_value,
|
||||
slot,
|
||||
materialized_family: materialized_family_value,
|
||||
payload_json,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn insert_outcome_total_counts_all_buckets() {
|
||||
let outcome = crate::InsertOutcome::new(1, 2, 3);
|
||||
assert_eq!(outcome.total_count(), 6);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn materialized_event_rejects_empty_family() {
|
||||
let result = crate::MaterializedEventInsert::new(
|
||||
"abc",
|
||||
1,
|
||||
" ",
|
||||
serde_json::json!({"kind": "trade"}),
|
||||
);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
}
|
||||
@@ -1,73 +0,0 @@
|
||||
// file: ks-store/src/contracts/dto/ledger.rs
|
||||
// version: 2
|
||||
|
||||
//! Processing ledger storage DTOs.
|
||||
|
||||
/// Processing ledger mark request contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct ProcessingLedgerMark {
|
||||
/// Processing stage name, for example `raw_ingest`, `core_extract` or `decode`.
|
||||
pub stage: std::string::String,
|
||||
/// Processing module name.
|
||||
pub module_name: std::string::String,
|
||||
/// Processing module version.
|
||||
pub module_version: std::string::String,
|
||||
/// Stable input key, usually a signature or notification id.
|
||||
pub input_key: std::string::String,
|
||||
}
|
||||
|
||||
impl ProcessingLedgerMark {
|
||||
/// Builds a processing ledger mark request after minimal validation.
|
||||
pub fn new(
|
||||
stage: impl std::convert::Into<std::string::String>,
|
||||
module_name: impl std::convert::Into<std::string::String>,
|
||||
module_version: impl std::convert::Into<std::string::String>,
|
||||
input_key: impl std::convert::Into<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let stage_value = stage.into();
|
||||
let module_name_value = module_name.into();
|
||||
let module_version_value = module_version.into();
|
||||
let input_key_value = input_key.into();
|
||||
if stage_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger stage must not be empty",
|
||||
));
|
||||
}
|
||||
if module_name_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger module name must not be empty",
|
||||
));
|
||||
}
|
||||
if module_version_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger module version must not be empty",
|
||||
));
|
||||
}
|
||||
if input_key_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"processing ledger input key must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
stage: stage_value,
|
||||
module_name: module_name_value,
|
||||
module_version: module_version_value,
|
||||
input_key: input_key_value,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn ledger_mark_rejects_empty_input_key() {
|
||||
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", " ");
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn ledger_mark_accepts_minimal_values() {
|
||||
let result = crate::ProcessingLedgerMark::new("decode", "module", "1", "signature");
|
||||
assert!(result.is_ok());
|
||||
}
|
||||
}
|
||||
40
ks-store/src/contracts/dto/outcome.rs
Normal file
40
ks-store/src/contracts/dto/outcome.rs
Normal file
@@ -0,0 +1,40 @@
|
||||
// file: ks-store/src/contracts/dto/outcome.rs
|
||||
// version: 4
|
||||
|
||||
//! Generic repository operation outcome DTOs.
|
||||
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct InsertOutcome {
|
||||
/// Number of rows inserted by the repository call.
|
||||
pub inserted_count: u64,
|
||||
/// Number of rows updated by the repository call.
|
||||
pub updated_count: u64,
|
||||
/// Number of rows skipped by the repository call.
|
||||
pub skipped_count: u64,
|
||||
}
|
||||
|
||||
impl crate::InsertOutcome {
|
||||
/// Builds an insert outcome from explicit counters.
|
||||
pub fn new(inserted_count: u64, updated_count: u64, skipped_count: u64) -> Self {
|
||||
return Self {
|
||||
inserted_count,
|
||||
updated_count,
|
||||
skipped_count,
|
||||
};
|
||||
}
|
||||
|
||||
/// Returns the sum of inserted, updated and skipped rows.
|
||||
pub fn total_count(&self) -> u64 {
|
||||
return self.inserted_count + self.updated_count + self.skipped_count;
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn insert_outcome_total_counts_all_buckets() {
|
||||
let outcome = crate::InsertOutcome::new(1, 2, 3);
|
||||
assert_eq!(outcome.total_count(), 6);
|
||||
}
|
||||
}
|
||||
@@ -1,20 +1,7 @@
|
||||
// file: ks-store/src/contracts/dto/store.rs
|
||||
// version: 2
|
||||
// version: 5
|
||||
|
||||
//! Store backend diagnostic DTOs.
|
||||
|
||||
/// Store backend kind contract.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreBackendKind {
|
||||
/// PostgreSQL backend.
|
||||
Postgres,
|
||||
/// SQLite backend retained for tests and legacy imports.
|
||||
Sqlite,
|
||||
/// In-memory backend used by offline tests.
|
||||
Memory,
|
||||
/// Backend is not known.
|
||||
Unknown,
|
||||
}
|
||||
//! Backend-agnostic store configuration and diagnostic DTOs.
|
||||
|
||||
/// Store migration status contract.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
@@ -33,38 +20,81 @@ pub enum StoreMigrationStatus {
|
||||
Failed,
|
||||
}
|
||||
|
||||
/// Store backend diagnostic contract.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreBackendDescriptor {
|
||||
/// Store backend kind.
|
||||
pub backend_kind: StoreBackendKind,
|
||||
/// Human-readable backend label.
|
||||
pub backend_label: std::string::String,
|
||||
/// Masked DSN or connection descriptor safe for diagnostics.
|
||||
pub masked_dsn: std::option::Option<std::string::String>,
|
||||
/// Current PostgreSQL schema or equivalent namespace when known.
|
||||
pub current_schema: std::option::Option<std::string::String>,
|
||||
/// Store initialization status exposed without backend-specific details.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreInitializationStatus {
|
||||
/// Persistence is disabled for the selected profile.
|
||||
Disabled,
|
||||
/// Every expected logical model is available.
|
||||
Ready,
|
||||
/// The backend is reachable but at least one expected logical model is incomplete.
|
||||
Partial,
|
||||
/// Initialization or verification failed.
|
||||
Failed,
|
||||
}
|
||||
|
||||
impl StoreBackendDescriptor {
|
||||
/// Store model verification status.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub enum StoreModelVerificationStatus {
|
||||
/// Every expected resource for the logical model is available.
|
||||
Ready,
|
||||
/// At least one expected resource is missing.
|
||||
Incomplete,
|
||||
}
|
||||
|
||||
/// Store configuration summary safe for UI and logs.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreConfigurationSummary {
|
||||
/// Whether persistence is enabled for the selected profile.
|
||||
pub enabled: bool,
|
||||
/// Stable backend code interpreted by `ks-store`.
|
||||
pub backend_code: std::string::String,
|
||||
/// Whether the selected backend supplied a connection/location value.
|
||||
pub connection_configured: bool,
|
||||
/// Whether schema auto-initialization is enabled for this backend.
|
||||
pub auto_initialize_schema: bool,
|
||||
/// Number of backend options supplied after configuration composition.
|
||||
pub backend_option_count: u32,
|
||||
}
|
||||
|
||||
/// Store backend diagnostic contract safe for UI display.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreBackendDescriptor {
|
||||
/// Stable backend code such as `postgres`.
|
||||
pub backend_code: std::string::String,
|
||||
/// Human-readable backend label.
|
||||
pub backend_label: std::string::String,
|
||||
/// Masked connection descriptor safe for diagnostics.
|
||||
pub masked_connection_descriptor: std::option::Option<std::string::String>,
|
||||
/// Backend namespace, schema or equivalent logical location when known.
|
||||
pub namespace: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl crate::StoreBackendDescriptor {
|
||||
/// Builds a store backend descriptor after minimal validation.
|
||||
pub fn new(
|
||||
backend_kind: StoreBackendKind,
|
||||
backend_code: impl std::convert::Into<std::string::String>,
|
||||
backend_label: impl std::convert::Into<std::string::String>,
|
||||
masked_dsn: std::option::Option<std::string::String>,
|
||||
current_schema: std::option::Option<std::string::String>,
|
||||
masked_connection_descriptor: std::option::Option<std::string::String>,
|
||||
namespace: std::option::Option<std::string::String>,
|
||||
) -> ks_core::Result<Self> {
|
||||
let backend_code_value = backend_code.into();
|
||||
let backend_label_value = backend_label.into();
|
||||
if backend_code_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"store backend code must not be empty",
|
||||
));
|
||||
}
|
||||
if backend_label_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"store backend label must not be empty",
|
||||
));
|
||||
}
|
||||
return std::result::Result::Ok(Self {
|
||||
backend_kind,
|
||||
backend_code: backend_code_value,
|
||||
backend_label: backend_label_value,
|
||||
masked_dsn,
|
||||
current_schema,
|
||||
masked_connection_descriptor,
|
||||
namespace,
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -73,7 +103,7 @@ impl StoreBackendDescriptor {
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreMigrationSnapshot {
|
||||
/// Current migration status.
|
||||
pub status: StoreMigrationStatus,
|
||||
pub status: crate::StoreMigrationStatus,
|
||||
/// Last applied migration identifier when known.
|
||||
pub current_version: std::option::Option<std::string::String>,
|
||||
/// Pending migration identifiers when known.
|
||||
@@ -82,10 +112,10 @@ pub struct StoreMigrationSnapshot {
|
||||
pub message: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
impl StoreMigrationSnapshot {
|
||||
impl crate::StoreMigrationSnapshot {
|
||||
/// Builds a migration snapshot from explicit values.
|
||||
pub fn new(
|
||||
status: StoreMigrationStatus,
|
||||
status: crate::StoreMigrationStatus,
|
||||
current_version: std::option::Option<std::string::String>,
|
||||
pending_versions: std::vec::Vec<std::string::String>,
|
||||
message: std::option::Option<std::string::String>,
|
||||
@@ -99,13 +129,114 @@ impl StoreMigrationSnapshot {
|
||||
}
|
||||
}
|
||||
|
||||
/// Backend-neutral diagnostic snapshot.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreBackendDiagnostics {
|
||||
/// Backend descriptor safe for UI display.
|
||||
pub descriptor: crate::StoreBackendDescriptor,
|
||||
/// Backend health snapshot.
|
||||
pub health: crate::StoreHealthSnapshot,
|
||||
/// Migration status snapshot.
|
||||
pub migrations: crate::StoreMigrationSnapshot,
|
||||
/// Backend version string when the backend exposes one.
|
||||
pub backend_version: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only statistics for one logical store resource.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreResourceStatistics {
|
||||
/// Number of records currently stored in the resource.
|
||||
pub record_count: i64,
|
||||
/// Lowest observed Solana slot when meaningful for the resource.
|
||||
pub min_slot: std::option::Option<i64>,
|
||||
/// Highest observed Solana slot when meaningful for the resource.
|
||||
pub max_slot: std::option::Option<i64>,
|
||||
/// Latest creation timestamp rendered by the backend for diagnostics.
|
||||
pub latest_created_at: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only diagnostics for one logical store resource.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreResourceDiagnostics {
|
||||
/// Stable backend-independent resource code.
|
||||
pub resource_code: std::string::String,
|
||||
/// Logical model containing this resource.
|
||||
pub model_code: std::string::String,
|
||||
/// Human-readable resource role.
|
||||
pub role: std::string::String,
|
||||
/// Whether the resource is available in the active backend.
|
||||
pub available: bool,
|
||||
/// Resource statistics when available and supported.
|
||||
pub statistics: std::option::Option<crate::StoreResourceStatistics>,
|
||||
}
|
||||
|
||||
/// Verification summary for one logical store model.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreModelVerificationSummary {
|
||||
/// Stable logical model code such as `raw`, `core`, `processing`, `decode` or `materialization`.
|
||||
pub model_code: std::string::String,
|
||||
/// Verification status for the model.
|
||||
pub status: crate::StoreModelVerificationStatus,
|
||||
/// Number of resources expected by the current store contract.
|
||||
pub expected_resource_count: u32,
|
||||
/// Number of expected resources available after verification.
|
||||
pub available_resource_count: u32,
|
||||
/// Number of resources created during this initialization pass.
|
||||
pub created_resource_count: u32,
|
||||
/// Number of expected resources still missing.
|
||||
pub missing_resource_count: u32,
|
||||
}
|
||||
|
||||
/// Count summary for one backend object category without exposing physical object names.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreObjectVerificationSummary {
|
||||
/// Stable object category such as `table`, `index` or a backend-specific equivalent.
|
||||
pub object_kind: std::string::String,
|
||||
/// Number of objects expected by the active backend contract.
|
||||
pub expected_count: u32,
|
||||
/// Number of expected objects available after verification.
|
||||
pub available_count: u32,
|
||||
/// Number of expected objects created during this initialization pass.
|
||||
pub created_count: u32,
|
||||
}
|
||||
|
||||
/// Initialization and model-verification summary captured while opening a store.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreInitializationSummary {
|
||||
/// Overall initialization status.
|
||||
pub status: crate::StoreInitializationStatus,
|
||||
/// Stable schema/model contract version.
|
||||
pub schema_contract_version: std::string::String,
|
||||
/// Per-model verification summaries.
|
||||
pub models: std::vec::Vec<crate::StoreModelVerificationSummary>,
|
||||
/// Total number of expected logical resources.
|
||||
pub expected_resource_count: u32,
|
||||
/// Total number of available logical resources.
|
||||
pub available_resource_count: u32,
|
||||
/// Total number of logical resources created during opening.
|
||||
pub created_resource_count: u32,
|
||||
/// Backend object counts grouped by category without physical object names.
|
||||
pub objects: std::vec::Vec<crate::StoreObjectVerificationSummary>,
|
||||
}
|
||||
|
||||
/// Complete runtime summary safe for desktop diagnostics.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct StoreRuntimeSummary {
|
||||
/// Sanitized selected configuration.
|
||||
pub configuration: crate::StoreConfigurationSummary,
|
||||
/// Backend diagnostics when persistence is enabled and opened.
|
||||
pub backend: std::option::Option<crate::StoreBackendDiagnostics>,
|
||||
/// Initialization/model verification result captured during opening.
|
||||
pub initialization: crate::StoreInitializationSummary,
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn store_descriptor_rejects_empty_label() {
|
||||
fn store_descriptor_rejects_empty_code() {
|
||||
let result = crate::StoreBackendDescriptor::new(
|
||||
crate::StoreBackendKind::Postgres,
|
||||
" ",
|
||||
"PostgreSQL",
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
);
|
||||
|
||||
@@ -1,32 +1,24 @@
|
||||
// file: ks-store/src/contracts/entity.rs
|
||||
// version: 3
|
||||
// version: 5
|
||||
|
||||
//! Backend-neutral SQL-like entity exports for storage adapters.
|
||||
//! Backend-neutral persisted entity exports for storage adapters.
|
||||
|
||||
mod core;
|
||||
mod event;
|
||||
mod ledger;
|
||||
mod raw;
|
||||
|
||||
/// Core account key SQL-like row contract.
|
||||
/// Core account key persisted row contract.
|
||||
pub use self::core::CoreAccountKeyRow;
|
||||
/// Core balance change SQL-like row contract.
|
||||
/// Core balance change persisted row contract.
|
||||
pub use self::core::CoreBalanceChangeRow;
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
/// Core inner instruction persisted row contract.
|
||||
pub use self::core::CoreInnerInstructionRow;
|
||||
/// Core instruction SQL-like row contract.
|
||||
/// Core instruction persisted row contract.
|
||||
pub use self::core::CoreInstructionRow;
|
||||
/// Core log SQL-like row contract.
|
||||
/// Core log persisted row contract.
|
||||
pub use self::core::CoreLogRow;
|
||||
/// Core transaction SQL-like row contract.
|
||||
/// Core transaction persisted row contract.
|
||||
pub use self::core::CoreTransactionRow;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::event::DecodedEventRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::event::MaterializedEventRow;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::ledger::ProcessingLedgerRow;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
/// Canonical raw Solana transaction persisted row contract.
|
||||
pub use self::raw::RawTransactionRow;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
/// Transaction acquisition observation persisted row contract.
|
||||
pub use self::raw::TransactionObservationRow;
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
// file: ks-store/src/contracts/entity/core.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! Core Solana SQL-like entities.
|
||||
//! Core Solana persisted entities.
|
||||
|
||||
/// Core transaction SQL-like row contract.
|
||||
/// Core transaction persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreTransactionRow {
|
||||
/// Technical primary key.
|
||||
@@ -24,7 +24,7 @@ pub struct CoreTransactionRow {
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core account key SQL-like row contract.
|
||||
/// Core account key persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreAccountKeyRow {
|
||||
/// Technical primary key.
|
||||
@@ -51,7 +51,7 @@ pub struct CoreAccountKeyRow {
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core instruction SQL-like row contract.
|
||||
/// Core instruction persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInstructionRow {
|
||||
/// Technical primary key.
|
||||
@@ -80,7 +80,7 @@ pub struct CoreInstructionRow {
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
/// Core inner instruction persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreInnerInstructionRow {
|
||||
/// Technical primary key.
|
||||
@@ -107,7 +107,7 @@ pub struct CoreInnerInstructionRow {
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core log SQL-like row contract.
|
||||
/// Core log persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreLogRow {
|
||||
/// Technical primary key.
|
||||
@@ -132,7 +132,7 @@ pub struct CoreLogRow {
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Core balance change SQL-like row contract.
|
||||
/// Core balance change persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct CoreBalanceChangeRow {
|
||||
/// Technical primary key.
|
||||
|
||||
@@ -1,46 +0,0 @@
|
||||
// file: ks-store/src/contracts/entity/event.rs
|
||||
// version: 2
|
||||
|
||||
//! Decoded and materialized event SQL-like entities.
|
||||
|
||||
/// Decoded event SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct DecodedEventRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Stable instruction path, for example `0` or `2/1`.
|
||||
pub instruction_path: std::string::String,
|
||||
/// Program id as non-empty base58 text.
|
||||
pub program_id: std::string::String,
|
||||
/// Protocol family code.
|
||||
pub protocol_code: std::string::String,
|
||||
/// Protocol surface code.
|
||||
pub surface_code: std::string::String,
|
||||
/// Canonical event code.
|
||||
pub event_code: std::string::String,
|
||||
/// Decoded payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Materialized event SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct MaterializedEventRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Source transaction signature as non-empty base58 text.
|
||||
pub signature: std::string::String,
|
||||
/// Source transaction slot stored as SQL `BIGINT`.
|
||||
pub slot: i64,
|
||||
/// Materialized event family code.
|
||||
pub materialized_family: std::string::String,
|
||||
/// Materialized payload JSON.
|
||||
pub payload_json: serde_json::Value,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
@@ -1,37 +0,0 @@
|
||||
// file: ks-store/src/contracts/entity/ledger.rs
|
||||
// version: 2
|
||||
|
||||
//! Processing ledger SQL-like entities.
|
||||
|
||||
/// Processing ledger SQL-like row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct ProcessingLedgerRow {
|
||||
/// Technical primary key.
|
||||
pub id: i64,
|
||||
/// Processing stage name.
|
||||
pub stage: std::string::String,
|
||||
/// Processor implementation name.
|
||||
pub processor_name: std::string::String,
|
||||
/// Processor semantic version.
|
||||
pub processor_version: std::string::String,
|
||||
/// Stable input key, usually a transaction signature.
|
||||
pub input_key: std::string::String,
|
||||
/// Deterministic input hash.
|
||||
pub input_hash: std::string::String,
|
||||
/// Current processing status.
|
||||
pub status: crate::ProcessingLedgerStatus,
|
||||
/// Number of started attempts for this processor identity.
|
||||
pub attempt_count: i32,
|
||||
/// Optional start timestamp for the latest attempt.
|
||||
pub started_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Optional finish timestamp for the latest terminal attempt.
|
||||
pub finished_at: std::option::Option<chrono::DateTime<chrono::Utc>>,
|
||||
/// Optional machine-readable error code.
|
||||
pub error_code: std::option::Option<std::string::String>,
|
||||
/// Optional diagnostic error message.
|
||||
pub error_message: std::option::Option<std::string::String>,
|
||||
/// Insert timestamp.
|
||||
pub created_at: chrono::DateTime<chrono::Utc>,
|
||||
/// Last update timestamp.
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
@@ -1,9 +1,9 @@
|
||||
// file: ks-store/src/contracts/entity/raw.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! Canonical Solana transaction and acquisition observation SQL-like entities.
|
||||
//! Canonical Solana transaction and acquisition observation persisted entities.
|
||||
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
/// Canonical raw Solana transaction persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct RawTransactionRow {
|
||||
/// Technical primary key.
|
||||
@@ -28,7 +28,7 @@ pub struct RawTransactionRow {
|
||||
pub updated_at: chrono::DateTime<chrono::Utc>,
|
||||
}
|
||||
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
/// Transaction acquisition observation persisted row contract.
|
||||
#[derive(Clone, Debug, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct TransactionObservationRow {
|
||||
/// Technical primary key.
|
||||
|
||||
@@ -1,30 +1,30 @@
|
||||
// file: ks-store/src/postgres/replay_candidates.rs
|
||||
// version: 3
|
||||
// file: ks-store/src/contracts/replay.rs
|
||||
// version: 6
|
||||
|
||||
//! Read-only replay candidate filters and PostgreSQL result rows.
|
||||
//! Backend-agnostic replay candidate filters and read models.
|
||||
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub const MAX_REPLAY_CANDIDATE_ROWS: u32 = 100_000;
|
||||
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum PostgresReplayProgramScope {
|
||||
/// Match outer instructions, inner instructions or reliably linked logs.
|
||||
pub enum ReplayProgramScope {
|
||||
/// Match top-level instructions, inner instructions or reliably linked logs.
|
||||
Any,
|
||||
/// Match only top-level instructions.
|
||||
Outer,
|
||||
TopLevel,
|
||||
/// Match only inner instructions.
|
||||
Inner,
|
||||
/// Match only logs with a reliably linked program id.
|
||||
Logs,
|
||||
}
|
||||
|
||||
impl PostgresReplayProgramScope {
|
||||
impl crate::ReplayProgramScope {
|
||||
/// Returns the stable SQL code for this scope.
|
||||
pub fn as_sql(self) -> &'static str {
|
||||
return match self {
|
||||
Self::Any => "any",
|
||||
Self::Outer => "outer",
|
||||
Self::TopLevel => "top_level",
|
||||
Self::Inner => "inner",
|
||||
Self::Logs => "logs",
|
||||
};
|
||||
@@ -33,7 +33,7 @@ impl PostgresReplayProgramScope {
|
||||
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum PostgresReplayEntityKind {
|
||||
pub enum ReplayEntityKind {
|
||||
/// SPL or Token-2022 mint address.
|
||||
Mint,
|
||||
/// Token account owner address.
|
||||
@@ -42,7 +42,7 @@ pub enum PostgresReplayEntityKind {
|
||||
AccountKey,
|
||||
}
|
||||
|
||||
impl PostgresReplayEntityKind {
|
||||
impl crate::ReplayEntityKind {
|
||||
/// Returns the stable SQL code for this entity kind.
|
||||
pub fn as_sql(self) -> &'static str {
|
||||
return match self {
|
||||
@@ -55,7 +55,7 @@ impl PostgresReplayEntityKind {
|
||||
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayTransactionFilter {
|
||||
pub struct ReplayTransactionFilter {
|
||||
/// Optional partial signature search.
|
||||
pub signature_contains: std::option::Option<std::string::String>,
|
||||
/// Optional inclusive minimum slot.
|
||||
@@ -69,9 +69,9 @@ pub struct PostgresReplayTransactionFilter {
|
||||
/// Optional exact program id.
|
||||
pub program_id: std::option::Option<std::string::String>,
|
||||
/// Program occurrence scope.
|
||||
pub program_scope: crate::PostgresReplayProgramScope,
|
||||
pub program_scope: crate::ReplayProgramScope,
|
||||
/// Optional core entity kind.
|
||||
pub entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
|
||||
pub entity_kind: std::option::Option<crate::ReplayEntityKind>,
|
||||
/// Optional exact entity value.
|
||||
pub entity_value: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
@@ -80,9 +80,8 @@ pub struct PostgresReplayTransactionFilter {
|
||||
pub newest_first: bool,
|
||||
}
|
||||
|
||||
impl PostgresReplayTransactionFilter {
|
||||
impl crate::ReplayTransactionFilter {
|
||||
/// Creates and validates a bounded transaction candidate filter.
|
||||
#[allow(clippy::too_many_arguments)]
|
||||
pub fn new(
|
||||
signature_contains: std::option::Option<std::string::String>,
|
||||
min_slot: std::option::Option<u64>,
|
||||
@@ -90,8 +89,8 @@ impl PostgresReplayTransactionFilter {
|
||||
raw_processing_state: std::option::Option<std::string::String>,
|
||||
ledger_status: std::option::Option<std::string::String>,
|
||||
program_id: std::option::Option<std::string::String>,
|
||||
program_scope: crate::PostgresReplayProgramScope,
|
||||
entity_kind: std::option::Option<crate::PostgresReplayEntityKind>,
|
||||
program_scope: crate::ReplayProgramScope,
|
||||
entity_kind: std::option::Option<crate::ReplayEntityKind>,
|
||||
entity_value: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
newest_first: bool,
|
||||
@@ -148,7 +147,7 @@ impl PostgresReplayTransactionFilter {
|
||||
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayTransactionCandidate {
|
||||
pub struct ReplayTransactionCandidate {
|
||||
/// Canonical transaction signature.
|
||||
pub signature: std::string::String,
|
||||
/// Transaction slot.
|
||||
@@ -168,27 +167,27 @@ pub struct PostgresReplayTransactionCandidate {
|
||||
/// Latest core extraction attempt count.
|
||||
pub attempt_count: i32,
|
||||
/// Number of top-level instructions.
|
||||
pub outer_instruction_count: i64,
|
||||
pub top_level_instruction_count: i64,
|
||||
/// Number of inner instructions.
|
||||
pub inner_instruction_count: i64,
|
||||
/// Number of distinct top-level programs.
|
||||
pub outer_program_count: i64,
|
||||
pub top_level_program_count: i64,
|
||||
/// Number of distinct inner programs.
|
||||
pub inner_program_count: i64,
|
||||
/// Raw row update timestamp rendered by PostgreSQL.
|
||||
/// Raw row update timestamp rendered by the active backend.
|
||||
pub updated_at: std::string::String,
|
||||
}
|
||||
|
||||
/// Bounded read-only filter for program summaries.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayProgramFilter {
|
||||
pub struct ReplayProgramFilter {
|
||||
/// Optional partial program id search.
|
||||
pub program_id_contains: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl PostgresReplayProgramFilter {
|
||||
impl crate::ReplayProgramFilter {
|
||||
/// Creates and validates a bounded program summary filter.
|
||||
pub fn new(
|
||||
program_id_contains: std::option::Option<std::string::String>,
|
||||
@@ -205,15 +204,15 @@ impl PostgresReplayProgramFilter {
|
||||
}
|
||||
}
|
||||
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
/// Aggregated program occurrences across top-level, inner and linked logs.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayProgramSummary {
|
||||
pub struct ReplayProgramSummary {
|
||||
/// Program id.
|
||||
pub program_id: std::string::String,
|
||||
/// Number of distinct transactions containing the program.
|
||||
pub transaction_count: i64,
|
||||
/// Number of top-level instruction occurrences.
|
||||
pub outer_instruction_count: i64,
|
||||
pub top_level_instruction_count: i64,
|
||||
/// Number of inner instruction occurrences.
|
||||
pub inner_instruction_count: i64,
|
||||
/// Number of reliably linked log occurrences.
|
||||
@@ -226,19 +225,19 @@ pub struct PostgresReplayProgramSummary {
|
||||
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresReplayEntityFilter {
|
||||
pub struct ReplayEntityFilter {
|
||||
/// Entity kind to aggregate.
|
||||
pub entity_kind: crate::PostgresReplayEntityKind,
|
||||
pub entity_kind: crate::ReplayEntityKind,
|
||||
/// Optional partial entity value search.
|
||||
pub entity_value_contains: std::option::Option<std::string::String>,
|
||||
/// Maximum returned rows.
|
||||
pub limit: u32,
|
||||
}
|
||||
|
||||
impl PostgresReplayEntityFilter {
|
||||
impl crate::ReplayEntityFilter {
|
||||
/// Creates and validates a bounded entity summary filter.
|
||||
pub fn new(
|
||||
entity_kind: crate::PostgresReplayEntityKind,
|
||||
entity_kind: crate::ReplayEntityKind,
|
||||
entity_value_contains: std::option::Option<std::string::String>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
@@ -256,7 +255,7 @@ impl PostgresReplayEntityFilter {
|
||||
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresReplayEntitySummary {
|
||||
pub struct ReplayEntitySummary {
|
||||
/// Stable entity kind code.
|
||||
pub entity_kind: std::string::String,
|
||||
/// Mint, owner or account-key address.
|
||||
@@ -335,14 +334,14 @@ fn validate_optional_code(
|
||||
mod tests {
|
||||
#[test]
|
||||
fn transaction_filter_rejects_inverted_slots() {
|
||||
let result = crate::PostgresReplayTransactionFilter::new(
|
||||
let result = crate::ReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::Some(20),
|
||||
std::option::Option::Some(10),
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
crate::ReplayProgramScope::Any,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
100,
|
||||
@@ -353,15 +352,15 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn transaction_filter_requires_complete_entity_pair() {
|
||||
let result = crate::PostgresReplayTransactionFilter::new(
|
||||
let result = crate::ReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
std::option::Option::Some(crate::PostgresReplayEntityKind::Mint),
|
||||
crate::ReplayProgramScope::Any,
|
||||
std::option::Option::Some(crate::ReplayEntityKind::Mint),
|
||||
std::option::Option::None,
|
||||
100,
|
||||
true,
|
||||
@@ -371,7 +370,7 @@ mod tests {
|
||||
|
||||
#[test]
|
||||
fn program_filter_rejects_limit_above_maximum() {
|
||||
let result = crate::PostgresReplayProgramFilter::new(
|
||||
let result = crate::ReplayProgramFilter::new(
|
||||
std::option::Option::None,
|
||||
crate::MAX_REPLAY_CANDIDATE_ROWS + 1,
|
||||
);
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/contracts/repository.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! Storage trait definitions shared by concrete stores.
|
||||
|
||||
@@ -192,46 +192,3 @@ pub trait DecodePipelineStore {
|
||||
filter: &crate::MaterializedEventFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>>;
|
||||
}
|
||||
|
||||
/// Program observation storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait ProgramObservationStore {
|
||||
/// Stores program observations.
|
||||
async fn store_observations(
|
||||
&self,
|
||||
observations: &[ks_lib::MdProgramObservation],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Decoded event storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait DecodedEventStore {
|
||||
/// Stores decoded protocol events.
|
||||
async fn store_decoded_events(
|
||||
&self,
|
||||
events: &[crate::DecodedEventInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Materialized event storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait MaterializedEventStore {
|
||||
/// Stores materialized business events.
|
||||
async fn store_materialized_events(
|
||||
&self,
|
||||
events: &[crate::MaterializedEventInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
}
|
||||
|
||||
/// Processing ledger storage behavior.
|
||||
#[async_trait::async_trait]
|
||||
pub trait ProcessingLedgerStore {
|
||||
/// Marks an input as processed for a module version.
|
||||
async fn mark_processed(
|
||||
&self,
|
||||
mark: &crate::ProcessingLedgerMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome>;
|
||||
|
||||
/// Returns true when an input was already processed for a module version.
|
||||
async fn is_processed(&self, mark: &crate::ProcessingLedgerMark) -> ks_core::Result<bool>;
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
// file: ks-store/src/lib.rs
|
||||
// version: 5
|
||||
// version: 10
|
||||
|
||||
//! Backend-neutral storage contracts and PostgreSQL implementation.
|
||||
//! Backend-agnostic Solana storage contracts and store facade.
|
||||
#![warn(missing_docs)]
|
||||
#![deny(unreachable_pub)]
|
||||
#![forbid(unsafe_code)]
|
||||
@@ -9,13 +9,177 @@
|
||||
mod constants;
|
||||
mod contracts;
|
||||
mod postgres;
|
||||
mod store;
|
||||
|
||||
/// Canonical tracing target for storage operations.
|
||||
/// Canonical crate-internal tracing target shared through the crate-root facade.
|
||||
pub(crate) use self::constants::TRACING_TARGET;
|
||||
/// Crate-internal storage symbol `CORE_ACCOUNT_KEYS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `CORE_BALANCE_CHANGES_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `CORE_INNER_INSTRUCTIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `CORE_INSTRUCTIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `CORE_LOGS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_LOGS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `CORE_TRANSACTIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `DECODE_COVERAGE_DECLARATIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `DECODE_EVENTS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::DECODE_EVENTS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `MATERIALIZED_EVENTS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `PROCESSING_LEDGER_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::PROCESSING_LEDGER_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `PostgresStore` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::PostgresStore;
|
||||
/// Crate-internal storage symbol `PostgresStoreOptions` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::PostgresStoreOptions;
|
||||
/// Crate-internal storage symbol `PostgresTableDiagnosticSpec` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::PostgresTableDiagnosticSpec;
|
||||
/// Crate-internal storage symbol `RAW_TRANSACTIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `STORE_SCHEMA_ADVISORY_LOCK_ID` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
/// Crate-internal storage symbol `TRANSACTION_OBSERVATIONS_TABLE_NAME` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
/// Crate-internal storage symbol `apply_core_store_schema` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::apply_core_store_schema;
|
||||
/// Crate-internal storage symbol `apply_decode_store_schema` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::apply_decode_store_schema;
|
||||
/// Crate-internal storage symbol `apply_raw_store_schema` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::apply_raw_store_schema;
|
||||
/// Crate-internal storage symbol `core_store_schema_statements` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::core_store_schema_statements;
|
||||
/// Crate-internal storage symbol `core_store_table_diagnostic_specs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::core_store_table_diagnostic_specs;
|
||||
/// Crate-internal storage symbol `decode_store_schema_statements` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::decode_store_schema_statements;
|
||||
/// Crate-internal storage symbol `decode_store_table_diagnostic_specs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::decode_store_table_diagnostic_specs;
|
||||
/// Crate-internal storage symbol `has_raw_transaction_signature` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::has_raw_transaction_signature;
|
||||
/// Crate-internal storage symbol `has_transaction_observation_key` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::has_transaction_observation_key;
|
||||
/// Crate-internal storage symbol `insert_core_account_keys` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_account_keys;
|
||||
/// Crate-internal storage symbol `insert_core_balance_changes` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_balance_changes;
|
||||
/// Crate-internal storage symbol `insert_core_inner_instructions` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_inner_instructions;
|
||||
/// Crate-internal storage symbol `insert_core_instructions` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_instructions;
|
||||
/// Crate-internal storage symbol `insert_core_logs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_logs;
|
||||
/// Crate-internal storage symbol `insert_core_transaction` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_core_transaction;
|
||||
/// Crate-internal storage symbol `insert_raw_transaction` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_raw_transaction;
|
||||
/// Crate-internal storage symbol `insert_transaction_observation` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::insert_transaction_observation;
|
||||
/// Crate-internal storage symbol `is_core_extraction_current` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::is_core_extraction_current;
|
||||
/// Crate-internal storage symbol `is_decode_current` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::is_decode_current;
|
||||
/// Crate-internal storage symbol `list_core_instruction_replay_inputs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_core_instruction_replay_inputs;
|
||||
/// Crate-internal storage symbol `list_core_instructions_for_replay` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_core_instructions_for_replay;
|
||||
/// Crate-internal storage symbol `list_decode_coverage_summary` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_decode_coverage_summary;
|
||||
/// Crate-internal storage symbol `list_decode_inputs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_decode_inputs;
|
||||
/// Crate-internal storage symbol `list_decode_replay_inputs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_decode_replay_inputs;
|
||||
/// Crate-internal storage symbol `list_materialized_events` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_materialized_events;
|
||||
/// Crate-internal storage symbol `list_raw_transactions_for_core_extraction` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_raw_transactions_for_core_extraction;
|
||||
/// Crate-internal storage symbol `list_replay_entity_summaries` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_replay_entity_summaries;
|
||||
/// Crate-internal storage symbol `list_replay_program_summaries` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_replay_program_summaries;
|
||||
/// Crate-internal storage symbol `list_replay_transaction_candidates` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::list_replay_transaction_candidates;
|
||||
/// Crate-internal storage symbol `load_current_schema` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::load_current_schema;
|
||||
/// Crate-internal storage symbol `load_latest_migration_version` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::load_latest_migration_version;
|
||||
/// Crate-internal storage symbol `load_migration_table_name` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::load_migration_table_name;
|
||||
/// Crate-internal storage symbol `load_server_version` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::load_server_version;
|
||||
/// Crate-internal storage symbol `load_table_statistics` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::load_table_statistics;
|
||||
/// Crate-internal storage symbol `mark_core_extraction_failed` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::mark_core_extraction_failed;
|
||||
/// Crate-internal storage symbol `mark_decode_failed` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::mark_decode_failed;
|
||||
/// Crate-internal storage symbol `persist_core_extraction` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::persist_core_extraction;
|
||||
/// Crate-internal storage symbol `persist_decode_coverage_declarations` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::persist_decode_coverage_declarations;
|
||||
/// Crate-internal storage symbol `persist_decode_result` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::persist_decode_result;
|
||||
/// Crate-internal storage symbol `persist_materialization_result` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::persist_materialization_result;
|
||||
#[cfg(test)]
|
||||
/// Crate-internal serialized PostgreSQL test guard shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::postgres_test_guard;
|
||||
/// Crate-internal storage symbol `raw_store_schema_statements` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::raw_store_schema_statements;
|
||||
/// Crate-internal storage symbol `raw_store_table_diagnostic_specs` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::raw_store_table_diagnostic_specs;
|
||||
/// Crate-internal storage symbol `run_health_check` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::run_health_check;
|
||||
/// Crate-internal storage symbol `table_exists` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_exists;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_account_keys_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_account_keys_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_balance_changes_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_balance_changes_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_inner_instructions_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_inner_instructions_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_instructions_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_instructions_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_logs_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_logs_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_core_transactions_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_core_transactions_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_decode_coverage_declarations_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_decode_coverage_declarations_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_decode_coverage_observations_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_decode_coverage_observations_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_decode_events_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_decode_events_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_mat_events_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_mat_events_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_obs_transaction_observations_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_obs_transaction_observations_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_ops_processing_ledger_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_ops_processing_ledger_sql;
|
||||
/// Crate-internal storage symbol `table_stats_kb_sol_raw_transactions_sql` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::table_stats_kb_sol_raw_transactions_sql;
|
||||
/// Crate-internal storage symbol `update_core_instruction_lifecycle` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::update_core_instruction_lifecycle;
|
||||
/// Crate-internal storage symbol `update_raw_payload_lifecycle` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::update_raw_payload_lifecycle;
|
||||
/// Crate-internal storage symbol `validate_core_store_table_names` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::validate_core_store_table_names;
|
||||
/// Crate-internal storage symbol `validate_decode_store_table_names` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::validate_decode_store_table_names;
|
||||
/// Crate-internal storage symbol `validate_raw_store_table_names` shared through the crate-root facade.
|
||||
pub(crate) use self::postgres::validate_raw_store_table_names;
|
||||
|
||||
/// Transitional logical schema contract identifier used until the `0.5.3-pre.3` rebuild.
|
||||
pub use self::constants::STORE_SCHEMA_CONTRACT_VERSION;
|
||||
/// Core account key insert contract.
|
||||
pub use self::contracts::CoreAccountKeyInsert;
|
||||
/// Core account key SQL-like row contract.
|
||||
/// Core account key persisted row contract.
|
||||
pub use self::contracts::CoreAccountKeyRow;
|
||||
/// Core account key source category.
|
||||
pub use self::contracts::CoreAccountKeySource;
|
||||
@@ -23,19 +187,19 @@ pub use self::contracts::CoreAccountKeySource;
|
||||
pub use self::contracts::CoreBalanceChangeInsert;
|
||||
/// Core balance change kind.
|
||||
pub use self::contracts::CoreBalanceChangeKind;
|
||||
/// Core balance change SQL-like row contract.
|
||||
/// Core balance change persisted row contract.
|
||||
pub use self::contracts::CoreBalanceChangeRow;
|
||||
/// Complete normalized core extraction write bundle.
|
||||
/// Complete normalized Core extraction write bundle.
|
||||
pub use self::contracts::CoreExtractionBundle;
|
||||
/// Failure details persisted for one canonical to core extraction attempt.
|
||||
/// Failure details persisted for one canonical-to-Core extraction attempt.
|
||||
pub use self::contracts::CoreExtractionFailure;
|
||||
/// Bounded canonical transaction selection filter for core extraction.
|
||||
/// Bounded canonical transaction selection filter for Core extraction.
|
||||
pub use self::contracts::CoreExtractionSelectionFilter;
|
||||
/// Canonical transaction to core extraction storage behavior.
|
||||
/// Canonical transaction-to-Core extraction storage behavior.
|
||||
pub use self::contracts::CoreExtractionStore;
|
||||
/// Core inner instruction insert contract.
|
||||
pub use self::contracts::CoreInnerInstructionInsert;
|
||||
/// Core inner instruction SQL-like row contract.
|
||||
/// Core inner instruction persisted row contract.
|
||||
pub use self::contracts::CoreInnerInstructionRow;
|
||||
/// Core instruction insert contract.
|
||||
pub use self::contracts::CoreInstructionInsert;
|
||||
@@ -45,15 +209,15 @@ pub use self::contracts::CoreInstructionLifecycleMark;
|
||||
pub use self::contracts::CoreInstructionProcessingState;
|
||||
/// Core instruction replay filter contract.
|
||||
pub use self::contracts::CoreInstructionReplayFilter;
|
||||
/// Core instruction SQL-like row contract.
|
||||
/// Core instruction persisted row contract.
|
||||
pub use self::contracts::CoreInstructionRow;
|
||||
/// Core log insert contract.
|
||||
pub use self::contracts::CoreLogInsert;
|
||||
/// Core log SQL-like row contract.
|
||||
/// Core log persisted row contract.
|
||||
pub use self::contracts::CoreLogRow;
|
||||
/// Core transaction insert contract.
|
||||
pub use self::contracts::CoreTransactionInsert;
|
||||
/// Core transaction SQL-like row contract.
|
||||
/// Core transaction persisted row contract.
|
||||
pub use self::contracts::CoreTransactionRow;
|
||||
/// Core Solana storage behavior.
|
||||
pub use self::contracts::CoreTransactionStore;
|
||||
@@ -65,7 +229,7 @@ pub use self::contracts::DecodeCoverageDeclarationInsert;
|
||||
pub use self::contracts::DecodeCoverageObservationInsert;
|
||||
/// One row of aggregated decoder coverage diagnostics.
|
||||
pub use self::contracts::DecodeCoverageSummaryRow;
|
||||
/// Failed decode attempt persisted in the common ledger.
|
||||
/// Failed decode attempt persisted in the common processing ledger.
|
||||
pub use self::contracts::DecodeFailure;
|
||||
/// One processor-owned decoded observation row.
|
||||
pub use self::contracts::DecodeObservationInsert;
|
||||
@@ -75,46 +239,28 @@ pub use self::contracts::DecodePersistenceBundle;
|
||||
pub use self::contracts::DecodePipelineStore;
|
||||
/// Bounded contextual instruction selection filter for decode campaigns.
|
||||
pub use self::contracts::DecodeSelectionFilter;
|
||||
/// Decoded event insert contract.
|
||||
pub use self::contracts::DecodedEventInsert;
|
||||
/// Decoded event SQL-like row contract.
|
||||
pub use self::contracts::DecodedEventRow;
|
||||
/// Decoded event storage behavior.
|
||||
pub use self::contracts::DecodedEventStore;
|
||||
/// Insert or upsert result contract returned by repositories.
|
||||
pub use self::contracts::InsertOutcome;
|
||||
/// Maximum number of materialized rows returned by one bounded query.
|
||||
pub use self::contracts::MAX_MATERIALIZED_EVENT_QUERY_ROWS;
|
||||
/// Maximum repository page size.
|
||||
pub use self::contracts::MAX_PAGE_SIZE;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::contracts::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// Atomic persistence bundle for one materializer and one decoded observation.
|
||||
pub use self::contracts::MaterializationPersistenceBundle;
|
||||
/// Bounded read-only materialized event selection.
|
||||
pub use self::contracts::MaterializedEventFilter;
|
||||
/// Materialized event insert contract.
|
||||
pub use self::contracts::MaterializedEventInsert;
|
||||
/// One materialized output returned by a bounded query.
|
||||
pub use self::contracts::MaterializedEventQueryRow;
|
||||
/// Materialized event SQL-like row contract.
|
||||
pub use self::contracts::MaterializedEventRow;
|
||||
/// Materialized event storage behavior.
|
||||
pub use self::contracts::MaterializedEventStore;
|
||||
/// One processor-owned materialized output row.
|
||||
/// One processor-owned generic materialized output row.
|
||||
pub use self::contracts::MaterializedOutputInsert;
|
||||
/// Page request contract for repository list operations.
|
||||
pub use self::contracts::PageRequest;
|
||||
/// Stable processing ledger identity.
|
||||
pub use self::contracts::ProcessingLedgerIdentity;
|
||||
/// Processing ledger mark request contract.
|
||||
pub use self::contracts::ProcessingLedgerMark;
|
||||
/// Processing ledger SQL-like row contract.
|
||||
pub use self::contracts::ProcessingLedgerRow;
|
||||
/// Stable processing ledger status.
|
||||
pub use self::contracts::ProcessingLedgerStatus;
|
||||
/// Processing ledger storage behavior.
|
||||
pub use self::contracts::ProcessingLedgerStore;
|
||||
/// Program observation storage behavior.
|
||||
pub use self::contracts::ProgramObservationStore;
|
||||
/// Raw payload lifecycle mark request.
|
||||
pub use self::contracts::RawPayloadLifecycleMark;
|
||||
/// Raw payload processing state.
|
||||
@@ -123,135 +269,75 @@ pub use self::contracts::RawPayloadProcessingState;
|
||||
pub use self::contracts::RawPayloadRetentionState;
|
||||
/// Canonical raw Solana transaction insert contract.
|
||||
pub use self::contracts::RawTransactionInsert;
|
||||
/// Canonical raw Solana transaction SQL-like row contract.
|
||||
/// Canonical raw Solana transaction persisted row contract.
|
||||
pub use self::contracts::RawTransactionRow;
|
||||
/// Raw transaction storage behavior.
|
||||
pub use self::contracts::RawTransactionStore;
|
||||
/// Bounded read-only filter for Core entity summaries.
|
||||
pub use self::contracts::ReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::contracts::ReplayEntityKind;
|
||||
/// Aggregated mint, owner, or account-key occurrences from Core facts.
|
||||
pub use self::contracts::ReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::contracts::ReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::contracts::ReplayProgramScope;
|
||||
/// Aggregated program occurrences across top-level instructions, inner instructions, and logs.
|
||||
pub use self::contracts::ReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with Core and processing diagnostics.
|
||||
pub use self::contracts::ReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::contracts::ReplayTransactionFilter;
|
||||
/// Sort direction for repository list operations.
|
||||
pub use self::contracts::SortDirection;
|
||||
/// Store backend diagnostic contract.
|
||||
/// Store backend diagnostic descriptor.
|
||||
pub use self::contracts::StoreBackendDescriptor;
|
||||
/// Store backend kind contract.
|
||||
pub use self::contracts::StoreBackendKind;
|
||||
/// Backend-neutral diagnostic snapshot.
|
||||
pub use self::contracts::StoreBackendDiagnostics;
|
||||
/// Store configuration summary safe for UI and logs.
|
||||
pub use self::contracts::StoreConfigurationSummary;
|
||||
/// Store backend health snapshot.
|
||||
pub use self::contracts::StoreHealthSnapshot;
|
||||
/// Store backend health status.
|
||||
pub use self::contracts::StoreHealthStatus;
|
||||
/// Store health storage behavior.
|
||||
pub use self::contracts::StoreHealthStore;
|
||||
/// Store initialization status.
|
||||
pub use self::contracts::StoreInitializationStatus;
|
||||
/// Initialization and model-verification summary captured while opening a store.
|
||||
pub use self::contracts::StoreInitializationSummary;
|
||||
/// Store migration diagnostic snapshot contract.
|
||||
pub use self::contracts::StoreMigrationSnapshot;
|
||||
/// Store migration status contract.
|
||||
pub use self::contracts::StoreMigrationStatus;
|
||||
/// Store model verification status.
|
||||
pub use self::contracts::StoreModelVerificationStatus;
|
||||
/// Verification summary for one logical store model.
|
||||
pub use self::contracts::StoreModelVerificationSummary;
|
||||
/// Count summary for one backend object category without exposing physical names.
|
||||
pub use self::contracts::StoreObjectVerificationSummary;
|
||||
/// Read-only diagnostics for one logical store resource.
|
||||
pub use self::contracts::StoreResourceDiagnostics;
|
||||
/// Read-only statistics for one logical store resource.
|
||||
pub use self::contracts::StoreResourceStatistics;
|
||||
/// Complete runtime summary safe for application diagnostics.
|
||||
pub use self::contracts::StoreRuntimeSummary;
|
||||
/// Transaction acquisition observation insert contract.
|
||||
pub use self::contracts::TransactionObservationInsert;
|
||||
/// Transaction acquisition observation origin.
|
||||
pub use self::contracts::TransactionObservationOrigin;
|
||||
/// Transaction acquisition observation SQL-like row contract.
|
||||
/// Transaction acquisition observation persisted row contract.
|
||||
pub use self::contracts::TransactionObservationRow;
|
||||
/// Transaction acquisition observation status.
|
||||
pub use self::contracts::TransactionObservationStatus;
|
||||
/// Storage error helper functions.
|
||||
pub use self::contracts::storage_contract_error;
|
||||
/// Allowed Solana table domains encoded in table names.
|
||||
pub use self::postgres::ALLOWED_SOLANA_TABLE_DOMAINS;
|
||||
/// Core account key table name.
|
||||
pub use self::postgres::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
/// Core balance change table name.
|
||||
pub use self::postgres::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
/// Core inner instruction table name.
|
||||
pub use self::postgres::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core instruction table name.
|
||||
pub use self::postgres::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core log table name.
|
||||
pub use self::postgres::CORE_LOGS_TABLE_NAME;
|
||||
/// Core Solana table names.
|
||||
pub use self::postgres::CORE_STORE_TABLE_NAMES;
|
||||
/// Core transaction table name.
|
||||
pub use self::postgres::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
/// Machine-readable decoder coverage declaration table name.
|
||||
pub use self::postgres::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
/// Observed decoder coverage classification table name.
|
||||
pub use self::postgres::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
/// Versioned decoded event table name.
|
||||
pub use self::postgres::DECODE_EVENTS_TABLE_NAME;
|
||||
/// Decode and materialization table names.
|
||||
pub use self::postgres::DECODE_STORE_TABLE_NAMES;
|
||||
/// Default PostgreSQL schema policy.
|
||||
pub use self::postgres::DEFAULT_SCHEMA_POLICY;
|
||||
/// Versioned materialized output table name.
|
||||
pub use self::postgres::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::postgres::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// PostgreSQL migration strategy used by this crate.
|
||||
pub use self::postgres::MIGRATION_STRATEGY;
|
||||
/// Migration table name used by sqlx.
|
||||
pub use self::postgres::MIGRATION_TABLE_NAME;
|
||||
/// Processing ledger table name.
|
||||
pub use self::postgres::PROCESSING_LEDGER_TABLE_NAME;
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
pub use self::postgres::PostgresBackendDiagnostics;
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
pub use self::postgres::PostgresReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::postgres::PostgresReplayEntityKind;
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
pub use self::postgres::PostgresReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::postgres::PostgresReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::postgres::PostgresReplayProgramScope;
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
pub use self::postgres::PostgresReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
pub use self::postgres::PostgresReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::postgres::PostgresReplayTransactionFilter;
|
||||
/// PostgreSQL store handle.
|
||||
pub use self::postgres::PostgresStore;
|
||||
/// PostgreSQL store connection options.
|
||||
pub use self::postgres::PostgresStoreOptions;
|
||||
/// PostgreSQL diagnostic table specification.
|
||||
pub use self::postgres::PostgresTableDiagnosticSpec;
|
||||
/// PostgreSQL table diagnostic snapshot.
|
||||
pub use self::postgres::PostgresTableDiagnostics;
|
||||
/// PostgreSQL table statistics snapshot.
|
||||
pub use self::postgres::PostgresTableStatistics;
|
||||
/// Canonical transaction acquisition table names.
|
||||
pub use self::postgres::RAW_STORE_TABLE_NAMES;
|
||||
/// Canonical raw transaction table name.
|
||||
pub use self::postgres::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
/// Solana application table prefix.
|
||||
pub use self::postgres::SOLANA_TABLE_PREFIX;
|
||||
/// Advisory lock id used while initializing store tables.
|
||||
pub use self::postgres::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
/// Transaction acquisition observation table name.
|
||||
pub use self::postgres::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
/// Core store SQL statements applied by the idempotent initializer.
|
||||
pub use self::postgres::core_store_schema_statements;
|
||||
/// Core table diagnostic specs.
|
||||
pub use self::postgres::core_store_table_diagnostic_specs;
|
||||
/// Decode and materialization SQL statements applied by the idempotent initializer.
|
||||
pub use self::postgres::decode_store_schema_statements;
|
||||
/// Decode and materialization table diagnostic specs.
|
||||
pub use self::postgres::decode_store_table_diagnostic_specs;
|
||||
/// Returns true when a Solana table name follows canonical rules.
|
||||
pub use self::postgres::is_valid_solana_table_name;
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub use self::postgres::mask_postgres_dsn;
|
||||
/// Canonical transaction acquisition SQL statements.
|
||||
pub use self::postgres::raw_store_schema_statements;
|
||||
/// Canonical transaction acquisition diagnostic specs.
|
||||
pub use self::postgres::raw_store_table_diagnostic_specs;
|
||||
/// Validates core store table names.
|
||||
pub use self::postgres::validate_core_store_table_names;
|
||||
/// Validates decode and materialization store table names.
|
||||
pub use self::postgres::validate_decode_store_table_names;
|
||||
/// Validates canonical transaction acquisition table names.
|
||||
pub use self::postgres::validate_raw_store_table_names;
|
||||
/// Validates a canonical Solana table name.
|
||||
pub use self::postgres::validate_solana_table_name;
|
||||
/// Current normalized core replay input contract version.
|
||||
/// Backend-agnostic persistent store facade.
|
||||
pub use self::store::Store;
|
||||
/// Backend-agnostic options used to open one persistent store.
|
||||
pub use self::store::StoreOpenOptions;
|
||||
/// Current normalized Core replay input contract version.
|
||||
pub use ks_lib::MD_CORE_REPLAY_INPUT_CONTRACT_VERSION;
|
||||
/// Decoder replay input containing one instruction plus extracted transaction context.
|
||||
pub use ks_lib::MdCoreInstructionReplayInput;
|
||||
|
||||
@@ -1,111 +1,94 @@
|
||||
// file: ks-store/src/postgres.rs
|
||||
// version: 4
|
||||
// version: 11
|
||||
|
||||
//! PostgreSQL storage implementation boundary.
|
||||
//! Private PostgreSQL implementation of the backend-agnostic store contracts.
|
||||
|
||||
mod migrations;
|
||||
mod query;
|
||||
mod replay_candidates;
|
||||
mod repository;
|
||||
mod store;
|
||||
#[cfg(test)]
|
||||
mod test_serial;
|
||||
|
||||
/// Allowed Solana table domains encoded in table names.
|
||||
pub use self::migrations::ALLOWED_SOLANA_TABLE_DOMAINS;
|
||||
/// Core account key table name.
|
||||
pub use self::migrations::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
/// Core balance change table name.
|
||||
pub use self::migrations::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
/// Core inner instruction table name.
|
||||
pub use self::migrations::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core instruction table name.
|
||||
pub use self::migrations::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
/// Core log table name.
|
||||
pub use self::migrations::CORE_LOGS_TABLE_NAME;
|
||||
/// Core Solana table names introduced by `0.2.4`.
|
||||
pub use self::migrations::CORE_STORE_TABLE_NAMES;
|
||||
/// Core transaction table name.
|
||||
pub use self::migrations::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
/// Machine-readable decoder coverage declaration table name.
|
||||
pub use self::migrations::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
/// Observed decoder coverage classification table name.
|
||||
pub use self::migrations::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
/// Versioned decoded event table name.
|
||||
pub use self::migrations::DECODE_EVENTS_TABLE_NAME;
|
||||
/// Decode and materialization table names introduced by `0.4.0`.
|
||||
pub use self::migrations::DECODE_STORE_TABLE_NAMES;
|
||||
/// Default PostgreSQL schema policy.
|
||||
pub use self::migrations::DEFAULT_SCHEMA_POLICY;
|
||||
/// Versioned materialized output table name.
|
||||
pub use self::migrations::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
/// PostgreSQL migration strategy used by this crate.
|
||||
pub use self::migrations::MIGRATION_STRATEGY;
|
||||
/// Migration table name used by sqlx when migrations are enabled later.
|
||||
pub use self::migrations::MIGRATION_TABLE_NAME;
|
||||
/// Processing ledger table name.
|
||||
pub use self::migrations::PROCESSING_LEDGER_TABLE_NAME;
|
||||
/// Table diagnostic metadata.
|
||||
pub use self::migrations::PostgresTableDiagnosticSpec;
|
||||
/// Canonical transaction acquisition table names active since `0.3.1`.
|
||||
pub use self::migrations::RAW_STORE_TABLE_NAMES;
|
||||
/// Canonical raw transaction table name.
|
||||
pub use self::migrations::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
/// Solana application table prefix.
|
||||
pub use self::migrations::SOLANA_TABLE_PREFIX;
|
||||
/// Advisory lock id used while initializing store schemas.
|
||||
pub use self::migrations::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
/// Transaction acquisition observation table name.
|
||||
pub use self::migrations::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
/// Core store SQL statements applied by the idempotent initializer.
|
||||
pub use self::migrations::core_store_schema_statements;
|
||||
/// Core table diagnostic specs.
|
||||
pub use self::migrations::core_store_table_diagnostic_specs;
|
||||
/// Decode and materialization SQL statements applied by the idempotent initializer.
|
||||
pub use self::migrations::decode_store_schema_statements;
|
||||
/// Decode and materialization table diagnostic specs.
|
||||
pub use self::migrations::decode_store_table_diagnostic_specs;
|
||||
/// Returns true when a Solana table name follows the canonical prefix and domain rules.
|
||||
pub use self::migrations::is_valid_solana_table_name;
|
||||
/// Canonical transaction acquisition SQL statements applied by the initializer.
|
||||
pub use self::migrations::raw_store_schema_statements;
|
||||
/// Canonical transaction acquisition diagnostic specs.
|
||||
pub use self::migrations::raw_store_table_diagnostic_specs;
|
||||
/// Validates core store table names.
|
||||
pub use self::migrations::validate_core_store_table_names;
|
||||
/// Validates decode and materialization store table names.
|
||||
pub use self::migrations::validate_decode_store_table_names;
|
||||
/// Validates canonical transaction acquisition table names.
|
||||
pub use self::migrations::validate_raw_store_table_names;
|
||||
/// Validates a canonical Solana table name.
|
||||
pub use self::migrations::validate_solana_table_name;
|
||||
/// Maximum number of rows returned by one replay candidate query.
|
||||
pub use self::replay_candidates::MAX_REPLAY_CANDIDATE_ROWS;
|
||||
/// Bounded read-only filter for core entity summaries.
|
||||
pub use self::replay_candidates::PostgresReplayEntityFilter;
|
||||
/// Core entity kind used while filtering replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayEntityKind;
|
||||
/// Aggregated mint, owner or account-key occurrences from core tables.
|
||||
pub use self::replay_candidates::PostgresReplayEntitySummary;
|
||||
/// Bounded read-only filter for program summaries.
|
||||
pub use self::replay_candidates::PostgresReplayProgramFilter;
|
||||
/// Program occurrence scope used while filtering replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayProgramScope;
|
||||
/// Aggregated program occurrences across outer, inner and linked logs.
|
||||
pub use self::replay_candidates::PostgresReplayProgramSummary;
|
||||
/// One raw transaction candidate enriched with core and ledger diagnostics.
|
||||
pub use self::replay_candidates::PostgresReplayTransactionCandidate;
|
||||
/// Bounded read-only filter for transaction replay candidates.
|
||||
pub use self::replay_candidates::PostgresReplayTransactionFilter;
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
pub use self::store::PostgresBackendDiagnostics;
|
||||
/// Minimal PostgreSQL store handle.
|
||||
pub use self::store::PostgresStore;
|
||||
/// PostgreSQL store connection options.
|
||||
pub use self::store::PostgresStoreOptions;
|
||||
/// PostgreSQL table diagnostic snapshot.
|
||||
pub use self::store::PostgresTableDiagnostics;
|
||||
/// PostgreSQL table statistics snapshot.
|
||||
pub use self::store::PostgresTableStatistics;
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub use self::store::mask_postgres_dsn;
|
||||
pub(crate) use self::migrations::CORE_ACCOUNT_KEYS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::CORE_BALANCE_CHANGES_TABLE_NAME;
|
||||
pub(crate) use self::migrations::CORE_INNER_INSTRUCTIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::CORE_INSTRUCTIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::CORE_LOGS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::CORE_TRANSACTIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::DECODE_EVENTS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::MATERIALIZED_EVENTS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::PROCESSING_LEDGER_TABLE_NAME;
|
||||
pub(crate) use self::migrations::PostgresTableDiagnosticSpec;
|
||||
pub(crate) use self::migrations::RAW_TRANSACTIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::STORE_SCHEMA_ADVISORY_LOCK_ID;
|
||||
pub(crate) use self::migrations::TRANSACTION_OBSERVATIONS_TABLE_NAME;
|
||||
pub(crate) use self::migrations::core_store_schema_statements;
|
||||
pub(crate) use self::migrations::core_store_table_diagnostic_specs;
|
||||
pub(crate) use self::migrations::decode_store_schema_statements;
|
||||
pub(crate) use self::migrations::decode_store_table_diagnostic_specs;
|
||||
pub(crate) use self::migrations::raw_store_schema_statements;
|
||||
pub(crate) use self::migrations::raw_store_table_diagnostic_specs;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_account_keys_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_balance_changes_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_inner_instructions_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_instructions_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_logs_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_core_transactions_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_decode_coverage_declarations_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_decode_coverage_observations_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_decode_events_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_mat_events_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_obs_transaction_observations_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_ops_processing_ledger_sql;
|
||||
pub(crate) use self::migrations::table_stats_kb_sol_raw_transactions_sql;
|
||||
pub(crate) use self::migrations::validate_core_store_table_names;
|
||||
pub(crate) use self::migrations::validate_decode_store_table_names;
|
||||
pub(crate) use self::migrations::validate_raw_store_table_names;
|
||||
pub(crate) use self::query::apply_core_store_schema;
|
||||
pub(crate) use self::query::apply_decode_store_schema;
|
||||
pub(crate) use self::query::apply_raw_store_schema;
|
||||
pub(crate) use self::query::has_raw_transaction_signature;
|
||||
pub(crate) use self::query::has_transaction_observation_key;
|
||||
pub(crate) use self::query::insert_core_account_keys;
|
||||
pub(crate) use self::query::insert_core_balance_changes;
|
||||
pub(crate) use self::query::insert_core_inner_instructions;
|
||||
pub(crate) use self::query::insert_core_instructions;
|
||||
pub(crate) use self::query::insert_core_logs;
|
||||
pub(crate) use self::query::insert_core_transaction;
|
||||
pub(crate) use self::query::insert_raw_transaction;
|
||||
pub(crate) use self::query::insert_transaction_observation;
|
||||
pub(crate) use self::query::is_core_extraction_current;
|
||||
pub(crate) use self::query::is_decode_current;
|
||||
pub(crate) use self::query::list_core_instruction_replay_inputs;
|
||||
pub(crate) use self::query::list_core_instructions_for_replay;
|
||||
pub(crate) use self::query::list_decode_coverage_summary;
|
||||
pub(crate) use self::query::list_decode_inputs;
|
||||
pub(crate) use self::query::list_decode_replay_inputs;
|
||||
pub(crate) use self::query::list_materialized_events;
|
||||
pub(crate) use self::query::list_raw_transactions_for_core_extraction;
|
||||
pub(crate) use self::query::list_replay_entity_summaries;
|
||||
pub(crate) use self::query::list_replay_program_summaries;
|
||||
pub(crate) use self::query::list_replay_transaction_candidates;
|
||||
pub(crate) use self::query::load_current_schema;
|
||||
pub(crate) use self::query::load_latest_migration_version;
|
||||
pub(crate) use self::query::load_migration_table_name;
|
||||
pub(crate) use self::query::load_server_version;
|
||||
pub(crate) use self::query::load_table_statistics;
|
||||
pub(crate) use self::query::mark_core_extraction_failed;
|
||||
pub(crate) use self::query::mark_decode_failed;
|
||||
pub(crate) use self::query::persist_core_extraction;
|
||||
pub(crate) use self::query::persist_decode_coverage_declarations;
|
||||
pub(crate) use self::query::persist_decode_result;
|
||||
pub(crate) use self::query::persist_materialization_result;
|
||||
pub(crate) use self::query::run_health_check;
|
||||
pub(crate) use self::query::table_exists;
|
||||
pub(crate) use self::query::update_core_instruction_lifecycle;
|
||||
pub(crate) use self::query::update_raw_payload_lifecycle;
|
||||
pub(crate) use self::store::PostgresStore;
|
||||
pub(crate) use self::store::PostgresStoreOptions;
|
||||
|
||||
#[cfg(test)]
|
||||
pub(crate) use self::test_serial::postgres_test_guard;
|
||||
|
||||
@@ -1,165 +1,175 @@
|
||||
// file: ks-store/src/postgres/migrations.rs
|
||||
// version: 3
|
||||
// version: 7
|
||||
|
||||
//! PostgreSQL migration conventions for the storage backend.
|
||||
|
||||
/// PostgreSQL schema policy used by application migrations.
|
||||
pub const DEFAULT_SCHEMA_POLICY: &str = "current_profile_schema";
|
||||
/// Canonical Solana table prefix.
|
||||
pub const SOLANA_TABLE_PREFIX: &str = "kb_sol_";
|
||||
/// Migration table name used by sqlx when migrations are enabled later.
|
||||
pub const MIGRATION_TABLE_NAME: &str = "_sqlx_migrations";
|
||||
/// Migration strategy used by the canonical transaction and core stores.
|
||||
pub const MIGRATION_STRATEGY: &str =
|
||||
"idempotent_canonical_transaction_observation_and_core_tables_without_application_schemas";
|
||||
const SOLANA_TABLE_PREFIX: &str = "kb_sol_";
|
||||
/// Advisory lock id used while applying idempotent store schema statements.
|
||||
pub const STORE_SCHEMA_ADVISORY_LOCK_ID: i64 = 2_024_000_204;
|
||||
pub(crate) const STORE_SCHEMA_ADVISORY_LOCK_ID: i64 = 2_024_000_204;
|
||||
/// Canonical raw transaction table name.
|
||||
pub const RAW_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_raw_transactions";
|
||||
pub(crate) const RAW_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_raw_transactions";
|
||||
/// Lightweight transaction acquisition observation table name.
|
||||
pub const TRANSACTION_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_obs_transaction_observations";
|
||||
pub(crate) const TRANSACTION_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_obs_transaction_observations";
|
||||
/// Core transaction table name.
|
||||
pub const CORE_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_core_transactions";
|
||||
pub(crate) const CORE_TRANSACTIONS_TABLE_NAME: &str = "kb_sol_core_transactions";
|
||||
/// Core account key table name.
|
||||
pub const CORE_ACCOUNT_KEYS_TABLE_NAME: &str = "kb_sol_core_account_keys";
|
||||
pub(crate) const CORE_ACCOUNT_KEYS_TABLE_NAME: &str = "kb_sol_core_account_keys";
|
||||
/// Core instruction table name.
|
||||
pub const CORE_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_instructions";
|
||||
pub(crate) const CORE_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_instructions";
|
||||
/// Core inner instruction table name.
|
||||
pub const CORE_INNER_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_inner_instructions";
|
||||
pub(crate) const CORE_INNER_INSTRUCTIONS_TABLE_NAME: &str = "kb_sol_core_inner_instructions";
|
||||
/// Core log table name.
|
||||
pub const CORE_LOGS_TABLE_NAME: &str = "kb_sol_core_logs";
|
||||
pub(crate) const CORE_LOGS_TABLE_NAME: &str = "kb_sol_core_logs";
|
||||
/// Core balance change table name.
|
||||
pub const CORE_BALANCE_CHANGES_TABLE_NAME: &str = "kb_sol_core_balance_changes";
|
||||
pub(crate) const CORE_BALANCE_CHANGES_TABLE_NAME: &str = "kb_sol_core_balance_changes";
|
||||
/// Processing ledger table name.
|
||||
pub const PROCESSING_LEDGER_TABLE_NAME: &str = "kb_sol_ops_processing_ledger";
|
||||
pub(crate) const PROCESSING_LEDGER_TABLE_NAME: &str = "kb_sol_ops_processing_ledger";
|
||||
/// Versioned decoded event table name.
|
||||
pub const DECODE_EVENTS_TABLE_NAME: &str = "kb_sol_decode_events";
|
||||
pub(crate) const DECODE_EVENTS_TABLE_NAME: &str = "kb_sol_decode_events";
|
||||
/// Machine-readable decoder coverage declaration table name.
|
||||
pub const DECODE_COVERAGE_DECLARATIONS_TABLE_NAME: &str = "kb_sol_decode_coverage_declarations";
|
||||
pub(crate) const DECODE_COVERAGE_DECLARATIONS_TABLE_NAME: &str =
|
||||
"kb_sol_decode_coverage_declarations";
|
||||
/// Observed decoder coverage classification table name.
|
||||
pub const DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME: &str = "kb_sol_decode_coverage_observations";
|
||||
pub(crate) const DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME: &str =
|
||||
"kb_sol_decode_coverage_observations";
|
||||
/// Versioned materialized output table name.
|
||||
pub const MATERIALIZED_EVENTS_TABLE_NAME: &str = "kb_sol_mat_events";
|
||||
pub(crate) const MATERIALIZED_EVENTS_TABLE_NAME: &str = "kb_sol_mat_events";
|
||||
/// Canonical transaction acquisition table names active since `0.3.1`.
|
||||
pub const RAW_STORE_TABLE_NAMES: &[&str] =
|
||||
&[crate::RAW_TRANSACTIONS_TABLE_NAME, crate::TRANSACTION_OBSERVATIONS_TABLE_NAME];
|
||||
const RAW_STORE_TABLE_NAMES: &[&str] =
|
||||
&[RAW_TRANSACTIONS_TABLE_NAME, TRANSACTION_OBSERVATIONS_TABLE_NAME];
|
||||
/// Core Solana table names introduced by `0.2.4`.
|
||||
pub const CORE_STORE_TABLE_NAMES: &[&str] = &[
|
||||
crate::CORE_TRANSACTIONS_TABLE_NAME,
|
||||
crate::CORE_ACCOUNT_KEYS_TABLE_NAME,
|
||||
crate::CORE_INSTRUCTIONS_TABLE_NAME,
|
||||
crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME,
|
||||
crate::CORE_LOGS_TABLE_NAME,
|
||||
crate::CORE_BALANCE_CHANGES_TABLE_NAME,
|
||||
crate::PROCESSING_LEDGER_TABLE_NAME,
|
||||
const CORE_STORE_TABLE_NAMES: &[&str] = &[
|
||||
CORE_TRANSACTIONS_TABLE_NAME,
|
||||
CORE_ACCOUNT_KEYS_TABLE_NAME,
|
||||
CORE_INSTRUCTIONS_TABLE_NAME,
|
||||
CORE_INNER_INSTRUCTIONS_TABLE_NAME,
|
||||
CORE_LOGS_TABLE_NAME,
|
||||
CORE_BALANCE_CHANGES_TABLE_NAME,
|
||||
PROCESSING_LEDGER_TABLE_NAME,
|
||||
];
|
||||
/// Decode and materialization table names introduced by `0.4.0`.
|
||||
pub const DECODE_STORE_TABLE_NAMES: &[&str] = &[
|
||||
crate::DECODE_EVENTS_TABLE_NAME,
|
||||
crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
|
||||
crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
|
||||
crate::MATERIALIZED_EVENTS_TABLE_NAME,
|
||||
const DECODE_STORE_TABLE_NAMES: &[&str] = &[
|
||||
DECODE_EVENTS_TABLE_NAME,
|
||||
DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
|
||||
DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
|
||||
MATERIALIZED_EVENTS_TABLE_NAME,
|
||||
];
|
||||
/// Allowed Solana table domains encoded after the `kb_sol_` prefix.
|
||||
pub const ALLOWED_SOLANA_TABLE_DOMAINS: &[&str] =
|
||||
const ALLOWED_SOLANA_TABLE_DOMAINS: &[&str] =
|
||||
&["raw", "core", "obs", "decode", "mat", "catalog", "agg", "ops", "wallet"];
|
||||
|
||||
/// Diagnostic metadata for one expected PostgreSQL table.
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresTableDiagnosticSpec {
|
||||
/// Expected table name.
|
||||
pub(crate) struct PostgresTableDiagnosticSpec {
|
||||
/// Physical PostgreSQL table name kept backend-private.
|
||||
pub table_name: &'static str,
|
||||
/// Logical Solana domain encoded in the table name.
|
||||
pub domain: &'static str,
|
||||
/// Human-readable table role.
|
||||
/// Stable backend-independent logical resource code.
|
||||
pub resource_code: &'static str,
|
||||
/// Logical store model code.
|
||||
pub model_code: &'static str,
|
||||
/// Human-readable resource role.
|
||||
pub role: &'static str,
|
||||
}
|
||||
|
||||
/// Diagnostic metadata for canonical transaction acquisition tables.
|
||||
pub fn raw_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 2] {
|
||||
pub(crate) fn raw_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 2] {
|
||||
return [
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::RAW_TRANSACTIONS_TABLE_NAME,
|
||||
domain: "raw",
|
||||
table_name: RAW_TRANSACTIONS_TABLE_NAME,
|
||||
resource_code: "raw_transactions",
|
||||
model_code: "raw",
|
||||
role: "Canonical transactions",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::TRANSACTION_OBSERVATIONS_TABLE_NAME,
|
||||
domain: "obs",
|
||||
table_name: TRANSACTION_OBSERVATIONS_TABLE_NAME,
|
||||
resource_code: "transaction_observations",
|
||||
model_code: "raw",
|
||||
role: "Transaction acquisition observations",
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/// Diagnostic metadata for core Solana store tables.
|
||||
pub fn core_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 7] {
|
||||
pub(crate) fn core_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 7] {
|
||||
return [
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_TRANSACTIONS_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_TRANSACTIONS_TABLE_NAME,
|
||||
resource_code: "transactions",
|
||||
model_code: "core",
|
||||
role: "Transactions",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_ACCOUNT_KEYS_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_ACCOUNT_KEYS_TABLE_NAME,
|
||||
resource_code: "account_keys",
|
||||
model_code: "core",
|
||||
role: "Resolved account keys",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_INSTRUCTIONS_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_INSTRUCTIONS_TABLE_NAME,
|
||||
resource_code: "top_level_instructions",
|
||||
model_code: "core",
|
||||
role: "Top-level instructions",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_INNER_INSTRUCTIONS_TABLE_NAME,
|
||||
resource_code: "inner_instructions",
|
||||
model_code: "core",
|
||||
role: "Inner instruction tree",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_LOGS_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_LOGS_TABLE_NAME,
|
||||
resource_code: "logs",
|
||||
model_code: "core",
|
||||
role: "Transaction logs",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::CORE_BALANCE_CHANGES_TABLE_NAME,
|
||||
domain: "core",
|
||||
table_name: CORE_BALANCE_CHANGES_TABLE_NAME,
|
||||
resource_code: "balance_changes",
|
||||
model_code: "core",
|
||||
role: "Balance changes",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::PROCESSING_LEDGER_TABLE_NAME,
|
||||
domain: "ops",
|
||||
table_name: PROCESSING_LEDGER_TABLE_NAME,
|
||||
resource_code: "processing_ledger",
|
||||
model_code: "processing",
|
||||
role: "Processing ledger",
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/// Diagnostic metadata for decode and materialization store tables.
|
||||
pub fn decode_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 4] {
|
||||
pub(crate) fn decode_store_table_diagnostic_specs() -> [crate::PostgresTableDiagnosticSpec; 4] {
|
||||
return [
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::DECODE_EVENTS_TABLE_NAME,
|
||||
domain: "decode",
|
||||
table_name: DECODE_EVENTS_TABLE_NAME,
|
||||
resource_code: "decoded_observations",
|
||||
model_code: "decode",
|
||||
role: "Versioned decoded observations",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
|
||||
domain: "decode",
|
||||
table_name: DECODE_COVERAGE_DECLARATIONS_TABLE_NAME,
|
||||
resource_code: "decode_coverage_declarations",
|
||||
model_code: "decode",
|
||||
role: "Declared decoder coverage",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
|
||||
domain: "decode",
|
||||
table_name: DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME,
|
||||
resource_code: "decode_coverage_observations",
|
||||
model_code: "decode",
|
||||
role: "Observed decoder coverage",
|
||||
},
|
||||
crate::PostgresTableDiagnosticSpec {
|
||||
table_name: crate::MATERIALIZED_EVENTS_TABLE_NAME,
|
||||
domain: "mat",
|
||||
table_name: MATERIALIZED_EVENTS_TABLE_NAME,
|
||||
resource_code: "materialized_outputs",
|
||||
model_code: "materialization",
|
||||
role: "Versioned materialized outputs",
|
||||
},
|
||||
];
|
||||
}
|
||||
|
||||
/// Idempotent SQL statements creating or upgrading the canonical transaction acquisition store.
|
||||
pub fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
|
||||
pub(crate) fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
|
||||
return vec![
|
||||
crate::postgres::migrations::migrate_legacy_raw_table_name_sql(),
|
||||
crate::postgres::migrations::migrate_legacy_raw_columns_sql(),
|
||||
@@ -185,7 +195,7 @@ pub fn raw_store_schema_statements() -> std::vec::Vec<&'static str> {
|
||||
}
|
||||
|
||||
/// Idempotent SQL statements creating the minimal core Solana store.
|
||||
pub fn core_store_schema_statements() -> [&'static str; 28] {
|
||||
pub(crate) fn core_store_schema_statements() -> [&'static str; 28] {
|
||||
return [
|
||||
crate::postgres::migrations::create_table_kb_sol_core_transactions_sql(),
|
||||
crate::postgres::migrations::create_ux_kb_sol_core_transactions_signature_sql(),
|
||||
@@ -219,7 +229,7 @@ pub fn core_store_schema_statements() -> [&'static str; 28] {
|
||||
}
|
||||
|
||||
/// Idempotent SQL statements creating the common decode and materialization store.
|
||||
pub fn decode_store_schema_statements() -> [&'static str; 15] {
|
||||
pub(crate) fn decode_store_schema_statements() -> [&'static str; 15] {
|
||||
return [
|
||||
crate::postgres::migrations::create_table_kb_sol_decode_events_sql(),
|
||||
crate::postgres::migrations::create_ux_kb_sol_decode_events_identity_sql(),
|
||||
@@ -240,7 +250,7 @@ pub fn decode_store_schema_statements() -> [&'static str; 15] {
|
||||
}
|
||||
|
||||
/// SQL conditionally renaming the historical raw RPC transaction table.
|
||||
pub(in crate::postgres) fn migrate_legacy_raw_table_name_sql() -> &'static str {
|
||||
fn migrate_legacy_raw_table_name_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('kb_sol_raw_rpc_transactions') IS NOT NULL
|
||||
@@ -255,7 +265,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL conditionally renaming historical raw payload columns and adding the format version.
|
||||
pub(in crate::postgres) fn migrate_legacy_raw_columns_sql() -> &'static str {
|
||||
fn migrate_legacy_raw_columns_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('kb_sol_raw_transactions') IS NOT NULL THEN
|
||||
@@ -274,7 +284,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL conditionally renaming historical canonical transaction constraints.
|
||||
pub(in crate::postgres) fn migrate_legacy_raw_constraints_sql() -> &'static str {
|
||||
fn migrate_legacy_raw_constraints_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('kb_sol_raw_transactions') IS NOT NULL THEN
|
||||
@@ -291,7 +301,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL conditionally renaming historical canonical transaction indexes.
|
||||
pub(in crate::postgres) fn migrate_legacy_raw_indexes_sql() -> &'static str {
|
||||
fn migrate_legacy_raw_indexes_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('ux_kb_sol_raw_rpc_transactions_signature') IS NOT NULL AND to_regclass('ux_kb_sol_raw_transactions_signature') IS NULL THEN ALTER INDEX ux_kb_sol_raw_rpc_transactions_signature RENAME TO ux_kb_sol_raw_transactions_signature; END IF;
|
||||
@@ -303,7 +313,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL conditionally renaming historical core-to-raw lineage.
|
||||
pub(in crate::postgres) fn migrate_legacy_core_lineage_sql() -> &'static str {
|
||||
fn migrate_legacy_core_lineage_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('kb_sol_core_transactions') IS NOT NULL THEN
|
||||
@@ -320,7 +330,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_raw_transactions`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_raw_transactions_sql() -> &'static str {
|
||||
fn create_table_kb_sol_raw_transactions_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_raw_transactions (
|
||||
id BIGSERIAL,
|
||||
signature TEXT NOT NULL,
|
||||
@@ -344,32 +354,32 @@ pub(in crate::postgres) fn create_table_kb_sol_raw_transactions_sql() -> &'stati
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature index for canonical raw transactions.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_raw_transactions_signature_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_raw_transactions_signature_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_raw_transactions_signature ON kb_sol_raw_transactions (signature)";
|
||||
}
|
||||
|
||||
/// SQL creating the slot index for canonical raw transactions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_slot_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_raw_transactions_slot_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_slot ON kb_sol_raw_transactions (slot)";
|
||||
}
|
||||
|
||||
/// SQL creating the created timestamp index for canonical raw transactions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_created_at_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_raw_transactions_created_at_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_created_at ON kb_sol_raw_transactions (created_at)";
|
||||
}
|
||||
|
||||
/// SQL creating the processing state index for canonical raw transactions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_processing_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_raw_transactions_processing_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_processing ON kb_sol_raw_transactions (processing_state)";
|
||||
}
|
||||
|
||||
/// SQL creating the optional canonical document hash index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_raw_transactions_canonical_hash_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_raw_transactions_canonical_hash_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_raw_transactions_canonical_hash ON kb_sol_raw_transactions (canonical_json_hash) WHERE canonical_json_hash IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_obs_transaction_observations`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_obs_transaction_observations_sql() -> &'static str {
|
||||
fn create_table_kb_sol_obs_transaction_observations_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_obs_transaction_observations (
|
||||
id BIGSERIAL,
|
||||
raw_transaction_id BIGINT,
|
||||
@@ -415,48 +425,42 @@ pub(in crate::postgres) fn create_table_kb_sol_obs_transaction_observations_sql(
|
||||
}
|
||||
|
||||
/// SQL creating the unique observation key index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_obs_transaction_observations_key_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_obs_transaction_observations_key_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_obs_transaction_observations_key ON kb_sol_obs_transaction_observations (observation_key)";
|
||||
}
|
||||
|
||||
/// SQL creating the optional signature index for transaction observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_signature_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_obs_transaction_observations_signature_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_signature ON kb_sol_obs_transaction_observations (signature) WHERE signature IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating the optional slot index for transaction observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_slot_sql() -> &'static str
|
||||
{
|
||||
fn create_ix_kb_sol_obs_transaction_observations_slot_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_slot ON kb_sol_obs_transaction_observations (slot) WHERE slot IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating the provider and acquisition method index for transaction observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_provider_method_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_obs_transaction_observations_provider_method_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_provider_method ON kb_sol_obs_transaction_observations (provider, acquisition_method)";
|
||||
}
|
||||
|
||||
/// SQL creating the received timestamp index for transaction observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_received_at_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_obs_transaction_observations_received_at_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_received_at ON kb_sol_obs_transaction_observations (received_at)";
|
||||
}
|
||||
|
||||
/// SQL creating the optional canonical transaction lineage index for observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_raw_transaction_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_obs_transaction_observations_raw_transaction_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_raw_transaction ON kb_sol_obs_transaction_observations (raw_transaction_id) WHERE raw_transaction_id IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating the status index for transaction observations.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_obs_transaction_observations_status_sql() -> &'static str
|
||||
{
|
||||
fn create_ix_kb_sol_obs_transaction_observations_status_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_obs_transaction_observations_status ON kb_sol_obs_transaction_observations (status)";
|
||||
}
|
||||
|
||||
/// SQL migrating lightweight metadata and dropping the historical WebSocket payload table.
|
||||
pub(in crate::postgres) fn migrate_and_drop_legacy_ws_notifications_sql() -> &'static str {
|
||||
fn migrate_and_drop_legacy_ws_notifications_sql() -> &'static str {
|
||||
return r#"DO $$
|
||||
BEGIN
|
||||
IF to_regclass('kb_sol_raw_ws_notifications') IS NOT NULL THEN
|
||||
@@ -471,7 +475,7 @@ $$"#;
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_transactions`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_transactions_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_transactions_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_transactions (
|
||||
id BIGSERIAL,
|
||||
raw_transaction_id BIGINT,
|
||||
@@ -489,22 +493,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_transactions_sql() -> &'stat
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature index for core transactions.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_transactions_signature_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_transactions_signature_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_transactions_signature ON kb_sol_core_transactions (signature)";
|
||||
}
|
||||
|
||||
/// SQL creating the slot index for core transactions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_transactions_slot_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_transactions_slot_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_slot ON kb_sol_core_transactions (slot)";
|
||||
}
|
||||
|
||||
/// SQL creating the created_at index for core transactions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_transactions_created_at_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_transactions_created_at_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_transactions_created_at ON kb_sol_core_transactions (created_at)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_account_keys`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_account_keys_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_account_keys_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_account_keys (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
@@ -528,17 +532,17 @@ pub(in crate::postgres) fn create_table_kb_sol_core_account_keys_sql() -> &'stat
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature/account index for core account keys.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_account_keys_sig_index_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_account_keys_sig_index_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_account_keys_sig_index ON kb_sol_core_account_keys (signature, account_index)";
|
||||
}
|
||||
|
||||
/// SQL creating the account key index for core account keys.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_account_keys_key_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_account_keys_key_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_account_keys_key ON kb_sol_core_account_keys (account_key)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_instructions`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_instructions_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_instructions_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_instructions (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
@@ -566,27 +570,27 @@ pub(in crate::postgres) fn create_table_kb_sol_core_instructions_sql() -> &'stat
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature/path index for core instructions.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_instructions_sig_path_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_instructions_sig_path_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_instructions_sig_path ON kb_sol_core_instructions (signature, instruction_path)";
|
||||
}
|
||||
|
||||
/// SQL creating the program index for core instructions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_program_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_instructions_program_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_program ON kb_sol_core_instructions (program_id)";
|
||||
}
|
||||
|
||||
/// SQL creating the slot index for core instructions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_slot_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_instructions_slot_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_slot ON kb_sol_core_instructions (slot)";
|
||||
}
|
||||
|
||||
/// SQL creating the processing state index for core instructions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_instructions_processing_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_instructions_processing_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_instructions_processing ON kb_sol_core_instructions (processing_state)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_inner_instructions`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_inner_instructions_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_inner_instructions_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_inner_instructions (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
@@ -610,22 +614,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_inner_instructions_sql() ->
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature/path index for core inner instructions.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_inner_instructions_sig_path_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_inner_instructions_sig_path_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_inner_instructions_sig_path ON kb_sol_core_inner_instructions (signature, instruction_path)";
|
||||
}
|
||||
|
||||
/// SQL creating the parent instruction index for core inner instructions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_inner_instructions_parent_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_inner_instructions_parent_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_parent ON kb_sol_core_inner_instructions (signature, parent_instruction_path)";
|
||||
}
|
||||
|
||||
/// SQL creating the program index for core inner instructions.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_inner_instructions_program_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_inner_instructions_program_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_inner_instructions_program ON kb_sol_core_inner_instructions (program_id)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_logs`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_logs_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_logs_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_logs (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
@@ -649,22 +653,22 @@ pub(in crate::postgres) fn create_table_kb_sol_core_logs_sql() -> &'static str {
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature/log index for core logs.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_logs_sig_index_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_logs_sig_index_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_logs_sig_index ON kb_sol_core_logs (signature, log_index)";
|
||||
}
|
||||
|
||||
/// SQL creating the program index for core logs.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_logs_program_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_logs_program_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_program ON kb_sol_core_logs (program_id) WHERE program_id IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating the instruction path index for core logs.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_logs_path_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_logs_path_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_logs_path ON kb_sol_core_logs (signature, instruction_path) WHERE instruction_path IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_core_balance_changes`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_core_balance_changes_sql() -> &'static str {
|
||||
fn create_table_kb_sol_core_balance_changes_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_core_balance_changes (
|
||||
id BIGSERIAL,
|
||||
transaction_id BIGINT NOT NULL,
|
||||
@@ -694,27 +698,28 @@ pub(in crate::postgres) fn create_table_kb_sol_core_balance_changes_sql() -> &'s
|
||||
}
|
||||
|
||||
/// SQL creating the unique signature/balance index for core balance changes.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_core_balance_changes_sig_index_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_core_balance_changes_sig_index_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_core_balance_changes_sig_index ON kb_sol_core_balance_changes (signature, balance_change_index)";
|
||||
}
|
||||
|
||||
/// SQL creating the account key index for core balance changes.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_balance_changes_account_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_balance_changes_account_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_account ON kb_sol_core_balance_changes (account_key) WHERE account_key IS NOT NULL";
|
||||
}
|
||||
|
||||
/// SQL creating the mint index for core balance changes.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_core_balance_changes_mint_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_core_balance_changes_mint_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_core_balance_changes_mint ON kb_sol_core_balance_changes (mint) WHERE mint IS NOT NULL";
|
||||
}
|
||||
|
||||
/// Returns true when a Solana table name follows the canonical prefix and domain rules.
|
||||
pub fn is_valid_solana_table_name(table_name: &str) -> bool {
|
||||
return crate::validate_solana_table_name(table_name).is_ok();
|
||||
#[cfg(test)]
|
||||
fn is_valid_solana_table_name(table_name: &str) -> bool {
|
||||
return validate_solana_table_name(table_name).is_ok();
|
||||
}
|
||||
|
||||
/// Validates a canonical Solana table name.
|
||||
pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
|
||||
fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
|
||||
let trimmed_table_name = table_name.trim();
|
||||
if trimmed_table_name.is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db("table name must not be empty"));
|
||||
@@ -724,12 +729,12 @@ pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
|
||||
"solana table names must not contain an explicit PostgreSQL schema",
|
||||
));
|
||||
}
|
||||
if !trimmed_table_name.starts_with(crate::SOLANA_TABLE_PREFIX) {
|
||||
if !trimmed_table_name.starts_with(SOLANA_TABLE_PREFIX) {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"solana table name must start with kb_sol_",
|
||||
));
|
||||
}
|
||||
let suffix = &trimmed_table_name[crate::SOLANA_TABLE_PREFIX.len()..];
|
||||
let suffix = &trimmed_table_name[SOLANA_TABLE_PREFIX.len()..];
|
||||
let domain = crate::postgres::migrations::first_segment(suffix);
|
||||
if domain.is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
@@ -755,23 +760,23 @@ pub fn validate_solana_table_name(table_name: &str) -> ks_core::Result<()> {
|
||||
}
|
||||
|
||||
/// Validates the raw store table names introduced by `0.2.3`.
|
||||
pub fn validate_raw_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(crate::RAW_STORE_TABLE_NAMES);
|
||||
pub(crate) fn validate_raw_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(RAW_STORE_TABLE_NAMES);
|
||||
}
|
||||
|
||||
/// Validates the core store table names introduced by `0.2.4`.
|
||||
pub fn validate_core_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(crate::CORE_STORE_TABLE_NAMES);
|
||||
pub(crate) fn validate_core_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(CORE_STORE_TABLE_NAMES);
|
||||
}
|
||||
|
||||
/// Validates the decode and materialization table names introduced by `0.4.0`.
|
||||
pub fn validate_decode_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(crate::DECODE_STORE_TABLE_NAMES);
|
||||
pub(crate) fn validate_decode_store_table_names() -> ks_core::Result<()> {
|
||||
return crate::postgres::migrations::validate_table_names(DECODE_STORE_TABLE_NAMES);
|
||||
}
|
||||
|
||||
fn validate_table_names(table_names: &[&str]) -> ks_core::Result<()> {
|
||||
for table_name in table_names {
|
||||
let validation_result = crate::validate_solana_table_name(table_name);
|
||||
let validation_result = validate_solana_table_name(table_name);
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
@@ -787,7 +792,7 @@ fn first_segment(value: &str) -> &str {
|
||||
}
|
||||
|
||||
fn is_allowed_domain(domain: &str) -> bool {
|
||||
for allowed_domain in crate::ALLOWED_SOLANA_TABLE_DOMAINS {
|
||||
for allowed_domain in ALLOWED_SOLANA_TABLE_DOMAINS {
|
||||
if domain == *allowed_domain {
|
||||
return true;
|
||||
}
|
||||
@@ -812,7 +817,7 @@ fn contains_only_table_name_chars(table_name: &str) -> bool {
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_ops_processing_ledger`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_ops_processing_ledger_sql() -> &'static str {
|
||||
fn create_table_kb_sol_ops_processing_ledger_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_ops_processing_ledger (
|
||||
id BIGSERIAL,
|
||||
stage TEXT NOT NULL,
|
||||
@@ -842,87 +847,87 @@ pub(in crate::postgres) fn create_table_kb_sol_ops_processing_ledger_sql() -> &'
|
||||
}
|
||||
|
||||
/// SQL creating the unique processing ledger identity index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_ops_processing_ledger_identity_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_ops_processing_ledger_identity_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_ops_processing_ledger_identity ON kb_sol_ops_processing_ledger (stage, processor_name, processor_version, input_key)";
|
||||
}
|
||||
|
||||
/// SQL creating the processing ledger status index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_ops_processing_ledger_status_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_ops_processing_ledger_status_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_status ON kb_sol_ops_processing_ledger (stage, processor_name, status, updated_at)";
|
||||
}
|
||||
|
||||
/// SQL creating the processing ledger input hash index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_ops_processing_ledger_input_hash_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_ops_processing_ledger_input_hash_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_ops_processing_ledger_input_hash ON kb_sol_ops_processing_ledger (input_hash)";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_raw_transactions`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_raw_transactions_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_raw_transactions_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_raw_transactions";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_obs_transaction_observations`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_obs_transaction_observations_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_obs_transaction_observations_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(persisted_at)::text AS latest_created_at FROM kb_sol_obs_transaction_observations";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_transactions`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_transactions_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_transactions_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_transactions";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_account_keys`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_account_keys_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_account_keys_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_account_keys";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_instructions`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_instructions_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_instructions_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_instructions";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_inner_instructions`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_inner_instructions_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_inner_instructions_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_inner_instructions";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_logs`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_logs_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_logs_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_logs";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_core_balance_changes`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_core_balance_changes_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_core_balance_changes_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_core_balance_changes";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_ops_processing_ledger`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_ops_processing_ledger_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_ops_processing_ledger_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, NULL::BIGINT AS min_slot, NULL::BIGINT AS max_slot, MAX(updated_at)::text AS latest_created_at FROM kb_sol_ops_processing_ledger";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_decode_events`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_decode_events_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_decode_events_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_events";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_decode_coverage_declarations`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_decode_coverage_declarations_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_decode_coverage_declarations_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, NULL::BIGINT AS min_slot, NULL::BIGINT AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_coverage_declarations";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_decode_coverage_observations`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_decode_coverage_observations_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_decode_coverage_observations_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_decode_coverage_observations";
|
||||
}
|
||||
|
||||
/// SQL statistics query for `kb_sol_mat_events`.
|
||||
pub(in crate::postgres) fn table_stats_kb_sol_mat_events_sql() -> &'static str {
|
||||
pub(crate) fn table_stats_kb_sol_mat_events_sql() -> &'static str {
|
||||
return "SELECT COUNT(*)::BIGINT AS row_count, MIN(slot) AS min_slot, MAX(slot) AS max_slot, MAX(created_at)::text AS latest_created_at FROM kb_sol_mat_events";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_decode_events`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_decode_events_sql() -> &'static str {
|
||||
fn create_table_kb_sol_decode_events_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_events (
|
||||
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, event_key TEXT NOT NULL,
|
||||
@@ -940,27 +945,27 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_events_sql() -> &'static s
|
||||
}
|
||||
|
||||
/// SQL creating the decoded observation stable identity index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_decode_events_identity_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_decode_events_identity_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_events_processor_input_event ON kb_sol_decode_events (processor_name, processor_version, input_key, event_key)";
|
||||
}
|
||||
|
||||
/// SQL creating the decoded observation signature and path index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_signature_path_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_decode_events_signature_path_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_signature_path ON kb_sol_decode_events (signature, instruction_path)";
|
||||
}
|
||||
|
||||
/// SQL creating the decoded observation program and surface index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_program_surface_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_decode_events_program_surface_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_program_surface ON kb_sol_decode_events (program_id, surface_code, event_code)";
|
||||
}
|
||||
|
||||
/// SQL creating the decoded observation family and commit index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_events_family_commit_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_decode_events_family_commit_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_events_family_commit ON kb_sol_decode_events (event_family, transaction_failed, observation_committed)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_decode_coverage_declarations`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_declarations_sql() -> &'static str {
|
||||
fn create_table_kb_sol_decode_coverage_declarations_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_declarations (
|
||||
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
|
||||
program_id TEXT NOT NULL, surface_code TEXT, entry_kind TEXT NOT NULL,
|
||||
@@ -971,19 +976,17 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_declarations_sql(
|
||||
}
|
||||
|
||||
/// SQL creating the declared coverage stable identity index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_decode_coverage_declarations_identity_sql()
|
||||
-> &'static str {
|
||||
fn create_ux_kb_sol_decode_coverage_declarations_identity_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_declarations_identity ON kb_sol_decode_coverage_declarations (processor_name, processor_version, program_id, COALESCE(surface_code, ''), entry_kind, entry_code, COALESCE(discriminator_hex, ''))";
|
||||
}
|
||||
|
||||
/// SQL creating the declared coverage program index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_declarations_program_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_decode_coverage_declarations_program_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_declarations_program ON kb_sol_decode_coverage_declarations (program_id, processor_name, processor_version)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_decode_coverage_observations`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_observations_sql() -> &'static str {
|
||||
fn create_table_kb_sol_decode_coverage_observations_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_decode_coverage_observations (
|
||||
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, signature TEXT NOT NULL,
|
||||
@@ -1000,25 +1003,22 @@ pub(in crate::postgres) fn create_table_kb_sol_decode_coverage_observations_sql(
|
||||
}
|
||||
|
||||
/// SQL creating the observed coverage stable identity index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_decode_coverage_observations_identity_sql()
|
||||
-> &'static str {
|
||||
fn create_ux_kb_sol_decode_coverage_observations_identity_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_decode_coverage_observations_identity ON kb_sol_decode_coverage_observations (processor_name, processor_version, input_key)";
|
||||
}
|
||||
|
||||
/// SQL creating the observed coverage program and status index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_observations_program_status_sql()
|
||||
-> &'static str {
|
||||
fn create_ix_kb_sol_decode_coverage_observations_program_status_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_program_status ON kb_sol_decode_coverage_observations (program_id, status, transaction_failed)";
|
||||
}
|
||||
|
||||
/// SQL creating the observed coverage entry index.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_decode_coverage_observations_entry_sql() -> &'static str
|
||||
{
|
||||
fn create_ix_kb_sol_decode_coverage_observations_entry_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_decode_coverage_observations_entry ON kb_sol_decode_coverage_observations (processor_name, processor_version, surface_code, entry_code)";
|
||||
}
|
||||
|
||||
/// SQL creating `kb_sol_mat_events`.
|
||||
pub(in crate::postgres) fn create_table_kb_sol_mat_events_sql() -> &'static str {
|
||||
fn create_table_kb_sol_mat_events_sql() -> &'static str {
|
||||
return r#"CREATE TABLE IF NOT EXISTS kb_sol_mat_events (
|
||||
id BIGSERIAL, processor_name TEXT NOT NULL, processor_version TEXT NOT NULL,
|
||||
input_key TEXT NOT NULL, input_hash TEXT NOT NULL, output_key TEXT NOT NULL,
|
||||
@@ -1033,12 +1033,12 @@ pub(in crate::postgres) fn create_table_kb_sol_mat_events_sql() -> &'static str
|
||||
}
|
||||
|
||||
/// SQL creating the materialized output stable identity index.
|
||||
pub(in crate::postgres) fn create_ux_kb_sol_mat_events_identity_sql() -> &'static str {
|
||||
fn create_ux_kb_sol_mat_events_identity_sql() -> &'static str {
|
||||
return "CREATE UNIQUE INDEX IF NOT EXISTS ux_kb_sol_mat_events_processor_input_output ON kb_sol_mat_events (processor_name, processor_version, input_key, output_key)";
|
||||
}
|
||||
|
||||
/// SQL creating materialized output signature and family indexes.
|
||||
pub(in crate::postgres) fn create_ix_kb_sol_mat_events_signature_family_sql() -> &'static str {
|
||||
fn create_ix_kb_sol_mat_events_signature_family_sql() -> &'static str {
|
||||
return "CREATE INDEX IF NOT EXISTS ix_kb_sol_mat_events_signature_family ON kb_sol_mat_events (source_decoder_name, source_decoder_version, source_decode_input_key, signature, materialized_family)";
|
||||
}
|
||||
|
||||
@@ -1046,28 +1046,28 @@ pub(in crate::postgres) fn create_ix_kb_sol_mat_events_signature_family_sql() ->
|
||||
mod tests {
|
||||
#[test]
|
||||
fn valid_table_name_accepts_domain_prefix() {
|
||||
assert!(crate::is_valid_solana_table_name("kb_sol_raw_transactions"));
|
||||
assert!(super::is_valid_solana_table_name("kb_sol_raw_transactions"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_name_rejects_explicit_schema() {
|
||||
let invalid_name = std::string::String::from("raw") + "." + "kb_sol_rpc_transactions";
|
||||
assert!(!crate::is_valid_solana_table_name(invalid_name.as_str()));
|
||||
assert!(!super::is_valid_solana_table_name(invalid_name.as_str()));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_name_rejects_unknown_domain() {
|
||||
assert!(!crate::is_valid_solana_table_name("kb_sol_unknown_rows"));
|
||||
assert!(!super::is_valid_solana_table_name("kb_sol_unknown_rows"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_name_rejects_missing_table_suffix() {
|
||||
assert!(!crate::is_valid_solana_table_name("kb_sol_raw"));
|
||||
assert!(!super::is_valid_solana_table_name("kb_sol_raw"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn table_name_rejects_uppercase() {
|
||||
assert!(!crate::is_valid_solana_table_name("kb_sol_raw_RPC_transactions"));
|
||||
assert!(!super::is_valid_solana_table_name("kb_sol_raw_RPC_transactions"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/query.rs
|
||||
// version: 3
|
||||
// version: 4
|
||||
|
||||
//! PostgreSQL query modules.
|
||||
|
||||
@@ -11,42 +11,43 @@ mod raw_queries;
|
||||
mod replay_candidate_queries;
|
||||
mod table_diagnostics_queries;
|
||||
|
||||
pub(in crate::postgres) use self::core_extraction_queries::is_core_extraction_current;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::list_raw_transactions_for_core_extraction;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::mark_core_extraction_failed;
|
||||
pub(in crate::postgres) use self::core_extraction_queries::persist_core_extraction;
|
||||
pub(in crate::postgres) use self::core_queries::apply_core_store_schema;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_account_keys;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_balance_changes;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_inner_instructions;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_instructions;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_logs;
|
||||
pub(in crate::postgres) use self::core_queries::insert_core_transaction;
|
||||
pub(in crate::postgres) use self::core_queries::list_core_instruction_replay_inputs;
|
||||
pub(in crate::postgres) use self::core_queries::list_core_instructions_for_replay;
|
||||
pub(in crate::postgres) use self::core_queries::update_core_instruction_lifecycle;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::apply_decode_store_schema;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::is_decode_current;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_coverage_summary;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_decode_inputs;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::list_materialized_events;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::mark_decode_failed;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_coverage_declarations;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_decode_result;
|
||||
pub(in crate::postgres) use self::decode_pipeline_queries::persist_materialization_result;
|
||||
pub(in crate::postgres) use self::health_queries::load_current_schema;
|
||||
pub(in crate::postgres) use self::health_queries::load_latest_migration_version;
|
||||
pub(in crate::postgres) use self::health_queries::load_migration_table_name;
|
||||
pub(in crate::postgres) use self::health_queries::load_server_version;
|
||||
pub(in crate::postgres) use self::health_queries::run_health_check;
|
||||
pub(in crate::postgres) use self::raw_queries::apply_raw_store_schema;
|
||||
pub(in crate::postgres) use self::raw_queries::has_raw_transaction_signature;
|
||||
pub(in crate::postgres) use self::raw_queries::has_transaction_observation_key;
|
||||
pub(in crate::postgres) use self::raw_queries::insert_raw_transaction;
|
||||
pub(in crate::postgres) use self::raw_queries::insert_transaction_observation;
|
||||
pub(in crate::postgres) use self::raw_queries::update_raw_payload_lifecycle;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_entity_summaries;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_program_summaries;
|
||||
pub(in crate::postgres) use self::replay_candidate_queries::list_replay_transaction_candidates;
|
||||
pub(in crate::postgres) use self::table_diagnostics_queries::load_table_statistics;
|
||||
pub(in crate::postgres) use self::table_diagnostics_queries::table_exists;
|
||||
pub(crate) use self::core_extraction_queries::is_core_extraction_current;
|
||||
pub(crate) use self::core_extraction_queries::list_raw_transactions_for_core_extraction;
|
||||
pub(crate) use self::core_extraction_queries::mark_core_extraction_failed;
|
||||
pub(crate) use self::core_extraction_queries::persist_core_extraction;
|
||||
pub(crate) use self::core_queries::apply_core_store_schema;
|
||||
pub(crate) use self::core_queries::insert_core_account_keys;
|
||||
pub(crate) use self::core_queries::insert_core_balance_changes;
|
||||
pub(crate) use self::core_queries::insert_core_inner_instructions;
|
||||
pub(crate) use self::core_queries::insert_core_instructions;
|
||||
pub(crate) use self::core_queries::insert_core_logs;
|
||||
pub(crate) use self::core_queries::insert_core_transaction;
|
||||
pub(crate) use self::core_queries::list_core_instruction_replay_inputs;
|
||||
pub(crate) use self::core_queries::list_core_instructions_for_replay;
|
||||
pub(crate) use self::core_queries::list_decode_replay_inputs;
|
||||
pub(crate) use self::core_queries::update_core_instruction_lifecycle;
|
||||
pub(crate) use self::decode_pipeline_queries::apply_decode_store_schema;
|
||||
pub(crate) use self::decode_pipeline_queries::is_decode_current;
|
||||
pub(crate) use self::decode_pipeline_queries::list_decode_coverage_summary;
|
||||
pub(crate) use self::decode_pipeline_queries::list_decode_inputs;
|
||||
pub(crate) use self::decode_pipeline_queries::list_materialized_events;
|
||||
pub(crate) use self::decode_pipeline_queries::mark_decode_failed;
|
||||
pub(crate) use self::decode_pipeline_queries::persist_decode_coverage_declarations;
|
||||
pub(crate) use self::decode_pipeline_queries::persist_decode_result;
|
||||
pub(crate) use self::decode_pipeline_queries::persist_materialization_result;
|
||||
pub(crate) use self::health_queries::load_current_schema;
|
||||
pub(crate) use self::health_queries::load_latest_migration_version;
|
||||
pub(crate) use self::health_queries::load_migration_table_name;
|
||||
pub(crate) use self::health_queries::load_server_version;
|
||||
pub(crate) use self::health_queries::run_health_check;
|
||||
pub(crate) use self::raw_queries::apply_raw_store_schema;
|
||||
pub(crate) use self::raw_queries::has_raw_transaction_signature;
|
||||
pub(crate) use self::raw_queries::has_transaction_observation_key;
|
||||
pub(crate) use self::raw_queries::insert_raw_transaction;
|
||||
pub(crate) use self::raw_queries::insert_transaction_observation;
|
||||
pub(crate) use self::raw_queries::update_raw_payload_lifecycle;
|
||||
pub(crate) use self::replay_candidate_queries::list_replay_entity_summaries;
|
||||
pub(crate) use self::replay_candidate_queries::list_replay_program_summaries;
|
||||
pub(crate) use self::replay_candidate_queries::list_replay_transaction_candidates;
|
||||
pub(crate) use self::table_diagnostics_queries::load_table_statistics;
|
||||
pub(crate) use self::table_diagnostics_queries::table_exists;
|
||||
|
||||
@@ -1,11 +1,11 @@
|
||||
// file: ks-store/src/postgres/query/core_extraction_queries.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
//! PostgreSQL queries for atomic canonical transaction to core extraction.
|
||||
|
||||
use sqlx::Row; // rust-rules: trait-import
|
||||
|
||||
pub(in crate::postgres) async fn list_raw_transactions_for_core_extraction(
|
||||
pub(crate) async fn list_raw_transactions_for_core_extraction(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::CoreExtractionSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>> {
|
||||
@@ -59,7 +59,7 @@ pub(in crate::postgres) async fn list_raw_transactions_for_core_extraction(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn is_core_extraction_current(
|
||||
pub(crate) async fn is_core_extraction_current(
|
||||
pool: &sqlx::PgPool,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
@@ -81,7 +81,7 @@ pub(in crate::postgres) async fn is_core_extraction_current(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn persist_core_extraction(
|
||||
pub(crate) async fn persist_core_extraction(
|
||||
pool: &sqlx::PgPool,
|
||||
bundle: &crate::CoreExtractionBundle,
|
||||
_force_replay: bool,
|
||||
@@ -126,11 +126,7 @@ pub(in crate::postgres) async fn persist_core_extraction(
|
||||
}
|
||||
}
|
||||
let transaction_id_result =
|
||||
crate::postgres::query::core_extraction_queries::insert_core_transaction_in_transaction(
|
||||
&mut transaction,
|
||||
&bundle.transaction,
|
||||
)
|
||||
.await;
|
||||
insert_core_transaction_in_transaction(&mut transaction, &bundle.transaction).await;
|
||||
let transaction_id = match transaction_id_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
@@ -226,7 +222,7 @@ pub(in crate::postgres) async fn persist_core_extraction(
|
||||
));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn mark_core_extraction_failed(
|
||||
pub(crate) async fn mark_core_extraction_failed(
|
||||
pool: &sqlx::PgPool,
|
||||
failure: &crate::CoreExtractionFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -783,7 +779,7 @@ mod tests {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/query/core_queries.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
//! PostgreSQL queries for normalized Solana core storage.
|
||||
|
||||
@@ -7,9 +7,7 @@ use sqlx::Row; // rust-rules: trait-import
|
||||
|
||||
const OUTER_INSTRUCTIONS_CONTEXT_SQL: &str = "SELECT COALESCE(jsonb_agg(jsonb_build_object('instructionIndex', instruction_path::BIGINT, 'instructionPath', instruction_path, 'programId', program_id, 'payloadJson', payload_json, 'payloadHash', payload_json_hash) ORDER BY instruction_path::BIGINT, instruction_path ASC), '[]'::jsonb) FROM kb_sol_core_instructions WHERE signature = $1 AND instruction_path ~ '^[0-9]+$'";
|
||||
|
||||
pub(in crate::postgres) async fn apply_core_store_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<()> {
|
||||
pub(crate) async fn apply_core_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
let validation_result = crate::validate_core_store_table_names();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
@@ -49,7 +47,7 @@ pub(in crate::postgres) async fn apply_core_store_schema(
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_transaction(
|
||||
pub(crate) async fn insert_core_transaction(
|
||||
pool: &sqlx::PgPool,
|
||||
input: &crate::CoreTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -74,7 +72,7 @@ pub(in crate::postgres) async fn insert_core_transaction(
|
||||
);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_account_keys(
|
||||
pub(crate) async fn insert_core_account_keys(
|
||||
pool: &sqlx::PgPool,
|
||||
inputs: &[crate::CoreAccountKeyInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -122,7 +120,7 @@ pub(in crate::postgres) async fn insert_core_account_keys(
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_instructions(
|
||||
pub(crate) async fn insert_core_instructions(
|
||||
pool: &sqlx::PgPool,
|
||||
inputs: &[crate::CoreInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -165,7 +163,7 @@ pub(in crate::postgres) async fn insert_core_instructions(
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_inner_instructions(
|
||||
pub(crate) async fn insert_core_inner_instructions(
|
||||
pool: &sqlx::PgPool,
|
||||
inputs: &[crate::CoreInnerInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -204,7 +202,7 @@ pub(in crate::postgres) async fn insert_core_inner_instructions(
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_logs(
|
||||
pub(crate) async fn insert_core_logs(
|
||||
pool: &sqlx::PgPool,
|
||||
inputs: &[crate::CoreLogInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -250,7 +248,7 @@ pub(in crate::postgres) async fn insert_core_logs(
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_core_balance_changes(
|
||||
pub(crate) async fn insert_core_balance_changes(
|
||||
pool: &sqlx::PgPool,
|
||||
inputs: &[crate::CoreBalanceChangeInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -310,7 +308,7 @@ pub(in crate::postgres) async fn insert_core_balance_changes(
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(inserted_count, 0, skipped_count));
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_core_instructions_for_replay(
|
||||
pub(crate) async fn list_core_instructions_for_replay(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
@@ -368,18 +366,13 @@ pub(in crate::postgres) async fn list_core_instructions_for_replay(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_core_instruction_replay_inputs(
|
||||
pub(crate) async fn list_core_instruction_replay_inputs(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
let instructions_result =
|
||||
crate::postgres::query::core_queries::list_core_instructions_for_replay(
|
||||
pool,
|
||||
filter,
|
||||
page_request,
|
||||
)
|
||||
.await;
|
||||
crate::list_core_instructions_for_replay(pool, filter, page_request).await;
|
||||
let instructions = match instructions_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
@@ -399,7 +392,7 @@ pub(in crate::postgres) async fn list_core_instruction_replay_inputs(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_decode_replay_inputs(
|
||||
pub(crate) async fn list_decode_replay_inputs(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::DecodeSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
@@ -482,7 +475,7 @@ pub(in crate::postgres) async fn list_decode_replay_inputs(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn update_core_instruction_lifecycle(
|
||||
pub(crate) async fn update_core_instruction_lifecycle(
|
||||
pool: &sqlx::PgPool,
|
||||
mark: &crate::CoreInstructionLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -507,7 +500,7 @@ pub(in crate::postgres) async fn update_core_instruction_lifecycle(
|
||||
);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) fn instruction_processing_state_to_sql(
|
||||
fn instruction_processing_state_to_sql(
|
||||
state: crate::CoreInstructionProcessingState,
|
||||
) -> &'static str {
|
||||
return match state {
|
||||
@@ -755,7 +748,7 @@ where
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_replay_input_for_instruction(
|
||||
async fn load_replay_input_for_instruction(
|
||||
pool: &sqlx::PgPool,
|
||||
instruction: &crate::CoreInstructionRow,
|
||||
) -> ks_core::Result<crate::MdCoreInstructionReplayInput> {
|
||||
@@ -990,7 +983,7 @@ mod tests {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/query/decode_pipeline_queries.rs
|
||||
// version: 7
|
||||
// version: 10
|
||||
|
||||
//! PostgreSQL queries for contextual decode, coverage and materialization persistence.
|
||||
|
||||
@@ -22,10 +22,8 @@ struct MaterializedEventDatabaseRow {
|
||||
updated_at: std::string::String,
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn apply_decode_store_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<()> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "apply_decode_store_schema", statement_count = crate::decode_store_schema_statements().len(), "apply PostgreSQL decode store schema");
|
||||
pub(crate) async fn apply_decode_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "apply_decode_store_schema", statement_count = crate::decode_store_schema_statements().len(), "apply PostgreSQL decode store schema");
|
||||
let validation_result = crate::validate_decode_store_table_names();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
@@ -62,17 +60,16 @@ pub(in crate::postgres) async fn apply_decode_store_schema(
|
||||
"postgres decode store schema commit failed: {error}"
|
||||
)));
|
||||
}
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "apply_decode_store_schema", committed = true, "PostgreSQL decode store schema applied");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "apply_decode_store_schema", committed = true, "PostgreSQL decode store schema applied");
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_decode_inputs(
|
||||
pub(crate) async fn list_decode_inputs(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::DecodeSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_inputs", signature_count = filter.signatures.len(), signature_sample = ?filter.signatures.iter().take(5).map(std::string::String::as_str).collect::<std::vec::Vec<_>>(), processing_states = ?filter.processing_states, min_slot = ?filter.min_slot, max_slot = ?filter.max_slot, program_ids = ?filter.program_ids, instruction_paths = ?filter.instruction_paths, limit = filter.limit, "query PostgreSQL contextual decode inputs");
|
||||
let result =
|
||||
crate::postgres::query::core_queries::list_decode_replay_inputs(pool, filter).await;
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", signature_count = filter.signatures.len(), signature_sample = ?filter.signatures.iter().take(5).map(std::string::String::as_str).collect::<std::vec::Vec<_>>(), processing_states = ?filter.processing_states, min_slot = ?filter.min_slot, max_slot = ?filter.max_slot, program_ids = ?filter.program_ids, instruction_paths = ?filter.instruction_paths, limit = filter.limit, "query PostgreSQL contextual decode inputs");
|
||||
let result = crate::list_decode_replay_inputs(pool, filter).await;
|
||||
return match result {
|
||||
std::result::Result::Ok(inputs) => {
|
||||
let selected_input_keys = inputs
|
||||
@@ -80,21 +77,21 @@ pub(in crate::postgres) async fn list_decode_inputs(
|
||||
.take(10)
|
||||
.map(|input| return input.replay_input_key.as_str())
|
||||
.collect::<std::vec::Vec<_>>();
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_inputs", selected_count = inputs.len(), input_key_sample = ?selected_input_keys, "PostgreSQL contextual decode inputs selected");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", selected_count = inputs.len(), input_key_sample = ?selected_input_keys, "PostgreSQL contextual decode inputs selected");
|
||||
std::result::Result::Ok(inputs)
|
||||
},
|
||||
std::result::Result::Err(error) => {
|
||||
tracing::error!(target: crate::TRACING_TARGET, action = "list_decode_inputs", error = %error, "PostgreSQL contextual decode input query failed");
|
||||
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_inputs", error = %error, "PostgreSQL contextual decode input query failed");
|
||||
std::result::Result::Err(error)
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_materialized_events(
|
||||
pub(crate) async fn list_materialized_events(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::MaterializedEventFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_materialized_events", processor_name = ?filter.processor_name, materialized_family = ?filter.materialized_family, signature_contains = ?filter.signature_contains, limit = filter.limit, "query bounded PostgreSQL materialized events");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_materialized_events", processor_name = ?filter.processor_name, materialized_family = ?filter.materialized_family, signature_contains = ?filter.signature_contains, limit = filter.limit, "query bounded PostgreSQL materialized events");
|
||||
if filter.limit == 0 || filter.limit > crate::MAX_MATERIALIZED_EVENT_QUERY_ROWS {
|
||||
return std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"materialized event query limit must be between 1 and {}",
|
||||
@@ -145,15 +142,15 @@ pub(in crate::postgres) async fn list_materialized_events(
|
||||
updated_at: row.updated_at,
|
||||
});
|
||||
}
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_materialized_events", row_count = output.len(), "bounded PostgreSQL materialized events loaded");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_materialized_events", row_count = output.len(), "bounded PostgreSQL materialized events loaded");
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn is_decode_current(
|
||||
pub(crate) async fn is_decode_current(
|
||||
pool: &sqlx::PgPool,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, "query PostgreSQL processing ledger current state");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, "query PostgreSQL processing ledger current state");
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, bool>(
|
||||
"SELECT EXISTS(SELECT 1 FROM kb_sol_ops_processing_ledger WHERE stage = $1 AND processor_name = $2 AND processor_version = $3 AND input_key = $4 AND input_hash = $5 AND status = 'succeeded')",
|
||||
)
|
||||
@@ -166,11 +163,11 @@ pub(in crate::postgres) async fn is_decode_current(
|
||||
.await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(value) => {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, current = value, "PostgreSQL processing ledger current state loaded");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, input_hash = %identity.input_hash, current = value, "PostgreSQL processing ledger current state loaded");
|
||||
std::result::Result::Ok(value)
|
||||
},
|
||||
std::result::Result::Err(error) => {
|
||||
tracing::error!(target: crate::TRACING_TARGET, action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, error = %error, "PostgreSQL processing ledger current check failed");
|
||||
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "is_decode_current", stage = %identity.stage, processor_name = %identity.processor_name, processor_version = %identity.processor_version, input_key = %identity.input_key, error = %error, "PostgreSQL processing ledger current check failed");
|
||||
std::result::Result::Err(ks_core::Error::db(format!(
|
||||
"postgres decode ledger current check failed: {error}"
|
||||
)))
|
||||
@@ -178,12 +175,12 @@ pub(in crate::postgres) async fn is_decode_current(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn persist_decode_coverage_declarations(
|
||||
pub(crate) async fn persist_decode_coverage_declarations(
|
||||
pool: &sqlx::PgPool,
|
||||
declarations: &[crate::DecodeCoverageDeclarationInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
if declarations.is_empty() {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", declaration_count = 0_usize, "skip empty PostgreSQL decode coverage declarations");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", declaration_count = 0_usize, "skip empty PostgreSQL decode coverage declarations");
|
||||
return std::result::Result::Ok(crate::InsertOutcome::new(0, 0, 0));
|
||||
}
|
||||
let processor_name = declarations[0].processor_name.as_str();
|
||||
@@ -192,7 +189,7 @@ pub(in crate::postgres) async fn persist_decode_coverage_declarations(
|
||||
.iter()
|
||||
.map(|entry| return entry.program_id.as_str())
|
||||
.collect::<std::vec::Vec<_>>();
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, declaration_count = declarations.len(), program_ids = ?program_ids, "persist PostgreSQL decode coverage declarations");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, declaration_count = declarations.len(), program_ids = ?program_ids, "persist PostgreSQL decode coverage declarations");
|
||||
if declarations.iter().any(|entry| {
|
||||
return entry.processor_name != processor_name
|
||||
|| entry.processor_version != processor_version
|
||||
@@ -351,16 +348,16 @@ pub(in crate::postgres) async fn persist_decode_coverage_declarations(
|
||||
)));
|
||||
}
|
||||
let outcome = crate::InsertOutcome::new(inserted_count, updated_count, skipped_count);
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, outcome = ?outcome, "PostgreSQL decode coverage declarations persisted");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_coverage_declarations", processor_name = %processor_name, processor_version = %processor_version, outcome = ?outcome, "PostgreSQL decode coverage declarations persisted");
|
||||
return std::result::Result::Ok(outcome);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn persist_decode_result(
|
||||
pub(crate) async fn persist_decode_result(
|
||||
pool: &sqlx::PgPool,
|
||||
bundle: &crate::DecodePersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, status = %bundle.status, observation_count = bundle.observations.len(), coverage_program_id = %bundle.coverage.program_id, force_replay, "persist PostgreSQL contextual decode result");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, status = %bundle.status, observation_count = bundle.observations.len(), coverage_program_id = %bundle.coverage.program_id, force_replay, "persist PostgreSQL contextual decode result");
|
||||
let validation_result = bundle.validate();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
@@ -513,15 +510,15 @@ pub(in crate::postgres) async fn persist_decode_result(
|
||||
},
|
||||
};
|
||||
let outcome = crate::InsertOutcome::new(count, 1, 0);
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL contextual decode result persisted");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_decode_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL contextual decode result persisted");
|
||||
return std::result::Result::Ok(outcome);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn mark_decode_failed(
|
||||
pub(crate) async fn mark_decode_failed(
|
||||
pool: &sqlx::PgPool,
|
||||
failure: &crate::DecodeFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
tracing::error!(target: crate::TRACING_TARGET, action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, input_key = %failure.ledger_identity.input_key, input_hash = %failure.ledger_identity.input_hash, error_code = %failure.error_code, error_message = %failure.error_message, "persist PostgreSQL contextual decode failure");
|
||||
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, input_key = %failure.ledger_identity.input_key, input_hash = %failure.ledger_identity.input_hash, error_code = %failure.error_code, error_message = %failure.error_message, "persist PostgreSQL contextual decode failure");
|
||||
if failure.ledger_identity.stage != "instruction_decode"
|
||||
|| failure.signature.trim().is_empty()
|
||||
|| failure.instruction_path.trim().is_empty()
|
||||
@@ -572,18 +569,18 @@ pub(in crate::postgres) async fn mark_decode_failed(
|
||||
)));
|
||||
}
|
||||
let outcome = crate::InsertOutcome::new(0, 1, 0);
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, outcome = ?outcome, committed = true, "PostgreSQL contextual decode failure persisted");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "mark_decode_failed", signature = %failure.signature, instruction_path = %failure.instruction_path, processor_name = %failure.ledger_identity.processor_name, processor_version = %failure.ledger_identity.processor_version, outcome = ?outcome, committed = true, "PostgreSQL contextual decode failure persisted");
|
||||
return std::result::Result::Ok(outcome);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn persist_materialization_result(
|
||||
pub(crate) async fn persist_materialization_result(
|
||||
pool: &sqlx::PgPool,
|
||||
bundle: &crate::MaterializationPersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
if bundle.status == "failed" {
|
||||
tracing::error!(
|
||||
target: crate::TRACING_TARGET,
|
||||
target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg",
|
||||
action = "persist_materialization_failure",
|
||||
signature = %bundle.signature,
|
||||
instruction_path = %bundle.instruction_path,
|
||||
@@ -598,7 +595,7 @@ pub(in crate::postgres) async fn persist_materialization_result(
|
||||
"persist PostgreSQL materialization failure"
|
||||
);
|
||||
}
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, source_decoder_name = %bundle.source_decoder_name, source_decoder_version = %bundle.source_decoder_version, source_decode_input_key = %bundle.source_decode_input_key, status = %bundle.status, output_count = bundle.outputs.len(), force_replay, "persist PostgreSQL materialization result");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, stage = %bundle.ledger_identity.stage, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, input_key = %bundle.ledger_identity.input_key, input_hash = %bundle.ledger_identity.input_hash, source_decoder_name = %bundle.source_decoder_name, source_decoder_version = %bundle.source_decoder_version, source_decode_input_key = %bundle.source_decode_input_key, status = %bundle.status, output_count = bundle.outputs.len(), force_replay, "persist PostgreSQL materialization result");
|
||||
let validation_result = bundle.validate();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
@@ -715,17 +712,17 @@ pub(in crate::postgres) async fn persist_materialization_result(
|
||||
},
|
||||
};
|
||||
let outcome = crate::InsertOutcome::new(count, 1, 0);
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL materialization result persisted");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "persist_materialization_result", signature = %bundle.signature, instruction_path = %bundle.instruction_path, processor_name = %bundle.ledger_identity.processor_name, processor_version = %bundle.ledger_identity.processor_version, status = %bundle.status, outcome = ?outcome, committed = true, "PostgreSQL materialization result persisted");
|
||||
return std::result::Result::Ok(outcome);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_decode_coverage_summary(
|
||||
pub(crate) async fn list_decode_coverage_summary(
|
||||
pool: &sqlx::PgPool,
|
||||
processor_name: std::option::Option<&str>,
|
||||
processor_version: std::option::Option<&str>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, limit, "query PostgreSQL decode coverage summary");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, limit, "query PostgreSQL decode coverage summary");
|
||||
if limit == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"decode coverage summary limit must be greater than zero",
|
||||
@@ -756,7 +753,7 @@ pub(in crate::postgres) async fn list_decode_coverage_summary(
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
}
|
||||
}
|
||||
tracing::debug!(target: crate::TRACING_TARGET, action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, row_count = output.len(), "PostgreSQL decode coverage summary loaded");
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "list_decode_coverage_summary", processor_name = ?processor_name, processor_version = ?processor_version, row_count = output.len(), "PostgreSQL decode coverage summary loaded");
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
@@ -1035,17 +1032,15 @@ mod tests {
|
||||
};
|
||||
let pool_result = sqlx::PgPool::connect(url.as_str()).await;
|
||||
let pool = result_or_panic(pool_result);
|
||||
result_or_panic(crate::postgres::query::raw_queries::apply_raw_store_schema(&pool).await);
|
||||
result_or_panic(crate::postgres::query::core_queries::apply_core_store_schema(&pool).await);
|
||||
result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::apply_decode_store_schema(&pool).await,
|
||||
);
|
||||
result_or_panic(crate::apply_raw_store_schema(&pool).await);
|
||||
result_or_panic(crate::apply_core_store_schema(&pool).await);
|
||||
result_or_panic(crate::apply_decode_store_schema(&pool).await);
|
||||
return std::option::Option::Some(pool);
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_coverage_declarations_report_insert_skip_and_update_from_env() {
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let pool = match test_pool_from_env().await {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return,
|
||||
@@ -1062,28 +1057,16 @@ mod tests {
|
||||
historical: false,
|
||||
};
|
||||
let first = result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
|
||||
&pool,
|
||||
&[declaration.clone()],
|
||||
)
|
||||
.await,
|
||||
crate::persist_decode_coverage_declarations(&pool, &[declaration.clone()]).await,
|
||||
);
|
||||
assert_eq!(first, crate::InsertOutcome::new(1, 0, 0));
|
||||
let second = result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
|
||||
&pool,
|
||||
&[declaration.clone()],
|
||||
)
|
||||
.await,
|
||||
crate::persist_decode_coverage_declarations(&pool, &[declaration.clone()]).await,
|
||||
);
|
||||
assert_eq!(second, crate::InsertOutcome::new(0, 0, 1));
|
||||
declaration.historical = true;
|
||||
let third = result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::persist_decode_coverage_declarations(
|
||||
&pool,
|
||||
&[declaration],
|
||||
)
|
||||
.await,
|
||||
crate::persist_decode_coverage_declarations(&pool, &[declaration]).await,
|
||||
);
|
||||
assert_eq!(third, crate::InsertOutcome::new(0, 1, 0));
|
||||
let cleanup_result = sqlx::query(
|
||||
@@ -1097,7 +1080,7 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_same_version_and_hash_is_current_from_env() {
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let pool = match test_pool_from_env().await {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return,
|
||||
@@ -1140,16 +1123,8 @@ mod tests {
|
||||
observations: std::vec::Vec::new(),
|
||||
coverage,
|
||||
};
|
||||
result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::persist_decode_result(
|
||||
&pool, &bundle, true,
|
||||
)
|
||||
.await,
|
||||
);
|
||||
let current = result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::is_decode_current(&pool, &identity)
|
||||
.await,
|
||||
);
|
||||
result_or_panic(crate::persist_decode_result(&pool, &bundle, true).await);
|
||||
let current = result_or_panic(crate::is_decode_current(&pool, &identity).await);
|
||||
assert!(current);
|
||||
let cleanup_coverage = sqlx::query(
|
||||
"DELETE FROM kb_sol_decode_coverage_observations WHERE processor_name = $1",
|
||||
@@ -1168,7 +1143,7 @@ mod tests {
|
||||
|
||||
#[tokio::test]
|
||||
async fn optional_postgres_materialized_event_query_is_bounded_and_typed_from_env() {
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let pool = match test_pool_from_env().await {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => return,
|
||||
@@ -1189,12 +1164,7 @@ mod tests {
|
||||
std::option::Option::Some(input_key.clone()),
|
||||
1,
|
||||
));
|
||||
let rows = result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::list_materialized_events(
|
||||
&pool, &filter,
|
||||
)
|
||||
.await,
|
||||
);
|
||||
let rows = result_or_panic(crate::list_materialized_events(&pool, &filter).await);
|
||||
assert_eq!(rows.len(), 1);
|
||||
assert_eq!(rows[0].slot, 42);
|
||||
assert_eq!(rows[0].payload_json["text"], "postgres annotation");
|
||||
@@ -1211,14 +1181,12 @@ mod tests {
|
||||
std::result::Result::Ok(value) if !value.trim().is_empty() => value,
|
||||
_ => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let pool_result = sqlx::PgPool::connect(url.as_str()).await;
|
||||
let pool = result_or_panic(pool_result);
|
||||
result_or_panic(crate::postgres::query::raw_queries::apply_raw_store_schema(&pool).await);
|
||||
result_or_panic(crate::postgres::query::core_queries::apply_core_store_schema(&pool).await);
|
||||
result_or_panic(
|
||||
crate::postgres::query::decode_pipeline_queries::apply_decode_store_schema(&pool).await,
|
||||
);
|
||||
result_or_panic(crate::apply_raw_store_schema(&pool).await);
|
||||
result_or_panic(crate::apply_core_store_schema(&pool).await);
|
||||
result_or_panic(crate::apply_decode_store_schema(&pool).await);
|
||||
execute_sql(
|
||||
&pool,
|
||||
"DELETE FROM kb_sol_decode_events WHERE processor_name = 'decode_atomic_rollback_test'",
|
||||
@@ -1310,11 +1278,7 @@ mod tests {
|
||||
observations: std::vec![observation],
|
||||
coverage,
|
||||
};
|
||||
let persistence_result =
|
||||
crate::postgres::query::decode_pipeline_queries::persist_decode_result(
|
||||
&pool, &bundle, true,
|
||||
)
|
||||
.await;
|
||||
let persistence_result = crate::persist_decode_result(&pool, &bundle, true).await;
|
||||
assert!(persistence_result.is_err());
|
||||
let event_count_result = sqlx::query_scalar::<sqlx::Postgres, i64>(
|
||||
"SELECT COUNT(*) FROM kb_sol_decode_events WHERE processor_name = 'decode_atomic_rollback_test'",
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
// file: ks-store/src/postgres/query/health_queries.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL health and diagnostic SQL queries.
|
||||
|
||||
pub(in crate::postgres) async fn run_health_check(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
pub(crate) async fn run_health_check(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, i32>("SELECT 1").fetch_one(pool).await;
|
||||
return match query_result {
|
||||
std::result::Result::Ok(_value) => std::result::Result::Ok(()),
|
||||
@@ -13,7 +13,7 @@ pub(in crate::postgres) async fn run_health_check(pool: &sqlx::PgPool) -> ks_cor
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_current_schema(
|
||||
pub(crate) async fn load_current_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::string::String> {
|
||||
let query_result =
|
||||
@@ -28,7 +28,7 @@ pub(in crate::postgres) async fn load_current_schema(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_server_version(
|
||||
pub(crate) async fn load_server_version(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::string::String> {
|
||||
let query_result =
|
||||
@@ -43,7 +43,7 @@ pub(in crate::postgres) async fn load_server_version(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_migration_table_name(
|
||||
pub(crate) async fn load_migration_table_name(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::option::Option<std::string::String>> {
|
||||
let query_result =
|
||||
@@ -60,7 +60,7 @@ pub(in crate::postgres) async fn load_migration_table_name(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_latest_migration_version(
|
||||
pub(crate) async fn load_latest_migration_version(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<std::option::Option<std::string::String>> {
|
||||
let query_result = sqlx::query_scalar::<sqlx::Postgres, std::option::Option<std::string::String>>(
|
||||
|
||||
@@ -1,11 +1,9 @@
|
||||
// file: ks-store/src/postgres/query/raw_queries.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
//! PostgreSQL canonical transaction and acquisition observation SQL queries.
|
||||
|
||||
pub(in crate::postgres) async fn apply_raw_store_schema(
|
||||
pool: &sqlx::PgPool,
|
||||
) -> ks_core::Result<()> {
|
||||
pub(crate) async fn apply_raw_store_schema(pool: &sqlx::PgPool) -> ks_core::Result<()> {
|
||||
let validation_result = crate::validate_raw_store_table_names();
|
||||
if let std::result::Result::Err(error) = validation_result {
|
||||
return std::result::Result::Err(error);
|
||||
@@ -45,7 +43,7 @@ pub(in crate::postgres) async fn apply_raw_store_schema(
|
||||
return std::result::Result::Ok(());
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn has_raw_transaction_signature(
|
||||
pub(crate) async fn has_raw_transaction_signature(
|
||||
pool: &sqlx::PgPool,
|
||||
signature: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
@@ -70,7 +68,7 @@ pub(in crate::postgres) async fn has_raw_transaction_signature(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn has_transaction_observation_key(
|
||||
pub(crate) async fn has_transaction_observation_key(
|
||||
pool: &sqlx::PgPool,
|
||||
observation_key: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
@@ -95,7 +93,7 @@ pub(in crate::postgres) async fn has_transaction_observation_key(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_raw_transaction(
|
||||
pub(crate) async fn insert_raw_transaction(
|
||||
pool: &sqlx::PgPool,
|
||||
input: &crate::RawTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -136,7 +134,7 @@ pub(in crate::postgres) async fn insert_raw_transaction(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn insert_transaction_observation(
|
||||
pub(crate) async fn insert_transaction_observation(
|
||||
pool: &sqlx::PgPool,
|
||||
input: &crate::TransactionObservationInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -193,7 +191,7 @@ pub(in crate::postgres) async fn insert_transaction_observation(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn update_raw_payload_lifecycle(
|
||||
pub(crate) async fn update_raw_payload_lifecycle(
|
||||
pool: &sqlx::PgPool,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
@@ -378,7 +376,7 @@ mod tests {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/query/replay_candidate_queries.rs
|
||||
// version: 4
|
||||
// version: 6
|
||||
|
||||
//! Read-only PostgreSQL queries for replay candidate discovery.
|
||||
|
||||
@@ -14,9 +14,9 @@ struct ReplayTransactionCandidateRow {
|
||||
ledger_status: std::string::String,
|
||||
processor_version: std::option::Option<std::string::String>,
|
||||
attempt_count: i32,
|
||||
outer_instruction_count: i64,
|
||||
top_level_instruction_count: i64,
|
||||
inner_instruction_count: i64,
|
||||
outer_program_count: i64,
|
||||
top_level_program_count: i64,
|
||||
inner_program_count: i64,
|
||||
updated_at: std::string::String,
|
||||
}
|
||||
@@ -25,7 +25,7 @@ struct ReplayTransactionCandidateRow {
|
||||
struct ReplayProgramSummaryRow {
|
||||
program_id: std::string::String,
|
||||
transaction_count: i64,
|
||||
outer_instruction_count: i64,
|
||||
top_level_instruction_count: i64,
|
||||
inner_instruction_count: i64,
|
||||
log_count: i64,
|
||||
min_slot: i64,
|
||||
@@ -42,10 +42,10 @@ struct ReplayEntitySummaryRow {
|
||||
max_slot: i64,
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_transaction_candidates(
|
||||
pub(crate) async fn list_replay_transaction_candidates(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
|
||||
filter: &crate::ReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayTransactionCandidate>> {
|
||||
let min_slot_result =
|
||||
crate::postgres::query::replay_candidate_queries::optional_sql_bigint(filter.min_slot);
|
||||
let min_slot = match min_slot_result {
|
||||
@@ -93,7 +93,7 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayTransactionCandidate {
|
||||
output.push(crate::ReplayTransactionCandidate {
|
||||
signature: row.signature,
|
||||
slot: row.slot,
|
||||
raw_processing_state: row.raw_processing_state,
|
||||
@@ -103,9 +103,9 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
|
||||
ledger_status: row.ledger_status,
|
||||
processor_version: row.processor_version,
|
||||
attempt_count: row.attempt_count,
|
||||
outer_instruction_count: row.outer_instruction_count,
|
||||
top_level_instruction_count: row.top_level_instruction_count,
|
||||
inner_instruction_count: row.inner_instruction_count,
|
||||
outer_program_count: row.outer_program_count,
|
||||
top_level_program_count: row.top_level_program_count,
|
||||
inner_program_count: row.inner_program_count,
|
||||
updated_at: row.updated_at,
|
||||
});
|
||||
@@ -113,13 +113,13 @@ pub(in crate::postgres) async fn list_replay_transaction_candidates(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_program_summaries(
|
||||
pub(crate) async fn list_replay_program_summaries(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
|
||||
filter: &crate::ReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayProgramSummary>> {
|
||||
let query_result = sqlx::query_as::<sqlx::Postgres, crate::postgres::query::replay_candidate_queries::ReplayProgramSummaryRow>(
|
||||
r#"WITH occurrences AS (
|
||||
SELECT program_id, signature, slot, 'outer'::TEXT AS scope FROM kb_sol_core_instructions
|
||||
SELECT program_id, signature, slot, 'top_level'::TEXT AS scope FROM kb_sol_core_instructions
|
||||
UNION ALL
|
||||
SELECT program_id, signature, slot, 'inner'::TEXT AS scope FROM kb_sol_core_inner_instructions
|
||||
UNION ALL
|
||||
@@ -127,7 +127,7 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
|
||||
)
|
||||
SELECT program_id,
|
||||
COUNT(DISTINCT signature)::BIGINT AS transaction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'outer')::BIGINT AS outer_instruction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'top_level')::BIGINT AS top_level_instruction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'inner')::BIGINT AS inner_instruction_count,
|
||||
COUNT(*) FILTER (WHERE scope = 'logs')::BIGINT AS log_count,
|
||||
MIN(slot)::BIGINT AS min_slot,
|
||||
@@ -152,10 +152,10 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayProgramSummary {
|
||||
output.push(crate::ReplayProgramSummary {
|
||||
program_id: row.program_id,
|
||||
transaction_count: row.transaction_count,
|
||||
outer_instruction_count: row.outer_instruction_count,
|
||||
top_level_instruction_count: row.top_level_instruction_count,
|
||||
inner_instruction_count: row.inner_instruction_count,
|
||||
log_count: row.log_count,
|
||||
min_slot: row.min_slot,
|
||||
@@ -165,10 +165,10 @@ pub(in crate::postgres) async fn list_replay_program_summaries(
|
||||
return std::result::Result::Ok(output);
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn list_replay_entity_summaries(
|
||||
pub(crate) async fn list_replay_entity_summaries(
|
||||
pool: &sqlx::PgPool,
|
||||
filter: &crate::PostgresReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
|
||||
filter: &crate::ReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayEntitySummary>> {
|
||||
let entity_kind = filter.entity_kind.as_sql();
|
||||
let query_result = sqlx::query_as::<
|
||||
sqlx::Postgres,
|
||||
@@ -214,7 +214,7 @@ pub(in crate::postgres) async fn list_replay_entity_summaries(
|
||||
};
|
||||
let mut output = std::vec::Vec::with_capacity(rows.len());
|
||||
for row in rows {
|
||||
output.push(crate::PostgresReplayEntitySummary {
|
||||
output.push(crate::ReplayEntitySummary {
|
||||
entity_kind: row.entity_kind,
|
||||
entity_value: row.entity_value,
|
||||
transaction_count: row.transaction_count,
|
||||
@@ -245,9 +245,9 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
COALESCE(ledger.status, 'not_started') AS ledger_status,
|
||||
ledger.processor_version,
|
||||
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
|
||||
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
|
||||
COALESCE(top_level_stats.instruction_count, 0)::BIGINT AS top_level_instruction_count,
|
||||
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
|
||||
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
|
||||
COALESCE(top_level_stats.program_count, 0)::BIGINT AS top_level_program_count,
|
||||
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
|
||||
raw.updated_at::TEXT AS updated_at
|
||||
FROM kb_sol_raw_transactions raw
|
||||
@@ -265,7 +265,7 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_instructions
|
||||
WHERE signature = raw.signature
|
||||
) outer_stats ON TRUE
|
||||
) top_level_stats ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_inner_instructions
|
||||
@@ -278,11 +278,11 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
|
||||
AND ($6::TEXT IS NULL OR
|
||||
($7 = 'any' AND (
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
|
||||
)) OR
|
||||
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
|
||||
($7 = 'top_level' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6)) OR
|
||||
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
|
||||
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
|
||||
AND ($8::TEXT IS NULL OR
|
||||
@@ -313,9 +313,9 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
COALESCE(ledger.status, 'not_started') AS ledger_status,
|
||||
ledger.processor_version,
|
||||
COALESCE(ledger.attempt_count, 0)::INTEGER AS attempt_count,
|
||||
COALESCE(outer_stats.instruction_count, 0)::BIGINT AS outer_instruction_count,
|
||||
COALESCE(top_level_stats.instruction_count, 0)::BIGINT AS top_level_instruction_count,
|
||||
COALESCE(inner_stats.instruction_count, 0)::BIGINT AS inner_instruction_count,
|
||||
COALESCE(outer_stats.program_count, 0)::BIGINT AS outer_program_count,
|
||||
COALESCE(top_level_stats.program_count, 0)::BIGINT AS top_level_program_count,
|
||||
COALESCE(inner_stats.program_count, 0)::BIGINT AS inner_program_count,
|
||||
raw.updated_at::TEXT AS updated_at
|
||||
FROM kb_sol_raw_transactions raw
|
||||
@@ -333,7 +333,7 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_instructions
|
||||
WHERE signature = raw.signature
|
||||
) outer_stats ON TRUE
|
||||
) top_level_stats ON TRUE
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT COUNT(*)::BIGINT AS instruction_count, COUNT(DISTINCT program_id)::BIGINT AS program_count
|
||||
FROM kb_sol_core_inner_instructions
|
||||
@@ -346,11 +346,11 @@ fn transaction_candidate_sql(order: &str) -> &'static str {
|
||||
AND ($5::TEXT IS NULL OR ($5 = 'not_started' AND ledger.status IS NULL) OR ledger.status = $5)
|
||||
AND ($6::TEXT IS NULL OR
|
||||
($7 = 'any' AND (
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6) OR
|
||||
EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)
|
||||
)) OR
|
||||
($7 = 'outer' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_outer WHERE candidate_outer.signature = raw.signature AND candidate_outer.program_id = $6)) OR
|
||||
($7 = 'top_level' AND EXISTS (SELECT 1 FROM kb_sol_core_instructions candidate_top_level WHERE candidate_top_level.signature = raw.signature AND candidate_top_level.program_id = $6)) OR
|
||||
($7 = 'inner' AND EXISTS (SELECT 1 FROM kb_sol_core_inner_instructions candidate_inner WHERE candidate_inner.signature = raw.signature AND candidate_inner.program_id = $6)) OR
|
||||
($7 = 'logs' AND EXISTS (SELECT 1 FROM kb_sol_core_logs candidate_log WHERE candidate_log.signature = raw.signature AND candidate_log.program_id = $6)))
|
||||
AND ($8::TEXT IS NULL OR
|
||||
@@ -400,7 +400,7 @@ mod tests {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
@@ -419,14 +419,14 @@ mod tests {
|
||||
if let std::result::Result::Err(error) = core_schema_result {
|
||||
panic!("unexpected core schema error: {error}");
|
||||
}
|
||||
let transaction_filter_result = crate::PostgresReplayTransactionFilter::new(
|
||||
let transaction_filter_result = crate::ReplayTransactionFilter::new(
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
crate::PostgresReplayProgramScope::Any,
|
||||
crate::ReplayProgramScope::Any,
|
||||
std::option::Option::None,
|
||||
std::option::Option::None,
|
||||
10,
|
||||
@@ -442,8 +442,7 @@ mod tests {
|
||||
if let std::result::Result::Err(error) = transaction_result {
|
||||
panic!("unexpected transaction candidate query error: {error}");
|
||||
}
|
||||
let program_filter_result =
|
||||
crate::PostgresReplayProgramFilter::new(std::option::Option::None, 10);
|
||||
let program_filter_result = crate::ReplayProgramFilter::new(std::option::Option::None, 10);
|
||||
let program_filter = match program_filter_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("unexpected program filter error: {error}"),
|
||||
@@ -453,12 +452,12 @@ mod tests {
|
||||
panic!("unexpected program summary query error: {error}");
|
||||
}
|
||||
for entity_kind in [
|
||||
crate::PostgresReplayEntityKind::Mint,
|
||||
crate::PostgresReplayEntityKind::Owner,
|
||||
crate::PostgresReplayEntityKind::AccountKey,
|
||||
crate::ReplayEntityKind::Mint,
|
||||
crate::ReplayEntityKind::Owner,
|
||||
crate::ReplayEntityKind::AccountKey,
|
||||
] {
|
||||
let entity_filter_result =
|
||||
crate::PostgresReplayEntityFilter::new(entity_kind, std::option::Option::None, 10);
|
||||
crate::ReplayEntityFilter::new(entity_kind, std::option::Option::None, 10);
|
||||
let entity_filter = match entity_filter_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => {
|
||||
|
||||
@@ -1,14 +1,11 @@
|
||||
// file: ks-store/src/postgres/query/table_diagnostics_queries.rs
|
||||
// version: 3
|
||||
// version: 6
|
||||
|
||||
//! Read-only PostgreSQL diagnostics for known Solana store tables.
|
||||
|
||||
use sqlx::Row; // rust-rules: trait-import
|
||||
|
||||
pub(in crate::postgres) async fn table_exists(
|
||||
pool: &sqlx::PgPool,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
pub(crate) async fn table_exists(pool: &sqlx::PgPool, table_name: &str) -> ks_core::Result<bool> {
|
||||
let query_result =
|
||||
sqlx::query_scalar::<sqlx::Postgres, bool>("SELECT to_regclass($1)::text IS NOT NULL")
|
||||
.bind(table_name)
|
||||
@@ -22,111 +19,111 @@ pub(in crate::postgres) async fn table_exists(
|
||||
};
|
||||
}
|
||||
|
||||
pub(in crate::postgres) async fn load_table_statistics(
|
||||
pub(crate) async fn load_table_statistics(
|
||||
pool: &sqlx::PgPool,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<crate::PostgresTableStatistics> {
|
||||
) -> ks_core::Result<crate::StoreResourceStatistics> {
|
||||
return match table_name {
|
||||
crate::RAW_TRANSACTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_raw_transactions_sql(),
|
||||
crate::table_stats_kb_sol_raw_transactions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::TRANSACTION_OBSERVATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_obs_transaction_observations_sql(),
|
||||
crate::table_stats_kb_sol_obs_transaction_observations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_TRANSACTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_transactions_sql(),
|
||||
crate::table_stats_kb_sol_core_transactions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_ACCOUNT_KEYS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_account_keys_sql(),
|
||||
crate::table_stats_kb_sol_core_account_keys_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_INSTRUCTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_instructions_sql(),
|
||||
crate::table_stats_kb_sol_core_instructions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_INNER_INSTRUCTIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_inner_instructions_sql(),
|
||||
crate::table_stats_kb_sol_core_inner_instructions_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_LOGS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_logs_sql(),
|
||||
crate::table_stats_kb_sol_core_logs_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::CORE_BALANCE_CHANGES_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_core_balance_changes_sql(),
|
||||
crate::table_stats_kb_sol_core_balance_changes_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::PROCESSING_LEDGER_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_ops_processing_ledger_sql(),
|
||||
crate::table_stats_kb_sol_ops_processing_ledger_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_EVENTS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_events_sql(),
|
||||
crate::table_stats_kb_sol_decode_events_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_COVERAGE_DECLARATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_declarations_sql(),
|
||||
crate::table_stats_kb_sol_decode_coverage_declarations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::DECODE_COVERAGE_OBSERVATIONS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_decode_coverage_observations_sql(),
|
||||
crate::table_stats_kb_sol_decode_coverage_observations_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
},
|
||||
crate::MATERIALIZED_EVENTS_TABLE_NAME => {
|
||||
crate::postgres::query::table_diagnostics_queries::load_table_statistics_from_sql(
|
||||
load_table_statistics_from_sql(
|
||||
pool,
|
||||
crate::postgres::migrations::table_stats_kb_sol_mat_events_sql(),
|
||||
crate::table_stats_kb_sol_mat_events_sql(),
|
||||
table_name,
|
||||
)
|
||||
.await
|
||||
@@ -141,7 +138,7 @@ async fn load_table_statistics_from_sql(
|
||||
pool: &sqlx::PgPool,
|
||||
sql: &'static str,
|
||||
table_name: &str,
|
||||
) -> ks_core::Result<crate::PostgresTableStatistics> {
|
||||
) -> ks_core::Result<crate::StoreResourceStatistics> {
|
||||
let query_result = sqlx::query(sql).fetch_one(pool).await;
|
||||
let row = match query_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
@@ -188,8 +185,8 @@ async fn load_table_statistics_from_sql(
|
||||
)));
|
||||
},
|
||||
};
|
||||
return std::result::Result::Ok(crate::PostgresTableStatistics {
|
||||
row_count,
|
||||
return std::result::Result::Ok(crate::StoreResourceStatistics {
|
||||
record_count: row_count,
|
||||
min_slot,
|
||||
max_slot,
|
||||
latest_created_at,
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/repository/core_extraction_repository.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL atomic canonical transaction to core extraction repository.
|
||||
|
||||
@@ -13,11 +13,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
|
||||
&self,
|
||||
filter: &crate::CoreExtractionSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::RawTransactionRow>> {
|
||||
return crate::postgres::query::list_raw_transactions_for_core_extraction(
|
||||
self.pool(),
|
||||
filter,
|
||||
)
|
||||
.await;
|
||||
return crate::list_raw_transactions_for_core_extraction(self.pool(), filter).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -28,7 +24,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::is_core_extraction_current(self.pool(), identity).await;
|
||||
return crate::is_core_extraction_current(self.pool(), identity).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -40,8 +36,7 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
|
||||
bundle: &crate::CoreExtractionBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_core_extraction(self.pool(), bundle, force_replay)
|
||||
.await;
|
||||
return crate::persist_core_extraction(self.pool(), bundle, force_replay).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -52,6 +47,6 @@ impl crate::CoreExtractionStore for crate::PostgresStore {
|
||||
&self,
|
||||
failure: &crate::CoreExtractionFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::mark_core_extraction_failed(self.pool(), failure).await;
|
||||
return crate::mark_core_extraction_failed(self.pool(), failure).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/repository/core_transaction_repository.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL core Solana repository implementation.
|
||||
|
||||
@@ -13,7 +13,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
input: &crate::CoreTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_transaction(self.pool(), input).await;
|
||||
return crate::insert_core_transaction(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -24,7 +24,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
inputs: &[crate::CoreAccountKeyInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_account_keys(self.pool(), inputs).await;
|
||||
return crate::insert_core_account_keys(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -35,7 +35,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
inputs: &[crate::CoreInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_instructions(self.pool(), inputs).await;
|
||||
return crate::insert_core_instructions(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -46,7 +46,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
inputs: &[crate::CoreInnerInstructionInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_inner_instructions(self.pool(), inputs).await;
|
||||
return crate::insert_core_inner_instructions(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -57,7 +57,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
inputs: &[crate::CoreLogInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_logs(self.pool(), inputs).await;
|
||||
return crate::insert_core_logs(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -68,7 +68,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
inputs: &[crate::CoreBalanceChangeInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_core_balance_changes(self.pool(), inputs).await;
|
||||
return crate::insert_core_balance_changes(self.pool(), inputs).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -80,12 +80,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::CoreInstructionRow>> {
|
||||
return crate::postgres::query::list_core_instructions_for_replay(
|
||||
self.pool(),
|
||||
filter,
|
||||
page_request,
|
||||
)
|
||||
.await;
|
||||
return crate::list_core_instructions_for_replay(self.pool(), filter, page_request).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -97,12 +92,7 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
filter: &crate::CoreInstructionReplayFilter,
|
||||
page_request: &crate::PageRequest,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
return crate::postgres::query::list_core_instruction_replay_inputs(
|
||||
self.pool(),
|
||||
filter,
|
||||
page_request,
|
||||
)
|
||||
.await;
|
||||
return crate::list_core_instruction_replay_inputs(self.pool(), filter, page_request).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -113,6 +103,6 @@ impl crate::CoreTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
mark: &crate::CoreInstructionLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::update_core_instruction_lifecycle(self.pool(), mark).await;
|
||||
return crate::update_core_instruction_lifecycle(self.pool(), mark).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/repository/decode_pipeline_repository.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL contextual decode and materialization repository.
|
||||
|
||||
@@ -13,7 +13,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
&self,
|
||||
filter: &crate::DecodeSelectionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MdCoreInstructionReplayInput>> {
|
||||
return crate::postgres::query::list_decode_inputs(self.pool(), filter).await;
|
||||
return crate::list_decode_inputs(self.pool(), filter).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -24,7 +24,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
&self,
|
||||
identity: &crate::ProcessingLedgerIdentity,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::is_decode_current(self.pool(), identity).await;
|
||||
return crate::is_decode_current(self.pool(), identity).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -35,11 +35,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
&self,
|
||||
declarations: &[crate::DecodeCoverageDeclarationInsert],
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_decode_coverage_declarations(
|
||||
self.pool(),
|
||||
declarations,
|
||||
)
|
||||
.await;
|
||||
return crate::persist_decode_coverage_declarations(self.pool(), declarations).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -51,8 +47,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
bundle: &crate::DecodePersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_decode_result(self.pool(), bundle, force_replay)
|
||||
.await;
|
||||
return crate::persist_decode_result(self.pool(), bundle, force_replay).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -63,7 +58,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
&self,
|
||||
failure: &crate::DecodeFailure,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::mark_decode_failed(self.pool(), failure).await;
|
||||
return crate::mark_decode_failed(self.pool(), failure).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -75,12 +70,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
bundle: &crate::MaterializationPersistenceBundle,
|
||||
force_replay: bool,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::persist_materialization_result(
|
||||
self.pool(),
|
||||
bundle,
|
||||
force_replay,
|
||||
)
|
||||
.await;
|
||||
return crate::persist_materialization_result(self.pool(), bundle, force_replay).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -93,7 +83,7 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
processor_version: std::option::Option<&str>,
|
||||
limit: u32,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::DecodeCoverageSummaryRow>> {
|
||||
return crate::postgres::query::list_decode_coverage_summary(
|
||||
return crate::list_decode_coverage_summary(
|
||||
self.pool(),
|
||||
processor_name,
|
||||
processor_version,
|
||||
@@ -110,6 +100,6 @@ impl crate::DecodePipelineStore for crate::PostgresStore {
|
||||
&self,
|
||||
filter: &crate::MaterializedEventFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::MaterializedEventQueryRow>> {
|
||||
return crate::postgres::query::list_materialized_events(self.pool(), filter).await;
|
||||
return crate::list_materialized_events(self.pool(), filter).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/repository/raw_transaction_repository.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! PostgreSQL canonical transaction and acquisition observation repository implementation.
|
||||
|
||||
@@ -13,11 +13,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
signature: &ks_lib::MdSignature,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::has_raw_transaction_signature(
|
||||
self.pool(),
|
||||
signature.0.as_str(),
|
||||
)
|
||||
.await;
|
||||
return crate::has_raw_transaction_signature(self.pool(), signature.0.as_str()).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -28,11 +24,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
observation_key: &str,
|
||||
) -> ks_core::Result<bool> {
|
||||
return crate::postgres::query::has_transaction_observation_key(
|
||||
self.pool(),
|
||||
observation_key,
|
||||
)
|
||||
.await;
|
||||
return crate::has_transaction_observation_key(self.pool(), observation_key).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -43,7 +35,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
input: &crate::RawTransactionInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_raw_transaction(self.pool(), input).await;
|
||||
return crate::insert_raw_transaction(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -54,7 +46,7 @@ impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
input: &crate::TransactionObservationInsert,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::insert_transaction_observation(self.pool(), input).await;
|
||||
return crate::insert_transaction_observation(self.pool(), input).await;
|
||||
}
|
||||
|
||||
#[expect(
|
||||
@@ -65,6 +57,6 @@ impl crate::RawTransactionStore for crate::PostgresStore {
|
||||
&self,
|
||||
mark: &crate::RawPayloadLifecycleMark,
|
||||
) -> ks_core::Result<crate::InsertOutcome> {
|
||||
return crate::postgres::query::update_raw_payload_lifecycle(self.pool(), mark).await;
|
||||
return crate::update_raw_payload_lifecycle(self.pool(), mark).await;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,42 +1,131 @@
|
||||
// file: ks-store/src/postgres/store.rs
|
||||
// version: 4
|
||||
// version: 9
|
||||
|
||||
//! Store implementation scaffold for the `ks-store` crate.
|
||||
//! PostgreSQL store implementation kept behind the backend-agnostic `Store` facade.
|
||||
|
||||
/// PostgreSQL store connection options.
|
||||
#[derive(Clone, Debug, Eq, PartialEq)]
|
||||
pub struct PostgresStoreOptions {
|
||||
/// Database URL or DSN.
|
||||
pub database_url: std::string::String,
|
||||
/// Maximum connection count.
|
||||
pub max_connections: u32,
|
||||
/// Connection timeout in milliseconds.
|
||||
pub connect_timeout_ms: u64,
|
||||
/// Enables idempotent raw schema initialization at startup.
|
||||
pub auto_initialize_schema: bool,
|
||||
/// PostgreSQL store connection options interpreted only inside `ks-store`.
|
||||
#[derive(Clone, Eq, PartialEq)]
|
||||
pub(crate) struct PostgresStoreOptions {
|
||||
database_url: std::string::String,
|
||||
max_connections: u32,
|
||||
connect_timeout_ms: u64,
|
||||
auto_initialize_schema: bool,
|
||||
option_count: u32,
|
||||
}
|
||||
|
||||
impl PostgresStoreOptions {
|
||||
/// Creates validated PostgreSQL store options.
|
||||
pub fn new(
|
||||
impl crate::PostgresStoreOptions {
|
||||
/// Creates validated PostgreSQL store options for crate-internal tests and adapters.
|
||||
#[cfg(test)]
|
||||
pub(crate) fn new(
|
||||
database_url: impl std::convert::Into<std::string::String>,
|
||||
max_connections: u32,
|
||||
connect_timeout_ms: u64,
|
||||
auto_initialize_schema: bool,
|
||||
) -> ks_core::Result<Self> {
|
||||
return Self::new_with_option_count(
|
||||
database_url,
|
||||
max_connections,
|
||||
connect_timeout_ms,
|
||||
auto_initialize_schema,
|
||||
4,
|
||||
);
|
||||
}
|
||||
|
||||
/// Interprets the selected opaque backend options supplied to `Store::open`.
|
||||
pub(crate) fn from_backend_options(options: &serde_json::Value) -> ks_core::Result<Self> {
|
||||
let object = match options.as_object() {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_invalid",
|
||||
"postgres backend options must be a JSON object",
|
||||
));
|
||||
},
|
||||
};
|
||||
let database_url = match object.get("url").and_then(serde_json::Value::as_str) {
|
||||
std::option::Option::Some(value) => value.to_string(),
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_incomplete",
|
||||
"postgres backend requires a non-empty connection URL",
|
||||
));
|
||||
},
|
||||
};
|
||||
let max_connections_u64 =
|
||||
match object.get("max_connections").and_then(serde_json::Value::as_u64) {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_incomplete",
|
||||
"postgres backend requires max_connections greater than zero",
|
||||
));
|
||||
},
|
||||
};
|
||||
let max_connections = match u32::try_from(max_connections_u64) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_invalid",
|
||||
"postgres max_connections does not fit into u32",
|
||||
));
|
||||
},
|
||||
};
|
||||
let connect_timeout_ms =
|
||||
match object.get("connect_timeout_ms").and_then(serde_json::Value::as_u64) {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_incomplete",
|
||||
"postgres backend requires connect_timeout_ms greater than zero",
|
||||
));
|
||||
},
|
||||
};
|
||||
let auto_initialize_schema =
|
||||
match object.get("auto_initialize_schema").and_then(serde_json::Value::as_bool) {
|
||||
std::option::Option::Some(value) => value,
|
||||
std::option::Option::None => {
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_incomplete",
|
||||
"postgres backend requires auto_initialize_schema",
|
||||
));
|
||||
},
|
||||
};
|
||||
let option_count = match u32::try_from(object.len()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_) => u32::MAX,
|
||||
};
|
||||
return Self::new_with_option_count(
|
||||
database_url,
|
||||
max_connections,
|
||||
connect_timeout_ms,
|
||||
auto_initialize_schema,
|
||||
option_count,
|
||||
);
|
||||
}
|
||||
|
||||
fn new_with_option_count(
|
||||
database_url: impl std::convert::Into<std::string::String>,
|
||||
max_connections: u32,
|
||||
connect_timeout_ms: u64,
|
||||
auto_initialize_schema: bool,
|
||||
option_count: u32,
|
||||
) -> ks_core::Result<Self> {
|
||||
let database_url_value = database_url.into();
|
||||
if database_url_value.trim().is_empty() {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
"postgres database url must not be empty",
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_incomplete",
|
||||
"postgres backend requires a non-empty connection URL",
|
||||
));
|
||||
}
|
||||
if max_connections == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_invalid",
|
||||
"postgres max_connections must be greater than zero",
|
||||
));
|
||||
}
|
||||
if connect_timeout_ms == 0 {
|
||||
return std::result::Result::Err(ks_core::Error::db(
|
||||
return std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_options_invalid",
|
||||
"postgres connect_timeout_ms must be greater than zero",
|
||||
));
|
||||
}
|
||||
@@ -45,173 +134,142 @@ impl PostgresStoreOptions {
|
||||
max_connections,
|
||||
connect_timeout_ms,
|
||||
auto_initialize_schema,
|
||||
option_count,
|
||||
});
|
||||
}
|
||||
|
||||
/// Returns whether automatic schema initialization is enabled.
|
||||
pub(crate) fn auto_initialize_schema(&self) -> bool {
|
||||
return self.auto_initialize_schema;
|
||||
}
|
||||
|
||||
/// Returns a backend-neutral safe configuration summary.
|
||||
pub(crate) fn configuration_summary(&self) -> crate::StoreConfigurationSummary {
|
||||
return crate::StoreConfigurationSummary {
|
||||
enabled: true,
|
||||
backend_code: "postgres".to_string(),
|
||||
connection_configured: !self.database_url.trim().is_empty(),
|
||||
auto_initialize_schema: self.auto_initialize_schema,
|
||||
backend_option_count: self.option_count,
|
||||
};
|
||||
}
|
||||
|
||||
/// Returns a DSN masked for diagnostics.
|
||||
pub fn masked_dsn(&self) -> std::string::String {
|
||||
return crate::mask_postgres_dsn(self.database_url.as_str());
|
||||
pub(crate) fn masked_connection_descriptor(&self) -> std::string::String {
|
||||
return mask_postgres_dsn(self.database_url.as_str());
|
||||
}
|
||||
}
|
||||
|
||||
/// PostgreSQL diagnostic snapshot.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresBackendDiagnostics {
|
||||
/// Backend descriptor safe for UI display.
|
||||
pub descriptor: crate::StoreBackendDescriptor,
|
||||
/// Backend health snapshot.
|
||||
pub health: crate::StoreHealthSnapshot,
|
||||
/// Migration status snapshot.
|
||||
pub migrations: crate::StoreMigrationSnapshot,
|
||||
/// Full PostgreSQL server version string when available.
|
||||
pub server_version: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only statistics for one PostgreSQL table.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresTableStatistics {
|
||||
/// Number of rows currently stored in the table.
|
||||
pub row_count: i64,
|
||||
/// Lowest observed Solana slot when the table contains a slot column and rows.
|
||||
pub min_slot: std::option::Option<i64>,
|
||||
/// Highest observed Solana slot when the table contains a slot column and rows.
|
||||
pub max_slot: std::option::Option<i64>,
|
||||
/// Latest insertion timestamp rendered by PostgreSQL for UI diagnostics.
|
||||
pub latest_created_at: std::option::Option<std::string::String>,
|
||||
}
|
||||
|
||||
/// Read-only diagnostics for one expected PostgreSQL table.
|
||||
#[derive(Clone, Debug, Eq, PartialEq, serde::Deserialize, serde::Serialize)]
|
||||
pub struct PostgresTableDiagnostics {
|
||||
/// Expected table name.
|
||||
pub table_name: std::string::String,
|
||||
/// Logical Solana domain encoded in the table name.
|
||||
pub domain: std::string::String,
|
||||
/// Human-readable role of the table.
|
||||
pub role: std::string::String,
|
||||
/// Whether the table exists in the current PostgreSQL search path.
|
||||
pub exists: bool,
|
||||
/// Table statistics when the table exists.
|
||||
pub statistics: std::option::Option<crate::PostgresTableStatistics>,
|
||||
}
|
||||
|
||||
/// PostgreSQL store handle.
|
||||
#[derive(Clone, Debug)]
|
||||
pub struct PostgresStore {
|
||||
/// PostgreSQL store handle kept private to `ks-store`.
|
||||
#[derive(Clone)]
|
||||
pub(crate) struct PostgresStore {
|
||||
options: crate::PostgresStoreOptions,
|
||||
pool: sqlx::PgPool,
|
||||
}
|
||||
|
||||
impl PostgresStore {
|
||||
/// Connects to PostgreSQL from typed store options.
|
||||
pub async fn connect(options: crate::PostgresStoreOptions) -> ks_core::Result<Self> {
|
||||
impl crate::PostgresStore {
|
||||
/// Connects to PostgreSQL from validated backend options.
|
||||
pub(crate) async fn connect(options: crate::PostgresStoreOptions) -> ks_core::Result<Self> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", "open PostgreSQL store connection");
|
||||
let pool_options = sqlx::postgres::PgPoolOptions::new()
|
||||
.max_connections(options.max_connections)
|
||||
.acquire_timeout(std::time::Duration::from_millis(options.connect_timeout_ms));
|
||||
let connect_result = pool_options.connect(options.database_url.as_str()).await;
|
||||
return match connect_result {
|
||||
std::result::Result::Ok(pool) => {
|
||||
let store = Self { options, pool };
|
||||
if store.options.auto_initialize_schema {
|
||||
let schema_result = store.initialize_store_schema().await;
|
||||
if let std::result::Result::Err(error) = schema_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
}
|
||||
std::result::Result::Ok(store)
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", connected = true, "PostgreSQL store connection opened");
|
||||
std::result::Result::Ok(Self { options, pool })
|
||||
},
|
||||
std::result::Result::Err(_error) => {
|
||||
tracing::error!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "connection_open", connected = false, "PostgreSQL store connection failed");
|
||||
std::result::Result::Err(crate::storage_contract_error(
|
||||
"store_backend_connection_failed",
|
||||
"postgres connection failed",
|
||||
))
|
||||
},
|
||||
std::result::Result::Err(error) => std::result::Result::Err(ks_core::Error::db(
|
||||
format!("postgres connection failed: {error}"),
|
||||
)),
|
||||
};
|
||||
}
|
||||
|
||||
/// Creates a store handle from an existing PostgreSQL pool.
|
||||
pub fn from_pool(options: crate::PostgresStoreOptions, pool: sqlx::PgPool) -> Self {
|
||||
return Self { options, pool };
|
||||
}
|
||||
|
||||
/// Returns the underlying PostgreSQL pool.
|
||||
pub fn pool(&self) -> &sqlx::PgPool {
|
||||
/// Returns the underlying PostgreSQL pool to backend-private repositories.
|
||||
pub(crate) fn pool(&self) -> &sqlx::PgPool {
|
||||
return &self.pool;
|
||||
}
|
||||
|
||||
/// Returns the connection options used to create this store.
|
||||
pub fn options(&self) -> &crate::PostgresStoreOptions {
|
||||
return &self.options;
|
||||
}
|
||||
|
||||
/// Applies each idempotent store schema once per invocation in dependency order.
|
||||
pub async fn initialize_store_schema(&self) -> ks_core::Result<()> {
|
||||
let raw_result = crate::postgres::query::apply_raw_store_schema(&self.pool).await;
|
||||
pub(crate) async fn initialize_store_schema(&self) -> ks_core::Result<()> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "schema_initialize", "initialize PostgreSQL store schema");
|
||||
let raw_result = crate::apply_raw_store_schema(&self.pool).await;
|
||||
if let std::result::Result::Err(error) = raw_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
let core_result = crate::postgres::query::apply_core_store_schema(&self.pool).await;
|
||||
let core_result = crate::apply_core_store_schema(&self.pool).await;
|
||||
if let std::result::Result::Err(error) = core_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
|
||||
let result = crate::apply_decode_store_schema(&self.pool).await;
|
||||
if result.is_ok() {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "schema_initialize", initialized = true, "PostgreSQL store schema initialized");
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
/// Applies the idempotent minimal raw Solana store schema.
|
||||
pub async fn initialize_raw_store_schema(&self) -> ks_core::Result<()> {
|
||||
return crate::postgres::query::apply_raw_store_schema(&self.pool).await;
|
||||
#[cfg(test)]
|
||||
pub(crate) async fn initialize_raw_store_schema(&self) -> ks_core::Result<()> {
|
||||
return crate::apply_raw_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Applies the idempotent minimal core Solana store schema.
|
||||
pub async fn initialize_core_store_schema(&self) -> ks_core::Result<()> {
|
||||
/// Applies the idempotent minimal Core Solana store schema.
|
||||
#[cfg(test)]
|
||||
pub(crate) async fn initialize_core_store_schema(&self) -> ks_core::Result<()> {
|
||||
let raw_result = self.initialize_raw_store_schema().await;
|
||||
if let std::result::Result::Err(error) = raw_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_core_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Applies the idempotent common decode and materialization store schema.
|
||||
pub async fn initialize_decode_store_schema(&self) -> ks_core::Result<()> {
|
||||
let core_result = self.initialize_core_store_schema().await;
|
||||
if let std::result::Result::Err(error) = core_result {
|
||||
return std::result::Result::Err(error);
|
||||
}
|
||||
return crate::postgres::query::apply_decode_store_schema(&self.pool).await;
|
||||
return crate::apply_core_store_schema(&self.pool).await;
|
||||
}
|
||||
|
||||
/// Reads a UI-safe backend descriptor.
|
||||
pub async fn backend_descriptor(&self) -> ks_core::Result<crate::StoreBackendDescriptor> {
|
||||
let schema_result = crate::postgres::query::load_current_schema(&self.pool).await;
|
||||
pub(crate) async fn backend_descriptor(
|
||||
&self,
|
||||
) -> ks_core::Result<crate::StoreBackendDescriptor> {
|
||||
let schema_result = crate::load_current_schema(&self.pool).await;
|
||||
return match schema_result {
|
||||
std::result::Result::Ok(schema) => crate::StoreBackendDescriptor::new(
|
||||
crate::StoreBackendKind::Postgres,
|
||||
"postgres",
|
||||
std::option::Option::Some(self.options.masked_dsn()),
|
||||
"PostgreSQL",
|
||||
std::option::Option::Some(self.options.masked_connection_descriptor()),
|
||||
std::option::Option::Some(schema),
|
||||
),
|
||||
std::result::Result::Err(error) => std::result::Result::Err(error),
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a PostgreSQL health snapshot.
|
||||
pub async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
|
||||
let health_result = crate::postgres::query::run_health_check(&self.pool).await;
|
||||
/// Reads a PostgreSQL health snapshot without exposing backend error details.
|
||||
pub(crate) async fn health_snapshot(&self) -> ks_core::Result<crate::StoreHealthSnapshot> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "health_check", "run PostgreSQL store health check");
|
||||
let health_result = crate::run_health_check(&self.pool).await;
|
||||
return match health_result {
|
||||
std::result::Result::Ok(()) => crate::StoreHealthSnapshot::new(
|
||||
"postgres",
|
||||
crate::StoreHealthStatus::Healthy,
|
||||
std::option::Option::Some(std::string::String::from("SELECT 1 succeeded")),
|
||||
std::option::Option::Some(std::string::String::from(
|
||||
"backend health check succeeded",
|
||||
)),
|
||||
),
|
||||
std::result::Result::Err(error) => crate::StoreHealthSnapshot::new(
|
||||
std::result::Result::Err(_error) => crate::StoreHealthSnapshot::new(
|
||||
"postgres",
|
||||
crate::StoreHealthStatus::Unhealthy,
|
||||
std::option::Option::Some(error.to_string()),
|
||||
std::option::Option::Some(std::string::String::from("backend health check failed")),
|
||||
),
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a non-destructive migration snapshot.
|
||||
pub async fn migration_snapshot(&self) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
let migration_table_result =
|
||||
crate::postgres::query::load_migration_table_name(&self.pool).await;
|
||||
pub(crate) async fn migration_snapshot(
|
||||
&self,
|
||||
) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
let migration_table_result = crate::load_migration_table_name(&self.pool).await;
|
||||
return match migration_table_result {
|
||||
std::result::Result::Ok(std::option::Option::None) => {
|
||||
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
|
||||
@@ -219,7 +277,7 @@ impl PostgresStore {
|
||||
std::option::Option::None,
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(std::string::String::from(
|
||||
"no sqlx migration table detected; 0.3.1 canonical acquisition/core schemas use idempotent crate-managed DDL",
|
||||
"no migration history table detected; crate-managed schema initialization is active",
|
||||
)),
|
||||
))
|
||||
},
|
||||
@@ -230,132 +288,121 @@ impl PostgresStore {
|
||||
};
|
||||
}
|
||||
|
||||
/// Reads a complete PostgreSQL diagnostic snapshot.
|
||||
pub async fn backend_diagnostics(&self) -> ks_core::Result<crate::PostgresBackendDiagnostics> {
|
||||
let descriptor_result = self.backend_descriptor().await;
|
||||
let descriptor = match descriptor_result {
|
||||
/// Reads a complete backend-neutral diagnostic snapshot.
|
||||
pub(crate) async fn backend_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<crate::StoreBackendDiagnostics> {
|
||||
let descriptor = match self.backend_descriptor().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let health_result = self.health_snapshot().await;
|
||||
let health = match health_result {
|
||||
let health = match self.health_snapshot().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let migrations_result = self.migration_snapshot().await;
|
||||
let migrations = match migrations_result {
|
||||
let migrations = match self.migration_snapshot().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let server_version = match crate::postgres::query::load_server_version(&self.pool).await {
|
||||
let backend_version = match crate::load_server_version(&self.pool).await {
|
||||
std::result::Result::Ok(value) => std::option::Option::Some(value),
|
||||
std::result::Result::Err(_error) => std::option::Option::None,
|
||||
};
|
||||
return std::result::Result::Ok(crate::PostgresBackendDiagnostics {
|
||||
return std::result::Result::Ok(crate::StoreBackendDiagnostics {
|
||||
descriptor,
|
||||
health,
|
||||
migrations,
|
||||
server_version,
|
||||
backend_version,
|
||||
});
|
||||
}
|
||||
|
||||
/// Lists bounded raw transaction candidates enriched with core and ledger diagnostics.
|
||||
pub async fn replay_transaction_candidates(
|
||||
/// Lists bounded raw transaction candidates enriched with Core and ledger diagnostics.
|
||||
pub(crate) async fn replay_transaction_candidates(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayTransactionCandidate>> {
|
||||
return crate::postgres::query::list_replay_transaction_candidates(&self.pool, filter)
|
||||
.await;
|
||||
filter: &crate::ReplayTransactionFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayTransactionCandidate>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_transaction_candidates", "query PostgreSQL replay transaction candidates");
|
||||
return crate::list_replay_transaction_candidates(&self.pool, filter).await;
|
||||
}
|
||||
|
||||
/// Lists bounded program summaries across outer, inner and reliably linked logs.
|
||||
pub async fn replay_program_summaries(
|
||||
/// Lists bounded program summaries across top-level, inner and reliably linked logs.
|
||||
pub(crate) async fn replay_program_summaries(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayProgramSummary>> {
|
||||
return crate::postgres::query::list_replay_program_summaries(&self.pool, filter).await;
|
||||
filter: &crate::ReplayProgramFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayProgramSummary>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_program_summaries", "query PostgreSQL replay program summaries");
|
||||
return crate::list_replay_program_summaries(&self.pool, filter).await;
|
||||
}
|
||||
|
||||
/// Lists bounded mint, owner or account-key summaries from core tables.
|
||||
pub async fn replay_entity_summaries(
|
||||
/// Lists bounded mint, owner or account-key summaries from Core facts.
|
||||
pub(crate) async fn replay_entity_summaries(
|
||||
&self,
|
||||
filter: &crate::PostgresReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresReplayEntitySummary>> {
|
||||
return crate::postgres::query::list_replay_entity_summaries(&self.pool, filter).await;
|
||||
filter: &crate::ReplayEntityFilter,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::ReplayEntitySummary>> {
|
||||
tracing::debug!(target: crate::TRACING_TARGET, backend = "postgres", domain = "ks-store.pg", action = "replay_entity_summaries", "query PostgreSQL replay entity summaries");
|
||||
return crate::list_replay_entity_summaries(&self.pool, filter).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for raw Solana store tables without changing the schema.
|
||||
pub async fn raw_table_diagnostics(
|
||||
/// Reads diagnostics for raw store resources without changing the schema.
|
||||
pub(crate) async fn raw_resource_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
|
||||
let specs = crate::raw_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
return self.resource_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for core Solana store tables without changing the schema.
|
||||
pub async fn core_table_diagnostics(
|
||||
/// Reads diagnostics for Core and processing store resources without changing the schema.
|
||||
pub(crate) async fn core_resource_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
|
||||
let specs = crate::core_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
return self.resource_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for decode and materialization store tables without changing the schema.
|
||||
pub async fn decode_table_diagnostics(
|
||||
/// Reads diagnostics for decode and materialization resources without changing the schema.
|
||||
pub(crate) async fn decode_resource_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
|
||||
let specs = crate::decode_store_table_diagnostic_specs();
|
||||
return self.table_diagnostics(&specs).await;
|
||||
return self.resource_diagnostics(&specs).await;
|
||||
}
|
||||
|
||||
/// Reads diagnostics for every known raw/core/decode Solana store table.
|
||||
pub async fn known_table_diagnostics(
|
||||
/// Reads diagnostics for every known logical store resource.
|
||||
pub(crate) async fn known_resource_diagnostics(
|
||||
&self,
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
|
||||
let mut diagnostics = std::vec::Vec::new();
|
||||
let raw_result = self.raw_table_diagnostics().await;
|
||||
let raw_tables = match raw_result {
|
||||
let raw_resources = match self.raw_resource_diagnostics().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in raw_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
let core_result = self.core_table_diagnostics().await;
|
||||
let core_tables = match core_result {
|
||||
diagnostics.extend(raw_resources);
|
||||
let core_resources = match self.core_resource_diagnostics().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in core_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
let decode_result = self.decode_table_diagnostics().await;
|
||||
let decode_tables = match decode_result {
|
||||
diagnostics.extend(core_resources);
|
||||
let decode_resources = match self.decode_resource_diagnostics().await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
for table in decode_tables {
|
||||
diagnostics.push(table);
|
||||
}
|
||||
diagnostics.extend(decode_resources);
|
||||
return std::result::Result::Ok(diagnostics);
|
||||
}
|
||||
|
||||
async fn table_diagnostics(
|
||||
async fn resource_diagnostics(
|
||||
&self,
|
||||
specs: &[crate::PostgresTableDiagnosticSpec],
|
||||
) -> ks_core::Result<std::vec::Vec<crate::PostgresTableDiagnostics>> {
|
||||
) -> ks_core::Result<std::vec::Vec<crate::StoreResourceDiagnostics>> {
|
||||
let mut diagnostics = std::vec::Vec::new();
|
||||
for spec in specs {
|
||||
let exists_result =
|
||||
crate::postgres::query::table_exists(&self.pool, spec.table_name).await;
|
||||
let exists = match exists_result {
|
||||
let available = match crate::table_exists(&self.pool, spec.table_name).await {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
};
|
||||
let statistics = if exists {
|
||||
let statistics = if available {
|
||||
let statistics_result =
|
||||
crate::postgres::query::load_table_statistics(&self.pool, spec.table_name)
|
||||
.await;
|
||||
crate::load_table_statistics(&self.pool, spec.table_name).await;
|
||||
match statistics_result {
|
||||
std::result::Result::Ok(value) => std::option::Option::Some(value),
|
||||
std::result::Result::Err(error) => return std::result::Result::Err(error),
|
||||
@@ -363,11 +410,11 @@ impl PostgresStore {
|
||||
} else {
|
||||
std::option::Option::None
|
||||
};
|
||||
diagnostics.push(crate::PostgresTableDiagnostics {
|
||||
table_name: spec.table_name.to_string(),
|
||||
domain: spec.domain.to_string(),
|
||||
diagnostics.push(crate::StoreResourceDiagnostics {
|
||||
resource_code: spec.resource_code.to_string(),
|
||||
model_code: spec.model_code.to_string(),
|
||||
role: spec.role.to_string(),
|
||||
exists,
|
||||
available,
|
||||
statistics,
|
||||
});
|
||||
}
|
||||
@@ -377,8 +424,7 @@ impl PostgresStore {
|
||||
async fn migration_snapshot_from_existing_table(
|
||||
&self,
|
||||
) -> ks_core::Result<crate::StoreMigrationSnapshot> {
|
||||
let version_result =
|
||||
crate::postgres::query::load_latest_migration_version(&self.pool).await;
|
||||
let version_result = crate::load_latest_migration_version(&self.pool).await;
|
||||
return match version_result {
|
||||
std::result::Result::Ok(current_version) => {
|
||||
std::result::Result::Ok(crate::StoreMigrationSnapshot::new(
|
||||
@@ -386,7 +432,7 @@ impl PostgresStore {
|
||||
current_version,
|
||||
std::vec::Vec::new(),
|
||||
std::option::Option::Some(std::string::String::from(
|
||||
"sqlx migration table detected; canonical acquisition/core schema remains idempotent and crate-managed in 0.3.1",
|
||||
"migration history table detected",
|
||||
)),
|
||||
))
|
||||
},
|
||||
@@ -395,22 +441,18 @@ impl PostgresStore {
|
||||
}
|
||||
}
|
||||
|
||||
/// Returns a DSN masked for logs and UI diagnostics.
|
||||
pub fn mask_postgres_dsn(dsn: &str) -> std::string::String {
|
||||
/// Returns a PostgreSQL connection descriptor masked for logs and diagnostics.
|
||||
fn mask_postgres_dsn(dsn: &str) -> std::string::String {
|
||||
let trimmed_dsn = dsn.trim();
|
||||
if trimmed_dsn.is_empty() {
|
||||
return std::string::String::from("");
|
||||
}
|
||||
let queryless = crate::postgres::store::strip_query(trimmed_dsn);
|
||||
let queryless = strip_query(trimmed_dsn);
|
||||
return match queryless.split_once("://") {
|
||||
std::option::Option::Some((scheme, remainder)) => {
|
||||
crate::postgres::store::mask_scheme_remainder(
|
||||
scheme,
|
||||
remainder,
|
||||
trimmed_dsn.contains('?'),
|
||||
)
|
||||
mask_scheme_remainder(scheme, remainder, trimmed_dsn.contains('?'))
|
||||
},
|
||||
std::option::Option::None => crate::postgres::store::mask_plain_dsn(queryless.as_str()),
|
||||
std::option::Option::None => mask_plain_dsn(queryless.as_str()),
|
||||
};
|
||||
}
|
||||
|
||||
@@ -422,7 +464,7 @@ fn strip_query(dsn: &str) -> std::string::String {
|
||||
}
|
||||
|
||||
fn mask_scheme_remainder(scheme: &str, remainder: &str, had_query: bool) -> std::string::String {
|
||||
let suffix = crate::postgres::store::query_suffix(had_query);
|
||||
let suffix = query_suffix(had_query);
|
||||
return match remainder.rsplit_once('@') {
|
||||
std::option::Option::Some((_userinfo, host_path)) => {
|
||||
format!("{scheme}://***:***@{host_path}{suffix}")
|
||||
@@ -448,21 +490,51 @@ fn query_suffix(had_query: bool) -> std::string::String {
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
#[test]
|
||||
fn options_reject_empty_database_url() {
|
||||
fn backend_options_reject_empty_database_url() {
|
||||
let result = crate::PostgresStoreOptions::new(" ", 1, 1000, false);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn options_reject_zero_max_connections() {
|
||||
fn backend_options_reject_zero_max_connections() {
|
||||
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 0, 1000, false);
|
||||
assert!(result.is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn options_reject_zero_connect_timeout() {
|
||||
let result = crate::PostgresStoreOptions::new("postgres://localhost/db", 1, 0, false);
|
||||
assert!(result.is_err());
|
||||
fn incomplete_backend_options_do_not_echo_secret_values() {
|
||||
let options = serde_json::json!({
|
||||
"url": "postgres://operator:STORE-SECRET-CANARY@localhost/db",
|
||||
"connect_timeout_ms": 5000,
|
||||
"auto_initialize_schema": true
|
||||
});
|
||||
let error = match crate::PostgresStoreOptions::from_backend_options(&options) {
|
||||
std::result::Result::Ok(_) => panic!("incomplete backend options must be rejected"),
|
||||
std::result::Result::Err(error) => error,
|
||||
};
|
||||
assert!(!error.to_string().contains("STORE-SECRET-CANARY"));
|
||||
assert!(!error.to_string().contains("postgres://"));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn opaque_backend_options_are_sanitized_in_summary() {
|
||||
let options = serde_json::json!({
|
||||
"url": "postgres://operator:STORE-SECRET-CANARY@localhost/db",
|
||||
"max_connections": 4,
|
||||
"connect_timeout_ms": 5000,
|
||||
"auto_initialize_schema": true
|
||||
});
|
||||
let parsed = crate::PostgresStoreOptions::from_backend_options(&options);
|
||||
let value = match parsed {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("backend options must parse: {error}"),
|
||||
};
|
||||
let serialized = match serde_json::to_string(&value.configuration_summary()) {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(error) => panic!("summary must serialize: {error}"),
|
||||
};
|
||||
assert!(!serialized.contains("STORE-SECRET-CANARY"));
|
||||
assert!(!serialized.contains("postgres://"));
|
||||
}
|
||||
|
||||
#[tokio::test]
|
||||
@@ -471,7 +543,7 @@ mod tests {
|
||||
std::result::Result::Ok(value) => value,
|
||||
std::result::Result::Err(_error) => return,
|
||||
};
|
||||
let _postgres_guard = crate::postgres::test_serial::postgres_test_guard().await;
|
||||
let _postgres_guard = crate::postgres_test_guard().await;
|
||||
let options_result = crate::PostgresStoreOptions::new(database_url, 1, 5000, false);
|
||||
let options = match options_result {
|
||||
std::result::Result::Ok(value) => value,
|
||||
@@ -488,25 +560,24 @@ mod tests {
|
||||
std::result::Result::Err(error) => panic!("unexpected health error: {error}"),
|
||||
};
|
||||
assert_eq!(health.status, crate::StoreHealthStatus::Healthy);
|
||||
return;
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_userinfo() {
|
||||
let masked = crate::mask_postgres_dsn("postgres://user:secret@localhost:5432/db");
|
||||
let masked = super::mask_postgres_dsn("postgres://user:secret@localhost:5432/db");
|
||||
assert_eq!(masked, "postgres://***:***@localhost:5432/db");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_query_string() {
|
||||
let masked =
|
||||
crate::mask_postgres_dsn("postgres://localhost/db?sslmode=require&password=secret");
|
||||
super::mask_postgres_dsn("postgres://localhost/db?sslmode=require&password=secret");
|
||||
assert_eq!(masked, "postgres://localhost/db?<redacted>");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn mask_postgres_dsn_masks_plain_password_dsn() {
|
||||
let masked = crate::mask_postgres_dsn("host=localhost password=secret dbname=khadhroony");
|
||||
let masked = super::mask_postgres_dsn("host=localhost password=secret dbname=khadhroony");
|
||||
assert_eq!(masked, "<postgres-dsn-redacted>");
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
// file: ks-store/src/postgres/test_serial.rs
|
||||
// version: 2
|
||||
// version: 3
|
||||
|
||||
//! Test-only serialization helpers for optional real PostgreSQL tests.
|
||||
|
||||
@@ -7,7 +7,7 @@ static POSTGRES_TEST_MUTEX: std::sync::OnceLock<std::sync::Arc<tokio::sync::Mute
|
||||
std::sync::OnceLock::new();
|
||||
|
||||
/// Acquires the process-local guard shared by optional real PostgreSQL tests.
|
||||
pub(in crate::postgres) async fn postgres_test_guard() -> tokio::sync::OwnedMutexGuard<()> {
|
||||
pub(crate) async fn postgres_test_guard() -> tokio::sync::OwnedMutexGuard<()> {
|
||||
let mutex = POSTGRES_TEST_MUTEX
|
||||
.get_or_init(|| return std::sync::Arc::new(tokio::sync::Mutex::new(())))
|
||||
.clone();
|
||||
|
||||
1023
ks-store/src/store.rs
Normal file
1023
ks-store/src/store.rs
Normal file
File diff suppressed because it is too large
Load Diff
Reference in New Issue
Block a user