Sasedev/khadhroony-bobobot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

0.7.28

Browse Source
This commit is contained in:
SinuS von SifriduS
2026-05-11 11:02:47 +02:00
parent d66afede28
commit 7f130dba6b
49 changed files with 10301 additions and 8481 deletions
Download Patch File Download Diff File Expand all files Collapse all files

1
CHANGELOG.md
View File

@@ -58,3 +58,4 @@
0.7.25 - Enrichissement metadata des tokens, avec résolution locale limitée à SOL / WSOL, résolution des autres mints via comptes on-chain, Token-2022, Metaplex ou payloads DEX, et conservation explicite des cas non résolus 0.7.25 - Enrichissement metadata des tokens, avec résolution locale limitée à SOL / WSOL, résolution des autres mints via comptes on-chain, Token-2022, Metaplex ou payloads DEX, et conservation explicite des cas non résolus
0.7.26 - Diagnostics locaux du pipeline persisté, correction de l’agrégation instruction-scoped des swaps Raydium, clarification des compteurs de replay/upsert, et validation qu’aucun trade candidate issu d’une transaction OK n’est perdu 0.7.26 - Diagnostics locaux du pipeline persisté, correction de l’agrégation instruction-scoped des swaps Raydium, clarification des compteurs de replay/upsert, et validation qu’aucun trade candidate issu d’une transaction OK n’est perdu
0.7.27 - Validation multi-DEX et non-régression du pipeline sur Pump.fun, PumpSwap, Raydium CPMM et Raydium CLMM, avec corpus de tests, diagnostics de référence et garanties sur les événements non pricés 0.7.27 - Validation multi-DEX et non-régression du pipeline sur Pump.fun, PumpSwap, Raydium CPMM et Raydium CLMM, avec corpus de tests, diagnostics de référence et garanties sur les événements non pricés
0.7.28 - nettoyer la couche DEX avant d’ajouter de nouveaux protocoles, sans modifier le transport HTTP/WS déjà stabilisé.

2
Cargo.toml
View File

@@ -8,7 +8,7 @@ members = [
] ]
[workspace.package] [workspace.package]
version = "0.7.27" version = "0.7.28"
edition = "2024" edition = "2024"
license = "MIT" license = "MIT"
repository = "https://git.sasedev.com/Sasedev/khadhroony-bobobot" repository = "https://git.sasedev.com/Sasedev/khadhroony-bobobot"

463
README.md
View File

@@ -2,256 +2,259 @@
# khadhroony-bobobot # khadhroony-bobobot
Projet personnel Rust de détection, d’analyse de patterns et de trading/swap semi-automatisé de tokens et meme-tokens sur la blockchain Solana. `khadhroony-bobobot` est un workspace Rust destiné à la détection, au décodage, à l’analyse et, à terme, au trading semi-automatisé de tokens Solana.
Le README précédent décrivait surtout l’état `0.3.1`. Ce fichier reflète l’état de reprise autour de `0.7.27` : le socle transport HTTP/WS, la résolution transactionnelle, le modèle SQLite, plusieurs connecteurs DEX, les candles, les signaux analytiques et l’application de démonstration existent déjà.
## 1. Objectif ## 1. Objectif
L’objectif du projet est de construire une application capable de : L’objectif opérationnel est de construire progressivement une application capable de :
- détecter la création de tokens et de paires sur Solana, - détecter l’apparition de nouveaux tokens, pools, paires et listings sur Solana ;
- suivre leur évolution technique et de marché, - identifier la première source de mint ou de lancement, même lorsque le token migre ensuite vers un autre DEX ;
- collecter des informations utiles au filtrage, - décoder les transactions pertinentes des DEX et launch surfaces ciblés ;
- analyser des patterns statistiques ou comportementaux, - séparer les swaps/candles des événements utiles seulement à l’analyse : liquidité, cycle de vie de pool, fees, rewards, administration, wallets observés ;
- préparer ensuite la gestion de wallets et les opérations de swap/trading. - produire des métriques exploitables : prix, volume, candles/OHLCV, activité, bursts, déséquilibres buy/sell, signaux analytiques ;
- préparer ensuite des règles déterministes de filtrage, d’achat, de vente, de stop-loss et de trailing stop ;
- conserver une traçabilité locale suffisante pour rejouer, diagnostiquer et améliorer les décodeurs.
Les cibles observées incluent notamment les DEX et protocoles tels que : Le but court terme n’est pas encore le live trading. Le but court terme est de fiabiliser le décodage multi-DEX et la matérialisation des objets métier nécessaires au trading.
- Pump.fun ## 2. Workspace
- PumpSwap
- Raydium
- Meteora
- Bags
- FluxBeam
- LaunchLab / LaunchBeam
- Heaven
- DexLab
- Moonit
- Zora
La liste exacte pourra évoluer au fil du projet. Le workspace contient deux crates principales.
## 2. Architecture générale | Crate | Rôle |
|---|---|
| `kb_lib` | Bibliothèque métier : configuration, tracing, clients réseau, pool HTTP, manager WS, résolution transactionnelle, décodage DEX, détection métier, persistance SQLite, backfill, metadata, candles, signaux analytiques. |
| `kb_demo_app` | Application Tauri V2 de démonstration et d’inspection : fenêtres `Demo Ws`, `Demo Ws Manager`, `Demo Http`, `Demo Pipeline`, `Demo Pipeline 2`, graphiques candles et commandes de backfill/replay. |
Le workspace Rust est organisé autour de deux sous-crates principales : La logique métier doit rester dans `kb_lib`. `kb_demo_app` doit rester une façade UI/Tauri et ne doit pas récupérer de logique Solana ou DEX profonde.
- `kb_lib` ## 3. État actuel autour de `0.7.27`
- `kb_demo_app`
### `kb_lib` ### 3.1. Socle stabilisé à ne pas refactorer maintenant
`kb_lib` porte la logique métier et technique : Ces éléments fonctionnent et ne sont pas bloquants pour les DEX. Ils ne doivent pas être remaniés dans la phase immédiate :
- configuration JSON, - `ws_client.rs` ;
- tracing, - `ws_manager.rs` ;
- constantes Solana, - `http_client.rs` ;
- clients réseau HTTP / WebSocket, - `http_pool.rs` ;
- gestion future de gRPC Yellowstone, - couches JSON-RPC WS/HTTP déjà stabilisées ;
- persistance, - orchestration réseau utilisée par les fenêtres de démonstration.
- analyse de patterns,
- gestion des wallets, Ils pourront être améliorés plus tard, mais la priorité actuelle est le décodage DEX, les événements métier et les tables d’analyse.
- logique de détection et de filtrage.
### 3.2. Pipeline métier existant
Le pipeline `0.7.x` couvre déjà les étapes suivantes :
### `kb_demo_app` 1. réception d’observations via RPC WS ou backfill HTTP ;
2. résolution des transactions via HTTP RPC ;
3. projection transactionnelle normalisée en base ;
4. décodage DEX dans `k_sol_dex_decoded_events` ;
5. détection métier vers tokens, pools, paires, listings, origins et wallets observés ;
6. matérialisation des trades exploitables ;
7. agrégation pair metrics ;
8. génération candles/OHLCV ;
9. signaux analytiques simples ;
10. inspection via l’application de démonstration.
### 3.3. Connecteurs validés manuellement via l’application de démo
`kb_demo_app` est l’application Demo Tauri V2 avec frontend TypeScript. Les connecteurs suivants ont déjà été testés via l’application de démonstration et doivent être verrouillés par corpus/replay avant d’ajouter de nouveaux DEX :
Son rôle est de : - `pump_fun` ;
- `pump_swap` ;
- `raydium_cpmm` ;
- `raydium_clmm`.
- afficher l’interface utilisateur, ### 3.4. Connecteurs déjà présents mais à consolider par corpus
- exposer les commandes Tauri,
- déléguer la logique à `kb_lib`, Les modules suivants existent ou sont partiellement représentés dans le code, mais doivent être consolidés par corpus local, invariants et documentation :
- afficher les états, logs et événements remontés par la bibliothèque.
Le crate expose une bibliothèque interne `kb_demo_app_lib` afin de limiter le couplage avec `main.rs` et de préparer une évolution mobile ultérieure. - `meteora_dbc` ;
- `meteora_damm_v1` ;
- `meteora_damm_v2` ;
- `orca_whirlpools` ;
- `fluxbeam` ;
- `dexlab` ;
- `raydium_amm_v4` legacy ;
- launch origins déjà amorcées : `meteora_fun_launch`, `bags`, `moonit`.
## 3. Contraintes techniques ## 4. Matrice DEX et launch surfaces
Le projet suit des contraintes strictes. La distinction importante est la suivante :
### Contraintes de style et de structure - un **DEX effectif** permet de détecter et décoder des swaps, pools, liquidité et candles ;
- une **launch surface** peut être la première source de mint ou de lancement, même si le token migre ensuite vers Raydium, Meteora ou un autre AMM ;
- pour le trading, la première source de mint est une information de filtrage et de ciblage aussi importante que le DEX final.
- Rust 2024. ### 4.1. Matrice de travail
- Pas de `mod.rs`.
- Les fichiers Rust commencent par une entête `// file: ...`. | Code cible | Type | Statut actuel | Prochaine action |
- Les fichiers `lib.rs` et `main.rs` activent `#![deny(unreachable_pub)]` et `#![warn(missing_docs)]`. |---|---:|---|---|
- Les éléments publics sont documentés. | `pump_fun` | Launch + bonding curve | testé via démo | verrouiller corpus, invariants et documentation |
- Les expositions publiques passent par la racine des crates. | `pump_swap` | AMM / swap | testé via démo | verrouiller corpus, invariants et candles |
| `raydium_cpmm` | AMM | testé via démo | verrouiller corpus, swaps et candles |
### Contraintes de code | `raydium_clmm` | CLMM | testé via démo | verrouiller corpus, swaps et candles |
| `raydium_launchlab` / `raydium_launchpad` | Launch surface + migration | manquant | ajouter comme origine de mint/lancement et migration vers Raydium |
- pas de `anyhow`, | `raydium_amm_v4` | AMM legacy | présent, à isoler | traiter après les autres Raydium avec corpus dédié |
- pas de `thiserror`, | `meteora_dbc` | Launch / bonding curve | présent, à consolider | corpus, lifecycle, migration et swaps exploitables |
- pas de `?` dans le code applicatif, | `meteora_damm_v1` | AMM legacy | présent, à consolider | corpus et séparation swaps/liquidité/events |
- pas de `unwrap` ni `expect` dans le code applicatif, | `meteora_damm_v2` | AMM | présent, à consolider | corpus et séparation swaps/liquidité/events |
- utilisation préférée de `match`, `if let Err`, `let Err = ... else`. | `meteora_dlmm` | DLMM | manquant | ajouter à la matrice, puis corpus avant décodage |
| `orca_whirlpools` | CLMM | présent, à consolider | corpus fiable et validation des instructions utiles |
Les tests unitaires peuvent utiliser `?`, `unwrap` et `expect` si nécessaire. | `fluxbeam` | DEX | présent, à consolider | corpus fiable avant validation |
| `dexlab` | DEX | présent, à consolider | corpus fiable avant validation |
### Contraintes d’import | `bags` | Launch surface / attribution | amorcé | conserver comme origine de lancement, relier à Meteora si prouvé |
| `letsbonk` / `bonk_fun` | Launch surface | manquant | ajouter comme origine LaunchLab/Raydium, pas comme AMM autonome tant que non prouvé |
- pas de `use` sur les types/fonctions des crates externes, | `boop_fun` | Launch surface | manquant | ajouter comme origine de mint/lancement et migration |
- seuls les imports de traits sont tolérés, | `moonshot` / `moonit` | Launch surface | amorcé partiellement | remplacer les heuristiques faibles par corpus et règles prouvées |
- les appels doivent utiliser les chemins complets, par exemple `std::string::String` ou `tokio::sync::mpsc::Sender`. | `believe` | Launch surface | manquant | ajouter comme origine associée à Meteora DBC si les comptes l’attestent |
| `heaven` | Launch + AMM candidat | manquant | ajouter corpus et déterminer séparation launch/swap |
## 4. Configuration | `zora_solana` | À vérifier | écarté maintenant | ne pas intégrer avant preuve de programme Solana pertinent |
L’application utilisera un fichier `config.json`. ## 5. Base de données
La configuration devra à terme permettre de définir : SQLite reste le stockage local initial.
- les endpoints HTTP, Organisation actuelle à conserver :
- les endpoints WebSocket,
- leur nom logique, - `kb_lib/src/db/schema.rs` crée les tables et index ;
- les rôles affectés à chaque endpoint, - chaque table/index est créée dans une fonction dédiée ;
- les limitations de débit, - les requêtes sont sous `kb_lib/src/db/queries/` ;
- les délais et timeouts, - les entités persistées sont sous `kb_lib/src/db/entities/` ;
- les répertoires locaux, - les DTO applicatifs sont sous `kb_lib/src/db/dtos/`.
- la base SQLite,
- le futur répertoire des wallets Solana, `schema.rs` n’est donc pas un fichier métier à splitter immédiatement. Il reste acceptable tant qu’il garde uniquement la responsabilité de création de schéma.
- les paramètres de tracing,
- la politique de reconnexion. ### 5.1. Tables existantes importantes
Chaque endpoint devra pouvoir être identifié et affecté à une tâche spécifique, par exemple : Le modèle actuel contient déjà notamment :
- réception des notifications de slots, - transactions et instructions Solana normalisées ;
- réception des program subscriptions, - DEX connus ;
- réception des logs, - événements DEX décodés ;
- exécution des requêtes HTTP, - tokens, pools, pool tokens, paires, listings ;
- endpoint de secours. - launch surfaces et attributions ;
- pool origins ;
Cela permet de répartir la charge ou d’adapter le provider selon son niveau de service, ses limitations ou l’usage d’une API key. - swaps et trade events ;
- liquidity events ;
## 5. Tracing et logs - wallets, participations, holdings ;
- candles ;
Le tracing sera centralisé dans `kb_lib`. - metrics et analytic signals ;
- diagnostics locaux.
Le système devra supporter :
### 5.2. Tables futures prioritaires
- sortie console,
- sortie fichier, Avant d’étendre trop agressivement les DEX, le modèle doit prévoir les cas non directement tradables :
- niveau configurable,
- format de message configurable, | Table cible | Rôle |
- format temporel configurable, |---|---|
- ANSI console activable/désactivable, | `k_sol_transaction_classifications` | classifier les transactions connues, inconnues, partielles, échouées, non-DEX, DEX-candidates, launch-candidates. |
- comportement spécifique pour les tests. | `k_sol_protocol_candidates` | conserver les programmes ou patterns suspects/récurrents qui ne correspondent pas encore à un DEX connu. |
| `k_sol_pool_lifecycle_events` | matérialiser initialize/create/migrate/open/close/status events. |
## 6. Clients réseau | `k_sol_fee_events` | conserver fees, creator fees, protocol fees, fund fees. |
| `k_sol_reward_events` | conserver reward params, init rewards, collect rewards. |
## 6.1. `WsClient` | `k_sol_pool_admin_events` | conserver changements de config, authority, pause/resume, paramètres de pool. |
`kb_lib` devra contenir un `WsClient` asynchrone basé sur `tokio-tungstenite`. `k_sol_liquidity_events` existe déjà et doit être stabilisée/étendue plutôt que recréée sans nécessité.
Exigences initiales : ## 6. Politique de refactor actuelle
- client duplicable, Le code et la documentation sont vivants. Les refactors agressifs sont acceptables lorsque cela rend le pipeline plus propre et plus durable, à condition de respecter ces limites :
- connexion à plusieurs serveurs WS RPC,
- identifiants de requêtes incrémentaux par instance, - ne pas casser les fonctionnalités déjà validées ;
- flux de lecture et flux d’écriture séparés, - ne pas toucher pour le moment aux clients et managers réseau stabilisés ;
- séparation des réponses RPC et des notifications, - faire des étapes courtes, testables et rejouables ;
- registre `subscribe` / `unsubscribe`, - conserver les invariants de replay local ;
- tentative d’unsubscribe avant fermeture, - ne pas transformer un événement non price-action en trade/candle ;
- timeout pour ne pas bloquer le disconnect. - documenter les nouveaux types publics avec une rustdoc utile mais pas surchargée ;
- laisser `local_pipeline_diagnostics` servir d’outil temporaire de validation tant que les DEX ne sont pas stabilisés.
Le client ne devra pas s’appuyer sur `solana-pubsub-client`, même si son comportement fonctionnel peut s’en inspirer.
Les fichiers à surveiller en priorité sont :
## 6.2. `HttpClient`
| Fichier | Action recommandée |
`kb_lib` devra contenir un `HttpClient` asynchrone basé sur `reqwest`. |---|---|
| `kb_lib/src/dex_decode.rs` | extraire classification, catégories d’événements et enrichissement commun. |
Exigences initiales : | `kb_lib/src/dex_detect.rs` | extraire helpers communs pool/pair/listing/origin/wallets et isoler les handlers par famille. |
| `kb_lib/src/trade_aggregation.rs` | isoler extraction de montants, normalisation trade et pricing. |
- client duplicable, | `kb_lib/src/dex/*.rs` | homogénéiser les contrats de décodeurs sans forcer un gros trait prématuré. |
- connexion à plusieurs endpoints HTTP RPC,
- limites de requêtes configurables, ## 7. Contraintes de code
- profils par endpoint,
- adaptation à des providers publics ou privés. Contraintes maintenues :
Le client ne devra pas s’appuyer sur le `RpcClient` officiel de `solana-client`. - Rust 2024 ;
- pas de `mod.rs` ;
## 6.3. `GrpcClient` - fichiers Rust avec entête `// file: ...` ;
- fichiers `.toml` avec entête `# file: ...` ;
Le support `GrpcClient` basé sur `yellowstone-grpc-client` et `yellowstone-grpc-proto` est prévu dans une phase ultérieure. - exposition centralisée via `lib.rs` ;
- `#![deny(unreachable_pub)]` et `#![warn(missing_docs)]` dans les racines concernées ;
## 7. Types et RPC - pas de `anyhow` ;
- pas de `thiserror` ;
Le projet cherchera à réutiliser autant que possible les types officiels fournis par l’écosystème Solana, notamment pour les payloads et surtout pour le parsing des réponses. - pas de `?`, `unwrap`, `expect` dans le code applicatif ;
- usage privilégié de `match`, `if let Err`, `let Err = ... else` ;
Objectif : - imports externes limités, sauf traits lorsque nécessaire ;
- tests unitaires et tests de replay maintenus.
- limiter l’invention de structures approximatives,
- réutiliser les types des crates officielles lorsque cela est pertinent, Les tests peuvent rester plus souples lorsque cela clarifie le test.
- encapsuler si besoin ces types dans une couche propre au projet.
## 8. Priorité immédiate
Le projet pourra embarquer son propre générateur de requêtes JSON-RPC 2.0 afin d’encapsuler proprement les appels HTTP et WS.
La reprise doit suivre cet ordre :
## 8. Base de données
1. finir/verrouiller `0.7.27` sur `pump_fun`, `pump_swap`, `raydium_cpmm`, `raydium_clmm` ;
Le stockage initial se fera dans SQLite. 2. démarrer `0.7.28` par un refactor commun DEX sans toucher au transport ;
3. ajouter la matrice DEX documentée et les corpus de validation ;
Cette première étape doit permettre : 4. ajouter les tables de classification des transactions inconnues et protocol candidates ;
5. matérialiser les événements non-trade : lifecycle, liquidité, fees, rewards, admin ;
- de conserver l’historique observé, 6. consolider Meteora, y compris `meteora_dlmm` dans la matrice ;
- de stocker des événements et états techniques, 7. ajouter les launch surfaces manquantes comme origines de mint : LaunchLab/Launchpad, LetsBonk/Bonk.fun, Boop.fun, Moonshot/Moonit, Believe ;
- de préparer l’analyse. 8. traiter Heaven ;
9. consolider Orca/FluxBeam/DexLab ;
Une migration vers PostgreSQL pourra être envisagée plus tard lorsque l’application aura stabilisé ses besoins. 10. isoler Raydium AMM v4 legacy ;
11. effectuer une validation DEX v1 consolidée ;
## 9. Frontend 12. reprendre ensuite l’UI analytique et les vues token/pair/pool.
L’application Tauri démarrera avec une interface volontairement simple. ## 9. Fichiers utiles pour reprendre dans une nouvelle session
### UI minimale prévue Pour reprendre rapidement le codage dans une nouvelle session, fournir au minimum :
- un bouton ou toggle de connexion, - `README.md` ;
- un bouton d’arrêt si nécessaire, - `ROADMAP.md` ;
- une zone de texte scrollable et en lecture seule, - `CHANGELOG.md` ;
- affichage des messages reçus depuis `kb_lib`. - `Cargo.toml` racine ;
- `kb_lib/Cargo.toml` ;
Cette première UI servira surtout à valider : - `kb_lib/src/lib.rs` ;
- `kb_lib/src/constants.rs` ;
- l’initialisation, - `kb_lib/src/dex.rs` ;
- le tracing, - `kb_lib/src/dex/*.rs` ;
- la délégation vers `kb_lib`, - `kb_lib/src/dex_decode.rs` ;
- la remontée d’événements depuis les clients réseau. - `kb_lib/src/dex_detect.rs` ;
- `kb_lib/src/trade_aggregation.rs` ;
## 10. Constantes Solana - `kb_lib/src/pair_candle_aggregation.rs` ;
- `kb_lib/src/local_pipeline_replay.rs` ;
Le projet devra réutiliser autant que possible les Program IDs et identifiants officiels fournis par les crates officielles au lieu de les recopier à la main. - `kb_lib/src/local_pipeline_validation.rs` ;
- `kb_lib/src/local_pipeline_diagnostics.rs` ;
Exemples : - `kb_lib/src/db/schema.rs` ;
- `kb_lib/src/db.rs` ;
- SPL Token - `kb_lib/src/db/entities.rs` et `kb_lib/src/db/entities/*` ;
- SPL Token-2022 - `kb_lib/src/db/dtos.rs` et `kb_lib/src/db/dtos/*` ;
- Associated Token Account - `kb_lib/src/db/queries.rs` et `kb_lib/src/db/queries/*`.
- Wrapped SOL mint
- System Program Ajouter `kb_demo_app/src/demo_pipeline*.rs` seulement si la tâche concerne l’UI ou les diagnostics affichés.
- Compute Budget Program
## 10. Prompt court de reprise
## 11. Génération de bindings TypeScript
```text
Les structures partagées entre Rust et le frontend devront être générées avec `ts-rs`. Je reprends le workspace Rust khadhroony-bobobot autour de la version 0.7.27.
Objectif actuel : finaliser le pipeline DEX Solana avant trading.
Le flux prévu est : Ne pas toucher pour le moment à ws_client/ws_manager/http_client/http_pool : ils fonctionnent et sont non bloquants.
Priorité : refactor DEX commun à partir de 0.7.28, matrice DEX, transactions inconnues/protocol candidates, événements non-trade, puis ajout/consolidation des DEX et launch surfaces.
```bash Respecter les contraintes : Rust 2024, pas de mod.rs, pas de anyhow/thiserror, pas de ?/unwrap/expect dans le code applicatif, rustdoc utile sur l’API publique.
cargo test export_bindings Les connecteurs à verrouiller avant extension sont pump_fun, pump_swap, raydium_cpmm et raydium_clmm.
Les launch surfaces sont importantes comme première source de mint, même si le token migre ensuite vers Raydium/Meteora/autre.
``` ```
Exemple déjà présent :
- `kb_demo_app/src/splash.rs`
- `kb_demo_app/frontend/ts/bindings/SplashOrder.ts`
## 12. État du projet
voir `CHANGELOG.md`
## 13. Feuille de route
La feuille de route détaillée est disponible dans :
- `ROADMAP.md`
## 14. Licence
Licence MIT.

342
ROADMAP.md
View File

@@ -749,9 +749,7 @@ Réalisé :
- seuls les trade candidates issus de transactions échouées restent ignorés. - seuls les trade candidates issus de transactions échouées restent ignorés.
### 6.059. Version `0.7.27` — Validation multi-DEX des connecteurs déjà branchés ### 6.059. Version `0.7.27` — Validation multi-DEX des connecteurs déjà branchés
Objectif : verrouiller la non-régression du pipeline actuel avant d’ajouter de nouveaux DEX ou d’ouvrir la phase d’analyse `0.8.x`. Réalisé :
À faire :
- rejouer des bases neuves de test pour `pump_fun`, `pump_swap`, `raydium_cpmm` et `raydium_clmm`, - rejouer des bases neuves de test pour `pump_fun`, `pump_swap`, `raydium_cpmm` et `raydium_clmm`,
- ne pas ajouter de nouveau DEX dans cette version ; cette version sert uniquement à valider les connecteurs déjà branchés, - ne pas ajouter de nouveau DEX dans cette version ; cette version sert uniquement à valider les connecteurs déjà branchés,
@@ -765,60 +763,147 @@ Objectif : verrouiller la non-régression du pipeline actuel avant d’ajouter d
- conserver la tolérance aux événements DEX partiels tout en refusant les trades sans montant ou prix exploitable, - conserver la tolérance aux événements DEX partiels tout en refusant les trades sans montant ou prix exploitable,
- valider que les transactions échouées restent traçables dans les événements décodés sans produire de `k_sol_trade_events`. - valider que les transactions échouées restent traçables dans les événements décodés sans produire de `k_sol_trade_events`.
### 6.060. Version `0.7.28` — Matérialisation des événements liquidité et cycle de vie pool ### 6.060. Version `0.7.28` — Refactor DEX commun et préparation extension
Objectif : exploiter les événements non buy/sell utiles à l’analyse et au trading semi-automatique sans les mélanger avec les trades/candles. Réalisé :
- ne pas toucher à `ws_client.rs`, `ws_manager.rs`, `http_client.rs`, `http_pool.rs` ni aux couches JSON-RPC déjà stabilisées,
- extraire depuis `dex_decode.rs` les catégories communes d’événements : trade, candle candidate, liquidity candidate, fee candidate, reward candidate, admin candidate, pool lifecycle candidate,
- créer une représentation interne documentée pour les familles d’événements DEX afin d’éviter les chaînes dispersées dans plusieurs fichiers,
- clarifier la différence entre événement décodé, événement actionnable, trade candidate, candle candidate et événement conservé seulement pour analyse,
- simplifier `dex_decode.rs` en gardant son rôle de service de persistance-orchestration,
- simplifier `dex_detect.rs` en extrayant les helpers communs pool/pair/listing/origin/wallet quand cela réduit la duplication,
- homogénéiser les contrats des modules `kb_lib/src/dex/*.rs` sans imposer trop tôt un trait générique lourd,
- vérifier la rustdoc publique : utile, courte, orientée responsabilité ; supprimer la documentation redondante ou trop chargée,
- conserver les tests verts et ajouter des tests de non-régression sur les catégories d’événements existantes.
Contraintes :
- refactor agressif autorisé si le résultat est plus propre,
- chaque changement doit rester rejouable et testable,
- aucun changement de comportement métier volontaire dans cette version,
- aucun événement non price-action ne doit devenir un trade ou une candle.
### 6.061. Version `0.7.29` — Matrice DEX, corpus et documentation de support
Objectif : rendre explicite ce qui est validé, présent, partiel, manquant ou volontairement ignoré.
À faire : À faire :
- stabiliser ou ajouter la table `k_sol_liquidity_events`, - ajouter et maintenir une matrice DEX dans `README.md` et `ROADMAP.md`,
- stabiliser ou ajouter la table `k_sol_pool_lifecycle_events`, - distinguer clairement `DEX effectif`, `launch surface`, `pool origin`, `launch origin` et `migration target`,
- matérialiser les événements de type `increase_liquidity`, `decrease_liquidity`, `add_liquidity`, `remove_liquidity`, `open_position`, `close_position`, `initialize`, `create_pool`, `migrate` et assimilés, - documenter que les launch surfaces sont importantes comme première source de mint/lancement même lorsqu’un token migre ensuite vers un autre DEX,
- rattacher chaque événement métier à `dex_id`, `pool_id`, `pair_id`, `transaction_id`, `decoded_event_id`, `signature` et `slot`, - constituer une liste de corpus par DEX/surface : signatures, pools, token mints, résultat attendu,
- conserver le `payload_json` source pour audit et extension future, - indiquer pour chaque protocole : statut, source de preuve, type d’événements couverts, tables alimentées, limites connues,
- alimenter les diagnostics locaux avec les compteurs liquidité et cycle de vie, - retirer `zora_solana` du phasage actif tant qu’aucun programme Solana pertinent n’est prouvé,
- garantir qu’un événement de liquidité ou de cycle de vie ne produit jamais de candle directement, - ajouter `meteora_dlmm` à la matrice comme variante Meteora manquante à couvrir plus tard,
- préparer les signaux futurs liés à la liquidité initiale, à l’ajout/retrait brutal de liquidité, aux migrations et aux changements de statut de pool. - préparer les diagnostics SQL de référence par protocole et par table métier.
### 6.061. Version `0.7.29` — Matérialisation des événements fees, rewards et administration Matrice cible initiale :
Objectif : conserver les événements non price-action utiles au risque, à l’analyse économique et à la traçabilité opérationnelle des pools.
| Code cible | Type | Statut | Objectif immédiat |
|---|---:|---|---|
| `pump_fun` | launch + bonding curve | testé via démo | verrouiller corpus et invariants |
| `pump_swap` | AMM / swap | testé via démo | verrouiller trades/candles |
| `raydium_cpmm` | AMM | testé via démo | verrouiller trades/candles |
| `raydium_clmm` | CLMM | testé via démo | verrouiller trades/candles |
| `raydium_launchlab` / `raydium_launchpad` | launch surface | manquant | détecter mint, launch, migration |
| `raydium_amm_v4` | AMM legacy | présent, à isoler | corpus dédié après autres DEX |
| `meteora_dbc` | launch / bonding curve | présent, à consolider | lifecycle, migration, swaps utiles |
| `meteora_damm_v1` | AMM legacy | présent, à consolider | corpus et séparation events |
| `meteora_damm_v2` | AMM | présent, à consolider | corpus et séparation events |
| `meteora_dlmm` | DLMM | manquant | ajouter corpus avant décodeur |
| `orca_whirlpools` | CLMM | présent, à consolider | validation par corpus |
| `fluxbeam` | DEX | présent, à consolider | validation par corpus |
| `dexlab` | DEX | présent, à consolider | validation par corpus |
| `bags` | launch surface | amorcé | attribution fiable, migration si prouvée |
| `letsbonk` / `bonk_fun` | launch surface | manquant | origine LaunchLab/Raydium |
| `boop_fun` | launch surface | manquant | origine mint/lancement/migration |
| `moonshot` / `moonit` | launch surface | amorcé partiellement | corpus, éviter heuristique faible |
| `believe` | launch surface | manquant | origine Meteora DBC si comptes probants |
| `heaven` | launch + AMM candidat | manquant | corpus et séparation launch/swap |
### 6.062. Version `0.7.30` — Transactions inconnues et protocol candidates
Objectif : ne plus perdre les transactions utiles qui ne correspondent pas encore à un DEX connu.
À faire : À faire :
- ajouter ou stabiliser `k_sol_fee_events`, - ajouter `k_sol_transaction_classifications`,
- ajouter ou stabiliser `k_sol_reward_events`, - ajouter `k_sol_protocol_candidates`,
- ajouter ou stabiliser `k_sol_pool_admin_events`, - classifier les transactions résolues en catégories : known dex, known launch surface, unknown program, non-dex, failed transaction, partial decode, ignored technical transaction,
- conserver les `program_id`, comptes, signatures, préfixes de `data`, logs et indices d’instructions utiles à l’analyse,
- créer des requêtes de diagnostic pour repérer les programmes inconnus fréquents,
- permettre de promouvoir plus tard un protocol candidate vers un vrai DEX/surface sans perdre l’historique,
- garantir que ces tables n’alimentent jamais directement les trades/candles.
### 6.063. Version `0.7.31` — Événements non-trade v1 : liquidité et cycle de vie pool
Objectif : exploiter les événements utiles à l’analyse et au trading semi-automatique sans les mélanger avec les swaps/candles.
À faire :
- stabiliser et étendre `k_sol_liquidity_events` au lieu de la recréer inutilement,
- ajouter `k_sol_pool_lifecycle_events`,
- matérialiser les événements `initialize`, `create_pool`, `migrate`, `open_position`, `close_position`, `increase_liquidity`, `decrease_liquidity`, `add_liquidity`, `remove_liquidity` et assimilés,
- rattacher chaque événement à `dex_id`, `pool_id`, `pair_id`, `transaction_id`, `decoded_event_id`, `signature` et `slot` lorsque les informations existent,
- conserver le `payload_json` source pour audit,
- alimenter les diagnostics locaux avec les compteurs liquidité/lifecycle,
- garantir qu’un événement de liquidité ou de cycle de vie ne produit jamais de candle directement.
### 6.064. Version `0.7.32` — Événements non-trade v2 : fees, rewards et administration
Objectif : conserver les événements utiles au risque, au scoring, à l’économie du pool et à la traçabilité opérationnelle.
À faire :
- ajouter `k_sol_fee_events`,
- ajouter `k_sol_reward_events`,
- ajouter `k_sol_pool_admin_events`,
- matérialiser les événements `collect_protocol_fee`, `collect_fund_fee`, `collect_creator_fee`, `collect_fee` et assimilés, - matérialiser les événements `collect_protocol_fee`, `collect_fund_fee`, `collect_creator_fee`, `collect_fee` et assimilés,
- matérialiser les événements `set_reward_params`, `initialize_reward`, `collect_reward`, `update_reward_infos` et assimilés, - matérialiser les événements `set_reward_params`, `initialize_reward`, `collect_reward`, `update_reward_infos` et assimilés,
- matérialiser les événements `set_config`, `update_config`, `set_authority`, `set_fee_rate`, `pause`, `resume` et assimilés, - matérialiser les événements `set_config`, `update_config`, `set_authority`, `set_fee_rate`, `pause`, `resume` et assimilés,
- rattacher chaque événement à la transaction, au decoded event, au pool, à la paire et aux wallets observés lorsque les comptes sont disponibles, - rattacher ces événements aux transactions, decoded events, pools, paires et wallets observés lorsque les comptes le permettent,
- documenter clairement que ces événements sont utiles à l’analyse et au scoring mais ne sont ni des trades ni des candles. - documenter clairement que ces événements ne sont ni des trades ni des candles.
### 6.062. Version `0.7.30` — Meteora : DBC / DAMM v1 / DAMM v2 ### 6.065. Version `0.7.33` — Meteora : DBC / DAMM v1 / DAMM v2 / DLMM
Objectif : ajouter ou stabiliser les connecteurs Meteora avec une séparation claire entre swaps, liquidité, fees, rewards et événements pool. Objectif : consolider Meteora comme famille multi-programmes au lieu de traiter chaque variante comme un cas isolé incomplet.
À faire : À faire :
- vérifier les programmes et discriminants réellement utilisés pour `Meteora DBC`, `Meteora DAMM v1` et `Meteora DAMM v2`, - vérifier les programmes et discriminants réellement utilisés pour `Meteora DBC`, `Meteora DAMM v1`, `Meteora DAMM v2` et `Meteora DLMM`,
- constituer un petit corpus local de pools et signatures fiables pour chaque variante, - ajouter `meteora_dlmm` à la couverture cible seulement après corpus fiable,
- décoder les créations de pool, swaps et événements de liquidité exploitables, - constituer un corpus local par variante,
- alimenter `k_sol_dex_decoded_events`, les tables métier pool/pair/listing et les nouvelles tables d’événements non-trade, - décoder les créations de pool, swaps, liquidités et événements lifecycle exploitables,
- vérifier l’idempotence du replay local sur un corpus mixte Meteora, - identifier les cas où `DBC` sert de launch origin avant migration vers un AMM,
- documenter les limites connues des variantes insuffisamment couvertes par le corpus. - alimenter `k_sol_dex_decoded_events`, les tables pool/pair/listing, les origins et les tables non-trade,
- vérifier l’idempotence du replay local sur un corpus Meteora mixte,
- documenter les limites connues des variantes insuffisamment couvertes.
### 6.063. Version `0.7.31` — Launch DEX : LaunchLab / Fun Launch / Bags / Moonit ### 6.066. Version `0.7.34` — Launch surfaces : LaunchLab, LetsBonk, Bags, Moonshot/Moonit, Boop.fun, Believe
Objectif : couvrir les surfaces de lancement et de migration de tokens sans confondre origine de lancement et protocole DEX final. Objectif : détecter la première source de mint/lancement des tokens même lorsque le swap final se fait ailleurs.
À faire : À faire :
- ajouter ou stabiliser les mappings `LaunchLab`, `Fun Launch`, `Bags` et `Moonit`, - ajouter ou stabiliser `raydium_launchlab` / `raydium_launchpad`,
- distinguer clairement launch origin, pool origin et DEX effectif, - ajouter `letsbonk` / `bonk_fun` comme surface d’origine rattachée à LaunchLab/Raydium si le corpus le prouve,
- détecter les créations de token, pools initiaux, migrations et listings dérivés, - ajouter `boop_fun` comme surface d’origine et suivre ses migrations,
- consolider `moonshot` / `moonit` avec corpus au lieu de simples heuristiques faibles,
- consolider `bags` comme surface d’origine, notamment lorsque le token passe par Meteora DBC/DAMM,
- ajouter `believe` comme surface d’origine associée à Meteora DBC si les comptes/authorities le prouvent,
- distinguer `launch_origin`, `pool_origin`, `dex_effective` et `migration_target`,
- rattacher les launch origins aux pools et paires lorsque les comptes permettent un matching fiable, - rattacher les launch origins aux pools et paires lorsque les comptes permettent un matching fiable,
- enrichir les diagnostics pour exposer les objets détectés par surface de lancement, - exposer les origins dans les diagnostics et l’UI d’inspection.
- éviter les heuristiques trop larges lorsqu’un suffixe de mint ou un label externe ne suffit pas à prouver l’origine.
### 6.064. Version `0.7.32` — Orca / FluxBeam / DexLab : corpus et validation ciblée ### 6.067. Version `0.7.35` — Heaven : corpus, launch et AMM
Objectif : ajouter les connecteurs restants à partir de corpus locaux vérifiables et garder les décodeurs heuristiques isolés tant qu’ils ne sont pas prouvés. Objectif : ajouter Heaven sans le classer trop tôt comme simple DEX ou simple launchpad.
À faire :
- vérifier le ou les programmes Heaven réellement observés sur mainnet,
- constituer un corpus local de mints, pools et swaps Heaven,
- séparer les événements de lancement des événements de swap,
- ajouter les decoded events, launch origins, pool/pair/listing et trade events seulement lorsque les instructions sont prouvées,
- documenter les limites si le corpus ne permet pas encore de matérialiser tous les événements,
- vérifier que Heaven ne crée pas de candles invalides en cas d’événement de launch non pricé.
### 6.068. Version `0.7.36` — Orca / FluxBeam / DexLab : corpus et validation ciblée
Objectif : consolider les connecteurs déjà présents à partir de corpus locaux vérifiables.
À faire : À faire :
@@ -829,32 +914,33 @@ Objectif : ajouter les connecteurs restants à partir de corpus locaux vérifiab
- marquer explicitement les variantes partiellement supportées ou heuristiques, - marquer explicitement les variantes partiellement supportées ou heuristiques,
- rejouer les corpus plusieurs fois pour vérifier l’idempotence et l’absence de trades/candles invalides. - rejouer les corpus plusieurs fois pour vérifier l’idempotence et l’absence de trades/candles invalides.
### 6.065. Version `0.7.33` — Validation DEX v1 consolidée ### 6.069. Version `0.7.37` — Raydium AMM v4 legacy : corpus et validation ciblée
Objectif : rejouer tous les DEX supportés et valider les invariants du pipeline complet avant de traiter Raydium AMM v4 legacy séparément. Objectif : traiter le vrai Raydium AMM v4 historique après les autres Raydium, afin de l’isoler de `raydium_cpmm`, `raydium_clmm` et des labels Raydium génériques.
À faire :
- rejouer des bases neuves couvrant tous les connecteurs DEX supportés hors `raydium_amm_v4` legacy,
- vérifier les compteurs globaux et par DEX : decoded events, trade events, liquidity events, lifecycle events, fee events, reward events, admin events, candles et analytic signals,
- contrôler que chaque famille d’événements alimente uniquement les tables métier prévues,
- vérifier les diagnostics bloquants et les samples d’anomalie,
- documenter les corpus utilisés pour chaque DEX,
- conserver une matrice de support par DEX, variante, instruction et type d’événement.
### 6.066. Version `0.7.34` — Raydium AMM v4 legacy : corpus et validation ciblée
Objectif : traiter le vrai Raydium AMM v4 historique après les autres DEX, afin de l’isoler correctement de `raydium_cpmm`, `raydium_clmm` et des labels Raydium génériques.
À faire : À faire :
- rechercher des pools réellement rattachés au programme `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8`, - rechercher des pools réellement rattachés au programme `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8`,
- constituer un petit corpus local de signatures/pools AMM v4 fiables pour les tests, - constituer un petit corpus local de signatures/pools AMM v4 fiables,
- vérifier que les adresses issues de Dexscreener ou d’autres explorateurs ne sont pas seulement catégorisées globalement comme `Raydium`, - vérifier que les adresses issues d’explorateurs ne sont pas seulement catégorisées globalement comme `Raydium`,
- ajouter des requêtes de diagnostic par `program_id`, `accounts_json` et préfixe `data_json`, - ajouter des requêtes de diagnostic par `program_id`, `accounts_json` et préfixe `data_json`,
- valider la prise en charge `initialize2` et identifier les instructions de swap AMM v4 à supporter si elles apparaissent dans les transactions testées, - valider `initialize2` et identifier les instructions de swap AMM v4 à supporter si elles apparaissent dans le corpus,
- renommer et stabiliser les fonctions internes autour de `raydium_amm_v4` afin d’éviter l’ambiguïté avec `raydium_cpmm` et `raydium_clmm`, - renommer/stabiliser les fonctions internes autour de `raydium_amm_v4` pour éviter l’ambiguïté avec `raydium_cpmm` et `raydium_clmm`,
- documenter les limites connues si le corpus AMM v4 reste trop faible. - documenter les limites connues si le corpus AMM v4 reste faible.
### 6.067. Version `0.7.35` — `kb_demo_app` : overlays analytiques ### 6.070. Version `0.7.38` — Validation DEX v1 consolidée
Objectif : rejouer tous les DEX et launch surfaces supportés et valider les invariants du pipeline complet.
À faire :
- rejouer des bases neuves couvrant tous les connecteurs DEX supportés,
- vérifier les compteurs globaux et par DEX : decoded events, trade events, liquidity events, lifecycle events, fee events, reward events, admin events, candles et analytic signals,
- contrôler que chaque famille d’événements alimente uniquement les tables métier prévues,
- vérifier les diagnostics bloquants et les samples d’anomalie,
- documenter les corpus utilisés pour chaque DEX/surface,
- conserver une matrice de support par DEX, variante, instruction et type d’événement,
- verrouiller les invariants avant d’ouvrir l’analyse `0.8.x`.
### 6.071. Version `0.7.39` — `kb_demo_app` : overlays analytiques
Objectif : rendre visibles les signaux analytiques directement sur les graphes et vues de marché. Objectif : rendre visibles les signaux analytiques directement sur les graphes et vues de marché.
À faire : À faire :
@@ -863,9 +949,9 @@ Objectif : rendre visibles les signaux analytiques directement sur les graphes e
- ajouter des marqueurs pour `first_trade_seen`, `trade_burst_60s`, `buy_sell_imbalance_60s`, `price_jump_up_60s`, `price_jump_down_60s` et `volume_spike_60s`, - ajouter des marqueurs pour `first_trade_seen`, `trade_burst_60s`, `buy_sell_imbalance_60s`, `price_jump_up_60s`, `price_jump_down_60s` et `volume_spike_60s`,
- permettre le filtrage par type de signal et par sévérité, - permettre le filtrage par type de signal et par sévérité,
- afficher un panneau latéral listant les signaux liés à une paire et à un timeframe, - afficher un panneau latéral listant les signaux liés à une paire et à un timeframe,
- préparer l’extension future vers des indicateurs Ichimoku, Kumo, projections ABCD et égalités temps/prix sans les mélanger au pipeline de décodage DEX. - préparer l’extension future vers Ichimoku, Kumo, projections ABCD et égalités temps/prix sans les mélanger au pipeline de décodage DEX.
### 6.068. Version `0.7.36` — `kb_demo_app` : vues consolidées token / pair / pool ### 6.072. Version `0.7.40` — `kb_demo_app` : vues consolidées token / pair / pool
Objectif : fournir une lecture métier plus confortable du modèle `0.7.x`. Objectif : fournir une lecture métier plus confortable du modèle `0.7.x`.
À faire : À faire :
@@ -873,11 +959,11 @@ Objectif : fournir une lecture métier plus confortable du modèle `0.7.x`.
- ajouter une fiche token avec mint, programme token, metadata, pools, paires et historique de découverte, - ajouter une fiche token avec mint, programme token, metadata, pools, paires et historique de découverte,
- ajouter une fiche paire avec base/quote, DEX, pool, métriques, candles, signaux et derniers trades, - ajouter une fiche paire avec base/quote, DEX, pool, métriques, candles, signaux et derniers trades,
- ajouter une fiche pool avec composition, vaults, origine, première signature vue, programme DEX et statut de décodage, - ajouter une fiche pool avec composition, vaults, origine, première signature vue, programme DEX et statut de décodage,
- relier dans l’UI les launch origins, pool origins, wallets observés, holdings observés, événements de liquidité, événements lifecycle, fees, rewards, admin, candles et analytic signals, - relier dans l’UI les launch origins, pool origins, wallets observés, holdings observés, événements de liquidité, lifecycle, fees, rewards, admin, candles et analytic signals,
- préparer une navigation transversale entre objets techniques et objets métier, - préparer une navigation transversale entre objets techniques et objets métier,
- rendre explicites les cas `tradeCount = null`, `lastPriceQuotePerBase = null`, tokens non enrichis et événements conservés uniquement pour analyse. - rendre explicites les cas `tradeCount = null`, `lastPriceQuotePerBase = null`, tokens non enrichis et événements conservés uniquement pour analyse.
### 6.069. Version `0.7.37` — Finition UI `0.7.x` ### 6.073. Version `0.7.41` — Finition UI `0.7.x`
Objectif : stabiliser la couche desktop de validation avant l’ouverture de `0.8.x`. Objectif : stabiliser la couche desktop de validation avant l’ouverture de `0.8.x`.
À faire : À faire :
@@ -886,41 +972,52 @@ Objectif : stabiliser la couche desktop de validation avant l’ouverture de `0.
- améliorer la navigation, les filtres et la pagination, - améliorer la navigation, les filtres et la pagination,
- ajouter les derniers raffinements de confort et de lisibilité, - ajouter les derniers raffinements de confort et de lisibilité,
- préparer une base UI suffisamment stable pour la future phase d’analyse et filtrage `0.8.x`, - préparer une base UI suffisamment stable pour la future phase d’analyse et filtrage `0.8.x`,
- vérifier que les commandes Tauri restent de simples façades vers `kb_lib` et ne récupèrent pas de logique métier. - vérifier que les commandes Tauri restent de simples façades vers `kb_lib`.
### 6.070. Version `0.7.x` — Couverture DEX v1 ### 6.074. Version `0.7.x` — Couverture DEX v1
Objectif : structurer les connecteurs DEX autour d’un pipeline complet de résolution, décodage et normalisation métier. Objectif : structurer les connecteurs DEX autour d’un pipeline complet de résolution, décodage, normalisation métier et classification des événements non-trade.
Protocoles cibles : Protocoles et surfaces cibles :
- Pump.fun - Pump.fun,
- PumpSwap - PumpSwap,
- Raydium CPMM - Raydium CPMM,
- Raydium CLMM - Raydium CLMM,
- Meteora DBC - Raydium LaunchLab / Launchpad,
- Meteora DAMM v2 - Raydium AMM v4 legacy,
- Meteora DAMM v1 - Meteora DBC,
- LaunchLab / Fun Launch - Meteora DAMM v1,
- Bags - Meteora DAMM v2,
- Moonit - Meteora DLMM,
- Orca - Orca Whirlpools,
- FluxBeam - FluxBeam,
- DexLab - DexLab,
- Raydium AMM v4 legacy - Bags,
- LetsBonk / Bonk.fun,
- Boop.fun,
- Moonshot / Moonit,
- Believe,
- Heaven.
Hors périmètre immédiat :
- `zora_solana`, tant qu’aucun programme Solana pertinent et exploitable n’est prouvé.
Résultat attendu : Résultat attendu :
- identification fiable des programmes et versions, - identification fiable des programmes et versions,
- résolution des signatures pertinentes, - résolution des signatures pertinentes,
- décodage des transactions utiles, - décodage des transactions utiles,
- création d’objets métier riches pour tokens, pools, paires, listings, participants et holdings observés, - conservation des transactions inconnues ou candidates sans perte d’information,
- création d’objets métier riches pour tokens, pools, paires, listings, participants, origins et holdings observés,
- distinction claire entre première source de mint, launch origin, pool origin, DEX effectif et migration target,
- enrichissement metadata des tokens découverts, - enrichissement metadata des tokens découverts,
- séparation claire entre événements candle/trade et événements utiles seulement à l’analyse, aux frais, à la liquidité, aux rewards, à l’administration ou au cycle de vie des pools, - séparation stricte entre événements candle/trade et événements utiles seulement à l’analyse,
- matérialisation progressive des événements non-trade dans des tables métier dédiées, - matérialisation progressive des événements non-trade dans des tables métier dédiées,
- préparation d’une détection temps réel hybride et d’un backfill ciblé compatible avec les mêmes objets métier, - préparation d’une détection temps réel hybride et d’un backfill ciblé compatible avec les mêmes objets métier,
- préparation d’agrégats DEX plus riches, de candles/OHLCV et d’une UI d’inspection du pipeline `0.7.x`. - préparation d’agrégats DEX plus riches, de candles/OHLCV et d’une UI d’inspection du pipeline `0.7.x`.
### 6.071. Version `0.8.x` — Analyse et filtrage ### 6.075. Version `0.8.x` — Analyse et filtrage
Objectif : transformer les événements bruts en signaux exploitables. Objectif : transformer les événements bruts en signaux exploitables.
À faire : À faire :
@@ -935,7 +1032,7 @@ Objectif : transformer les événements bruts en signaux exploitables.
- outils de sélection manuelle de points ABC et projection d’un point D selon des règles temps/prix explicites, - outils de sélection manuelle de points ABC et projection d’un point D selon des règles temps/prix explicites,
- séparation stricte entre signaux analytiques observés, projections hypothétiques et décisions de trading. - séparation stricte entre signaux analytiques observés, projections hypothétiques et décisions de trading.
### 6.072. Version `1.x.y` — Wallets et swap préparatoire ### 6.076. Version `1.x.y` — Wallets et swap préparatoire
Objectif : préparer la couche d’action. Objectif : préparer la couche d’action.
À faire : À faire :
@@ -946,7 +1043,7 @@ Objectif : préparer la couche d’action.
- préparation d’ordres et de swaps, - préparation d’ordres et de swaps,
- simulation et garde-fous. - simulation et garde-fous.
### 6.073. Version `2.x.y` — Trading semi-automatisé ### 6.077. Version `2.x.y` — Trading semi-automatisé
Objectif : brancher l’analyse à l’action tout en gardant des garde-fous explicites. Objectif : brancher l’analyse à l’action tout en gardant des garde-fous explicites.
À faire : À faire :
@@ -957,7 +1054,7 @@ Objectif : brancher l’analyse à l’action tout en gardant des garde-fous exp
- confirmations explicites ou semi-automatiques, - confirmations explicites ou semi-automatiques,
- journaux d’exécution. - journaux d’exécution.
### 6.074. Version `3.x.y` — Yellowstone gRPC ### 6.078. Version `3.x.y` — Yellowstone gRPC
Objectif : ajouter le connecteur gRPC dédié. Objectif : ajouter le connecteur gRPC dédié.
À faire : À faire :
@@ -970,41 +1067,59 @@ Objectif : ajouter le connecteur gRPC dédié.
## 7. Organisation des modules ciblés ## 7. Organisation des modules ciblés
### 7.1. `kb_lib` ### 7.1. `kb_lib`
Modules cibles à court terme : Modules stables à ne pas remanier dans la phase immédiate :
- `error.rs`
- `config.rs`
- `tracing.rs`
- `constants.rs`
- `types.rs`
- `ws_client.rs` - `ws_client.rs`
- `ws_manager.rs` - `ws_manager.rs`
- `http_client.rs` - `http_client.rs`
- `http_pool.rs` - `http_pool.rs`
- `json_rpc_ws.rs` - `json_rpc_ws.rs`
- `solana_pubsub_ws.rs` - `solana_pubsub_ws.rs`
- `detect.rs`
Modules ciblés par le refactor et la consolidation DEX :
- `dex.rs`
- `dex/*.rs`
- `dex_decode.rs` - `dex_decode.rs`
- `dex_detect.rs` - `dex_detect.rs`
- `trade_aggregation.rs` - `trade_aggregation.rs`
- `pair_candle_aggregation.rs` - `pair_candle_aggregation.rs`
- `pair_analytic_signal.rs` - `pair_analytic_signal.rs`
- `launch_origin.rs`
- `pool_origin.rs`
- `wallet_observation.rs`
- `wallet_holding_observation.rs`
- `token_metadata.rs` - `token_metadata.rs`
- `local_pipeline_replay.rs` - `local_pipeline_replay.rs`
- `local_pipeline_validation.rs`
- `local_pipeline_diagnostics.rs` - `local_pipeline_diagnostics.rs`
- `db/entities/*`
- `db/dtos/*`
- `db/queries/*`
### 7.2. `kb_demo_app` `local_pipeline_diagnostics.rs` est volontairement conservé comme outil temporaire de validation. Il pourra devenir obsolète ou être remplacé lorsque les tests DEX seront stabilisés. Il n’est pas prioritaire de le refactorer maintenant.
### 7.2. Base de données
Organisation de la couche DB à conserver :
- `db/schema.rs` : création des tables et index uniquement ; chaque table ou index reste dans une fonction dédiée,
- `db/entities/*` : entités proches des lignes persistées,
- `db/dtos/*` : DTOs applicatifs,
- `db/queries/*` : requêtes SQL regroupées par table ou usage,
- `db/queries/local_pipeline_diagnostics.rs` : requêtes de diagnostic local, utiles pendant la validation DEX.
`schema.rs` peut rester long tant qu’il reste strictement un fichier de schéma. Le split prioritaire concerne plutôt les responsabilités métier dans `dex_decode.rs`, `dex_detect.rs` et `trade_aggregation.rs`.
### 7.3. `kb_demo_app`
Responsabilités cibles : Responsabilités cibles :
- lancement Tauri, - lancement Tauri,
- commandes UI, - commandes UI,
- affichage des états et messages, - affichage des états et messages,
- réception des événements venant de `kb_lib`, - réception des événements venant de `kb_lib`,
- persistance future des préférences UI, - fenêtres de démonstration / diagnostic isolées,
- fenêtres de démonstration / diagnostic isolées. - inspection du pipeline persisté,
- affichage candles et futurs overlays analytiques.
`kb_demo_app` ne doit pas contenir de logique métier DEX profonde.
## 8. Ligne de conduite sur le `WsClient` ## 8. Ligne de conduite sur le `WsClient`
Le `WsClient` doit être conçu en plusieurs couches : Le `WsClient` doit être conçu en plusieurs couches :
@@ -1048,7 +1163,7 @@ Le projet doit maintenir au minimum :
- un `README.md` global, - un `README.md` global,
- un `ROADMAP.md` global, - un `ROADMAP.md` global,
- un `CHANGELOG.md` global, - un `CHANGELOG.md` global,
- des `README.md` et `TODO.md` par crate à mesure de l’évolution, - des `README.md` et `TODO.md` par crate à mesure de l’évolution (surtout en version 1.0),
- des tests unitaires robustes, - des tests unitaires robustes,
- les bindings TS générés via `cargo test export_bindings` lorsque les types partagés évoluent. - les bindings TS générés via `cargo test export_bindings` lorsque les types partagés évoluent.
@@ -1056,16 +1171,21 @@ Le projet doit maintenir au minimum :
La priorité immédiate est désormais la suivante : La priorité immédiate est désormais la suivante :
1. terminer la validation `0.7.27` sur les connecteurs déjà branchés : `pump_fun`, `pump_swap`, `raydium_cpmm` et `raydium_clmm`, sans ajouter de nouveau DEX dans cette étape, 1. terminer la validation `0.7.27` sur `pump_fun`, `pump_swap`, `raydium_cpmm` et `raydium_clmm`, sans ajouter de nouveau DEX dans cette étape,
2. vérifier sur bases neuves et après replay local les invariants bloquants du pipeline : `diagnosticsClean = true`, `blockingIssueCount = 0`, aucun trade candidate exploitable perdu, aucun trade event invalide, aucun doublon réel par `decoded_event_id`, aucune candle dupliquée par bucket, 2. vérifier sur bases neuves et après replay local les invariants bloquants du pipeline : `diagnosticsClean = true`, `blockingIssueCount = 0`, aucun trade candidate exploitable perdu, aucun trade event invalide, aucun doublon réel par `decoded_event_id`, aucune candle dupliquée par bucket,
3. documenter les requêtes SQL de diagnostic de référence et les résultats attendus pour les tables clés du pipeline, 3. démarrer `0.7.28` par le refactor DEX commun, sans toucher aux clients HTTP/WS ni aux managers réseau stabilisés,
4. matérialiser ensuite les événements non buy/sell utiles au trading et à l’analyse : liquidité, cycle de vie des pools, fees, rewards et administration, 4. ajouter la matrice DEX et launch surfaces, avec statut, corpus, limites et prochaine action pour chaque protocole,
5. garantir que ces événements non-trade restent séparés des `k_sol_trade_events` et des candles tout en restant rattachés aux transactions, decoded events, pools, pairs et wallets observés, 5. ajouter les tables de classification des transactions inconnues et des protocol candidates afin de ne plus perdre les transactions utiles non encore décodables,
6. ajouter ou stabiliser les autres DEX par lots vérifiables : Meteora, surfaces de lancement, Orca, FluxBeam et DexLab, 6. matérialiser ensuite les événements non-trade : liquidité, cycle de vie des pools, fees, rewards et administration,
7. traiter `raydium_amm_v4` legacy seulement après les autres DEX, avec un corpus dédié prouvant le programme `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8`, 7. garantir que ces événements non-trade restent séparés des `k_sol_trade_events` et des candles tout en restant rattachés aux transactions, decoded events, pools, pairs et wallets observés,
8. effectuer une validation DEX v1 consolidée sur tous les connecteurs supportés avant de considérer la couche DEX `0.7.x` comme stable, 8. consolider Meteora, y compris `meteora_dlmm`, après corpus fiable,
9. ajouter ensuite les overlays des signaux analytiques sur les candles, 9. ajouter les launch surfaces manquantes comme premières sources de mint/lancement : Raydium LaunchLab/Launchpad, LetsBonk/Bonk.fun, Boop.fun, Moonshot/Moonit, Believe et Bags,
10. consolider les vues métier `token / pair / pool` dans `kb_demo_app`, y compris les événements liquidité, lifecycle, fees, rewards et admin, 10. traiter Heaven avec séparation launch/swap,
11. stabiliser l’ergonomie, les filtres, la pagination et la navigation de l’UI d’inspection, 11. consolider Orca, FluxBeam et DexLab sur corpus,
12. préparer ensuite l’ouverture de `0.8.x` pour l’analyse, les filtres, les patterns et les projections graphiques, 12. traiter `raydium_amm_v4` legacy seulement après les autres Raydium, avec corpus dédié prouvant le programme `675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8`,
13. préparer enfin Yellowstone gRPC comme extension de capacité, et non comme remplacement du socle HTTP / WS existant. 13. effectuer une validation DEX v1 consolidée sur tous les connecteurs supportés avant de considérer la couche DEX `0.7.x` comme stable,
14. ajouter ensuite les overlays des signaux analytiques sur les candles,
15. consolider les vues métier `token / pair / pool` dans `kb_demo_app`, y compris les événements liquidité, lifecycle, fees, rewards et admin,
16. stabiliser l’ergonomie, les filtres, la pagination et la navigation de l’UI d’inspection,
17. préparer ensuite l’ouverture de `0.8.x` pour l’analyse, les filtres, les patterns et les projections graphiques,
18. préparer enfin Yellowstone gRPC comme extension de capacité, et non comme remplacement du socle HTTP / WS existant.

47
kb_demo_app/frontend/demo_pipeline2.html
View File

@@ -29,11 +29,11 @@
<div class="accordion" id="demoPipeline2LeftAccordion"> <div class="accordion" id="demoPipeline2LeftAccordion">
<div class="accordion-item border-0 shadow-sm mb-3"> <div class="accordion-item border-0 shadow-sm mb-3">
<h1 class="accordion-header" id="demoPipeline2CatalogHeading"> <h1 class="accordion-header" id="demoPipeline2CatalogHeading">
<button class="accordion-button" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2CatalogCollapse" aria-expanded="true" aria-controls="demoPipeline2CatalogCollapse"> <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2CatalogCollapse" aria-expanded="false" aria-controls="demoPipeline2CatalogCollapse">
Catalogue local Catalogue local
</button> </button>
</h1> </h1>
<div id="demoPipeline2CatalogCollapse" class="accordion-collapse collapse show" aria-labelledby="demoPipeline2CatalogHeading" data-bs-parent="#demoPipeline2LeftAccordion"> <div id="demoPipeline2CatalogCollapse" class="accordion-collapse collapse" aria-labelledby="demoPipeline2CatalogHeading" data-bs-parent="#demoPipeline2LeftAccordion">
<div class="accordion-body"> <div class="accordion-body">
<div class="d-flex gap-2 mb-3"> <div class="d-flex gap-2 mb-3">
<button id="demoPipeline2RefreshCatalogButton" type="button" class="btn btn-primary"> <button id="demoPipeline2RefreshCatalogButton" type="button" class="btn btn-primary">
@@ -175,6 +175,32 @@
</div> </div>
</div> </div>
<div class="accordion-item border-0 shadow-sm mb-3">
<h2 class="accordion-header" id="demoPipeline2ProtocolCandidatesHeading">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2ProtocolCandidatesCollapse" aria-expanded="false" aria-controls="demoPipeline2ProtocolCandidatesCollapse">
Protocol candidates
</button>
</h2>
<div id="demoPipeline2ProtocolCandidatesCollapse" class="accordion-collapse collapse" aria-labelledby="demoPipeline2ProtocolCandidatesHeading" data-bs-parent="#demoPipeline2LeftAccordion">
<div class="accordion-body">
<p class="small text-body-secondary mb-3">
Résume les programmes candidats par priorité : transaction_count, occurrence_count, dernier slot et dernière signature.
</p>
<div class="mb-3">
<label for="demoPipeline2ProtocolCandidateLimitInput" class="form-label">Summary limit</label>
<input id="demoPipeline2ProtocolCandidateLimitInput" type="number" min="1" step="1" class="form-control" value="50" />
</div>
<div class="d-flex gap-2">
<button id="demoPipeline2RefreshProtocolCandidatesButton" type="button" class="btn btn-outline-warning">
Refresh protocol candidates
</button>
</div>
</div>
</div>
</div>
<div class="accordion-item border-0 shadow-sm"> <div class="accordion-item border-0 shadow-sm">
<h2 class="accordion-header" id="demoPipeline2CandlesControlHeading"> <h2 class="accordion-header" id="demoPipeline2CandlesControlHeading">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2CandlesControlCollapse" aria-expanded="false" aria-controls="demoPipeline2CandlesControlCollapse"> <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2CandlesControlCollapse" aria-expanded="false" aria-controls="demoPipeline2CandlesControlCollapse">
@@ -265,13 +291,26 @@
</div> </div>
</div> </div>
<div class="accordion-item border-0 shadow-sm mb-3">
<h2 class="accordion-header" id="demoPipeline2ProtocolCandidateSummaryHeading">
<button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2ProtocolCandidateSummaryCollapse" aria-expanded="false" aria-controls="demoPipeline2ProtocolCandidateSummaryCollapse">
Protocol candidate summaries
</button>
</h2>
<div id="demoPipeline2ProtocolCandidateSummaryCollapse" class="accordion-collapse collapse" aria-labelledby="demoPipeline2ProtocolCandidateSummaryHeading" data-bs-parent="#demoPipeline2ContentAccordion">
<div class="accordion-body">
<textarea id="demoPipeline2ProtocolCandidateSummariesTextarea" class="form-control font-monospace" rows="16" readonly spellcheck="false"></textarea>
</div>
</div>
</div>
<div class="accordion-item border-0 shadow-sm mb-3"> <div class="accordion-item border-0 shadow-sm mb-3">
<h2 class="accordion-header" id="demoPipeline2ChartHeading"> <h2 class="accordion-header" id="demoPipeline2ChartHeading">
<button class="accordion-button" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2ChartCollapse" aria-expanded="true" aria-controls="demoPipeline2ChartCollapse"> <button class="accordion-button collapsed" type="button" data-bs-toggle="collapse" data-bs-target="#demoPipeline2ChartCollapse" aria-expanded="false" aria-controls="demoPipeline2ChartCollapse">
Candles / OHLCV Candles / OHLCV
</button> </button>
</h2> </h2>
<div id="demoPipeline2ChartCollapse" class="accordion-collapse collapse show" aria-labelledby="demoPipeline2ChartHeading" data-bs-parent="#demoPipeline2ContentAccordion"> <div id="demoPipeline2ChartCollapse" class="accordion-collapse collapse" aria-labelledby="demoPipeline2ChartHeading" data-bs-parent="#demoPipeline2ContentAccordion">
<div class="accordion-body"> <div class="accordion-body">
<div class="d-flex flex-wrap justify-content-between align-items-center gap-3 mb-3"> <div class="d-flex flex-wrap justify-content-between align-items-center gap-3 mb-3">
<div id="demoPipeline2ChartMeta" class="small text-body-secondary"> <div id="demoPipeline2ChartMeta" class="small text-body-secondary">

10
kb_demo_app/frontend/ts/bindings/DemoPipeline2ProtocolCandidateSummaryPayload.ts Normal file
View File

@@ -0,0 +1,10 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
/**
* Response payload for protocol candidate summary diagnostics.
*/
export type DemoPipeline2ProtocolCandidateSummaryPayload = {
/**
* Pretty JSON summary rows.
*/
summariesJson: string, };

10
kb_demo_app/frontend/ts/bindings/DemoPipeline2ProtocolCandidateSummaryRequest.ts Normal file
View File

@@ -0,0 +1,10 @@
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
/**
* Request payload for protocol candidate summary diagnostics.
*/
export type DemoPipeline2ProtocolCandidateSummaryRequest = {
/**
* Maximum number of summary rows to return.
*/
limit: number, };

49
kb_demo_app/frontend/ts/demo_pipeline2.ts
View File

@@ -15,6 +15,8 @@ import type { DemoPipeline2PairCandlesRequest } from "./bindings/DemoPipeline2Pa
import type { DemoPipeline2PairCandlesPayload } from "./bindings/DemoPipeline2PairCandlesPayload.ts"; import type { DemoPipeline2PairCandlesPayload } from "./bindings/DemoPipeline2PairCandlesPayload.ts";
import type { DemoPipeline2LocalDiagnosticsPayload } from "./bindings/DemoPipeline2LocalDiagnosticsPayload.ts"; import type { DemoPipeline2LocalDiagnosticsPayload } from "./bindings/DemoPipeline2LocalDiagnosticsPayload.ts";
import type { DemoPipeline2LocalValidationPayload } from "./bindings/DemoPipeline2LocalValidationPayload.ts"; import type { DemoPipeline2LocalValidationPayload } from "./bindings/DemoPipeline2LocalValidationPayload.ts";
import { DemoPipeline2ProtocolCandidateSummaryRequest } from './bindings/DemoPipeline2ProtocolCandidateSummaryRequest.ts';
import { DemoPipeline2ProtocolCandidateSummaryPayload } from './bindings/DemoPipeline2ProtocolCandidateSummaryPayload.ts';
(window as Window & typeof globalThis & { bootstrap?: typeof bootstrap }).bootstrap = bootstrap; (window as Window & typeof globalThis & { bootstrap?: typeof bootstrap }).bootstrap = bootstrap;
(window as Window & typeof globalThis & { ResizeObserver?: typeof ResizeObserver }).ResizeObserver = ResizeObserver; (window as Window & typeof globalThis & { ResizeObserver?: typeof ResizeObserver }).ResizeObserver = ResizeObserver;
@@ -354,6 +356,9 @@ document.addEventListener("DOMContentLoaded", async () => {
const diagnoseLocalPipelineButton = document.querySelector<HTMLButtonElement>("#demoPipeline2DiagnoseLocalPipelineButton"); const diagnoseLocalPipelineButton = document.querySelector<HTMLButtonElement>("#demoPipeline2DiagnoseLocalPipelineButton");
const validateLocalPipelineButton = document.querySelector<HTMLButtonElement>("#demoPipeline2ValidateLocalPipelineButton"); const validateLocalPipelineButton = document.querySelector<HTMLButtonElement>("#demoPipeline2ValidateLocalPipelineButton");
const protocolCandidateLimitInput = document.querySelector<HTMLInputElement>("#demoPipeline2ProtocolCandidateLimitInput");
const refreshProtocolCandidatesButton = document.querySelector<HTMLButtonElement>("#demoPipeline2RefreshProtocolCandidatesButton");
const pairSelect = document.querySelector<HTMLSelectElement>("#demoPipeline2PairSelect"); const pairSelect = document.querySelector<HTMLSelectElement>("#demoPipeline2PairSelect");
const timeframeSelect = document.querySelector<HTMLSelectElement>("#demoPipeline2TimeframeSelect"); const timeframeSelect = document.querySelector<HTMLSelectElement>("#demoPipeline2TimeframeSelect");
const customTimeframeInput = document.querySelector<HTMLInputElement>("#demoPipeline2CustomTimeframeInput"); const customTimeframeInput = document.querySelector<HTMLInputElement>("#demoPipeline2CustomTimeframeInput");
@@ -366,6 +371,8 @@ document.addEventListener("DOMContentLoaded", async () => {
const localDiagnosticsTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LocalDiagnosticsTextarea"); const localDiagnosticsTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LocalDiagnosticsTextarea");
const localValidationTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LocalValidationTextarea"); const localValidationTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LocalValidationTextarea");
const protocolCandidateSummariesTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2ProtocolCandidateSummariesTextarea");
const clearLogButton = document.querySelector<HTMLButtonElement>("#demoPipeline2ClearLogButton"); const clearLogButton = document.querySelector<HTMLButtonElement>("#demoPipeline2ClearLogButton");
const logTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LogTextarea"); const logTextarea = document.querySelector<HTMLTextAreaElement>("#demoPipeline2LogTextarea");
@@ -388,6 +395,8 @@ document.addEventListener("DOMContentLoaded", async () => {
!replayLocalPipelineButton || !replayLocalPipelineButton ||
!diagnoseLocalPipelineButton || !diagnoseLocalPipelineButton ||
!validateLocalPipelineButton || !validateLocalPipelineButton ||
!protocolCandidateLimitInput ||
!refreshProtocolCandidatesButton ||
!pairSelect || !pairSelect ||
!timeframeSelect || !timeframeSelect ||
!customTimeframeInput || !customTimeframeInput ||
@@ -396,6 +405,7 @@ document.addEventListener("DOMContentLoaded", async () => {
!backfillSummaryTextarea || !backfillSummaryTextarea ||
!localDiagnosticsTextarea || !localDiagnosticsTextarea ||
!localValidationTextarea || !localValidationTextarea ||
!protocolCandidateSummariesTextarea ||
!chartElement || !chartElement ||
!chartMeta || !chartMeta ||
!clearLogButton || !clearLogButton ||
@@ -644,6 +654,45 @@ document.addEventListener("DOMContentLoaded", async () => {
} }
}); });
refreshProtocolCandidatesButton.addEventListener("click", async () => {
const limit = readPositiveIntegerInput(
protocolCandidateLimitInput,
logTextarea,
"protocolCandidateSummaryLimit",
);
if (limit === undefined) {
return;
}
appendLogLine(
logTextarea,
`[ui] loading protocol candidate summaries with limit '${limit.toString()}'`,
);
const request: DemoPipeline2ProtocolCandidateSummaryRequest = {
limit,
};
try {
const payload = await invoke<DemoPipeline2ProtocolCandidateSummaryPayload>(
"demo_pipeline2_get_protocol_candidate_summaries",
{ request },
);
protocolCandidateSummariesTextarea.value = payload.summariesJson;
appendLogLine(
logTextarea,
"[ui] protocol candidate summaries loaded",
);
} catch (error) {
appendLogLine(
logTextarea,
`[ui] protocol candidate summary error: ${String(error)}`,
);
}
});
loadCandlesButton.addEventListener("click", async () => { loadCandlesButton.addEventListener("click", async () => {
const pairIdText = pairSelect.value.trim(); const pairIdText = pairSelect.value.trim();
if (pairIdText === "") { if (pairIdText === "") {

2
kb_demo_app/package.json
View File

@@ -1,7 +1,7 @@
{ {
"name": "kb-demo-app", "name": "kb-demo-app",
"private": true, "private": true,
"version": "0.7.27", "version": "0.7.28",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite", "dev": "vite",

64
kb_demo_app/src/demo_pipeline2.rs
View File

@@ -10,6 +10,34 @@
use tauri::Manager; use tauri::Manager;
use ts_rs::TS; use ts_rs::TS;
/// Request payload for protocol candidate summary diagnostics.
#[derive(Clone, Debug, serde::Deserialize, ts_rs::TS)]
#[ts(
export,
export_to = "../frontend/ts/bindings/DemoPipeline2ProtocolCandidateSummaryRequest.ts"
)]
#[serde(rename_all = "camelCase")]
pub(crate) struct DemoPipeline2ProtocolCandidateSummaryRequest {
/// Maximum number of summary rows to return.
pub limit: u32,
}
/// Response payload for protocol candidate summary diagnostics.
#[derive(Clone, Debug, serde::Serialize, ts_rs::TS)]
#[ts(
export,
export_to = "../frontend/ts/bindings/DemoPipeline2ProtocolCandidateSummaryPayload.ts"
)]
#[serde(rename_all = "camelCase")]
pub(crate) struct DemoPipeline2ProtocolCandidateSummaryPayload {
/// Pretty JSON summary rows.
pub summaries_json: std::string::String,
}
/// Local diagnostics payload returned to the UI. /// Local diagnostics payload returned to the UI.
#[derive(Clone, Debug, serde::Serialize, TS)] #[derive(Clone, Debug, serde::Serialize, TS)]
#[ts( #[ts(
@@ -690,6 +718,42 @@ pub(crate) struct DemoPipeline2PairCandlesPayload {
pub candles_json: std::string::String, pub candles_json: std::string::String,
} }
/// Lists protocol candidate summaries ordered by investigation priority.
#[tauri::command]
pub(crate) async fn demo_pipeline2_get_protocol_candidate_summaries(
state: tauri::State<'_, crate::AppState>,
request: DemoPipeline2ProtocolCandidateSummaryRequest,
) -> Result<DemoPipeline2ProtocolCandidateSummaryPayload, std::string::String> {
if request.limit == 0 {
return Err("protocol candidate summary limit must be > 0".to_string());
}
let summaries_result = kb_lib::query_protocol_candidate_summaries_list_by_priority(
state.database.as_ref(),
request.limit,
)
.await;
let summaries = match summaries_result {
Ok(summaries) => summaries,
Err(error) => {
return Err(format!(
"cannot list protocol candidate summaries with limit '{}': {}",
request.limit, error
));
},
};
let summaries_json_result = serde_json::to_string_pretty(&summaries);
let summaries_json = match summaries_json_result {
Ok(summaries_json) => summaries_json,
Err(error) => {
return Err(format!(
"cannot serialize protocol candidate summaries: {}",
error
));
},
};
return Ok(DemoPipeline2ProtocolCandidateSummaryPayload { summaries_json });
}
/// Runs local pipeline diagnostics from persisted data only. /// Runs local pipeline diagnostics from persisted data only.
#[tauri::command] #[tauri::command]
pub(crate) async fn demo_pipeline2_diagnose_local_pipeline( pub(crate) async fn demo_pipeline2_diagnose_local_pipeline(

1
kb_demo_app/src/lib.rs
View File

@@ -153,6 +153,7 @@ pub async fn run() -> Result<(), kb_lib::Error> {
crate::demo_pipeline2::demo_pipeline2_replay_local_pipeline, crate::demo_pipeline2::demo_pipeline2_replay_local_pipeline,
crate::demo_pipeline2::demo_pipeline2_diagnose_local_pipeline, crate::demo_pipeline2::demo_pipeline2_diagnose_local_pipeline,
crate::demo_pipeline2::demo_pipeline2_validate_local_pipeline, crate::demo_pipeline2::demo_pipeline2_validate_local_pipeline,
crate::demo_pipeline2::demo_pipeline2_get_protocol_candidate_summaries,
]); ]);
tauri_builder = tauri_builder.plugin(tracing_builder.build::<tauri::Wry>()); tauri_builder = tauri_builder.plugin(tracing_builder.build::<tauri::Wry>());
tauri_builder = tauri_builder.setup(|app| { tauri_builder = tauri_builder.setup(|app| {

2
kb_demo_app/tauri.conf.json
View File

@@ -1,7 +1,7 @@
{ {
"$schema": "https://schema.tauri.app/config/2", "$schema": "https://schema.tauri.app/config/2",
"productName": "kb-demo-app", "productName": "kb-demo-app",
"version": "0.7.27", "version": "0.7.28",
"identifier": "com.sasedev.kb-demo-app", "identifier": "com.sasedev.kb-demo-app",
"build": { "build": {
"beforeDevCommand": "npm run dev", "beforeDevCommand": "npm run dev",

149
kb_lib/src/constants.rs
View File

@@ -14,17 +14,134 @@ pub const SPL_TOKEN_2022_PROGRAM_ID: &str = "TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqC
/// @see solana_sdk::pubkey::Pubkey = spl_associated_token_account_interface::program::ID /// @see solana_sdk::pubkey::Pubkey = spl_associated_token_account_interface::program::ID
pub const ASSOCIATED_TOKEN_PROGRAM_ID: &str = "ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL"; pub const ASSOCIATED_TOKEN_PROGRAM_ID: &str = "ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL";
/// Wrapped SOL mint identifier. ("So11111111111111111111111111111111111111112"). /// Address Lookup Table program identifier. ("AddressLookupTab1e1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = spl_token_interface::native_mint::ID /// @see solana_sdk_ids::address_lookup_table::ID
pub const WSOL_MINT_ID: &str = "So11111111111111111111111111111111111111112"; pub const ADDRESS_LOOKUP_TABLE_PROGRAM_ID: &str = "AddressLookupTab1e1111111111111111111111111";
/// BPF Loader program identifier. ("BPFLoader1111111111111111111111111111111111").
/// @see solana_sdk_ids::bpf_loader_deprecated::ID
pub const BPF_LOADER_DEPRECATED_PROGRAM_ID: &str = "BPFLoader1111111111111111111111111111111111";
/// BPF Loader program identifier. ("BPFLoaderUpgradeab1e11111111111111111111111").
/// @see solana_sdk_ids::bpf_loader_upgradeable::ID
pub const BPF_LOADER_UPGRADEABLE_PROGRAM_ID: &str = "BPFLoaderUpgradeab1e11111111111111111111111";
/// Compute Budget program identifier. ("ComputeBudget111111111111111111111111111111").
/// @see solana_sdk_ids::compute_budget::ID
pub const COMPUTE_BUDGET_PROGRAM_ID: &str = "ComputeBudget111111111111111111111111111111";
/// Config program identifier. ("Config1111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::config::ID
pub const CONFIG_PROGRAM_ID: &str = "Config1111111111111111111111111111111111111";
/// ED25519 program identifier. ("Ed25519SigVerify111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::ed25519_program::ID
pub const ED25519_PROGRAM_ID: &str = "Ed25519SigVerify111111111111111111111111111";
/// Feature program identifier. ("Feature111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::feature::ID
pub const FEATURE_PROGRAM_ID: &str = "Feature111111111111111111111111111111111111";
/// Incinerator program identifier. ("1nc1nerator11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::incinerator::ID
pub const INCINERATOR_PROGRAM_ID: &str = "1nc1nerator11111111111111111111111111111111";
/// Loader V4 program identifier. ("LoaderV411111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::loader_v4::ID
pub const LOADER_V4_PROGRAM_ID: &str = "LoaderV411111111111111111111111111111111111";
/// Native Loader program identifier. ("NativeLoader1111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::native_loader::ID
pub const NATIVE_LOADER_PROGRAM_ID: &str = "NativeLoader1111111111111111111111111111111";
/// Secp256k1 program identifier. ("KeccakSecp256k11111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::secp256k1_program::ID
pub const SECP256K1_PROGRAM_ID: &str = "KeccakSecp256k11111111111111111111111111111";
/// Secp256r1 program identifier. ("Secp256r1SigVerify1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::secp256r1_program::ID
pub const SECP256R1_PROGRAM_ID: &str = "Secp256r1SigVerify1111111111111111111111111";
/// Stake program identifier. ("Stake11111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::stake::ID
pub const STAKE_PROGRAM_ID: &str = "Stake11111111111111111111111111111111111111";
/// Stake Config program identifier. ("StakeConfig11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::stake::config::ID
pub const STAKE_CONFIG_PROGRAM_ID: &str = "StakeConfig11111111111111111111111111111111";
/// System program identifier. ("11111111111111111111111111111111"). /// System program identifier. ("11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::system_program::ID /// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::system_program::ID
pub const SYSTEM_PROGRAM_ID: &str = "11111111111111111111111111111111"; pub const SYSTEM_PROGRAM_ID: &str = "11111111111111111111111111111111";
/// Compute Budget program identifier. ("ComputeBudget111111111111111111111111111111"). /// Vote program identifier. ("Vote111111111111111111111111111111111111111").
/// @see solana_sdk_ids::compute_budget::ID /// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::vote::ID
pub const COMPUTE_BUDGET_PROGRAM_ID: &str = "ComputeBudget111111111111111111111111111111"; pub const VOTE_PROGRAM_ID: &str = "Vote111111111111111111111111111111111111111";
/// Sysvar program identifier. ("Sysvar1111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::ID
pub const SYSVAR_PROGRAM_ID: &str = "Sysvar1111111111111111111111111111111111111";
/// Sysvar Clock program identifier. ("SysvarC1ock11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::clock::ID
pub const SYSVAR_CLOCK_PROGRAM_ID: &str = "SysvarC1ock11111111111111111111111111111111";
/// Sysvar Epoch Rewards program identifier. ("SysvarEpochRewards1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::epoch_rewards::ID
pub const SYSVAR_EPOCH_REWARDS_PROGRAM_ID: &str = "SysvarEpochRewards1111111111111111111111111";
/// Sysvar Epoch Schedule program identifier. ("SysvarEpochSchedu1e111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::epoch_schedule::ID
pub const SYSVAR_EPOCH_SCHEDULE_PROGRAM_ID: &str = "SysvarEpochSchedu1e111111111111111111111111";
/// Sysvar Fees program identifier. ("SysvarFees111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::fees::ID
pub const SYSVAR_FEES_PROGRAM_ID: &str = "SysvarFees111111111111111111111111111111111";
/// Sysvar Instructions program identifier. ("Sysvar1nstructions1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::instructions::ID
pub const SYSVAR_INSTRUCTIONS_PROGRAM_ID: &str = "Sysvar1nstructions1111111111111111111111111";
/// Sysvar Last Restart Slot program identifier. ("SysvarLastRestartS1ot1111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::last_restart_slot::ID
pub const SYSVAR_LAST_RESTART_SLOT_PROGRAM_ID: &str = "SysvarLastRestartS1ot1111111111111111111111";
/// Sysvar Recent Blockhashes program identifier. ("SysvarRecentB1ockHashes11111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::recent_blockhashes::ID
pub const SYSVAR_RECENT_BLOCKHASHES_PROGRAM_ID: &str =
"SysvarRecentB1ockHashes11111111111111111111";
/// Sysvar Rent program identifier. ("SysvarRent111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::rent::ID
pub const SYSVAR_RENT_PROGRAM_ID: &str = "SysvarRent111111111111111111111111111111111";
/// Sysvar Rewards program identifier. ("SysvarRewards111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::rewards::ID
pub const SYSVAR_REWARDS_PROGRAM_ID: &str = "SysvarRewards111111111111111111111111111111";
/// Sysvar Slot Hashes program identifier. ("SysvarS1otHashes111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::slot_hashes::ID
pub const SYSVAR_SLOT_HASHES_PROGRAM_ID: &str = "SysvarS1otHashes111111111111111111111111111";
/// Sysvar Slot History program identifier. ("SysvarS1otHistory11111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::slot_history::ID
pub const SYSVAR_SLOT_HISTORY_PROGRAM_ID: &str = "SysvarS1otHistory11111111111111111111111111";
/// Sysvar Stake History program identifier. ("SysvarStakeHistory1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::stake_history::ID
pub const SYSVAR_STAKE_HISTORY_PROGRAM_ID: &str = "SysvarStakeHistory1111111111111111111111111";
/// Zk Token Proof program identifier. ("ZkTokenProof1111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::zk_token_proof_program::ID
pub const ZK_TOKEN_PROOF_PROGRAM_ID: &str = "ZkTokenProof1111111111111111111111111111111";
/// Zk El Gamal Proof program identifier. ("ZkE1Gama1Proof11111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::zk_elgamal_proof_program::ID
pub const ZK_ELGAMAL_PROOF_PROGRAM_ID: &str = "ZkE1Gama1Proof11111111111111111111111111111";
/// Wrapped SOL mint identifier. ("So11111111111111111111111111111111111111112").
/// @see solana_sdk::pubkey::Pubkey = spl_token_interface::native_mint::ID
pub const WSOL_MINT_ID: &str = "So11111111111111111111111111111111111111112";
/// DexLab Swap/Pool program id. ("DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N"). /// DexLab Swap/Pool program id. ("DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N").
pub const DEXLAB_PROGRAM_ID: &str = "DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N"; pub const DEXLAB_PROGRAM_ID: &str = "DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N";
@@ -41,6 +158,11 @@ pub const METEORA_DAMM_V2_PROGRAM_ID: &str = "cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWA
/// Meteora DBC program id. ("dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN"). /// Meteora DBC program id. ("dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN").
pub const METEORA_DBC_PROGRAM_ID: &str = "dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN"; pub const METEORA_DBC_PROGRAM_ID: &str = "dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN";
/// Meteora DLMM program id. ("LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo").
///
/// DLMM = Dynamic Liquidity Market Maker.
pub const METEORA_DLMM_PROGRAM_ID: &str = "LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo";
/// Orca Whirlpools program id. ("whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"). /// Orca Whirlpools program id. ("whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc").
pub const ORCA_WHIRLPOOLS_PROGRAM_ID: &str = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"; pub const ORCA_WHIRLPOOLS_PROGRAM_ID: &str = "whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc";
@@ -58,3 +180,18 @@ pub const RAYDIUM_CLMM_PROGRAM_ID: &str = "CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7gr
/// Raydium CPMM mainnet program id. ("CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C"). /// Raydium CPMM mainnet program id. ("CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C").
pub const RAYDIUM_CPMM_PROGRAM_ID: &str = "CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C"; pub const RAYDIUM_CPMM_PROGRAM_ID: &str = "CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C";
/// Raydium LaunchLab program id. ("LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj").
pub const RAYDIUM_LAUNCHLAB_PROGRAM_ID: &str = "LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj";
/// Raydium AMM routing program id. ("routeUGWgWzqBWFcrCfv8tritsqukccJPu3q5GPP3xS").
pub const RAYDIUM_AMM_ROUTING_PROGRAM_ID: &str = "routeUGWgWzqBWFcrCfv8tritsqukccJPu3q5GPP3xS";
/// Raydium Stable Swap AMM program id, deprecated. ("5quBtoiQqxF9Jv6KYKctB59NT3gtJD2Y65kdnB1Uev3h").
pub const RAYDIUM_STABLE_SWAP_AMM_PROGRAM_ID: &str = "5quBtoiQqxF9Jv6KYKctB59NT3gtJD2Y65kdnB1Uev3h";
/// Known Solana arbitrage/sandwich bot program id observed in local corpus.
///
/// This is not treated as a DEX program. It is used only to tag protocol
/// candidates with `candidate_surface = "arbitrage_bot"`.
pub const ARBITRAGE_BOT_6MWVT_PROGRAM_ID: &str = "6MWVTis8rmmk6Vt9zmAJJbmb3VuLpzoQ1aHH4N6wQEGh";

16
kb_lib/src/db.rs
View File

@@ -50,11 +50,14 @@ pub use dtos::PoolDto;
pub use dtos::PoolListingDto; pub use dtos::PoolListingDto;
pub use dtos::PoolOriginDto; pub use dtos::PoolOriginDto;
pub use dtos::PoolTokenDto; pub use dtos::PoolTokenDto;
pub use dtos::ProtocolCandidateDto;
pub use dtos::ProtocolCandidateSummaryDto;
pub use dtos::SwapDto; pub use dtos::SwapDto;
pub use dtos::TokenBurnEventDto; pub use dtos::TokenBurnEventDto;
pub use dtos::TokenDto; pub use dtos::TokenDto;
pub use dtos::TokenMintEventDto; pub use dtos::TokenMintEventDto;
pub use dtos::TradeEventDto; pub use dtos::TradeEventDto;
pub use dtos::TransactionClassificationDto;
pub use dtos::WalletDto; pub use dtos::WalletDto;
pub use dtos::WalletHoldingDto; pub use dtos::WalletHoldingDto;
pub use dtos::WalletParticipationDto; pub use dtos::WalletParticipationDto;
@@ -82,11 +85,14 @@ pub use entities::PoolEntity;
pub use entities::PoolListingEntity; pub use entities::PoolListingEntity;
pub use entities::PoolOriginEntity; pub use entities::PoolOriginEntity;
pub use entities::PoolTokenEntity; pub use entities::PoolTokenEntity;
pub use entities::ProtocolCandidateEntity;
pub use entities::ProtocolCandidateSummaryEntity;
pub use entities::SwapEntity; pub use entities::SwapEntity;
pub use entities::TokenBurnEventEntity; pub use entities::TokenBurnEventEntity;
pub use entities::TokenEntity; pub use entities::TokenEntity;
pub use entities::TokenMintEventEntity; pub use entities::TokenMintEventEntity;
pub use entities::TradeEventEntity; pub use entities::TradeEventEntity;
pub use entities::TransactionClassificationEntity;
pub use entities::WalletEntity; pub use entities::WalletEntity;
pub use entities::WalletHoldingEntity; pub use entities::WalletHoldingEntity;
pub use entities::WalletParticipationEntity; pub use entities::WalletParticipationEntity;
@@ -172,6 +178,12 @@ pub use queries::query_pool_tokens_upsert;
pub use queries::query_pools_get_by_address; pub use queries::query_pools_get_by_address;
pub use queries::query_pools_list; pub use queries::query_pools_list;
pub use queries::query_pools_upsert; pub use queries::query_pools_upsert;
pub use queries::query_protocol_candidate_summaries_list_by_priority;
pub use queries::query_protocol_candidates_delete_by_transaction_id;
pub use queries::query_protocol_candidates_insert;
pub use queries::query_protocol_candidates_list_by_program_id;
pub use queries::query_protocol_candidates_list_by_transaction_id;
pub use queries::query_protocol_candidates_list_recent;
pub use queries::query_swaps_list_recent; pub use queries::query_swaps_list_recent;
pub use queries::query_swaps_upsert; pub use queries::query_swaps_upsert;
pub use queries::query_token_burn_events_list_recent; pub use queries::query_token_burn_events_list_recent;
@@ -187,6 +199,10 @@ pub use queries::query_trade_events_get_by_decoded_event_id;
pub use queries::query_trade_events_list_by_pair_id; pub use queries::query_trade_events_list_by_pair_id;
pub use queries::query_trade_events_list_by_transaction_id; pub use queries::query_trade_events_list_by_transaction_id;
pub use queries::query_trade_events_upsert; pub use queries::query_trade_events_upsert;
pub use queries::query_transaction_classifications_get_by_signature;
pub use queries::query_transaction_classifications_get_by_transaction_id;
pub use queries::query_transaction_classifications_list_recent;
pub use queries::query_transaction_classifications_upsert;
pub use queries::query_wallet_holdings_get_by_wallet_and_token; pub use queries::query_wallet_holdings_get_by_wallet_and_token;
pub use queries::query_wallet_holdings_list_by_wallet_id; pub use queries::query_wallet_holdings_list_by_wallet_id;
pub use queries::query_wallet_holdings_upsert; pub use queries::query_wallet_holdings_upsert;

6
kb_lib/src/db/dtos.rs
View File

@@ -27,11 +27,14 @@ mod pool;
mod pool_listing; mod pool_listing;
mod pool_origin; mod pool_origin;
mod pool_token; mod pool_token;
mod protocol_candidate;
mod protocol_candidate_summary;
mod swap; mod swap;
mod token; mod token;
mod token_burn_event; mod token_burn_event;
mod token_mint_event; mod token_mint_event;
mod trade_event; mod trade_event;
mod transaction_classification;
mod wallet; mod wallet;
mod wallet_holding; mod wallet_holding;
mod wallet_participation; mod wallet_participation;
@@ -82,11 +85,14 @@ pub use pool::PoolDto;
pub use pool_listing::PoolListingDto; pub use pool_listing::PoolListingDto;
pub use pool_origin::PoolOriginDto; pub use pool_origin::PoolOriginDto;
pub use pool_token::PoolTokenDto; pub use pool_token::PoolTokenDto;
pub use protocol_candidate::ProtocolCandidateDto;
pub use protocol_candidate_summary::ProtocolCandidateSummaryDto;
pub use swap::SwapDto; pub use swap::SwapDto;
pub use token::TokenDto; pub use token::TokenDto;
pub use token_burn_event::TokenBurnEventDto; pub use token_burn_event::TokenBurnEventDto;
pub use token_mint_event::TokenMintEventDto; pub use token_mint_event::TokenMintEventDto;
pub use trade_event::TradeEventDto; pub use trade_event::TradeEventDto;
pub use transaction_classification::TransactionClassificationDto;
pub use wallet::WalletDto; pub use wallet::WalletDto;
pub use wallet_holding::WalletHoldingDto; pub use wallet_holding::WalletHoldingDto;
pub use wallet_participation::WalletParticipationDto; pub use wallet_participation::WalletParticipationDto;

114
kb_lib/src/db/dtos/protocol_candidate.rs Normal file
View File

@@ -0,0 +1,114 @@
// file: kb_lib/src/db/dtos/protocol_candidate.rs
//! Protocol candidate DTO.
/// Application-facing protocol candidate DTO.
///
/// A protocol candidate records a program/instruction that should be inspected
/// later because it may correspond to an unsupported DEX, launch surface,
/// migration path or protocol-specific non-trade event.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ProtocolCandidateDto {
/// Optional numeric primary key.
pub id: std::option::Option<i64>,
/// Related chain transaction id.
pub transaction_id: i64,
/// Optional related chain instruction id.
pub instruction_id: std::option::Option<i64>,
/// Transaction signature.
pub signature: std::string::String,
/// Optional Solana slot.
pub slot: std::option::Option<u64>,
/// Program id observed in the transaction instruction.
pub program_id: std::string::String,
/// Optional program name hint from parsed transaction data.
pub program_name_hint: std::option::Option<std::string::String>,
/// Optional candidate protocol code.
pub candidate_protocol: std::option::Option<std::string::String>,
/// Optional candidate surface code.
pub candidate_surface: std::option::Option<std::string::String>,
/// Human-readable reason.
pub reason: std::string::String,
/// Serialized JSON evidence.
pub evidence_json: std::string::String,
/// Creation timestamp.
pub created_at: chrono::DateTime<chrono::Utc>,
}
impl ProtocolCandidateDto {
/// Creates a protocol candidate DTO.
#[allow(clippy::too_many_arguments)]
pub fn new(
transaction_id: i64,
instruction_id: std::option::Option<i64>,
signature: std::string::String,
slot: std::option::Option<u64>,
program_id: std::string::String,
program_name_hint: std::option::Option<std::string::String>,
candidate_protocol: std::option::Option<std::string::String>,
candidate_surface: std::option::Option<std::string::String>,
reason: std::string::String,
evidence_json: std::string::String,
) -> Self {
return Self {
id: None,
transaction_id,
instruction_id,
signature,
slot,
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
evidence_json,
created_at: chrono::Utc::now(),
};
}
}
impl TryFrom<crate::ProtocolCandidateEntity> for ProtocolCandidateDto {
type Error = crate::Error;
fn try_from(entity: crate::ProtocolCandidateEntity) -> Result<Self, Self::Error> {
let slot = match entity.slot {
Some(slot) => {
let slot_result = u64::try_from(slot);
match slot_result {
Ok(slot) => Some(slot),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert protocol candidate slot '{}' to u64: {}",
slot, error
)));
},
}
},
None => None,
};
let created_at_result = chrono::DateTime::parse_from_rfc3339(&entity.created_at);
let created_at = match created_at_result {
Ok(created_at) => created_at.with_timezone(&chrono::Utc),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot parse protocol candidate created_at '{}': {}",
entity.created_at, error
)));
},
};
return Ok(Self {
id: Some(entity.id),
transaction_id: entity.transaction_id,
instruction_id: entity.instruction_id,
signature: entity.signature,
slot,
program_id: entity.program_id,
program_name_hint: entity.program_name_hint,
candidate_protocol: entity.candidate_protocol,
candidate_surface: entity.candidate_surface,
reason: entity.reason,
evidence_json: entity.evidence_json,
created_at,
});
}
}

96
kb_lib/src/db/dtos/protocol_candidate_summary.rs Normal file
View File

@@ -0,0 +1,96 @@
// file: kb_lib/src/db/dtos/protocol_candidate_summary.rs
//! Protocol candidate summary DTO.
/// Aggregated protocol candidate diagnostic row.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct ProtocolCandidateSummaryDto {
/// Program id observed in protocol candidates.
pub program_id: std::string::String,
/// Optional program name hint.
pub program_name_hint: std::option::Option<std::string::String>,
/// Optional candidate protocol.
pub candidate_protocol: std::option::Option<std::string::String>,
/// Optional candidate surface.
pub candidate_surface: std::option::Option<std::string::String>,
/// Candidate reason.
pub reason: std::string::String,
/// Number of candidate rows.
pub occurrence_count: u64,
/// Number of distinct transactions.
pub transaction_count: u64,
/// Latest observed slot.
pub last_slot: std::option::Option<u64>,
/// Latest candidate row id in this group.
pub latest_candidate_id: i64,
/// Latest signature in this group.
pub latest_signature: std::string::String,
/// Latest candidate creation timestamp.
pub latest_created_at: chrono::DateTime<chrono::Utc>,
}
impl TryFrom<crate::ProtocolCandidateSummaryEntity> for ProtocolCandidateSummaryDto {
type Error = crate::Error;
fn try_from(entity: crate::ProtocolCandidateSummaryEntity) -> Result<Self, Self::Error> {
let occurrence_count_result = u64::try_from(entity.occurrence_count);
let occurrence_count = match occurrence_count_result {
Ok(occurrence_count) => occurrence_count,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert protocol candidate occurrence_count '{}' to u64: {}",
entity.occurrence_count, error
)));
},
};
let transaction_count_result = u64::try_from(entity.transaction_count);
let transaction_count = match transaction_count_result {
Ok(transaction_count) => transaction_count,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert protocol candidate transaction_count '{}' to u64: {}",
entity.transaction_count, error
)));
},
};
let last_slot = match entity.last_slot {
Some(last_slot) => {
let slot_result = u64::try_from(last_slot);
match slot_result {
Ok(slot) => Some(slot),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert protocol candidate last_slot '{}' to u64: {}",
last_slot, error
)));
},
}
},
None => None,
};
let latest_created_at_result =
chrono::DateTime::parse_from_rfc3339(entity.latest_created_at.as_str());
let latest_created_at = match latest_created_at_result {
Ok(latest_created_at) => latest_created_at.with_timezone(&chrono::Utc),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot parse protocol candidate latest_created_at '{}': {}",
entity.latest_created_at, error
)));
},
};
return Ok(Self {
program_id: entity.program_id,
program_name_hint: entity.program_name_hint,
candidate_protocol: entity.candidate_protocol,
candidate_surface: entity.candidate_surface,
reason: entity.reason,
occurrence_count,
transaction_count,
last_slot,
latest_candidate_id: entity.latest_candidate_id,
latest_signature: entity.latest_signature,
latest_created_at,
});
}
}

130
kb_lib/src/db/dtos/transaction_classification.rs Normal file
View File

@@ -0,0 +1,130 @@
// file: kb_lib/src/db/dtos/transaction_classification.rs
//! Transaction classification DTO.
/// Application-facing transaction classification DTO.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize)]
pub struct TransactionClassificationDto {
/// Optional numeric primary key.
pub id: std::option::Option<i64>,
/// Related chain transaction id.
pub transaction_id: i64,
/// Transaction signature.
pub signature: std::string::String,
/// Optional Solana slot.
pub slot: std::option::Option<u64>,
/// Stable classification kind.
pub classification_kind: std::string::String,
/// Optional primary protocol name.
pub primary_protocol: std::option::Option<std::string::String>,
/// Optional primary program id.
pub primary_program_id: std::option::Option<std::string::String>,
/// Confidence level from 0 to 100.
pub confidence_level: i16,
/// Human-readable reason.
pub reason: std::string::String,
/// Serialized JSON evidence.
pub evidence_json: std::string::String,
/// Creation timestamp.
pub created_at: chrono::DateTime<chrono::Utc>,
/// Update timestamp.
pub updated_at: chrono::DateTime<chrono::Utc>,
}
impl TransactionClassificationDto {
/// Creates a new transaction classification DTO.
#[allow(clippy::too_many_arguments)]
pub fn new(
transaction_id: i64,
signature: std::string::String,
slot: std::option::Option<u64>,
classification_kind: std::string::String,
primary_protocol: std::option::Option<std::string::String>,
primary_program_id: std::option::Option<std::string::String>,
confidence_level: i16,
reason: std::string::String,
evidence_json: std::string::String,
) -> Self {
let now = chrono::Utc::now();
return Self {
id: None,
transaction_id,
signature,
slot,
classification_kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
created_at: now,
updated_at: now,
};
}
}
impl TryFrom<crate::TransactionClassificationEntity> for TransactionClassificationDto {
type Error = crate::Error;
fn try_from(entity: crate::TransactionClassificationEntity) -> Result<Self, Self::Error> {
let slot = match entity.slot {
Some(slot) => {
let slot_result = u64::try_from(slot);
match slot_result {
Ok(slot) => Some(slot),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert transaction classification slot '{}' to u64: {}",
slot, error
)));
},
}
},
None => None,
};
let created_at_result = chrono::DateTime::parse_from_rfc3339(&entity.created_at);
let created_at = match created_at_result {
Ok(created_at) => created_at.with_timezone(&chrono::Utc),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot parse transaction classification created_at '{}': {}",
entity.created_at, error
)));
},
};
let updated_at_result = chrono::DateTime::parse_from_rfc3339(&entity.updated_at);
let updated_at = match updated_at_result {
Ok(updated_at) => updated_at.with_timezone(&chrono::Utc),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot parse transaction classification updated_at '{}': {}",
entity.updated_at, error
)));
},
};
let confidence_level_result = i16::try_from(entity.confidence_level);
let confidence_level = match confidence_level_result {
Ok(confidence_level) => confidence_level,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert transaction classification confidence_level '{}' to i16: {}",
entity.confidence_level, error
)));
},
};
return Ok(Self {
id: Some(entity.id),
transaction_id: entity.transaction_id,
signature: entity.signature,
slot,
classification_kind: entity.classification_kind,
primary_protocol: entity.primary_protocol,
primary_program_id: entity.primary_program_id,
confidence_level,
reason: entity.reason,
evidence_json: entity.evidence_json,
created_at,
updated_at,
});
}
}

6
kb_lib/src/db/entities.rs
View File

@@ -28,11 +28,14 @@ mod pool;
mod pool_listing; mod pool_listing;
mod pool_origin; mod pool_origin;
mod pool_token; mod pool_token;
mod protocol_candidate;
mod protocol_candidate_summary;
mod swap; mod swap;
mod token; mod token;
mod token_burn_event; mod token_burn_event;
mod token_mint_event; mod token_mint_event;
mod trade_event; mod trade_event;
mod transaction_classification;
mod wallet; mod wallet;
mod wallet_holding; mod wallet_holding;
mod wallet_participation; mod wallet_participation;
@@ -61,11 +64,14 @@ pub use pool::PoolEntity;
pub use pool_listing::PoolListingEntity; pub use pool_listing::PoolListingEntity;
pub use pool_origin::PoolOriginEntity; pub use pool_origin::PoolOriginEntity;
pub use pool_token::PoolTokenEntity; pub use pool_token::PoolTokenEntity;
pub use protocol_candidate::ProtocolCandidateEntity;
pub use protocol_candidate_summary::ProtocolCandidateSummaryEntity;
pub use swap::SwapEntity; pub use swap::SwapEntity;
pub use token::TokenEntity; pub use token::TokenEntity;
pub use token_burn_event::TokenBurnEventEntity; pub use token_burn_event::TokenBurnEventEntity;
pub use token_mint_event::TokenMintEventEntity; pub use token_mint_event::TokenMintEventEntity;
pub use trade_event::TradeEventEntity; pub use trade_event::TradeEventEntity;
pub use transaction_classification::TransactionClassificationEntity;
pub use wallet::WalletEntity; pub use wallet::WalletEntity;
pub use wallet_holding::WalletHoldingEntity; pub use wallet_holding::WalletHoldingEntity;
pub use wallet_participation::WalletParticipationEntity; pub use wallet_participation::WalletParticipationEntity;

32
kb_lib/src/db/entities/protocol_candidate.rs Normal file
View File

@@ -0,0 +1,32 @@
// file: kb_lib/src/db/entities/protocol_candidate.rs
//! Protocol candidate entity.
/// Persisted protocol candidate row.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, sqlx::FromRow)]
pub struct ProtocolCandidateEntity {
/// Numeric primary key.
pub id: i64,
/// Related chain transaction id.
pub transaction_id: i64,
/// Optional related chain instruction id.
pub instruction_id: std::option::Option<i64>,
/// Transaction signature.
pub signature: std::string::String,
/// Optional Solana slot.
pub slot: std::option::Option<i64>,
/// Program id observed in the transaction instruction.
pub program_id: std::string::String,
/// Optional program name hint from parsed transaction data.
pub program_name_hint: std::option::Option<std::string::String>,
/// Optional candidate protocol code.
pub candidate_protocol: std::option::Option<std::string::String>,
/// Optional candidate surface code.
pub candidate_surface: std::option::Option<std::string::String>,
/// Human-readable reason.
pub reason: std::string::String,
/// Serialized JSON evidence.
pub evidence_json: std::string::String,
/// Creation timestamp encoded as RFC3339 UTC text.
pub created_at: std::string::String,
}

30
kb_lib/src/db/entities/protocol_candidate_summary.rs Normal file
View File

@@ -0,0 +1,30 @@
// file: kb_lib/src/db/entities/protocol_candidate_summary.rs
//! Protocol candidate summary entity.
/// Aggregated protocol candidate diagnostic row.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, sqlx::FromRow)]
pub struct ProtocolCandidateSummaryEntity {
/// Program id observed in protocol candidates.
pub program_id: std::string::String,
/// Optional program name hint.
pub program_name_hint: std::option::Option<std::string::String>,
/// Optional candidate protocol.
pub candidate_protocol: std::option::Option<std::string::String>,
/// Optional candidate surface.
pub candidate_surface: std::option::Option<std::string::String>,
/// Candidate reason.
pub reason: std::string::String,
/// Number of candidate rows.
pub occurrence_count: i64,
/// Number of distinct transactions.
pub transaction_count: i64,
/// Latest observed slot.
pub last_slot: std::option::Option<i64>,
/// Latest candidate row id in this group.
pub latest_candidate_id: i64,
/// Latest signature in this group.
pub latest_signature: std::string::String,
/// Latest candidate creation timestamp encoded as RFC3339 UTC text.
pub latest_created_at: std::string::String,
}

32
kb_lib/src/db/entities/transaction_classification.rs Normal file
View File

@@ -0,0 +1,32 @@
// file: kb_lib/src/db/entities/transaction_classification.rs
//! Transaction classification entity.
/// Persisted transaction classification row.
#[derive(Debug, Clone, serde::Serialize, serde::Deserialize, sqlx::FromRow)]
pub struct TransactionClassificationEntity {
/// Numeric primary key.
pub id: i64,
/// Related chain transaction id.
pub transaction_id: i64,
/// Transaction signature.
pub signature: std::string::String,
/// Optional Solana slot.
pub slot: std::option::Option<i64>,
/// Stable classification kind.
pub classification_kind: std::string::String,
/// Optional primary protocol name.
pub primary_protocol: std::option::Option<std::string::String>,
/// Optional primary program id.
pub primary_program_id: std::option::Option<std::string::String>,
/// Confidence level from 0 to 100.
pub confidence_level: i64,
/// Human-readable reason.
pub reason: std::string::String,
/// Serialized JSON evidence.
pub evidence_json: std::string::String,
/// Creation timestamp encoded as RFC3339 UTC text.
pub created_at: std::string::String,
/// Update timestamp encoded as RFC3339 UTC text.
pub updated_at: std::string::String,
}

12
kb_lib/src/db/queries.rs
View File

@@ -27,11 +27,13 @@ mod pool;
mod pool_listing; mod pool_listing;
mod pool_origin; mod pool_origin;
mod pool_token; mod pool_token;
mod protocol_candidate;
mod swap; mod swap;
mod token; mod token;
mod token_burn_event; mod token_burn_event;
mod token_mint_event; mod token_mint_event;
mod trade_event; mod trade_event;
mod transaction_classification;
mod wallet; mod wallet;
mod wallet_holding; mod wallet_holding;
mod wallet_participation; mod wallet_participation;
@@ -118,6 +120,12 @@ pub use pool_origin::query_pool_origins_list;
pub use pool_origin::query_pool_origins_upsert; pub use pool_origin::query_pool_origins_upsert;
pub use pool_token::query_pool_tokens_list_by_pool_id; pub use pool_token::query_pool_tokens_list_by_pool_id;
pub use pool_token::query_pool_tokens_upsert; pub use pool_token::query_pool_tokens_upsert;
pub use protocol_candidate::query_protocol_candidate_summaries_list_by_priority;
pub use protocol_candidate::query_protocol_candidates_delete_by_transaction_id;
pub use protocol_candidate::query_protocol_candidates_insert;
pub use protocol_candidate::query_protocol_candidates_list_by_program_id;
pub use protocol_candidate::query_protocol_candidates_list_by_transaction_id;
pub use protocol_candidate::query_protocol_candidates_list_recent;
pub use swap::query_swaps_list_recent; pub use swap::query_swaps_list_recent;
pub use swap::query_swaps_upsert; pub use swap::query_swaps_upsert;
pub use token::query_tokens_get_by_id; pub use token::query_tokens_get_by_id;
@@ -133,6 +141,10 @@ pub use trade_event::query_trade_events_get_by_decoded_event_id;
pub use trade_event::query_trade_events_list_by_pair_id; pub use trade_event::query_trade_events_list_by_pair_id;
pub use trade_event::query_trade_events_list_by_transaction_id; pub use trade_event::query_trade_events_list_by_transaction_id;
pub use trade_event::query_trade_events_upsert; pub use trade_event::query_trade_events_upsert;
pub use transaction_classification::query_transaction_classifications_get_by_signature;
pub use transaction_classification::query_transaction_classifications_get_by_transaction_id;
pub use transaction_classification::query_transaction_classifications_list_recent;
pub use transaction_classification::query_transaction_classifications_upsert;
pub use wallet::query_wallets_get_by_address; pub use wallet::query_wallets_get_by_address;
pub use wallet::query_wallets_list; pub use wallet::query_wallets_list;
pub use wallet::query_wallets_upsert; pub use wallet::query_wallets_upsert;

337
kb_lib/src/db/queries/protocol_candidate.rs Normal file
View File

@@ -0,0 +1,337 @@
// file: kb_lib/src/db/queries/protocol_candidate.rs
//! Queries for `k_sol_protocol_candidates`.
/// Inserts one protocol candidate row.
pub async fn query_protocol_candidates_insert(
database: &crate::Database,
dto: &crate::ProtocolCandidateDto,
) -> Result<i64, crate::Error> {
let slot_i64 = match dto.slot {
Some(slot) => {
let slot_result = i64::try_from(slot);
match slot_result {
Ok(slot) => Some(slot),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert protocol candidate slot '{}' to i64: {}",
slot, error
)));
},
}
},
None => None,
};
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query(
r#"
INSERT INTO k_sol_protocol_candidates (
transaction_id,
instruction_id,
signature,
slot,
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
evidence_json,
created_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
"#,
)
.bind(dto.transaction_id)
.bind(dto.instruction_id)
.bind(dto.signature.clone())
.bind(slot_i64)
.bind(dto.program_id.clone())
.bind(dto.program_name_hint.clone())
.bind(dto.candidate_protocol.clone())
.bind(dto.candidate_surface.clone())
.bind(dto.reason.clone())
.bind(dto.evidence_json.clone())
.bind(dto.created_at.to_rfc3339())
.execute(pool)
.await;
match query_result {
Ok(query_result) => return Ok(query_result.last_insert_rowid()),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot insert k_sol_protocol_candidates on sqlite: {}",
error
)));
},
}
},
}
}
/// Deletes protocol candidates for one transaction.
///
/// This is useful before recomputing candidates for a replayed transaction.
pub async fn query_protocol_candidates_delete_by_transaction_id(
database: &crate::Database,
transaction_id: i64,
) -> Result<u64, crate::Error> {
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query(
r#"
DELETE FROM k_sol_protocol_candidates
WHERE transaction_id = ?
"#,
)
.bind(transaction_id)
.execute(pool)
.await;
match query_result {
Ok(query_result) => return Ok(query_result.rows_affected()),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot delete k_sol_protocol_candidates for transaction_id '{}' on sqlite: {}",
transaction_id, error
)));
},
}
},
}
}
/// Lists protocol candidates for one transaction.
pub async fn query_protocol_candidates_list_by_transaction_id(
database: &crate::Database,
transaction_id: i64,
) -> Result<std::vec::Vec<crate::ProtocolCandidateDto>, crate::Error> {
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query_as::<sqlx::Sqlite, crate::ProtocolCandidateEntity>(
r#"
SELECT
id,
transaction_id,
instruction_id,
signature,
slot,
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
evidence_json,
created_at
FROM k_sol_protocol_candidates
WHERE transaction_id = ?
ORDER BY id ASC
"#,
)
.bind(transaction_id)
.fetch_all(pool)
.await;
let entities = match query_result {
Ok(entities) => entities,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot list k_sol_protocol_candidates for transaction_id '{}' on sqlite: {}",
transaction_id, error
)));
},
};
return protocol_candidate_entities_to_dtos(entities);
},
}
}
/// Lists protocol candidates for one program id.
pub async fn query_protocol_candidates_list_by_program_id(
database: &crate::Database,
program_id: &str,
limit: u32,
) -> Result<std::vec::Vec<crate::ProtocolCandidateDto>, crate::Error> {
if limit == 0 {
return Ok(std::vec::Vec::new());
}
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query_as::<sqlx::Sqlite, crate::ProtocolCandidateEntity>(
r#"
SELECT
id,
transaction_id,
instruction_id,
signature,
slot,
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
evidence_json,
created_at
FROM k_sol_protocol_candidates
WHERE program_id = ?
ORDER BY id DESC
LIMIT ?
"#,
)
.bind(program_id.to_string())
.bind(i64::from(limit))
.fetch_all(pool)
.await;
let entities = match query_result {
Ok(entities) => entities,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot list k_sol_protocol_candidates for program_id '{}' on sqlite: {}",
program_id, error
)));
},
};
return protocol_candidate_entities_to_dtos(entities);
},
}
}
/// Lists recent protocol candidates ordered from newest to oldest.
pub async fn query_protocol_candidates_list_recent(
database: &crate::Database,
limit: u32,
) -> Result<std::vec::Vec<crate::ProtocolCandidateDto>, crate::Error> {
if limit == 0 {
return Ok(std::vec::Vec::new());
}
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query_as::<sqlx::Sqlite, crate::ProtocolCandidateEntity>(
r#"
SELECT
id,
transaction_id,
instruction_id,
signature,
slot,
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
evidence_json,
created_at
FROM k_sol_protocol_candidates
ORDER BY id DESC
LIMIT ?
"#,
)
.bind(i64::from(limit))
.fetch_all(pool)
.await;
let entities = match query_result {
Ok(entities) => entities,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot list recent k_sol_protocol_candidates on sqlite: {}",
error
)));
},
};
return protocol_candidate_entities_to_dtos(entities);
},
}
}
fn protocol_candidate_entities_to_dtos(
entities: std::vec::Vec<crate::ProtocolCandidateEntity>,
) -> Result<std::vec::Vec<crate::ProtocolCandidateDto>, crate::Error> {
let mut dtos = std::vec::Vec::new();
for entity in entities {
let dto_result = crate::ProtocolCandidateDto::try_from(entity);
let dto = match dto_result {
Ok(dto) => dto,
Err(error) => return Err(error),
};
dtos.push(dto);
}
return Ok(dtos);
}
/// Lists protocol candidate summaries ordered by investigation priority.
pub async fn query_protocol_candidate_summaries_list_by_priority(
database: &crate::Database,
limit: u32,
) -> Result<std::vec::Vec<crate::ProtocolCandidateSummaryDto>, crate::Error> {
if limit == 0 {
return Ok(std::vec::Vec::new());
}
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result =
sqlx::query_as::<sqlx::Sqlite, crate::ProtocolCandidateSummaryEntity>(
r#"
WITH grouped AS (
SELECT
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason,
COUNT(*) AS occurrence_count,
COUNT(DISTINCT signature) AS transaction_count,
MAX(slot) AS last_slot,
MAX(id) AS latest_candidate_id
FROM k_sol_protocol_candidates
GROUP BY
program_id,
program_name_hint,
candidate_protocol,
candidate_surface,
reason
)
SELECT
grouped.program_id,
grouped.program_name_hint,
grouped.candidate_protocol,
grouped.candidate_surface,
grouped.reason,
grouped.occurrence_count,
grouped.transaction_count,
grouped.last_slot,
grouped.latest_candidate_id,
latest.signature AS latest_signature,
latest.created_at AS latest_created_at
FROM grouped
JOIN k_sol_protocol_candidates latest
ON latest.id = grouped.latest_candidate_id
ORDER BY
grouped.transaction_count DESC,
grouped.occurrence_count DESC,
grouped.last_slot DESC,
grouped.latest_candidate_id DESC
LIMIT ?
"#,
)
.bind(i64::from(limit))
.fetch_all(pool)
.await;
let entities = match query_result {
Ok(entities) => entities,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot list k_sol_protocol_candidates summaries on sqlite: {}",
error
)));
},
};
let mut dtos = std::vec::Vec::new();
for entity in entities {
let dto_result = crate::ProtocolCandidateSummaryDto::try_from(entity);
let dto = match dto_result {
Ok(dto) => dto,
Err(error) => return Err(error),
};
dtos.push(dto);
}
return Ok(dtos);
},
}
}

263
kb_lib/src/db/queries/transaction_classification.rs Normal file
View File

@@ -0,0 +1,263 @@
// file: kb_lib/src/db/queries/transaction_classification.rs
//! Queries for `k_sol_transaction_classifications`.
/// Inserts or updates one transaction classification row.
pub async fn query_transaction_classifications_upsert(
database: &crate::Database,
dto: &crate::TransactionClassificationDto,
) -> Result<i64, crate::Error> {
let slot_i64 = match dto.slot {
Some(slot) => {
let slot_result = i64::try_from(slot);
match slot_result {
Ok(slot) => Some(slot),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot convert transaction classification slot '{}' to i64: {}",
slot, error
)));
},
}
},
None => None,
};
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result = sqlx::query(
r#"
INSERT INTO k_sol_transaction_classifications (
transaction_id,
signature,
slot,
classification_kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
created_at,
updated_at
)
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
ON CONFLICT(transaction_id) DO UPDATE SET
signature = excluded.signature,
slot = excluded.slot,
classification_kind = excluded.classification_kind,
primary_protocol = excluded.primary_protocol,
primary_program_id = excluded.primary_program_id,
confidence_level = excluded.confidence_level,
reason = excluded.reason,
evidence_json = excluded.evidence_json,
updated_at = excluded.updated_at
"#,
)
.bind(dto.transaction_id)
.bind(dto.signature.clone())
.bind(slot_i64)
.bind(dto.classification_kind.clone())
.bind(dto.primary_protocol.clone())
.bind(dto.primary_program_id.clone())
.bind(i64::from(dto.confidence_level))
.bind(dto.reason.clone())
.bind(dto.evidence_json.clone())
.bind(dto.created_at.to_rfc3339())
.bind(dto.updated_at.to_rfc3339())
.execute(pool)
.await;
if let Err(error) = query_result {
return Err(crate::Error::Db(format!(
"cannot upsert k_sol_transaction_classifications on sqlite: {}",
error
)));
}
let id_result = sqlx::query_scalar::<sqlx::Sqlite, i64>(
r#"
SELECT id
FROM k_sol_transaction_classifications
WHERE transaction_id = ?
LIMIT 1
"#,
)
.bind(dto.transaction_id)
.fetch_one(pool)
.await;
match id_result {
Ok(id) => return Ok(id),
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot fetch k_sol_transaction_classifications id for transaction_id '{}' on sqlite: {}",
dto.transaction_id, error
)));
},
}
},
}
}
/// Reads one transaction classification by transaction id.
pub async fn query_transaction_classifications_get_by_transaction_id(
database: &crate::Database,
transaction_id: i64,
) -> Result<std::option::Option<crate::TransactionClassificationDto>, crate::Error> {
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result =
sqlx::query_as::<sqlx::Sqlite, crate::TransactionClassificationEntity>(
r#"
SELECT
id,
transaction_id,
signature,
slot,
classification_kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
created_at,
updated_at
FROM k_sol_transaction_classifications
WHERE transaction_id = ?
LIMIT 1
"#,
)
.bind(transaction_id)
.fetch_optional(pool)
.await;
let entity_option = match query_result {
Ok(entity_option) => entity_option,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot fetch k_sol_transaction_classifications for transaction_id '{}' on sqlite: {}",
transaction_id, error
)));
},
};
match entity_option {
Some(entity) => {
let dto_result = crate::TransactionClassificationDto::try_from(entity);
match dto_result {
Ok(dto) => return Ok(Some(dto)),
Err(error) => return Err(error),
}
},
None => return Ok(None),
}
},
}
}
/// Reads one transaction classification by signature.
pub async fn query_transaction_classifications_get_by_signature(
database: &crate::Database,
signature: &str,
) -> Result<std::option::Option<crate::TransactionClassificationDto>, crate::Error> {
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result =
sqlx::query_as::<sqlx::Sqlite, crate::TransactionClassificationEntity>(
r#"
SELECT
id,
transaction_id,
signature,
slot,
classification_kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
created_at,
updated_at
FROM k_sol_transaction_classifications
WHERE signature = ?
LIMIT 1
"#,
)
.bind(signature.to_string())
.fetch_optional(pool)
.await;
let entity_option = match query_result {
Ok(entity_option) => entity_option,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot fetch k_sol_transaction_classifications for signature '{}' on sqlite: {}",
signature, error
)));
},
};
match entity_option {
Some(entity) => {
let dto_result = crate::TransactionClassificationDto::try_from(entity);
match dto_result {
Ok(dto) => return Ok(Some(dto)),
Err(error) => return Err(error),
}
},
None => return Ok(None),
}
},
}
}
/// Lists recent transaction classifications ordered from newest to oldest.
pub async fn query_transaction_classifications_list_recent(
database: &crate::Database,
limit: u32,
) -> Result<std::vec::Vec<crate::TransactionClassificationDto>, crate::Error> {
if limit == 0 {
return Ok(std::vec::Vec::new());
}
match database.connection() {
crate::DatabaseConnection::Sqlite(pool) => {
let query_result =
sqlx::query_as::<sqlx::Sqlite, crate::TransactionClassificationEntity>(
r#"
SELECT
id,
transaction_id,
signature,
slot,
classification_kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
created_at,
updated_at
FROM k_sol_transaction_classifications
ORDER BY id DESC
LIMIT ?
"#,
)
.bind(i64::from(limit))
.fetch_all(pool)
.await;
let entities = match query_result {
Ok(entities) => entities,
Err(error) => {
return Err(crate::Error::Db(format!(
"cannot list k_sol_transaction_classifications on sqlite: {}",
error
)));
},
};
let mut dtos = std::vec::Vec::new();
for entity in entities {
let dto_result = crate::TransactionClassificationDto::try_from(entity);
let dto = match dto_result {
Ok(dto) => dto,
Err(error) => return Err(error),
};
dtos.push(dto);
}
return Ok(dtos);
},
}
}

498
kb_lib/src/db/schema.rs
View File

@@ -230,6 +230,94 @@ pub(crate) async fn ensure_schema(database: &crate::Database) -> Result<(), crat
if let Err(error) = result { if let Err(error) = result {
return Err(error); return Err(error);
} }
let result = create_tbl_transaction_classifications(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_uix_transaction_classifications_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_transaction_classifications_kind(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_protocol_candidates(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_protocol_candidates_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_protocol_candidates_program_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_pool_lifecycle_events(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_pool_lifecycle_events_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_pool_lifecycle_events_pool_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_uix_pool_lifecycle_events_decoded_event_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_fee_events(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_fee_events_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_fee_events_pool_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_uix_fee_events_decoded_event_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_reward_events(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_reward_events_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_reward_events_pool_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_uix_reward_events_decoded_event_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_pool_admin_events(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_pool_admin_events_transaction_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_idx_pool_admin_events_pool_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_uix_pool_admin_events_decoded_event_id(pool).await;
if let Err(error) = result {
return Err(error);
}
let result = create_tbl_launch_surfaces(pool).await; let result = create_tbl_launch_surfaces(pool).await;
if let Err(error) = result { if let Err(error) = result {
return Err(error); return Err(error);
@@ -1878,3 +1966,413 @@ ON k_sol_pair_analytic_signals(pair_id)
) )
.await; .await;
} }
/// Creates `k_sol_transaction_classifications`.
async fn create_tbl_transaction_classifications(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_transaction_classifications",
r#"
CREATE TABLE IF NOT EXISTS k_sol_transaction_classifications (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
classification_kind TEXT NOT NULL,
primary_protocol TEXT NULL,
primary_program_id TEXT NULL,
confidence_level INTEGER NOT NULL,
reason TEXT NOT NULL,
evidence_json TEXT NOT NULL,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates unique index on `k_sol_transaction_classifications(transaction_id)`.
async fn create_uix_transaction_classifications_transaction_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_uix_transaction_classifications_transaction_id",
r#"
CREATE UNIQUE INDEX IF NOT EXISTS uix_transaction_classifications_transaction_id
ON k_sol_transaction_classifications (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_transaction_classifications(classification_kind)`.
async fn create_idx_transaction_classifications_kind(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_transaction_classifications_kind",
r#"
CREATE INDEX IF NOT EXISTS idx_transaction_classifications_kind
ON k_sol_transaction_classifications (classification_kind)
"#,
)
.await;
}
/// Creates `k_sol_protocol_candidates`.
async fn create_tbl_protocol_candidates(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_protocol_candidates",
r#"
CREATE TABLE IF NOT EXISTS k_sol_protocol_candidates (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
instruction_id INTEGER NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
program_id TEXT NOT NULL,
program_name_hint TEXT NULL,
candidate_protocol TEXT NULL,
candidate_surface TEXT NULL,
reason TEXT NOT NULL,
evidence_json TEXT NOT NULL,
created_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates index on `k_sol_protocol_candidates(transaction_id)`.
async fn create_idx_protocol_candidates_transaction_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_protocol_candidates_transaction_id",
r#"
CREATE INDEX IF NOT EXISTS idx_protocol_candidates_transaction_id
ON k_sol_protocol_candidates (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_protocol_candidates(program_id)`.
async fn create_idx_protocol_candidates_program_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_protocol_candidates_program_id",
r#"
CREATE INDEX IF NOT EXISTS idx_protocol_candidates_program_id
ON k_sol_protocol_candidates (program_id)
"#,
)
.await;
}
/// Creates `k_sol_pool_lifecycle_events`.
async fn create_tbl_pool_lifecycle_events(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_pool_lifecycle_events",
r#"
CREATE TABLE IF NOT EXISTS k_sol_pool_lifecycle_events (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
decoded_event_id INTEGER NULL,
dex_id INTEGER NULL,
pool_id INTEGER NULL,
pair_id INTEGER NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
protocol_name TEXT NOT NULL,
program_id TEXT NOT NULL,
event_kind TEXT NOT NULL,
pool_account TEXT NULL,
token_a_mint TEXT NULL,
token_b_mint TEXT NULL,
payload_json TEXT NOT NULL,
executed_at TEXT NOT NULL,
created_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates index on `k_sol_pool_lifecycle_events(transaction_id)`.
async fn create_idx_pool_lifecycle_events_transaction_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_pool_lifecycle_events_transaction_id",
r#"
CREATE INDEX IF NOT EXISTS idx_pool_lifecycle_events_transaction_id
ON k_sol_pool_lifecycle_events (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_pool_lifecycle_events(pool_id)`.
async fn create_idx_pool_lifecycle_events_pool_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_pool_lifecycle_events_pool_id",
r#"
CREATE INDEX IF NOT EXISTS idx_pool_lifecycle_events_pool_id
ON k_sol_pool_lifecycle_events (pool_id)
"#,
)
.await;
}
/// Creates partial unique index on `k_sol_pool_lifecycle_events(decoded_event_id)`.
async fn create_uix_pool_lifecycle_events_decoded_event_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_uix_pool_lifecycle_events_decoded_event_id",
r#"
CREATE UNIQUE INDEX IF NOT EXISTS uix_pool_lifecycle_events_decoded_event_id
ON k_sol_pool_lifecycle_events (decoded_event_id)
WHERE decoded_event_id IS NOT NULL
"#,
)
.await;
}
/// Creates `k_sol_fee_events`.
async fn create_tbl_fee_events(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_fee_events",
r#"
CREATE TABLE IF NOT EXISTS k_sol_fee_events (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
decoded_event_id INTEGER NULL,
dex_id INTEGER NULL,
pool_id INTEGER NULL,
pair_id INTEGER NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
protocol_name TEXT NOT NULL,
program_id TEXT NOT NULL,
event_kind TEXT NOT NULL,
pool_account TEXT NULL,
actor_wallet TEXT NULL,
fee_token_mint TEXT NULL,
fee_amount_raw TEXT NULL,
payload_json TEXT NOT NULL,
executed_at TEXT NOT NULL,
created_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates index on `k_sol_fee_events(transaction_id)`.
async fn create_idx_fee_events_transaction_id(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_fee_events_transaction_id",
r#"
CREATE INDEX IF NOT EXISTS idx_fee_events_transaction_id
ON k_sol_fee_events (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_fee_events(pool_id)`.
async fn create_idx_fee_events_pool_id(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_fee_events_pool_id",
r#"
CREATE INDEX IF NOT EXISTS idx_fee_events_pool_id
ON k_sol_fee_events (pool_id)
"#,
)
.await;
}
/// Creates partial unique index on `k_sol_fee_events(decoded_event_id)`.
async fn create_uix_fee_events_decoded_event_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_uix_fee_events_decoded_event_id",
r#"
CREATE UNIQUE INDEX IF NOT EXISTS uix_fee_events_decoded_event_id
ON k_sol_fee_events (decoded_event_id)
WHERE decoded_event_id IS NOT NULL
"#,
)
.await;
}
/// Creates `k_sol_reward_events`.
async fn create_tbl_reward_events(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_reward_events",
r#"
CREATE TABLE IF NOT EXISTS k_sol_reward_events (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
decoded_event_id INTEGER NULL,
dex_id INTEGER NULL,
pool_id INTEGER NULL,
pair_id INTEGER NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
protocol_name TEXT NOT NULL,
program_id TEXT NOT NULL,
event_kind TEXT NOT NULL,
pool_account TEXT NULL,
actor_wallet TEXT NULL,
reward_token_mint TEXT NULL,
reward_amount_raw TEXT NULL,
payload_json TEXT NOT NULL,
executed_at TEXT NOT NULL,
created_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates index on `k_sol_reward_events(transaction_id)`.
async fn create_idx_reward_events_transaction_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_reward_events_transaction_id",
r#"
CREATE INDEX IF NOT EXISTS idx_reward_events_transaction_id
ON k_sol_reward_events (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_reward_events(pool_id)`.
async fn create_idx_reward_events_pool_id(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_reward_events_pool_id",
r#"
CREATE INDEX IF NOT EXISTS idx_reward_events_pool_id
ON k_sol_reward_events (pool_id)
"#,
)
.await;
}
/// Creates partial unique index on `k_sol_reward_events(decoded_event_id)`.
async fn create_uix_reward_events_decoded_event_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_uix_reward_events_decoded_event_id",
r#"
CREATE UNIQUE INDEX IF NOT EXISTS uix_reward_events_decoded_event_id
ON k_sol_reward_events (decoded_event_id)
WHERE decoded_event_id IS NOT NULL
"#,
)
.await;
}
/// Creates `k_sol_pool_admin_events`.
async fn create_tbl_pool_admin_events(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_tbl_pool_admin_events",
r#"
CREATE TABLE IF NOT EXISTS k_sol_pool_admin_events (
id INTEGER NOT NULL PRIMARY KEY AUTOINCREMENT,
transaction_id INTEGER NOT NULL,
decoded_event_id INTEGER NULL,
dex_id INTEGER NULL,
pool_id INTEGER NULL,
pair_id INTEGER NULL,
signature TEXT NOT NULL,
slot INTEGER NULL,
protocol_name TEXT NOT NULL,
program_id TEXT NOT NULL,
event_kind TEXT NOT NULL,
pool_account TEXT NULL,
actor_wallet TEXT NULL,
admin_action TEXT NULL,
payload_json TEXT NOT NULL,
executed_at TEXT NOT NULL,
created_at TEXT NOT NULL
)
"#,
)
.await;
}
/// Creates index on `k_sol_pool_admin_events(transaction_id)`.
async fn create_idx_pool_admin_events_transaction_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_pool_admin_events_transaction_id",
r#"
CREATE INDEX IF NOT EXISTS idx_pool_admin_events_transaction_id
ON k_sol_pool_admin_events (transaction_id)
"#,
)
.await;
}
/// Creates index on `k_sol_pool_admin_events(pool_id)`.
async fn create_idx_pool_admin_events_pool_id(pool: &sqlx::SqlitePool) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_idx_pool_admin_events_pool_id",
r#"
CREATE INDEX IF NOT EXISTS idx_pool_admin_events_pool_id
ON k_sol_pool_admin_events (pool_id)
"#,
)
.await;
}
/// Creates partial unique index on `k_sol_pool_admin_events(decoded_event_id)`.
async fn create_uix_pool_admin_events_decoded_event_id(
pool: &sqlx::SqlitePool,
) -> Result<(), crate::Error> {
return execute_sqlite_schema_statement(
pool,
"create_uix_pool_admin_events_decoded_event_id",
r#"
CREATE UNIQUE INDEX IF NOT EXISTS uix_pool_admin_events_decoded_event_id
ON k_sol_pool_admin_events (decoded_event_id)
WHERE decoded_event_id IS NOT NULL
"#,
)
.await;
}

18
kb_lib/src/detect/solana_ws.rs
View File

@@ -155,8 +155,8 @@ impl SolanaWsDetectionService {
Some(token_program) => token_program, Some(token_program) => token_program,
None => return Ok(None), None => return Ok(None),
}; };
if token_program != crate::SPL_TOKEN_PROGRAM_ID.to_string() if token_program.as_str() != crate::SPL_TOKEN_PROGRAM_ID
&& token_program != crate::SPL_TOKEN_2022_PROGRAM_ID.to_string() && token_program.as_str() != crate::SPL_TOKEN_2022_PROGRAM_ID
{ {
return Ok(None); return Ok(None);
} }
@@ -181,7 +181,7 @@ impl SolanaWsDetectionService {
let slot = let slot =
extract_slot_from_result(notification.method.as_str(), &notification.params.result); extract_slot_from_result(notification.method.as_str(), &notification.params.result);
let payload = build_notification_payload(notification); let payload = build_notification_payload(notification);
let is_quote_token = mint == crate::WSOL_MINT_ID.to_string(); let is_quote_token = mint.as_str() == crate::WSOL_MINT_ID;
let input = crate::DetectionTokenCandidateInput::new( let input = crate::DetectionTokenCandidateInput::new(
mint, mint,
None, None,
@@ -230,8 +230,8 @@ impl SolanaWsDetectionService {
Some(owner) => owner, Some(owner) => owner,
None => return Ok(None), None => return Ok(None),
}; };
if owner == crate::SPL_TOKEN_PROGRAM_ID.to_string() if owner.as_str() == crate::SPL_TOKEN_PROGRAM_ID
|| owner == crate::SPL_TOKEN_2022_PROGRAM_ID.to_string() || owner.as_str() == crate::SPL_TOKEN_2022_PROGRAM_ID
{ {
return Ok(None); return Ok(None);
} }
@@ -603,10 +603,10 @@ fn build_signal_kind_for_notification(
} }
let owner_option = extract_account_owner(account_value); let owner_option = extract_account_owner(account_value);
if let Some(owner) = owner_option { if let Some(owner) = owner_option {
if owner == crate::SPL_TOKEN_PROGRAM_ID.to_string() { if owner.as_str() == crate::SPL_TOKEN_PROGRAM_ID {
return "signal.account_notification.spl_token".to_string(); return "signal.account_notification.spl_token".to_string();
} }
if owner == crate::SPL_TOKEN_2022_PROGRAM_ID.to_string() { if owner.as_str() == crate::SPL_TOKEN_2022_PROGRAM_ID {
return "signal.account_notification.spl_token_2022".to_string(); return "signal.account_notification.spl_token_2022".to_string();
} }
} }
@@ -650,10 +650,10 @@ fn build_signal_kind_for_notification(
Some(owner) => owner, Some(owner) => owner,
None => return "signal.program_notification.generic".to_string(), None => return "signal.program_notification.generic".to_string(),
}; };
if owner == crate::SPL_TOKEN_PROGRAM_ID.to_string() { if owner.as_str() == crate::SPL_TOKEN_PROGRAM_ID {
return "signal.program_notification.spl_token".to_string(); return "signal.program_notification.spl_token".to_string();
} }
if owner == crate::SPL_TOKEN_2022_PROGRAM_ID.to_string() { if owner.as_str() == crate::SPL_TOKEN_2022_PROGRAM_ID {
return "signal.program_notification.spl_token_2022".to_string(); return "signal.program_notification.spl_token_2022".to_string();
} }
return "signal.program_notification.generic".to_string(); return "signal.program_notification.generic".to_string();

295
kb_lib/src/dex_catalog.rs Normal file
View File

@@ -0,0 +1,295 @@
// file: kb_lib/src/dex_catalog.rs
//! Internal DEX catalog and persistence helpers.
//!
//! This module centralizes known DEX metadata used by detection,
//! decoding and future launch-surface integrations.
//!
//! It does not decode instructions and does not classify events.
/// Static metadata for one known DEX entry.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub(crate) struct DexCatalogItem {
/// Stable internal DEX code.
pub(crate) code: &'static str,
/// Human-readable DEX name.
pub(crate) name: &'static str,
/// Main Solana program id.
pub(crate) program_id: std::option::Option<&'static str>,
/// Optional router program id.
pub(crate) router_program_id: std::option::Option<&'static str>,
/// Whether this DEX is currently enabled for detection.
pub(crate) is_enabled: bool,
}
/// Returns metadata for one known DEX or launch-backed swap surface.
pub(crate) fn dex_catalog_item_by_code(
code: &str,
) -> std::option::Option<crate::dex_catalog::DexCatalogItem> {
match code {
"raydium" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "raydium",
name: "Raydium AMM v4",
program_id: Some(crate::RAYDIUM_AMM_V4_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"raydium_cpmm" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "raydium_cpmm",
name: "Raydium CPMM",
program_id: Some(crate::RAYDIUM_CPMM_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"raydium_clmm" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "raydium_clmm",
name: "Raydium CLMM",
program_id: Some(crate::RAYDIUM_CLMM_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"pump_fun" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "pump_fun",
name: "Pump.fun",
program_id: Some(crate::PUMP_FUN_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"pump_swap" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "pump_swap",
name: "PumpSwap",
program_id: Some(crate::PUMP_SWAP_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"meteora_dbc" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "meteora_dbc",
name: "Meteora DBC",
program_id: Some(crate::METEORA_DBC_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"meteora_damm_v1" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "meteora_damm_v1",
name: "Meteora DAMM v1",
program_id: Some(crate::METEORA_DAMM_V1_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"meteora_damm_v2" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "meteora_damm_v2",
name: "Meteora DAMM v2",
program_id: Some(crate::METEORA_DAMM_V2_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"orca_whirlpools" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "orca_whirlpools",
name: "Orca Whirlpools",
program_id: Some(crate::ORCA_WHIRLPOOLS_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"fluxbeam" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "fluxbeam",
name: "FluxBeam",
program_id: Some(crate::FLUXBEAM_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
"dexlab" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "dexlab",
name: "DexLab Swap/Pool",
program_id: Some(crate::DEXLAB_PROGRAM_ID),
router_program_id: None,
is_enabled: true,
});
},
// Planned launch/swap surfaces.
//
// These entries are intentionally present before decoder support so that
// the roadmap can evolve without duplicating DEX metadata later.
//
// Program ids should be filled only after validation against live
// transactions and official or otherwise trustworthy references.
"raydium_launchlab" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "raydium_launchlab",
name: "Raydium LaunchLab",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
"letsbonk" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "letsbonk",
name: "LetsBonk / Bonk.fun",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
"boop_fun" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "boop_fun",
name: "Boop.fun",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
"moonshot" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "moonshot",
name: "Moonshot",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
"believe" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "believe",
name: "Believe",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
"heaven" => {
return Some(crate::dex_catalog::DexCatalogItem {
code: "heaven",
name: "Heaven",
program_id: None,
router_program_id: None,
is_enabled: false,
});
},
_ => return None,
}
}
/// Ensures that one known DEX exists in storage and returns its internal id.
pub(crate) async fn ensure_known_dex(
database: &crate::Database,
code: &str,
) -> Result<i64, crate::Error> {
let dex_result = crate::query_dexs_get_by_code(database, code).await;
let dex_option = match dex_result {
Ok(dex_option) => dex_option,
Err(error) => return Err(error),
};
match dex_option {
Some(dex) => match dex.id {
Some(dex_id) => return Ok(dex_id),
None => {
return Err(crate::Error::InvalidState(format!("{} dex has no internal id", code)));
},
},
None => {
let catalog_item_option = crate::dex_catalog::dex_catalog_item_by_code(code);
let catalog_item = match catalog_item_option {
Some(catalog_item) => catalog_item,
None => {
return Err(crate::Error::InvalidState(format!(
"unknown dex catalog code '{}'",
code
)));
},
};
let program_id = match catalog_item.program_id {
Some(program_id) => Some(program_id.to_string()),
None => None,
};
let router_program_id = match catalog_item.router_program_id {
Some(router_program_id) => Some(router_program_id.to_string()),
None => None,
};
let dex_dto = crate::DexDto::new(
catalog_item.code.to_string(),
catalog_item.name.to_string(),
program_id,
router_program_id,
catalog_item.is_enabled,
);
return crate::query_dexs_upsert(database, &dex_dto).await;
},
}
}
#[cfg(test)]
mod tests {
#[test]
fn known_active_dexes_are_available_from_catalog() {
let codes = [
"raydium",
"raydium_cpmm",
"raydium_clmm",
"pump_fun",
"pump_swap",
"meteora_dbc",
"meteora_damm_v1",
"meteora_damm_v2",
"orca_whirlpools",
"fluxbeam",
"dexlab",
];
for code in codes {
let item_option = crate::dex_catalog::dex_catalog_item_by_code(code);
let item = match item_option {
Some(item) => item,
None => {
panic!("expected known dex catalog item for '{}'", code);
},
};
assert_eq!(item.code, code);
assert!(item.is_enabled);
assert!(item.program_id.is_some());
}
}
#[test]
fn planned_launch_surfaces_are_present_but_disabled() {
let codes = ["raydium_launchlab", "letsbonk", "boop_fun", "moonshot", "believe", "heaven"];
for code in codes {
let item_option = crate::dex_catalog::dex_catalog_item_by_code(code);
let item = match item_option {
Some(item) => item,
None => {
panic!("expected planned launch surface catalog item for '{}'", code);
},
};
assert_eq!(item.code, code);
assert!(!item.is_enabled);
}
}
#[test]
fn unknown_dex_code_is_not_silently_accepted() {
let item_option = crate::dex_catalog::dex_catalog_item_by_code("zora_solana");
assert!(item_option.is_none());
}
}

2586
kb_lib/src/dex_decode.rs
View File

File diff suppressed because it is too large Load Diff

55
kb_lib/src/dex_decode_context.rs Normal file
View File

@@ -0,0 +1,55 @@
// file: kb_lib/src/dex_decode_context.rs
//! Transaction context loading for DEX decoding.
//!
//! This module loads the persisted transaction and projected instructions
//! required by `DexDecodeService`.
/// Transaction context required by DEX decoding.
pub(crate) struct DexDecodeTransactionContext {
/// Persisted chain transaction.
pub(crate) transaction: crate::ChainTransactionDto,
/// Projected transaction instructions.
pub(crate) instructions: std::vec::Vec<crate::ChainInstructionDto>,
}
/// Loads the transaction and its projected instructions for DEX decoding.
pub(crate) async fn load_dex_decode_transaction_context(
database: &crate::Database,
signature: &str,
) -> Result<crate::dex_decode_context::DexDecodeTransactionContext, crate::Error> {
let transaction_result =
crate::query_chain_transactions_get_by_signature(database, signature).await;
let transaction_option = match transaction_result {
Ok(transaction_option) => transaction_option,
Err(error) => return Err(error),
};
let transaction = match transaction_option {
Some(transaction) => transaction,
None => {
return Err(crate::Error::InvalidState(format!(
"cannot decode unknown chain transaction '{}'",
signature
)));
},
};
let transaction_id = match transaction.id {
Some(transaction_id) => transaction_id,
None => {
return Err(crate::Error::InvalidState(format!(
"chain transaction '{}' has no internal id",
signature
)));
},
};
let instructions_result =
crate::query_chain_instructions_list_by_transaction_id(database, transaction_id).await;
let instructions = match instructions_result {
Ok(instructions) => instructions,
Err(error) => return Err(error),
};
return Ok(crate::dex_decode_context::DexDecodeTransactionContext {
transaction,
instructions,
});
}

140
kb_lib/src/dex_decoded_event_materialization.rs Normal file
View File

@@ -0,0 +1,140 @@
// file: kb_lib/src/dex_decoded_event_materialization.rs
//! Decoded DEX event materialization helpers.
//!
//! This module centralizes persistence of decoded DEX events:
//! payload enrichment, upsert, fetch-after-upsert, observation recording
//! and signal recording.
/// Input required to persist one decoded DEX event.
pub(crate) struct DexDecodedEventMaterializationInput<'a> {
/// Database connection.
pub(crate) database: &'a crate::Database,
/// Detection persistence service.
pub(crate) persistence: &'a crate::DetectionPersistenceService,
/// Parent transaction.
pub(crate) transaction: &'a crate::ChainTransactionDto,
/// Internal transaction id.
pub(crate) transaction_id: i64,
/// Optional internal instruction id.
pub(crate) instruction_id: std::option::Option<i64>,
/// Stable protocol name.
pub(crate) protocol_name: std::string::String,
/// Program id that produced the event.
pub(crate) program_id: std::string::String,
/// Stable decoded event kind.
pub(crate) event_kind: std::string::String,
/// Optional pool account.
pub(crate) pool_account: std::option::Option<std::string::String>,
/// Optional market account.
pub(crate) market_account: std::option::Option<std::string::String>,
/// Optional token A mint.
pub(crate) token_a_mint: std::option::Option<std::string::String>,
/// Optional token B mint.
pub(crate) token_b_mint: std::option::Option<std::string::String>,
/// Optional LP mint or protocol-specific secondary mint.
pub(crate) lp_mint: std::option::Option<std::string::String>,
/// Payload used for classification enrichment and DB storage.
pub(crate) enrichment_payload_json: serde_json::Value,
/// Payload recorded in the detection observation.
pub(crate) observation_payload_json: serde_json::Value,
/// Detection observation kind.
pub(crate) observation_kind: std::string::String,
/// Detection signal kind.
pub(crate) signal_kind: std::string::String,
/// Diagnostic message emitted when fetch-after-upsert fails.
pub(crate) missing_after_upsert_message: std::string::String,
}
/// Persists one decoded DEX event and records its first-seen observation/signal.
pub(crate) async fn materialize_dex_decoded_event(
input: crate::dex_decoded_event_materialization::DexDecodedEventMaterializationInput<'_>,
) -> Result<crate::DexDecodedEventDto, crate::Error> {
let payload_json_result = crate::enrich_and_serialize_dex_decoded_payload(
input.protocol_name.as_str(),
input.event_kind.as_str(),
input.enrichment_payload_json,
);
let payload_json = match payload_json_result {
Ok(payload_json) => payload_json,
Err(error) => return Err(error),
};
let existing_result = crate::query_dex_decoded_events_get_by_key(
input.database,
input.transaction_id,
input.instruction_id,
input.event_kind.as_str(),
)
.await;
let existing_option = match existing_result {
Ok(existing_option) => existing_option,
Err(error) => return Err(error),
};
let already_present = existing_option.is_some();
let dto = crate::DexDecodedEventDto::new(
input.transaction_id,
input.instruction_id,
input.protocol_name,
input.program_id,
input.event_kind.clone(),
input.pool_account,
input.market_account,
input.token_a_mint,
input.token_b_mint,
input.lp_mint,
payload_json,
);
let upsert_result = crate::query_dex_decoded_events_upsert(input.database, &dto).await;
if let Err(error) = upsert_result {
return Err(error);
}
let fetched_result = crate::query_dex_decoded_events_get_by_key(
input.database,
input.transaction_id,
input.instruction_id,
input.event_kind.as_str(),
)
.await;
let fetched_option = match fetched_result {
Ok(fetched_option) => fetched_option,
Err(error) => return Err(error),
};
let fetched = match fetched_option {
Some(fetched) => fetched,
None => {
return Err(crate::Error::InvalidState(input.missing_after_upsert_message));
},
};
if !already_present {
let observation_result = input
.persistence
.record_observation(&crate::DetectionObservationInput::new(
input.observation_kind,
crate::ObservationSourceKind::HttpRpc,
input.transaction.source_endpoint_name.clone(),
input.transaction.signature.clone(),
input.transaction.slot,
input.observation_payload_json.clone(),
))
.await;
let observation_id = match observation_result {
Ok(observation_id) => observation_id,
Err(error) => return Err(error),
};
let signal_result = input
.persistence
.record_signal(&crate::DetectionSignalInput::new(
input.signal_kind,
crate::AnalysisSignalSeverity::Low,
input.transaction.signature.clone(),
Some(observation_id),
None,
input.observation_payload_json,
))
.await;
if let Err(error) = signal_result {
return Err(error);
}
}
return Ok(fetched);
}

3409
kb_lib/src/dex_detect.rs
View File

File diff suppressed because it is too large Load Diff

131
kb_lib/src/dex_detection_route.rs Normal file
View File

@@ -0,0 +1,131 @@
// file: kb_lib/src/dex_detection_route.rs
//! Routing from decoded DEX events to business-level detection handlers.
/// Internal route selected for one decoded DEX event.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub(crate) enum DexDetectionRoute {
/// Raydium AMM v4 initialize2 pool route.
RaydiumAmmV4Initialize2Pool,
/// Raydium CPMM trade route.
RaydiumCpmmTrade,
/// Raydium CLMM trade route.
RaydiumClmmTrade,
/// Pump.fun create token route.
PumpFunCreateV2Token,
/// Pump.fun trade route.
PumpFunTrade,
/// PumpSwap trade route.
PumpSwapTrade,
/// Incomplete PumpSwap event route.
SkipIncompletePumpSwapTrade,
/// Meteora DBC pool route.
MeteoraDbcPool,
/// Meteora DAMM v2 pool route.
MeteoraDammV2Pool,
/// Meteora DAMM v1 pool route.
MeteoraDammV1Pool,
/// Orca Whirlpools pool route.
OrcaWhirlpoolsPool,
/// FluxBeam pool route.
FluxbeamPool,
/// DexLab pool route.
DexlabPool,
}
/// Selects the business-level detection route for one decoded DEX event.
pub(crate) fn dex_detection_route(
decoded_event: &crate::DexDecodedEventDto,
) -> std::option::Option<crate::dex_detection_route::DexDetectionRoute> {
match (decoded_event.protocol_name.as_str(), decoded_event.event_kind.as_str()) {
("raydium_amm_v4", "raydium_amm_v4.initialize2_pool") => {
return Some(
crate::dex_detection_route::DexDetectionRoute::RaydiumAmmV4Initialize2Pool,
);
},
("raydium_cpmm", "raydium_cpmm.swap_base_input") => {
return Some(crate::dex_detection_route::DexDetectionRoute::RaydiumCpmmTrade);
},
("raydium_cpmm", "raydium_cpmm.swap_base_output") => {
return Some(crate::dex_detection_route::DexDetectionRoute::RaydiumCpmmTrade);
},
("raydium_clmm", "raydium_clmm.swap_v2") => {
return Some(crate::dex_detection_route::DexDetectionRoute::RaydiumClmmTrade);
},
("pump_fun", "pump_fun.create_v2_token") => {
return Some(crate::dex_detection_route::DexDetectionRoute::PumpFunCreateV2Token);
},
("pump_fun", "pump_fun.buy") => {
return Some(crate::dex_detection_route::DexDetectionRoute::PumpFunTrade);
},
("pump_fun", "pump_fun.sell") => {
return Some(crate::dex_detection_route::DexDetectionRoute::PumpFunTrade);
},
("pump_swap", "pump_swap.buy") => {
if crate::dex_detection_route::is_incomplete_pump_swap_decoded_event(decoded_event) {
return Some(
crate::dex_detection_route::DexDetectionRoute::SkipIncompletePumpSwapTrade,
);
}
return Some(crate::dex_detection_route::DexDetectionRoute::PumpSwapTrade);
},
("pump_swap", "pump_swap.sell") => {
if crate::dex_detection_route::is_incomplete_pump_swap_decoded_event(decoded_event) {
return Some(
crate::dex_detection_route::DexDetectionRoute::SkipIncompletePumpSwapTrade,
);
}
return Some(crate::dex_detection_route::DexDetectionRoute::PumpSwapTrade);
},
("meteora_dbc", "meteora_dbc.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDbcPool);
},
("meteora_dbc", "meteora_dbc.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDbcPool);
},
("meteora_damm_v2", "meteora_damm_v2.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDammV2Pool);
},
("meteora_damm_v2", "meteora_damm_v2.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDammV2Pool);
},
("meteora_damm_v1", "meteora_damm_v1.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDammV1Pool);
},
("meteora_damm_v1", "meteora_damm_v1.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::MeteoraDammV1Pool);
},
("orca_whirlpools", "orca_whirlpools.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::OrcaWhirlpoolsPool);
},
("orca_whirlpools", "orca_whirlpools.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::OrcaWhirlpoolsPool);
},
("fluxbeam", "fluxbeam.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::FluxbeamPool);
},
("fluxbeam", "fluxbeam.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::FluxbeamPool);
},
("dexlab", "dexlab.create_pool") => {
return Some(crate::dex_detection_route::DexDetectionRoute::DexlabPool);
},
("dexlab", "dexlab.swap") => {
return Some(crate::dex_detection_route::DexDetectionRoute::DexlabPool);
},
_ => return None,
}
}
fn is_incomplete_pump_swap_decoded_event(decoded_event: &crate::DexDecodedEventDto) -> bool {
if decoded_event.pool_account.is_none() {
return true;
}
if decoded_event.token_a_mint.is_none() {
return true;
}
if decoded_event.token_b_mint.is_none() {
return true;
}
return false;
}

509
kb_lib/src/dex_event_classification.rs Normal file
View File

@@ -0,0 +1,509 @@
// file: kb_lib/src/dex_event_classification.rs
//! Shared DEX event classification and decoded-payload enrichment.
//!
//! This module contains deterministic helpers used by DEX decoding,
//! trade aggregation and future non-trade event materialization.
//!
//! It intentionally does not decode protocol instructions and does not
//! perform database access.
/// Stable business category assigned to one decoded DEX event kind.
#[derive(Debug, Copy, Clone, Eq, PartialEq, serde::Serialize, serde::Deserialize)]
pub enum DexEventCategory {
/// Swap-like event that can potentially become a normalized trade.
Trade,
/// Liquidity deposit, withdraw, position open or position close event.
Liquidity,
/// Fee collection event.
Fee,
/// Reward or emission event.
Reward,
/// Pool creation, initialization or migration event.
PoolLifecycle,
/// Protocol administration, configuration or permission update event.
Admin,
/// Event kind that is not classified yet.
Unknown,
}
impl DexEventCategory {
/// Returns the stable string code persisted inside decoded payload metadata.
pub fn as_str(self) -> &'static str {
match self {
Self::Trade => return "trade",
Self::Liquidity => return "liquidity",
Self::Fee => return "fee",
Self::Reward => return "reward",
Self::PoolLifecycle => return "pool_lifecycle",
Self::Admin => return "admin",
Self::Unknown => return "unknown",
}
}
}
/// Classifies a DEX event kind into a stable business category.
pub fn classify_dex_event_category(event_kind: &str) -> DexEventCategory {
if is_dex_reward_event_kind(event_kind) {
return DexEventCategory::Reward;
}
if is_dex_fee_event_kind(event_kind) {
return DexEventCategory::Fee;
}
if is_dex_liquidity_event_kind(event_kind) {
return DexEventCategory::Liquidity;
}
if is_dex_pool_lifecycle_event_kind(event_kind) {
return DexEventCategory::PoolLifecycle;
}
if is_dex_admin_event_kind(event_kind) {
return DexEventCategory::Admin;
}
if is_dex_trade_event_kind(event_kind) {
return DexEventCategory::Trade;
}
return DexEventCategory::Unknown;
}
/// Classifies a DEX event kind and returns the persisted category code.
pub fn classify_dex_event_category_code(event_kind: &str) -> &'static str {
return classify_dex_event_category(event_kind).as_str();
}
/// Returns true when the event kind represents a swap-like event.
pub fn is_dex_trade_event_kind(event_kind: &str) -> bool {
if event_kind.ends_with(".buy") {
return true;
}
if event_kind.ends_with(".sell") {
return true;
}
if event_kind.ends_with(".swap") {
return true;
}
if event_kind.contains(".swap_") {
return true;
}
if event_kind.ends_with(".exact_input") {
return true;
}
if event_kind.ends_with(".exact_output") {
return true;
}
return false;
}
/// Returns true when the event kind can directly produce a candle candidate.
pub fn is_dex_candle_candidate_event_kind(event_kind: &str) -> bool {
if event_kind.contains("router") {
return false;
}
if event_kind.contains("route") {
return false;
}
return is_dex_trade_event_kind(event_kind);
}
/// Returns true for liquidity lifecycle changes that must not become candles.
pub fn is_dex_liquidity_event_kind(event_kind: &str) -> bool {
if event_kind.contains(".deposit") {
return true;
}
if event_kind.contains(".withdraw") {
return true;
}
if event_kind.contains(".increase_liquidity") {
return true;
}
if event_kind.contains(".decrease_liquidity") {
return true;
}
if event_kind.contains(".open_position") {
return true;
}
if event_kind.contains(".close_position") {
return true;
}
return false;
}
/// Returns true for fee collection events.
pub fn is_dex_fee_event_kind(event_kind: &str) -> bool {
if event_kind.contains("collect_creator_fee") {
return true;
}
if event_kind.contains("collect_protocol_fee") {
return true;
}
if event_kind.contains("collect_fund_fee") {
return true;
}
if event_kind.contains("collect_fee") {
return true;
}
return false;
}
/// Returns true for reward or incentive events.
pub fn is_dex_reward_event_kind(event_kind: &str) -> bool {
if event_kind.contains("reward") {
return true;
}
if event_kind.contains("emission") {
return true;
}
return false;
}
/// Returns true for pool creation, initialization or migration events.
pub fn is_dex_pool_lifecycle_event_kind(event_kind: &str) -> bool {
if event_kind.contains(".initialize") {
return true;
}
if event_kind.contains(".initialize_with_permission") {
return true;
}
if event_kind.contains(".create_pool") {
return true;
}
if event_kind.contains(".create_v2_token") {
return true;
}
if event_kind.contains(".migrate") {
return true;
}
return false;
}
/// Returns true for admin, configuration or permission changes.
pub fn is_dex_admin_event_kind(event_kind: &str) -> bool {
if event_kind.contains("admin") {
return true;
}
if event_kind.contains("config") {
return true;
}
if event_kind.contains("permission") {
return true;
}
if event_kind.contains("set_") {
return true;
}
if event_kind.contains("update_") {
return true;
}
return false;
}
/// Returns true when a decoded payload is marked as a trade candidate.
///
/// Explicit payload metadata wins over event-kind inference. This allows
/// incomplete decoded events, such as incomplete PumpSwap trades, to opt out.
pub fn is_decoded_event_trade_candidate(event_kind: &str, payload: &serde_json::Value) -> bool {
let trade_candidate_option =
extract_top_level_bool_by_candidate_keys(payload, &["tradeCandidate", "trade_candidate"]);
if let Some(trade_candidate) = trade_candidate_option {
return trade_candidate;
}
let event_category_option =
extract_string_by_candidate_keys(payload, &["eventCategory", "event_category"]);
if let Some(event_category) = event_category_option {
return event_category.as_str() == DexEventCategory::Trade.as_str();
}
return is_dex_trade_event_kind(event_kind);
}
/// Returns true when a decoded payload can be materialized as a candle candidate.
pub fn is_decoded_event_candle_candidate(event_kind: &str, payload: &serde_json::Value) -> bool {
let candle_candidate_option =
extract_top_level_bool_by_candidate_keys(payload, &["candleCandidate", "candle_candidate"]);
if let Some(candle_candidate) = candle_candidate_option {
return candle_candidate;
}
if !is_decoded_event_trade_candidate(event_kind, payload) {
return false;
}
return is_dex_candle_candidate_event_kind(event_kind);
}
/// Enriches a decoded payload with non-destructive classification metadata.
pub fn enrich_dex_decoded_payload(
protocol_name: &str,
event_kind: &str,
payload_json: serde_json::Value,
) -> serde_json::Value {
let event_category = classify_dex_event_category_code(event_kind);
let trade_candidate = is_dex_trade_event_kind(event_kind);
let candle_candidate = is_dex_candle_candidate_event_kind(event_kind);
let mut object = match payload_json {
serde_json::Value::Object(object) => object,
other => {
let mut object = serde_json::Map::new();
object.insert("rawPayload".to_owned(), other);
object
},
};
json_insert_string_if_missing(&mut object, "protocolName", protocol_name);
json_insert_string_if_missing(&mut object, "eventKind", event_kind);
json_insert_string_if_missing(&mut object, "eventCategory", event_category);
json_insert_bool_if_missing(&mut object, "tradeCandidate", trade_candidate);
json_insert_bool_if_missing(&mut object, "candleCandidate", candle_candidate);
json_insert_i64_if_missing(&mut object, "eventClassificationVersion", 1);
if !trade_candidate {
json_insert_string_if_missing(&mut object, "skipTradeReason", "non_trade_event");
} else if !candle_candidate {
json_insert_string_if_missing(
&mut object,
"skipCandleReason",
"route_or_multihop_event_requires_leg_resolution",
);
}
return serde_json::Value::Object(object);
}
/// Enriches a decoded payload and serializes it as JSON.
pub fn enrich_and_serialize_dex_decoded_payload(
protocol_name: &str,
event_kind: &str,
payload_json: serde_json::Value,
) -> Result<std::string::String, crate::Error> {
let enriched_payload = enrich_dex_decoded_payload(protocol_name, event_kind, payload_json);
let payload_json_result = serde_json::to_string(&enriched_payload);
match payload_json_result {
Ok(payload_json) => return Ok(payload_json),
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot serialize enriched decoded payload for '{}': {}",
event_kind, error
)));
},
}
}
/// Parses, enriches and serializes a decoded payload.
pub fn enrich_serialized_dex_decoded_payload(
protocol_name: &str,
event_kind: &str,
payload_json: &str,
) -> Result<std::string::String, crate::Error> {
let payload_value_result = serde_json::from_str::<serde_json::Value>(payload_json);
let payload_value = match payload_value_result {
Ok(payload_value) => payload_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse decoded payload for '{}': {}",
event_kind, error
)));
},
};
return enrich_and_serialize_dex_decoded_payload(protocol_name, event_kind, payload_value);
}
fn json_insert_string_if_missing(
object: &mut serde_json::Map<std::string::String, serde_json::Value>,
key: &str,
value: &str,
) {
if object.contains_key(key) {
return;
}
object.insert(key.to_owned(), serde_json::Value::String(value.to_owned()));
}
fn json_insert_bool_if_missing(
object: &mut serde_json::Map<std::string::String, serde_json::Value>,
key: &str,
value: bool,
) {
if object.contains_key(key) {
return;
}
object.insert(key.to_owned(), serde_json::Value::Bool(value));
}
fn json_insert_i64_if_missing(
object: &mut serde_json::Map<std::string::String, serde_json::Value>,
key: &str,
value: i64,
) {
if object.contains_key(key) {
return;
}
object.insert(key.to_owned(), serde_json::Value::Number(serde_json::Number::from(value)));
}
fn extract_top_level_bool_by_candidate_keys(
payload: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<bool> {
let object = match payload.as_object() {
Some(object) => object,
None => return None,
};
for candidate_key in candidate_keys {
let value_option = object.get(*candidate_key);
let value = match value_option {
Some(value) => value,
None => continue,
};
if let Some(value_bool) = value.as_bool() {
return Some(value_bool);
}
if let Some(value_i64) = value.as_i64() {
return Some(value_i64 != 0);
}
if let Some(value_u64) = value.as_u64() {
return Some(value_u64 != 0);
}
if let Some(value_text) = value.as_str() {
let normalized = value_text.trim().to_ascii_lowercase();
if normalized.as_str() == "true" {
return Some(true);
}
if normalized.as_str() == "false" {
return Some(false);
}
if normalized.as_str() == "1" {
return Some(true);
}
if normalized.as_str() == "0" {
return Some(false);
}
}
}
return None;
}
fn extract_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
let direct_text_option = direct.as_str();
if let Some(direct_text) = direct_text_option {
return Some(direct_text.to_string());
}
}
}
for nested_value in object.values() {
let nested_result = extract_string_by_candidate_keys(nested_value, candidate_keys);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result = extract_string_by_candidate_keys(nested_value, candidate_keys);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}
#[cfg(test)]
mod tests {
#[test]
fn classifies_swap_events_as_trade_candidates() {
assert_eq!(
super::classify_dex_event_category_code("raydium_cpmm.swap_base_input"),
"trade"
);
assert_eq!(
super::classify_dex_event_category_code("raydium_cpmm.swap_base_output"),
"trade"
);
assert_eq!(super::classify_dex_event_category_code("raydium_clmm.swap"), "trade");
assert_eq!(super::classify_dex_event_category_code("raydium_clmm.swap_v2"), "trade");
assert_eq!(super::classify_dex_event_category_code("raydium_clmm.exact_output"), "trade");
assert_eq!(super::classify_dex_event_category_code("pump_fun.buy"), "trade");
assert!(super::is_dex_trade_event_kind("raydium_cpmm.swap_base_input"));
assert!(super::is_dex_candle_candidate_event_kind("raydium_cpmm.swap_base_input"));
}
#[test]
fn classifies_router_swap_as_trade_but_not_direct_candle_candidate() {
assert_eq!(
super::classify_dex_event_category_code("raydium_clmm.swap_router_base_in"),
"trade"
);
assert!(super::is_dex_trade_event_kind("raydium_clmm.swap_router_base_in"));
assert!(!super::is_dex_candle_candidate_event_kind("raydium_clmm.swap_router_base_in"));
}
#[test]
fn classifies_fee_reward_liquidity_and_lifecycle_events() {
assert_eq!(
super::classify_dex_event_category_code("raydium_cpmm.collect_creator_fee"),
"fee"
);
assert_eq!(
super::classify_dex_event_category_code("raydium_clmm.collect_protocol_fee"),
"fee"
);
assert_eq!(
super::classify_dex_event_category_code("raydium_clmm.set_reward_params"),
"reward"
);
assert_eq!(
super::classify_dex_event_category_code("raydium_clmm.increase_liquidity_v2"),
"liquidity"
);
assert_eq!(
super::classify_dex_event_category_code("raydium_cpmm.initialize"),
"pool_lifecycle"
);
}
#[test]
fn enriched_payload_keeps_existing_fields() {
let payload_json = serde_json::json!({
"eventCategory": "custom",
"amountIn": "10"
});
let enriched_payload = super::enrich_dex_decoded_payload(
"raydium_cpmm",
"raydium_cpmm.swap_base_input",
payload_json,
);
let object_option = enriched_payload.as_object();
let object = match object_option {
Some(object) => object,
None => {
panic!("expected enriched payload object");
},
};
assert_eq!(
object.get("eventCategory"),
Some(&serde_json::Value::String("custom".to_owned()))
);
assert_eq!(
object.get("protocolName"),
Some(&serde_json::Value::String("raydium_cpmm".to_owned()))
);
assert_eq!(
object.get("eventKind"),
Some(&serde_json::Value::String("raydium_cpmm.swap_base_input".to_owned()))
);
assert_eq!(object.get("tradeCandidate"), Some(&serde_json::Value::Bool(true)));
assert_eq!(object.get("candleCandidate"), Some(&serde_json::Value::Bool(true)));
}
#[test]
fn decoded_event_payload_candidate_flags_win_over_event_kind() {
let payload_json = serde_json::json!({
"tradeCandidate": false,
"candleCandidate": false
});
assert!(!super::is_decoded_event_trade_candidate("pump_swap.buy", &payload_json));
assert!(!super::is_decoded_event_candle_candidate("pump_swap.buy", &payload_json));
}
}

662
kb_lib/src/dex_pool_materialization.rs Normal file
View File

@@ -0,0 +1,662 @@
// file: kb_lib/src/dex_pool_materialization.rs
//! Shared DEX pool materialization helpers.
//!
//! This module persists normalized pool, pair, pool token and pool listing
//! records from decoded DEX events.
//!
//! It intentionally does not decode instructions and does not record analysis
//! signals. Detection services remain responsible for deciding which signals
//! to emit.
/// Token ordering strategy used when materializing a decoded DEX pool.
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
pub(crate) enum DexPoolTokenOrder {
/// `token_a_mint` is already the base token and `token_b_mint` is already the quote token.
AlreadyBaseQuote,
/// Base/quote order should be chosen from token A/B using known quote-token hints.
ChooseBaseQuoteFromTokenAB,
}
/// Input required to materialize a normalized DEX pool.
#[derive(Debug, Clone)]
pub(crate) struct DexPoolMaterializationInput {
/// Parent decoded event id.
pub(crate) decoded_event_id: i64,
/// Internal DEX id.
pub(crate) dex_id: i64,
/// Pool account address.
pub(crate) pool_address: std::string::String,
/// Token A mint, or already-base mint depending on `token_order`.
pub(crate) token_a_mint: std::string::String,
/// Token B mint, or already-quote mint depending on `token_order`.
pub(crate) token_b_mint: std::string::String,
/// Optional LP mint.
pub(crate) lp_mint: std::option::Option<std::string::String>,
/// Optional token A vault address, or base vault when `token_order` is `AlreadyBaseQuote`.
pub(crate) token_a_vault_address: std::option::Option<std::string::String>,
/// Optional token B vault address, or quote vault when `token_order` is `AlreadyBaseQuote`.
pub(crate) token_b_vault_address: std::option::Option<std::string::String>,
/// Pool kind to persist.
pub(crate) pool_kind: crate::PoolKind,
/// Pool status to persist.
pub(crate) pool_status: crate::PoolStatus,
/// Token ordering strategy.
pub(crate) token_order: crate::dex_pool_materialization::DexPoolTokenOrder,
/// Listing source kind.
pub(crate) listing_source_kind: crate::ObservationSourceKind,
/// Optional source endpoint logical name.
pub(crate) source_endpoint_name: std::option::Option<std::string::String>,
}
impl DexPoolMaterializationInput {
/// Creates a materialization input from a decoded event requiring pool and two token mints.
pub(crate) fn from_decoded_event(
decoded_event: &crate::DexDecodedEventDto,
dex_id: i64,
pool_kind: crate::PoolKind,
pool_status: crate::PoolStatus,
token_order: crate::dex_pool_materialization::DexPoolTokenOrder,
token_a_vault_address: std::option::Option<std::string::String>,
token_b_vault_address: std::option::Option<std::string::String>,
source_endpoint_name: std::option::Option<std::string::String>,
) -> Result<Self, crate::Error> {
let decoded_event_id_result =
crate::dex_pool_materialization::required_decoded_event_id(decoded_event);
let decoded_event_id = match decoded_event_id_result {
Ok(decoded_event_id) => decoded_event_id,
Err(error) => return Err(error),
};
let pool_address_result =
crate::dex_pool_materialization::required_pool_account(decoded_event);
let pool_address = match pool_address_result {
Ok(pool_address) => pool_address,
Err(error) => return Err(error),
};
let token_a_mint_result =
crate::dex_pool_materialization::required_token_a_mint(decoded_event);
let token_a_mint = match token_a_mint_result {
Ok(token_a_mint) => token_a_mint,
Err(error) => return Err(error),
};
let token_b_mint_result =
crate::dex_pool_materialization::required_token_b_mint(decoded_event);
let token_b_mint = match token_b_mint_result {
Ok(token_b_mint) => token_b_mint,
Err(error) => return Err(error),
};
return Ok(Self {
decoded_event_id,
dex_id,
pool_address,
token_a_mint,
token_b_mint,
lp_mint: decoded_event.lp_mint.clone(),
token_a_vault_address,
token_b_vault_address,
pool_kind,
pool_status,
token_order,
listing_source_kind: crate::ObservationSourceKind::Dex,
source_endpoint_name,
});
}
/// Creates a materialization input from a decoded event and explicit token mints.
///
/// This is used by launch or bonding-curve events where the decoded event
/// may expose only the launched token mint and the quote mint is inferred
/// by the detector.
pub(crate) fn from_decoded_event_with_mints(
decoded_event: &crate::DexDecodedEventDto,
dex_id: i64,
token_a_mint: std::string::String,
token_b_mint: std::string::String,
lp_mint: std::option::Option<std::string::String>,
pool_kind: crate::PoolKind,
pool_status: crate::PoolStatus,
token_order: crate::dex_pool_materialization::DexPoolTokenOrder,
token_a_vault_address: std::option::Option<std::string::String>,
token_b_vault_address: std::option::Option<std::string::String>,
source_endpoint_name: std::option::Option<std::string::String>,
) -> Result<Self, crate::Error> {
let decoded_event_id_result =
crate::dex_pool_materialization::required_decoded_event_id(decoded_event);
let decoded_event_id = match decoded_event_id_result {
Ok(decoded_event_id) => decoded_event_id,
Err(error) => return Err(error),
};
let pool_address_result =
crate::dex_pool_materialization::required_pool_account(decoded_event);
let pool_address = match pool_address_result {
Ok(pool_address) => pool_address,
Err(error) => return Err(error),
};
return Ok(Self {
decoded_event_id,
dex_id,
pool_address,
token_a_mint,
token_b_mint,
lp_mint,
token_a_vault_address,
token_b_vault_address,
pool_kind,
pool_status,
token_order,
listing_source_kind: crate::ObservationSourceKind::Dex,
source_endpoint_name,
});
}
}
/// Returns the decoded event id or fails with a stable diagnostic.
pub(crate) fn required_decoded_event_id(
decoded_event: &crate::DexDecodedEventDto,
) -> Result<i64, crate::Error> {
match decoded_event.id {
Some(decoded_event_id) => return Ok(decoded_event_id),
None => {
return Err(crate::Error::InvalidState(
"decoded dex event has no internal id".to_string(),
));
},
}
}
/// Returns the pool account or fails with a stable diagnostic.
pub(crate) fn required_pool_account(
decoded_event: &crate::DexDecodedEventDto,
) -> Result<std::string::String, crate::Error> {
let decoded_event_id_result =
crate::dex_pool_materialization::required_decoded_event_id(decoded_event);
let decoded_event_id = match decoded_event_id_result {
Ok(decoded_event_id) => decoded_event_id,
Err(error) => return Err(error),
};
match decoded_event.pool_account.clone() {
Some(pool_account) => return Ok(pool_account),
None => {
return Err(crate::Error::InvalidState(format!(
"decoded event '{}' has no pool_account",
decoded_event_id
)));
},
}
}
/// Returns token A mint or fails with a stable diagnostic.
pub(crate) fn required_token_a_mint(
decoded_event: &crate::DexDecodedEventDto,
) -> Result<std::string::String, crate::Error> {
let decoded_event_id_result =
crate::dex_pool_materialization::required_decoded_event_id(decoded_event);
let decoded_event_id = match decoded_event_id_result {
Ok(decoded_event_id) => decoded_event_id,
Err(error) => return Err(error),
};
match decoded_event.token_a_mint.clone() {
Some(token_a_mint) => return Ok(token_a_mint),
None => {
return Err(crate::Error::InvalidState(format!(
"decoded event '{}' has no token_a_mint",
decoded_event_id
)));
},
}
}
/// Returns token B mint or fails with a stable diagnostic.
pub(crate) fn required_token_b_mint(
decoded_event: &crate::DexDecodedEventDto,
) -> Result<std::string::String, crate::Error> {
let decoded_event_id_result =
crate::dex_pool_materialization::required_decoded_event_id(decoded_event);
let decoded_event_id = match decoded_event_id_result {
Ok(decoded_event_id) => decoded_event_id,
Err(error) => return Err(error),
};
match decoded_event.token_b_mint.clone() {
Some(token_b_mint) => return Ok(token_b_mint),
None => {
return Err(crate::Error::InvalidState(format!(
"decoded event '{}' has no token_b_mint",
decoded_event_id
)));
},
}
}
/// Persists pool, pair, pool tokens and listing, returning the materialized ids.
pub(crate) async fn materialize_dex_pool(
database: &crate::Database,
input: &crate::dex_pool_materialization::DexPoolMaterializationInput,
) -> Result<crate::DexPoolDetectionResult, crate::Error> {
let ordered_tokens = crate::dex_pool_materialization::ordered_pool_tokens_from_input(input);
let base_token_id_result =
crate::dex_pool_materialization::ensure_token(database, ordered_tokens.base_mint.as_str())
.await;
let base_token_id = match base_token_id_result {
Ok(base_token_id) => base_token_id,
Err(error) => return Err(error),
};
let quote_token_id_result =
crate::dex_pool_materialization::ensure_token(database, ordered_tokens.quote_mint.as_str())
.await;
let quote_token_id = match quote_token_id_result {
Ok(quote_token_id) => quote_token_id,
Err(error) => return Err(error),
};
let lp_token_id = match input.lp_mint.clone() {
Some(lp_mint) => {
let lp_token_id_result =
crate::dex_pool_materialization::ensure_token(database, lp_mint.as_str()).await;
match lp_token_id_result {
Ok(lp_token_id) => Some(lp_token_id),
Err(error) => return Err(error),
}
},
None => None,
};
let pool_result = crate::dex_pool_materialization::ensure_pool(database, input).await;
let pool_materialization = match pool_result {
Ok(pool_materialization) => pool_materialization,
Err(error) => return Err(error),
};
let pair_result = crate::dex_pool_materialization::ensure_pair(
database,
input.dex_id,
pool_materialization.pool_id,
base_token_id,
quote_token_id,
ordered_tokens.base_mint.as_str(),
ordered_tokens.quote_mint.as_str(),
)
.await;
let pair_materialization = match pair_result {
Ok(pair_materialization) => pair_materialization,
Err(error) => return Err(error),
};
let upsert_base_pool_token_result = crate::query_pool_tokens_upsert(
database,
&crate::PoolTokenDto::new(
pool_materialization.pool_id,
base_token_id,
crate::PoolTokenRole::Base,
ordered_tokens.base_vault_address,
Some(0),
),
)
.await;
if let Err(error) = upsert_base_pool_token_result {
return Err(error);
}
let upsert_quote_pool_token_result = crate::query_pool_tokens_upsert(
database,
&crate::PoolTokenDto::new(
pool_materialization.pool_id,
quote_token_id,
crate::PoolTokenRole::Quote,
ordered_tokens.quote_vault_address,
Some(1),
),
)
.await;
if let Err(error) = upsert_quote_pool_token_result {
return Err(error);
}
if let Some(lp_token_id) = lp_token_id {
let upsert_lp_pool_token_result = crate::query_pool_tokens_upsert(
database,
&crate::PoolTokenDto::new(
pool_materialization.pool_id,
lp_token_id,
crate::PoolTokenRole::LpMint,
None,
None,
),
)
.await;
if let Err(error) = upsert_lp_pool_token_result {
return Err(error);
}
}
let listing_result = crate::dex_pool_materialization::ensure_pool_listing(
database,
input,
pool_materialization.pool_id,
pair_materialization.pair_id,
)
.await;
let listing_materialization = match listing_result {
Ok(listing_materialization) => listing_materialization,
Err(error) => return Err(error),
};
return Ok(crate::DexPoolDetectionResult {
decoded_event_id: input.decoded_event_id,
dex_id: input.dex_id,
pool_id: pool_materialization.pool_id,
pair_id: pair_materialization.pair_id,
pool_listing_id: listing_materialization.pool_listing_id,
created_pool: pool_materialization.created_pool,
created_pair: pair_materialization.created_pair,
created_listing: listing_materialization.created_listing,
});
}
#[derive(Debug, Clone)]
struct OrderedPoolTokens {
base_mint: std::string::String,
quote_mint: std::string::String,
base_vault_address: std::option::Option<std::string::String>,
quote_vault_address: std::option::Option<std::string::String>,
}
fn ordered_pool_tokens_from_input(
input: &crate::dex_pool_materialization::DexPoolMaterializationInput,
) -> OrderedPoolTokens {
match input.token_order {
crate::dex_pool_materialization::DexPoolTokenOrder::AlreadyBaseQuote => {
return OrderedPoolTokens {
base_mint: input.token_a_mint.clone(),
quote_mint: input.token_b_mint.clone(),
base_vault_address: input.token_a_vault_address.clone(),
quote_vault_address: input.token_b_vault_address.clone(),
};
},
crate::dex_pool_materialization::DexPoolTokenOrder::ChooseBaseQuoteFromTokenAB => {
let base_is_token_a = crate::dex_pool_materialization::choose_base_quote_order(
input.token_a_mint.as_str(),
input.token_b_mint.as_str(),
);
let base_mint = if base_is_token_a {
input.token_a_mint.clone()
} else {
input.token_b_mint.clone()
};
let quote_mint = if base_is_token_a {
input.token_b_mint.clone()
} else {
input.token_a_mint.clone()
};
let base_vault_address = if base_is_token_a {
input.token_a_vault_address.clone()
} else {
input.token_b_vault_address.clone()
};
let quote_vault_address = if base_is_token_a {
input.token_b_vault_address.clone()
} else {
input.token_a_vault_address.clone()
};
return OrderedPoolTokens {
base_mint,
quote_mint,
base_vault_address,
quote_vault_address,
};
},
}
}
async fn ensure_pool(
database: &crate::Database,
input: &crate::dex_pool_materialization::DexPoolMaterializationInput,
) -> Result<PoolMaterialization, crate::Error> {
let existing_pool_result =
crate::query_pools_get_by_address(database, input.pool_address.as_str()).await;
let existing_pool_option = match existing_pool_result {
Ok(existing_pool_option) => existing_pool_option,
Err(error) => return Err(error),
};
let created_pool = existing_pool_option.is_none();
let pool_id = match existing_pool_option {
Some(pool) => match pool.id {
Some(pool_id) => pool_id,
None => {
return Err(crate::Error::InvalidState(format!(
"pool '{}' has no internal id",
pool.address
)));
},
},
None => {
let pool_dto = crate::PoolDto::new(
input.dex_id,
input.pool_address.clone(),
input.pool_kind,
input.pool_status,
);
let upsert_result = crate::query_pools_upsert(database, &pool_dto).await;
match upsert_result {
Ok(pool_id) => pool_id,
Err(error) => return Err(error),
}
},
};
return Ok(PoolMaterialization { pool_id, created_pool });
}
async fn ensure_pair(
database: &crate::Database,
dex_id: i64,
pool_id: i64,
base_token_id: i64,
quote_token_id: i64,
base_mint: &str,
quote_mint: &str,
) -> Result<PairMaterialization, crate::Error> {
let existing_pair_result = crate::query_pairs_get_by_pool_id(database, pool_id).await;
let existing_pair_option = match existing_pair_result {
Ok(existing_pair_option) => existing_pair_option,
Err(error) => return Err(error),
};
let created_pair = existing_pair_option.is_none();
let pair_symbol = crate::dex_pool_materialization::build_pair_symbol(base_mint, quote_mint);
let pair_id = match existing_pair_option {
Some(pair) => match pair.id {
Some(pair_id) => pair_id,
None => {
return Err(crate::Error::InvalidState(format!(
"pair for pool '{}' has no internal id",
pool_id
)));
},
},
None => {
let pair_dto =
crate::PairDto::new(dex_id, pool_id, base_token_id, quote_token_id, pair_symbol);
let upsert_result = crate::query_pairs_upsert(database, &pair_dto).await;
match upsert_result {
Ok(pair_id) => pair_id,
Err(error) => return Err(error),
}
},
};
return Ok(PairMaterialization { pair_id, created_pair });
}
async fn ensure_pool_listing(
database: &crate::Database,
input: &crate::dex_pool_materialization::DexPoolMaterializationInput,
pool_id: i64,
pair_id: i64,
) -> Result<ListingMaterialization, crate::Error> {
let existing_listing_result =
crate::query_pool_listings_get_by_pool_id(database, pool_id).await;
let existing_listing_option = match existing_listing_result {
Ok(existing_listing_option) => existing_listing_option,
Err(error) => return Err(error),
};
let created_listing = existing_listing_option.is_none();
let pool_listing_id = match existing_listing_option {
Some(pool_listing) => pool_listing.id,
None => {
let listing_dto = crate::PoolListingDto::new(
input.dex_id,
pool_id,
Some(pair_id),
input.listing_source_kind,
input.source_endpoint_name.clone(),
None,
None,
None,
);
let upsert_result = crate::query_pool_listings_upsert(database, &listing_dto).await;
match upsert_result {
Ok(listing_id) => Some(listing_id),
Err(error) => return Err(error),
}
},
};
return Ok(ListingMaterialization { pool_listing_id, created_listing });
}
async fn ensure_token(database: &crate::Database, mint: &str) -> Result<i64, crate::Error> {
let token_result = crate::query_tokens_get_by_mint(database, mint).await;
let token_option = match token_result {
Ok(token_option) => token_option,
Err(error) => return Err(error),
};
match token_option {
Some(token) => match token.id {
Some(token_id) => return Ok(token_id),
None => {
return Err(crate::Error::InvalidState(format!(
"token '{}' has no internal id",
mint
)));
},
},
None => {
let token_dto = crate::TokenDto::new(
mint.to_string(),
None,
None,
None,
crate::SPL_TOKEN_PROGRAM_ID.to_string(),
crate::dex_pool_materialization::is_quote_mint(mint),
);
return crate::query_tokens_upsert(database, &token_dto).await;
},
}
}
#[derive(Debug, Clone)]
struct PoolMaterialization {
pool_id: i64,
created_pool: bool,
}
#[derive(Debug, Clone)]
struct PairMaterialization {
pair_id: i64,
created_pair: bool,
}
#[derive(Debug, Clone)]
struct ListingMaterialization {
pool_listing_id: std::option::Option<i64>,
created_listing: bool,
}
fn is_quote_mint(mint: &str) -> bool {
return mint == crate::WSOL_MINT_ID;
}
fn choose_base_quote_order(token_a_mint: &str, token_b_mint: &str) -> bool {
let token_a_is_quote = crate::dex_pool_materialization::is_quote_mint(token_a_mint);
let token_b_is_quote = crate::dex_pool_materialization::is_quote_mint(token_b_mint);
if token_a_is_quote && !token_b_is_quote {
return false;
}
if token_b_is_quote && !token_a_is_quote {
return true;
}
return true;
}
fn build_pair_symbol(
base_mint: &str,
quote_mint: &str,
) -> std::option::Option<std::string::String> {
let base_symbol = crate::dex_pool_materialization::symbol_hint_from_mint(base_mint);
let quote_symbol = crate::dex_pool_materialization::symbol_hint_from_mint(quote_mint);
match (base_symbol, quote_symbol) {
(Some(base_symbol), Some(quote_symbol)) => {
return Some(format!("{base_symbol}/{quote_symbol}"));
},
_ => return None,
}
}
fn symbol_hint_from_mint(mint: &str) -> std::option::Option<std::string::String> {
if mint == crate::WSOL_MINT_ID {
return Some("WSOL".to_string());
}
return None;
}
#[cfg(test)]
mod tests {
#[test]
fn quote_token_is_moved_to_quote_side_when_order_is_chosen() {
let input = crate::dex_pool_materialization::DexPoolMaterializationInput {
decoded_event_id: 1,
dex_id: 2,
pool_address: "Pool111".to_string(),
token_a_mint: crate::WSOL_MINT_ID.to_string(),
token_b_mint: "TokenB111".to_string(),
lp_mint: None,
token_a_vault_address: Some("VaultA111".to_string()),
token_b_vault_address: Some("VaultB111".to_string()),
pool_kind: crate::PoolKind::Amm,
pool_status: crate::PoolStatus::Active,
token_order:
crate::dex_pool_materialization::DexPoolTokenOrder::ChooseBaseQuoteFromTokenAB,
listing_source_kind: crate::ObservationSourceKind::Dex,
source_endpoint_name: Some("test".to_string()),
};
let ordered = super::ordered_pool_tokens_from_input(&input);
assert_eq!(ordered.base_mint, "TokenB111");
assert_eq!(ordered.quote_mint, crate::WSOL_MINT_ID);
assert_eq!(ordered.base_vault_address, Some("VaultB111".to_string()));
assert_eq!(ordered.quote_vault_address, Some("VaultA111".to_string()));
}
#[test]
fn already_base_quote_order_is_preserved() {
let input = crate::dex_pool_materialization::DexPoolMaterializationInput {
decoded_event_id: 1,
dex_id: 2,
pool_address: "Pool111".to_string(),
token_a_mint: crate::WSOL_MINT_ID.to_string(),
token_b_mint: "TokenB111".to_string(),
lp_mint: None,
token_a_vault_address: Some("BaseVault111".to_string()),
token_b_vault_address: Some("QuoteVault111".to_string()),
pool_kind: crate::PoolKind::Amm,
pool_status: crate::PoolStatus::Active,
token_order: crate::dex_pool_materialization::DexPoolTokenOrder::AlreadyBaseQuote,
listing_source_kind: crate::ObservationSourceKind::Dex,
source_endpoint_name: Some("test".to_string()),
};
let ordered = super::ordered_pool_tokens_from_input(&input);
assert_eq!(ordered.base_mint, crate::WSOL_MINT_ID);
assert_eq!(ordered.quote_mint, "TokenB111");
assert_eq!(ordered.base_vault_address, Some("BaseVault111".to_string()));
assert_eq!(ordered.quote_vault_address, Some("QuoteVault111".to_string()));
}
#[test]
fn pair_symbol_is_none_when_only_one_symbol_is_known() {
let pair_symbol = super::build_pair_symbol(crate::WSOL_MINT_ID, "TokenB111");
assert_eq!(pair_symbol, None);
}
#[test]
fn wsol_symbol_hint_is_known() {
let symbol = super::symbol_hint_from_mint(crate::WSOL_MINT_ID);
assert_eq!(symbol, Some("WSOL".to_string()));
}
}

199
kb_lib/src/lib.rs
View File

@@ -23,10 +23,22 @@ mod db;
mod detect; mod detect;
/// DEX-specific transaction decoders. /// DEX-specific transaction decoders.
mod dex; mod dex;
/// Internal known DEX catalog.
mod dex_catalog;
/// Persistence-oriented DEX decoding service. /// Persistence-oriented DEX decoding service.
mod dex_decode; mod dex_decode;
/// Transaction context loading for DEX decoding.
mod dex_decode_context;
/// Decoded DEX event materialization helpers.
mod dex_decoded_event_materialization;
/// Business-level detection built from decoded DEX events. /// Business-level detection built from decoded DEX events.
mod dex_detect; mod dex_detect;
/// Decoded DEX event to business-detection routing.
mod dex_detection_route;
/// Shared DEX event classification and decoded-payload enrichment helpers.
mod dex_event_classification;
/// Shared DEX pool materialization helpers.
mod dex_pool_materialization;
/// Shared error type for `kb_lib`. /// Shared error type for `kb_lib`.
mod error; mod error;
/// Generic asynchronous HTTP JSON-RPC client. /// Generic asynchronous HTTP JSON-RPC client.
@@ -55,6 +67,8 @@ mod pair_candle_query;
mod pair_symbol; mod pair_symbol;
/// Cross-DEX pool-origin recording service. /// Cross-DEX pool-origin recording service.
mod pool_origin; mod pool_origin;
/// Protocol candidate recording.
mod protocol_candidate_recording;
/// Typed Solana WebSocket PubSub helpers built on top of the generic JSON-RPC transport. /// Typed Solana WebSocket PubSub helpers built on top of the generic JSON-RPC transport.
mod solana_pubsub_ws; mod solana_pubsub_ws;
/// Historical token backfill service. /// Historical token backfill service.
@@ -65,6 +79,22 @@ mod token_metadata;
mod tracing; mod tracing;
/// Cross-DEX trade aggregation service. /// Cross-DEX trade aggregation service.
mod trade_aggregation; mod trade_aggregation;
/// Database context loading for trade aggregation.
mod trade_aggregation_context;
/// Trade amount resolution orchestration.
mod trade_amount_resolution;
/// Trade-event materialization.
mod trade_event_materialization;
/// Trade metric update and pricing helpers.
mod trade_metric_update;
/// PumpSwap trade amount recovery helpers.
mod trade_pump_swap_amounts;
/// Trade-side resolution helpers.
mod trade_side_resolution;
/// Solana transaction/meta trade amount extraction helpers.
mod trade_solana_amounts;
/// Transaction classification service.
mod transaction_classification;
/// Projection of resolved transactions into normalized internal DB tables. /// Projection of resolved transactions into normalized internal DB tables.
mod tx_model; mod tx_model;
/// Transaction resolution pipeline. /// Transaction resolution pipeline.
@@ -104,46 +134,143 @@ pub use config::SolanaConfig;
pub use config::SqliteDatabaseConfig; pub use config::SqliteDatabaseConfig;
/// WebSocket endpoint configuration. /// WebSocket endpoint configuration.
pub use config::WsEndpointConfig; pub use config::WsEndpointConfig;
/// Address Lookup Table program identifier. ("AddressLookupTab1e1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::address_lookup_table::ID
pub use constants::ADDRESS_LOOKUP_TABLE_PROGRAM_ID;
/// Arbitrage Bot (6MWVT) / Arbitrage or Sandwich Bot. ("6MWVTis8rmmk6Vt9zmAJJbmb3VuLpzoQ1aHH4N6wQEGh").
pub use constants::ARBITRAGE_BOT_6MWVT_PROGRAM_ID;
/// Associated Token Account program identifier. ("ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL"). /// Associated Token Account program identifier. ("ATokenGPvbdGVxr1b2hvZbsiqW5xWH25efTNsLJA8knL").
/// @see solana_sdk::pubkey::Pubkey = spl_associated_token_account_interface::program::ID /// @see solana_sdk::pubkey::Pubkey = spl_associated_token_account_interface::program::ID
pub use constants::ASSOCIATED_TOKEN_PROGRAM_ID; pub use constants::ASSOCIATED_TOKEN_PROGRAM_ID;
/// BPF Loader program identifier. ("BPFLoader1111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::bpf_loader_deprecated::ID
pub use constants::BPF_LOADER_DEPRECATED_PROGRAM_ID;
/// BPF Loader program identifier. ("BPFLoaderUpgradeab1e11111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::bpf_loader_upgradeable::ID
pub use constants::BPF_LOADER_UPGRADEABLE_PROGRAM_ID;
/// Compute Budget program identifier. ("ComputeBudget111111111111111111111111111111"). /// Compute Budget program identifier. ("ComputeBudget111111111111111111111111111111").
/// @see solana_sdk_ids::compute_budget::ID /// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::compute_budget::ID
pub use constants::COMPUTE_BUDGET_PROGRAM_ID; pub use constants::COMPUTE_BUDGET_PROGRAM_ID;
/// Config program identifier. ("Config1111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::config::ID
pub use constants::CONFIG_PROGRAM_ID;
/// DexLab Swap/Pool program id. ("DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N"). /// DexLab Swap/Pool program id. ("DSwpgjMvXhtGn6BsbqmacdBZyfLj6jSWf3HJpdJtmg6N").
pub use constants::DEXLAB_PROGRAM_ID; pub use constants::DEXLAB_PROGRAM_ID;
/// ED25519 program identifier. ("Ed25519SigVerify111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::ed25519_program::ID
pub use constants::ED25519_PROGRAM_ID;
/// Feature program identifier. ("Feature111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::feature::ID
pub use constants::FEATURE_PROGRAM_ID;
/// FluxBeam program id. ("FLUXubRmkEi2q6K3Y9kBPg9248ggaZVsoSFhtJHSrm1X"). /// FluxBeam program id. ("FLUXubRmkEi2q6K3Y9kBPg9248ggaZVsoSFhtJHSrm1X").
pub use constants::FLUXBEAM_PROGRAM_ID; pub use constants::FLUXBEAM_PROGRAM_ID;
/// Incinerator program identifier. ("1nc1nerator11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::incinerator::ID
pub use constants::INCINERATOR_PROGRAM_ID;
/// Loader V4 program identifier. ("LoaderV411111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::loader_v4::ID
pub use constants::LOADER_V4_PROGRAM_ID;
/// Meteora DAMM v1 program id. ("Eo7WjKq67rjJQSZxS6z3YkapzY3eMj6Xy8X5EQVn5UaB"). /// Meteora DAMM v1 program id. ("Eo7WjKq67rjJQSZxS6z3YkapzY3eMj6Xy8X5EQVn5UaB").
pub use constants::METEORA_DAMM_V1_PROGRAM_ID; pub use constants::METEORA_DAMM_V1_PROGRAM_ID;
/// Meteora DAMM v2 program id. ("cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG"). /// Meteora DAMM v2 program id. ("cpamdpZCGKUy5JxQXB4dcpGPiikHawvSWAd6mEn1sGG").
pub use constants::METEORA_DAMM_V2_PROGRAM_ID; pub use constants::METEORA_DAMM_V2_PROGRAM_ID;
/// Meteora DBC program id. ("dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN"). /// Meteora DBC program id. ("dbcij3LWUppWqq96dh6gJWwBifmcGfLSB5D4DuSMaqN").
pub use constants::METEORA_DBC_PROGRAM_ID; pub use constants::METEORA_DBC_PROGRAM_ID;
/// Meteora DLMM program id. ("LBUZKhRxPF3XUpBCjp4YzTKgLccjZhTSDM9YuVaPwxo").
pub use constants::METEORA_DLMM_PROGRAM_ID;
/// Native Loader program identifier. ("NativeLoader1111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::native_loader::ID
pub use constants::NATIVE_LOADER_PROGRAM_ID;
/// Orca Whirlpools program id. ("whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc"). /// Orca Whirlpools program id. ("whirLbMiicVdio4qvUfM5KAg6Ct8VwpYzGff3uctyCc").
pub use constants::ORCA_WHIRLPOOLS_PROGRAM_ID; pub use constants::ORCA_WHIRLPOOLS_PROGRAM_ID;
/// Pump.fun program id. ("6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P"). /// Pump.fun program id. ("6EF8rrecthR5Dkzon8Nwu78hRvfCKubJ14M5uBEwF6P").
pub use constants::PUMP_FUN_PROGRAM_ID; pub use constants::PUMP_FUN_PROGRAM_ID;
/// PumpSwap / PumpAMM program id. ("pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA"). /// PumpSwap / PumpAMM program id. ("pAMMBay6oceH9fJKBRHGP5D4bD4sWpmSwMn52FMfXEA").
pub use constants::PUMP_SWAP_PROGRAM_ID; pub use constants::PUMP_SWAP_PROGRAM_ID;
/// Raydium AMM routing program id. ("routeUGWgWzqBWFcrCfv8tritsqukccJPu3q5GPP3xS").
pub use constants::RAYDIUM_AMM_ROUTING_PROGRAM_ID;
/// Raydium AmmV4 program id. ("675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8"). /// Raydium AmmV4 program id. ("675kPX9MHTjS2zt1qfr1NYHuzeLXfQM9H24wFSUt1Mp8").
pub use constants::RAYDIUM_AMM_V4_PROGRAM_ID; pub use constants::RAYDIUM_AMM_V4_PROGRAM_ID;
/// Raydium CLMM program id. ("CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK"). /// Raydium CLMM program id. ("CAMMCzo5YL8w4VFF8KVHrK22GGUsp5VTaW7grrKgrWqK").
pub use constants::RAYDIUM_CLMM_PROGRAM_ID; pub use constants::RAYDIUM_CLMM_PROGRAM_ID;
/// Raydium CPMM mainnet program id. ("CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C"). /// Raydium CPMM mainnet program id. ("CPMMoo8L3F4NbTegBCKVNunggL7H1ZpdTHKxQB5qKP1C").
pub use constants::RAYDIUM_CPMM_PROGRAM_ID; pub use constants::RAYDIUM_CPMM_PROGRAM_ID;
/// Raydium LaunchLab program id. ("LanMV9sAd7wArD4vJFi2qDdfnVhFxYSUg6eADduJ3uj").
pub use constants::RAYDIUM_LAUNCHLAB_PROGRAM_ID;
/// Raydium Stable Swap AMM program id, deprecated. ("5quBtoiQqxF9Jv6KYKctB59NT3gtJD2Y65kdnB1Uev3h").
pub use constants::RAYDIUM_STABLE_SWAP_AMM_PROGRAM_ID;
/// Secp256k1 program identifier. ("KeccakSecp256k11111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::secp256k1_program::ID
pub use constants::SECP256K1_PROGRAM_ID;
/// Secp256r1 program identifier. ("Secp256r1SigVerify1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::secp256r1_program::ID
pub use constants::SECP256R1_PROGRAM_ID;
/// SPL Token-2022 program identifier. ("TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb"). /// SPL Token-2022 program identifier. ("TokenzQdBNbLqP5VEhdkAS6EPFLC1PHnBqCXEpPxuEb").
/// @see solana_sdk::pubkey::Pubkey = spl_token_2022_interface::ID /// @see solana_sdk::pubkey::Pubkey = spl_token_2022_interface::ID
pub use constants::SPL_TOKEN_2022_PROGRAM_ID; pub use constants::SPL_TOKEN_2022_PROGRAM_ID;
/// SPL Token program identifier. ("TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA"). /// SPL Token program identifier. ("TokenkegQfeZyiNwAJbNbGKPFXCWuBvf9Ss623VQ5DA").
/// @see solana_sdk::pubkey::Pubkey = spl_token_interface::ID /// @see solana_sdk::pubkey::Pubkey = spl_token_interface::ID
pub use constants::SPL_TOKEN_PROGRAM_ID; pub use constants::SPL_TOKEN_PROGRAM_ID;
/// Stake Config program identifier. ("StakeConfig11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::stake::config::ID
pub use constants::STAKE_CONFIG_PROGRAM_ID;
/// Stake program identifier. ("Stake11111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::stake::ID
pub use constants::STAKE_PROGRAM_ID;
/// System program identifier. ("11111111111111111111111111111111"). /// System program identifier. ("11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::system_program::ID /// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::system_program::ID
pub use constants::SYSTEM_PROGRAM_ID; pub use constants::SYSTEM_PROGRAM_ID;
/// Sysvar Clock program identifier. ("SysvarC1ock11111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::clock::ID
pub use constants::SYSVAR_CLOCK_PROGRAM_ID;
/// Sysvar Epoch Rewards program identifier. ("SysvarEpochRewards1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::epoch_rewards::ID
pub use constants::SYSVAR_EPOCH_REWARDS_PROGRAM_ID;
/// Sysvar Epoch Schedule program identifier. ("SysvarEpochSchedu1e111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::epoch_schedule::ID
pub use constants::SYSVAR_EPOCH_SCHEDULE_PROGRAM_ID;
/// Sysvar Fees program identifier. ("SysvarFees111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::fees::ID
pub use constants::SYSVAR_FEES_PROGRAM_ID;
/// Sysvar Instructions program identifier. ("Sysvar1nstructions1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::instructions::ID
pub use constants::SYSVAR_INSTRUCTIONS_PROGRAM_ID;
/// Sysvar Last Restart Slot program identifier. ("SysvarLastRestartS1ot1111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::last_restart_slot::ID
pub use constants::SYSVAR_LAST_RESTART_SLOT_PROGRAM_ID;
/// Sysvar program identifier. ("Sysvar1111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::ID
pub use constants::SYSVAR_PROGRAM_ID;
/// Sysvar Recent Blockhashes program identifier. ("SysvarRecentB1ockHashes11111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::recent_blockhashes::ID
pub use constants::SYSVAR_RECENT_BLOCKHASHES_PROGRAM_ID;
/// Sysvar Rent program identifier. ("SysvarRent111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::rent::ID
pub use constants::SYSVAR_RENT_PROGRAM_ID;
/// Sysvar Rewards program identifier. ("SysvarRewards111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::rewards::ID
pub use constants::SYSVAR_REWARDS_PROGRAM_ID;
/// Sysvar Slot Hashes program identifier. ("SysvarS1otHashes111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::slot_hashes::ID
pub use constants::SYSVAR_SLOT_HASHES_PROGRAM_ID;
/// Sysvar Slot History program identifier. ("SysvarS1otHistory11111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::slot_history::ID
pub use constants::SYSVAR_SLOT_HISTORY_PROGRAM_ID;
/// Sysvar Stake History program identifier. ("SysvarStakeHistory1111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::sysvar::stake_history::ID
pub use constants::SYSVAR_STAKE_HISTORY_PROGRAM_ID;
/// Vote program identifier. ("Vote111111111111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::vote::ID
pub use constants::VOTE_PROGRAM_ID;
/// Wrapped SOL mint identifier. ("So11111111111111111111111111111111111111112"). /// Wrapped SOL mint identifier. ("So11111111111111111111111111111111111111112").
/// @see solana_sdk::pubkey::Pubkey = spl_token_interface::native_mint::ID /// @see solana_sdk::pubkey::Pubkey = spl_token_interface::native_mint::ID
pub use constants::WSOL_MINT_ID; pub use constants::WSOL_MINT_ID;
/// Zk El Gamal Proof program identifier. ("ZkE1Gama1Proof11111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::zk_elgamal_proof_program::ID
pub use constants::ZK_ELGAMAL_PROOF_PROGRAM_ID;
/// Zk Token Proof program identifier. ("ZkTokenProof1111111111111111111111111111111").
/// @see solana_sdk::pubkey::Pubkey = solana_sdk_ids::zk_token_proof_program::ID
pub use constants::ZK_TOKEN_PROOF_PROGRAM_ID;
/// Application-facing analysis signal DTO. /// Application-facing analysis signal DTO.
pub use db::AnalysisSignalDto; pub use db::AnalysisSignalDto;
/// Persisted analysis signal row. /// Persisted analysis signal row.
@@ -284,6 +411,18 @@ pub use db::PoolTokenDto;
pub use db::PoolTokenEntity; pub use db::PoolTokenEntity;
/// Role of one token inside a normalized pool. /// Role of one token inside a normalized pool.
pub use db::PoolTokenRole; pub use db::PoolTokenRole;
/// Application-facing protocol candidate DTO.
///
/// A protocol candidate records a program/instruction that should be inspected
/// later because it may correspond to an unsupported DEX, launch surface,
/// migration path or protocol-specific non-trade event.
pub use db::ProtocolCandidateDto;
/// Persisted protocol candidate row.
pub use db::ProtocolCandidateEntity;
/// Aggregated protocol candidate diagnostic row.
pub use db::ProtocolCandidateSummaryDto;
/// Aggregated protocol candidate diagnostic row.
pub use db::ProtocolCandidateSummaryEntity;
/// Application-facing normalized swap DTO. /// Application-facing normalized swap DTO.
pub use db::SwapDto; pub use db::SwapDto;
/// Persisted normalized swap row. /// Persisted normalized swap row.
@@ -306,6 +445,10 @@ pub use db::TokenMintEventEntity;
pub use db::TradeEventDto; pub use db::TradeEventDto;
/// Persisted trade-event row. /// Persisted trade-event row.
pub use db::TradeEventEntity; pub use db::TradeEventEntity;
/// Application-facing transaction classification DTO.
pub use db::TransactionClassificationDto;
/// Persisted transaction classification row.
pub use db::TransactionClassificationEntity;
/// Application-facing wallet DTO. /// Application-facing wallet DTO.
pub use db::WalletDto; pub use db::WalletDto;
/// Persisted wallet row. /// Persisted wallet row.
@@ -482,6 +625,20 @@ pub use db::query_pools_get_by_address;
pub use db::query_pools_list; pub use db::query_pools_list;
/// Inserts or updates one normalized pool row by address. /// Inserts or updates one normalized pool row by address.
pub use db::query_pools_upsert; pub use db::query_pools_upsert;
/// Lists protocol candidate summaries ordered by investigation priority.
pub use db::query_protocol_candidate_summaries_list_by_priority;
/// Deletes protocol candidates for one transaction.
///
/// This is useful before recomputing candidates for a replayed transaction.
pub use db::query_protocol_candidates_delete_by_transaction_id;
/// Inserts one protocol candidate row.
pub use db::query_protocol_candidates_insert;
/// Lists protocol candidates for one program id.
pub use db::query_protocol_candidates_list_by_program_id;
/// Lists protocol candidates for one transaction.
pub use db::query_protocol_candidates_list_by_transaction_id;
/// Lists recent protocol candidates ordered from newest to oldest.
pub use db::query_protocol_candidates_list_recent;
/// Lists recent swaps ordered from newest to oldest. /// Lists recent swaps ordered from newest to oldest.
pub use db::query_swaps_list_recent; pub use db::query_swaps_list_recent;
/// Inserts or updates one normalized swap row. /// Inserts or updates one normalized swap row.
@@ -512,6 +669,14 @@ pub use db::query_trade_events_list_by_pair_id;
pub use db::query_trade_events_list_by_transaction_id; pub use db::query_trade_events_list_by_transaction_id;
/// Inserts or updates one trade-event row and returns its stable internal id. /// Inserts or updates one trade-event row and returns its stable internal id.
pub use db::query_trade_events_upsert; pub use db::query_trade_events_upsert;
/// Reads one transaction classification by signature.
pub use db::query_transaction_classifications_get_by_signature;
/// Reads one transaction classification by transaction id.
pub use db::query_transaction_classifications_get_by_transaction_id;
/// Lists recent transaction classifications ordered from newest to oldest.
pub use db::query_transaction_classifications_list_recent;
/// Inserts or updates one transaction classification row.
pub use db::query_transaction_classifications_upsert;
/// Returns one wallet-holding row identified by `(wallet_id, token_id)`, if it exists. /// Returns one wallet-holding row identified by `(wallet_id, token_id)`, if it exists.
pub use db::query_wallet_holdings_get_by_wallet_and_token; pub use db::query_wallet_holdings_get_by_wallet_and_token;
/// Lists wallet-holding rows for one wallet id. /// Lists wallet-holding rows for one wallet id.
@@ -644,6 +809,36 @@ pub use dex_decode::DexDecodeService;
pub use dex_detect::DexDetectService; pub use dex_detect::DexDetectService;
/// Result of one business-level DEX pool detection. /// Result of one business-level DEX pool detection.
pub use dex_detect::DexPoolDetectionResult; pub use dex_detect::DexPoolDetectionResult;
/// Stable DEX event business category.
pub use dex_event_classification::DexEventCategory;
/// Classifies a DEX event kind into a stable category.
pub use dex_event_classification::classify_dex_event_category;
/// Classifies a DEX event kind and returns its persisted category code.
pub use dex_event_classification::classify_dex_event_category_code;
/// Enriches and serializes a decoded DEX payload.
pub use dex_event_classification::enrich_and_serialize_dex_decoded_payload;
/// Enriches a decoded DEX payload with classification metadata.
pub use dex_event_classification::enrich_dex_decoded_payload;
/// Parses, enriches and serializes a decoded DEX payload.
pub use dex_event_classification::enrich_serialized_dex_decoded_payload;
/// Returns true when a decoded payload is a candle candidate.
pub use dex_event_classification::is_decoded_event_candle_candidate;
/// Returns true when a decoded payload is a trade candidate.
pub use dex_event_classification::is_decoded_event_trade_candidate;
/// Returns true for admin/config/permission DEX events.
pub use dex_event_classification::is_dex_admin_event_kind;
/// Returns true when a DEX event kind can directly feed candle materialization.
pub use dex_event_classification::is_dex_candle_candidate_event_kind;
/// Returns true for fee collection DEX events.
pub use dex_event_classification::is_dex_fee_event_kind;
/// Returns true for liquidity lifecycle DEX events.
pub use dex_event_classification::is_dex_liquidity_event_kind;
/// Returns true for pool lifecycle DEX events.
pub use dex_event_classification::is_dex_pool_lifecycle_event_kind;
/// Returns true for reward or emission DEX events.
pub use dex_event_classification::is_dex_reward_event_kind;
/// Returns true for swap-like DEX events.
pub use dex_event_classification::is_dex_trade_event_kind;
/// Global error type used by the `kb_lib` crate. /// Global error type used by the `kb_lib` crate.
/// ///
/// The project intentionally avoids `anyhow` and `thiserror`, so this /// The project intentionally avoids `anyhow` and `thiserror`, so this
@@ -793,6 +988,8 @@ pub use tracing::init_tracing;
pub use trade_aggregation::TradeAggregationResult; pub use trade_aggregation::TradeAggregationResult;
/// Trade-aggregation service. /// Trade-aggregation service.
pub use trade_aggregation::TradeAggregationService; pub use trade_aggregation::TradeAggregationService;
/// Service used to classify projected Solana transactions.
pub use transaction_classification::TransactionClassificationService;
/// Service projecting resolved transaction JSON into internal chain tables. /// Service projecting resolved transaction JSON into internal chain tables.
pub use tx_model::TransactionModelService; pub use tx_model::TransactionModelService;
/// Result of one transaction resolution pass. /// Result of one transaction resolution pass.

22
kb_lib/src/local_pipeline_replay.rs
View File

@@ -63,6 +63,10 @@ pub struct LocalPipelineReplayResult {
/// This is a replay write/result counter, not the number of distinct rows /// This is a replay write/result counter, not the number of distinct rows
/// currently persisted in the analytic signal table. /// currently persisted in the analytic signal table.
pub analytic_signal_upsert_count: usize, pub analytic_signal_upsert_count: usize,
/// Total transaction classification rows upserted during replay.
pub transaction_classification_count: usize,
/// Number of transactions that produced a classification error.
pub transaction_classification_error_count: usize,
/// Number of token metadata rows updated after replay. /// Number of token metadata rows updated after replay.
pub token_metadata_updated_count: usize, pub token_metadata_updated_count: usize,
/// Number of pair symbols updated after replay. /// Number of pair symbols updated after replay.
@@ -122,6 +126,8 @@ impl LocalPipelineReplayService {
let pair_candle_aggregation = let pair_candle_aggregation =
crate::PairCandleAggregationService::new(self.database.clone()); crate::PairCandleAggregationService::new(self.database.clone());
let pair_analytic_signal = crate::PairAnalyticSignalService::new(self.database.clone()); let pair_analytic_signal = crate::PairAnalyticSignalService::new(self.database.clone());
let transaction_classification =
crate::TransactionClassificationService::new(self.database.clone());
let mut result = LocalPipelineReplayResult { let mut result = LocalPipelineReplayResult {
selected_transaction_count: signatures.len(), selected_transaction_count: signatures.len(),
..Default::default() ..Default::default()
@@ -209,6 +215,22 @@ impl LocalPipelineReplayService {
); );
}, },
} }
let classification_result = transaction_classification
.classify_transaction_by_signature(signature.as_str())
.await;
match classification_result {
Ok(_) => {
result.transaction_classification_count += 1;
},
Err(error) => {
result.transaction_classification_error_count += 1;
tracing::warn!(
signature = %signature,
error = %error,
"local pipeline replay transaction classification step failed"
);
},
}
result.replayed_transaction_count += 1; result.replayed_transaction_count += 1;
} }
if config.refresh_missing_token_metadata { if config.refresh_missing_token_metadata {

530
kb_lib/src/protocol_candidate_recording.rs Normal file
View File

@@ -0,0 +1,530 @@
// file: kb_lib/src/protocol_candidate_recording.rs
//! Protocol candidate recording.
//!
//! This module records candidate protocol/program instructions for transactions
//! that were not fully decoded by the current DEX decoders.
/// Input used to record protocol candidates for one classified transaction.
pub(crate) struct ProtocolCandidateRecordingInput<'a> {
/// Database connection.
pub(crate) database: &'a crate::Database,
/// Persisted transaction.
pub(crate) transaction: &'a crate::ChainTransactionDto,
/// Internal transaction id.
pub(crate) transaction_id: i64,
/// Projected instructions for the transaction.
pub(crate) instructions: &'a [crate::ChainInstructionDto],
/// Persisted classification kind.
pub(crate) classification_kind: &'a str,
}
/// Records protocol candidates for one classified transaction.
///
/// Existing candidates for the same transaction are deleted first so replay is
/// deterministic.
pub(crate) async fn record_protocol_candidates_for_classification(
input: crate::protocol_candidate_recording::ProtocolCandidateRecordingInput<'_>,
) -> Result<usize, crate::Error> {
let delete_result = crate::query_protocol_candidates_delete_by_transaction_id(
input.database,
input.transaction_id,
)
.await;
if let Err(error) = delete_result {
return Err(error);
}
let candidate_specs =
crate::protocol_candidate_recording::build_protocol_candidate_specs_for_classification(
input.transaction,
input.transaction_id,
input.instructions,
input.classification_kind,
);
let mut inserted_count = 0_usize;
for candidate_spec in candidate_specs {
let dto = crate::ProtocolCandidateDto::new(
input.transaction_id,
candidate_spec.instruction_id,
input.transaction.signature.clone(),
input.transaction.slot,
candidate_spec.program_id,
candidate_spec.program_name_hint,
candidate_spec.candidate_protocol,
candidate_spec.candidate_surface,
candidate_spec.reason,
candidate_spec.evidence_json,
);
let insert_result = crate::query_protocol_candidates_insert(input.database, &dto).await;
match insert_result {
Ok(_) => {
inserted_count += 1;
},
Err(error) => return Err(error),
}
}
return Ok(inserted_count);
}
struct ProtocolCandidateSpec {
instruction_id: std::option::Option<i64>,
program_id: std::string::String,
program_name_hint: std::option::Option<std::string::String>,
candidate_protocol: std::option::Option<std::string::String>,
candidate_surface: std::option::Option<std::string::String>,
reason: std::string::String,
evidence_json: std::string::String,
}
fn build_protocol_candidate_specs_for_classification(
transaction: &crate::ChainTransactionDto,
transaction_id: i64,
instructions: &[crate::ChainInstructionDto],
classification_kind: &str,
) -> std::vec::Vec<ProtocolCandidateSpec> {
if classification_kind == "known_dex_program_unclassified" {
return build_known_dex_program_candidate_specs(transaction, transaction_id, instructions);
}
if classification_kind == "unknown_or_unclassified" {
return build_unknown_program_candidate_specs(transaction, transaction_id, instructions);
}
return std::vec::Vec::new();
}
fn build_known_dex_program_candidate_specs(
transaction: &crate::ChainTransactionDto,
transaction_id: i64,
instructions: &[crate::ChainInstructionDto],
) -> std::vec::Vec<ProtocolCandidateSpec> {
let mut specs = std::vec::Vec::new();
for instruction in instructions {
let program_id = match instruction.program_id.clone() {
Some(program_id) => program_id,
None => continue,
};
let known_protocol = known_dex_protocol_name(program_id.as_str());
let known_protocol = match known_protocol {
Some(known_protocol) => known_protocol,
None => continue,
};
let evidence_json = build_instruction_evidence_json(
transaction,
transaction_id,
instruction,
"known_dex_program_without_decoded_event",
);
specs.push(ProtocolCandidateSpec {
instruction_id: instruction.id,
program_id,
program_name_hint: instruction.program_name.clone(),
candidate_protocol: Some(known_protocol.to_string()),
candidate_surface: None,
reason: "known DEX program instruction did not produce a decoded DEX event".to_string(),
evidence_json,
});
}
return specs;
}
fn build_unknown_program_candidate_specs(
transaction: &crate::ChainTransactionDto,
transaction_id: i64,
instructions: &[crate::ChainInstructionDto],
) -> std::vec::Vec<ProtocolCandidateSpec> {
let mut specs = std::vec::Vec::new();
for instruction in instructions {
let program_id = match instruction.program_id.clone() {
Some(program_id) => program_id,
None => continue,
};
if should_ignore_program_id(program_id.as_str()) {
continue;
}
let surface_hint = infer_candidate_surface(program_id.as_str(), instruction);
let evidence_json = build_instruction_evidence_json(
transaction,
transaction_id,
instruction,
"unknown_or_unclassified_program_instruction",
);
specs.push(ProtocolCandidateSpec {
instruction_id: instruction.id,
program_id,
program_name_hint: instruction.program_name.clone(),
candidate_protocol: None,
candidate_surface: surface_hint,
reason: "transaction has no decoded DEX event and includes a non-ignored program instruction".to_string(),
evidence_json,
});
}
return specs;
}
fn build_instruction_evidence_json(
transaction: &crate::ChainTransactionDto,
transaction_id: i64,
instruction: &crate::ChainInstructionDto,
reason_code: &str,
) -> std::string::String {
let evidence_value = serde_json::json!({
"reasonCode": reason_code,
"transactionId": transaction_id,
"signature": transaction.signature,
"slot": transaction.slot,
"instructionId": instruction.id,
"parentInstructionId": instruction.parent_instruction_id,
"instructionIndex": instruction.instruction_index,
"programId": instruction.program_id,
"programName": instruction.program_name,
"stackHeight": instruction.stack_height,
"parsedType": instruction.parsed_type
});
let evidence_json_result = serde_json::to_string(&evidence_value);
match evidence_json_result {
Ok(evidence_json) => return evidence_json,
Err(error) => {
return format!(
"{{\"reasonCode\":\"evidence_serialization_failed\",\"error\":\"{}\"}}",
error
);
},
}
}
fn known_dex_protocol_name(program_id: &str) -> std::option::Option<&'static str> {
if program_id == crate::RAYDIUM_AMM_V4_PROGRAM_ID {
return Some("raydium_amm_v4");
}
if program_id == crate::RAYDIUM_CPMM_PROGRAM_ID {
return Some("raydium_cpmm");
}
if program_id == crate::RAYDIUM_CLMM_PROGRAM_ID {
return Some("raydium_clmm");
}
if program_id == crate::RAYDIUM_LAUNCHLAB_PROGRAM_ID {
return Some("raydium_launchlab");
}
if program_id == crate::RAYDIUM_AMM_ROUTING_PROGRAM_ID {
return Some("raydium_router");
}
if program_id == crate::RAYDIUM_STABLE_SWAP_AMM_PROGRAM_ID {
return Some("raydium_stable_swap");
}
if program_id == crate::PUMP_FUN_PROGRAM_ID {
return Some("pump_fun");
}
if program_id == crate::PUMP_SWAP_PROGRAM_ID {
return Some("pump_swap");
}
if program_id == crate::METEORA_DBC_PROGRAM_ID {
return Some("meteora_dbc");
}
if program_id == crate::METEORA_DLMM_PROGRAM_ID {
return Some("meteora_dlmm");
}
if program_id == crate::METEORA_DAMM_V1_PROGRAM_ID {
return Some("meteora_damm_v1");
}
if program_id == crate::METEORA_DAMM_V2_PROGRAM_ID {
return Some("meteora_damm_v2");
}
if program_id == crate::ORCA_WHIRLPOOLS_PROGRAM_ID {
return Some("orca_whirlpools");
}
if program_id == crate::FLUXBEAM_PROGRAM_ID {
return Some("fluxbeam");
}
if program_id == crate::DEXLAB_PROGRAM_ID {
return Some("dexlab");
}
return None;
}
fn should_ignore_program_id(program_id: &str) -> bool {
if program_id == crate::SYSTEM_PROGRAM_ID {
return true;
}
if program_id == crate::SPL_TOKEN_PROGRAM_ID {
return true;
}
if program_id == crate::SPL_TOKEN_2022_PROGRAM_ID {
return true;
}
if program_id == crate::ASSOCIATED_TOKEN_PROGRAM_ID {
return true;
}
if program_id == crate::COMPUTE_BUDGET_PROGRAM_ID {
return true;
}
if program_id == crate::ADDRESS_LOOKUP_TABLE_PROGRAM_ID {
return true;
}
if program_id == crate::BPF_LOADER_DEPRECATED_PROGRAM_ID {
return true;
}
if program_id == crate::BPF_LOADER_UPGRADEABLE_PROGRAM_ID {
return true;
}
if program_id == crate::LOADER_V4_PROGRAM_ID {
return true;
}
if program_id == crate::NATIVE_LOADER_PROGRAM_ID {
return true;
}
if program_id == crate::CONFIG_PROGRAM_ID {
return true;
}
if program_id == crate::VOTE_PROGRAM_ID {
return true;
}
if program_id == crate::STAKE_PROGRAM_ID {
return true;
}
if program_id == crate::STAKE_CONFIG_PROGRAM_ID {
return true;
}
if program_id == crate::ED25519_PROGRAM_ID {
return true;
}
if program_id == crate::SECP256K1_PROGRAM_ID {
return true;
}
if program_id == crate::SECP256R1_PROGRAM_ID {
return true;
}
if program_id == crate::ZK_TOKEN_PROOF_PROGRAM_ID {
return true;
}
if program_id == crate::ZK_ELGAMAL_PROOF_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_CLOCK_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_RENT_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_INSTRUCTIONS_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_EPOCH_REWARDS_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_EPOCH_SCHEDULE_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_FEES_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_LAST_RESTART_SLOT_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_RECENT_BLOCKHASHES_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_REWARDS_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_SLOT_HASHES_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_SLOT_HISTORY_PROGRAM_ID {
return true;
}
if program_id == crate::SYSVAR_STAKE_HISTORY_PROGRAM_ID {
return true;
}
return false;
}
fn infer_candidate_surface(
program_id: &str,
instruction: &crate::ChainInstructionDto,
) -> std::option::Option<std::string::String> {
if program_id == crate::ARBITRAGE_BOT_6MWVT_PROGRAM_ID {
return Some("arbitrage_bot".to_string());
}
if is_known_launch_surface_program_id(program_id) {
return Some("launch_surface".to_string());
}
if let Some(program_name) = instruction.program_name.as_deref() {
let normalized = program_name.to_ascii_lowercase();
if normalized.contains("arbitrage") {
return Some("arbitrage_bot".to_string());
}
if normalized.contains("sandwich") {
return Some("arbitrage_bot".to_string());
}
if normalized.contains("launch") {
return Some("launch_surface".to_string());
}
if normalized.contains("meteora") {
return Some("meteora_related".to_string());
}
if normalized.contains("raydium") {
return Some("raydium_related".to_string());
}
if normalized.contains("pump") {
return Some("pump_related".to_string());
}
if normalized.contains("swap") {
return Some("swap_related".to_string());
}
}
return None;
}
fn is_known_launch_surface_program_id(_program_id: &str) -> bool {
// Filled in later after program ids are verified from live corpus and
// official or sufficiently reliable references.
return false;
}
#[cfg(test)]
mod tests {
#[test]
fn associated_token_program_is_ignored() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some(crate::ASSOCIATED_TOKEN_PROGRAM_ID.to_string()),
Some("spl-associated-token-account".to_string()),
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"unknown_or_unclassified",
);
assert_eq!(specs.len(), 0);
}
#[test]
fn meteora_dlmm_is_known_dex_protocol() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some(crate::METEORA_DLMM_PROGRAM_ID.to_string()),
Some("Meteora DLMM".to_string()),
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"known_dex_program_unclassified",
);
assert_eq!(specs.len(), 1);
assert_eq!(specs[0].program_id, crate::METEORA_DLMM_PROGRAM_ID);
assert_eq!(specs[0].candidate_protocol, Some("meteora_dlmm".to_string()));
}
#[test]
fn known_arbitrage_bot_gets_surface_hint() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some(crate::ARBITRAGE_BOT_6MWVT_PROGRAM_ID.to_string()),
None,
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"unknown_or_unclassified",
);
assert_eq!(specs.len(), 1);
assert_eq!(specs[0].candidate_surface, Some("arbitrage_bot".to_string()));
}
fn test_instruction(
instruction_index: u32,
program_id: std::option::Option<std::string::String>,
program_name: std::option::Option<std::string::String>,
) -> crate::ChainInstructionDto {
return crate::ChainInstructionDto::new(
1,
None,
instruction_index,
None,
program_id,
program_name,
None,
"[]".to_string(),
None,
None,
Some(serde_json::json!({}).to_string()),
);
}
fn test_transaction() -> crate::ChainTransactionDto {
let mut transaction = crate::ChainTransactionDto::new(
"signature_1".to_string(),
Some(123),
None,
Some("test".to_string()),
None,
None,
None,
serde_json::json!({}).to_string(),
);
transaction.id = Some(1);
return transaction;
}
#[test]
fn known_dex_candidate_is_built_for_unclassified_known_program() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some(crate::METEORA_DAMM_V2_PROGRAM_ID.to_string()),
Some("Meteora".to_string()),
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"known_dex_program_unclassified",
);
assert_eq!(specs.len(), 1);
assert_eq!(specs[0].program_id, crate::METEORA_DAMM_V2_PROGRAM_ID);
assert_eq!(specs[0].candidate_protocol, Some("meteora_damm_v2".to_string()));
}
#[test]
fn ignored_program_is_not_recorded_as_unknown_candidate() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some(crate::SPL_TOKEN_PROGRAM_ID.to_string()),
Some("spl-token".to_string()),
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"unknown_or_unclassified",
);
assert_eq!(specs.len(), 0);
}
#[test]
fn unknown_non_ignored_program_is_recorded() {
let transaction = test_transaction();
let instructions = vec![test_instruction(
0,
Some("UnknownProgram111111111111111111111111111111111".to_string()),
Some("unknown swap program".to_string()),
)];
let specs = super::build_protocol_candidate_specs_for_classification(
&transaction,
1,
&instructions,
"unknown_or_unclassified",
);
assert_eq!(specs.len(), 1);
assert_eq!(specs[0].candidate_surface, Some("swap_related".to_string()));
}
}

11
kb_lib/src/token_backfill.rs
View File

@@ -82,6 +82,7 @@ pub struct TokenBackfillService {
wallet_observation_service: crate::WalletObservationService, wallet_observation_service: crate::WalletObservationService,
trade_aggregation_service: crate::TradeAggregationService, trade_aggregation_service: crate::TradeAggregationService,
pair_candle_aggregation_service: crate::PairCandleAggregationService, pair_candle_aggregation_service: crate::PairCandleAggregationService,
transaction_classification_service: crate::TransactionClassificationService,
token_metadata_service: crate::TokenMetadataBackfillService, token_metadata_service: crate::TokenMetadataBackfillService,
} }
@@ -102,6 +103,8 @@ impl TokenBackfillService {
let trade_aggregation_service = crate::TradeAggregationService::new(database.clone()); let trade_aggregation_service = crate::TradeAggregationService::new(database.clone());
let pair_candle_aggregation_service = let pair_candle_aggregation_service =
crate::PairCandleAggregationService::new(database.clone()); crate::PairCandleAggregationService::new(database.clone());
let transaction_classification_service =
crate::TransactionClassificationService::new(database.clone());
let token_metadata_service = crate::TokenMetadataBackfillService::new( let token_metadata_service = crate::TokenMetadataBackfillService::new(
http_pool.clone(), http_pool.clone(),
database.clone(), database.clone(),
@@ -120,6 +123,7 @@ impl TokenBackfillService {
wallet_observation_service, wallet_observation_service,
trade_aggregation_service, trade_aggregation_service,
pair_candle_aggregation_service, pair_candle_aggregation_service,
transaction_classification_service,
token_metadata_service, token_metadata_service,
}; };
} }
@@ -436,6 +440,13 @@ impl TokenBackfillService {
Ok(pair_candle_aggregations) => pair_candle_aggregations, Ok(pair_candle_aggregations) => pair_candle_aggregations,
Err(error) => return Err(error), Err(error) => return Err(error),
}; };
let transaction_classification_result = self
.transaction_classification_service
.classify_transaction_by_signature(signature.as_str())
.await;
if let Err(error) = transaction_classification_result {
return Err(error);
}
return Ok(TokenBackfillSignatureResult { return Ok(TokenBackfillSignatureResult {
resolved_transaction_count: 1, resolved_transaction_count: 1,
missing_transaction_count: 0, missing_transaction_count: 0,

2825
kb_lib/src/trade_aggregation.rs
View File

File diff suppressed because it is too large Load Diff

216
kb_lib/src/trade_aggregation_context.rs Normal file
View File

@@ -0,0 +1,216 @@
// file: kb_lib/src/trade_aggregation_context.rs
//! Database context loading for trade aggregation.
//!
//! This module resolves the normalized database context required by
//! `TradeAggregationService`: transaction, decoded events, pool, pair,
//! base/quote token metadata and pool token vault addresses.
/// Transaction-level context used by trade aggregation.
pub(crate) struct TradeAggregationTransactionContext {
/// Persisted transaction row.
pub(crate) transaction: crate::ChainTransactionDto,
/// Internal transaction id.
pub(crate) transaction_id: i64,
/// Decoded DEX events attached to the transaction.
pub(crate) decoded_events: std::vec::Vec<crate::DexDecodedEventDto>,
}
/// Decoded-event-level context used by trade aggregation.
pub(crate) struct TradeAggregationDecodedEventContext {
/// Internal decoded event id.
pub(crate) decoded_event_id: i64,
/// Existing trade event, when this decoded event was already materialized.
pub(crate) existing_trade_event: std::option::Option<crate::TradeEventDto>,
/// Pool account address from the decoded event.
pub(crate) pool_address: std::string::String,
/// Persisted pool row.
pub(crate) pool: crate::PoolDto,
/// Internal pool id.
pub(crate) pool_id: i64,
/// Persisted pair row.
pub(crate) pair: crate::PairDto,
/// Internal pair id.
pub(crate) pair_id: i64,
/// Base token mint, when the token row exists.
pub(crate) base_token_mint: std::option::Option<std::string::String>,
/// Base token decimals, when known.
pub(crate) base_token_decimals: std::option::Option<u8>,
/// Quote token mint, when the token row exists.
pub(crate) quote_token_mint: std::option::Option<std::string::String>,
/// Quote token decimals, when known.
pub(crate) quote_token_decimals: std::option::Option<u8>,
/// Base token vault address, when known.
pub(crate) base_vault_address: std::option::Option<std::string::String>,
/// Quote token vault address, when known.
pub(crate) quote_vault_address: std::option::Option<std::string::String>,
}
/// Loads a transaction and its decoded DEX events from one signature.
pub(crate) async fn load_trade_aggregation_transaction_context(
database: &crate::Database,
signature: &str,
) -> Result<crate::trade_aggregation_context::TradeAggregationTransactionContext, crate::Error> {
let transaction_result =
crate::query_chain_transactions_get_by_signature(database, signature).await;
let transaction_option = match transaction_result {
Ok(transaction_option) => transaction_option,
Err(error) => return Err(error),
};
let transaction = match transaction_option {
Some(transaction) => transaction,
None => {
return Err(crate::Error::InvalidState(format!(
"cannot aggregate trades for unknown transaction '{}'",
signature
)));
},
};
let transaction_id = match transaction.id {
Some(transaction_id) => transaction_id,
None => {
return Err(crate::Error::InvalidState(format!(
"transaction '{}' has no internal id",
signature
)));
},
};
let decoded_events_result =
crate::query_dex_decoded_events_list_by_transaction_id(database, transaction_id).await;
let decoded_events = match decoded_events_result {
Ok(decoded_events) => decoded_events,
Err(error) => return Err(error),
};
return Ok(crate::trade_aggregation_context::TradeAggregationTransactionContext {
transaction,
transaction_id,
decoded_events,
});
}
/// Loads the normalized DB context for one decoded event.
///
/// Returns `Ok(None)` when the decoded event is not materializable yet:
/// missing pool account, pool row or pair row.
pub(crate) async fn load_trade_aggregation_decoded_event_context(
database: &crate::Database,
decoded_event: &crate::DexDecodedEventDto,
) -> Result<
std::option::Option<crate::trade_aggregation_context::TradeAggregationDecodedEventContext>,
crate::Error,
> {
let decoded_event_id = match decoded_event.id {
Some(decoded_event_id) => decoded_event_id,
None => {
return Err(crate::Error::InvalidState("decoded event has no internal id".to_string()));
},
};
let existing_trade_result =
crate::query_trade_events_get_by_decoded_event_id(database, decoded_event_id).await;
let existing_trade_event = match existing_trade_result {
Ok(existing_trade_event) => existing_trade_event,
Err(error) => return Err(error),
};
let pool_address = match decoded_event.pool_account.clone() {
Some(pool_address) => pool_address,
None => return Ok(None),
};
let pool_result = crate::query_pools_get_by_address(database, pool_address.as_str()).await;
let pool_option = match pool_result {
Ok(pool_option) => pool_option,
Err(error) => return Err(error),
};
let pool = match pool_option {
Some(pool) => pool,
None => return Ok(None),
};
let pool_id = match pool.id {
Some(pool_id) => pool_id,
None => {
return Err(crate::Error::InvalidState(format!(
"pool '{}' has no internal id",
pool.address
)));
},
};
let pair_result = crate::query_pairs_get_by_pool_id(database, pool_id).await;
let pair_option = match pair_result {
Ok(pair_option) => pair_option,
Err(error) => return Err(error),
};
let pair = match pair_option {
Some(pair) => pair,
None => return Ok(None),
};
let pair_id = match pair.id {
Some(pair_id) => pair_id,
None => {
return Err(crate::Error::InvalidState(format!(
"pair for pool '{}' has no internal id",
pool_id
)));
},
};
let base_token_result = crate::query_tokens_get_by_id(database, pair.base_token_id).await;
let (base_token_mint, base_token_decimals) = match base_token_result {
Ok(Some(token)) => (Some(token.mint), token.decimals),
Ok(None) => (None, None),
Err(error) => return Err(error),
};
let quote_token_result = crate::query_tokens_get_by_id(database, pair.quote_token_id).await;
let (quote_token_mint, quote_token_decimals) = match quote_token_result {
Ok(Some(token)) => (Some(token.mint), token.decimals),
Ok(None) => (None, None),
Err(error) => return Err(error),
};
let pool_tokens_result = crate::query_pool_tokens_list_by_pool_id(database, pool_id).await;
let pool_tokens = match pool_tokens_result {
Ok(pool_tokens) => pool_tokens,
Err(error) => return Err(error),
};
let base_vault_address =
crate::trade_aggregation_context::find_pool_token_vault_address_by_token_id(
&pool_tokens,
pair.base_token_id,
);
let quote_vault_address =
crate::trade_aggregation_context::find_pool_token_vault_address_by_token_id(
&pool_tokens,
pair.quote_token_id,
);
return Ok(Some(crate::trade_aggregation_context::TradeAggregationDecodedEventContext {
decoded_event_id,
existing_trade_event,
pool_address,
pool,
pool_id,
pair,
pair_id,
base_token_mint,
base_token_decimals,
quote_token_mint,
quote_token_decimals,
base_vault_address,
quote_vault_address,
}));
}
fn find_pool_token_vault_address_by_token_id(
pool_tokens: &[crate::PoolTokenDto],
token_id: i64,
) -> std::option::Option<std::string::String> {
for pool_token in pool_tokens {
if pool_token.token_id != token_id {
continue;
}
let vault_address = match pool_token.vault_address.clone() {
Some(vault_address) => vault_address.trim().to_string(),
None => continue,
};
if vault_address.is_empty() {
continue;
}
return Some(vault_address);
}
return None;
}

650
kb_lib/src/trade_amount_resolution.rs Normal file
View File

@@ -0,0 +1,650 @@
// file: kb_lib/src/trade_amount_resolution.rs
//! Trade amount resolution orchestration.
//!
//! This module resolves base/quote raw amounts and quote/base price for one
//! decoded trade candidate by applying protocol-specific and generic fallback
//! strategies in deterministic order.
/// Input context required to resolve trade amounts.
pub(crate) struct TradeAmountResolutionInput<'a> {
/// Database connection.
pub(crate) database: &'a crate::Database,
/// Persisted transaction row.
pub(crate) transaction: &'a crate::ChainTransactionDto,
/// Decoded DEX event row.
pub(crate) decoded_event: &'a crate::DexDecodedEventDto,
/// Decoded event payload.
pub(crate) payload: &'a serde_json::Value,
/// Pool account address.
pub(crate) pool_address: &'a str,
/// Base token mint, when known.
pub(crate) base_token_mint: std::option::Option<&'a str>,
/// Quote token mint, when known.
pub(crate) quote_token_mint: std::option::Option<&'a str>,
/// Base token decimals, when known.
pub(crate) base_token_decimals: std::option::Option<u8>,
/// Quote token decimals, when known.
pub(crate) quote_token_decimals: std::option::Option<u8>,
/// Base token vault address, when known.
pub(crate) base_vault_address: std::option::Option<&'a str>,
/// Quote token vault address, when known.
pub(crate) quote_vault_address: std::option::Option<&'a str>,
}
/// Resolved raw trade amounts and quote/base price.
#[derive(Debug, Clone)]
pub(crate) struct TradeAmountResolution {
/// Base amount in raw token units.
pub(crate) base_amount_raw: std::option::Option<std::string::String>,
/// Quote amount in raw token units.
pub(crate) quote_amount_raw: std::option::Option<std::string::String>,
/// Quote/base price.
pub(crate) price_quote_per_base: std::option::Option<f64>,
}
/// Resolves trade amounts from payload and protocol-specific fallbacks.
pub(crate) async fn resolve_trade_amounts(
input: &crate::trade_amount_resolution::TradeAmountResolutionInput<'_>,
) -> Result<crate::trade_amount_resolution::TradeAmountResolution, crate::Error> {
let mut base_amount_raw = crate::trade_amount_resolution::extract_amount_string(
input.payload,
&["baseAmountRaw", "base_amount_raw", "baseAmount", "amountBase", "amountInBase"],
);
let mut quote_amount_raw = crate::trade_amount_resolution::extract_amount_string(
input.payload,
&[
"quoteAmountRaw",
"quote_amount_raw",
"quoteAmount",
"amountQuote",
"amountOutQuote",
],
);
let mut price_quote_per_base = None;
if input.decoded_event.event_kind.starts_with("pump_swap.")
&& (base_amount_raw.is_none()
|| quote_amount_raw.is_none()
|| price_quote_per_base.is_none())
{
let resolution_result = crate::trade_amount_resolution::apply_pump_swap_amount_fallbacks(
input,
&mut base_amount_raw,
&mut quote_amount_raw,
&mut price_quote_per_base,
)
.await;
if let Err(error) = resolution_result {
return Err(error);
}
}
if input.decoded_event.event_kind.starts_with("pump_fun.")
&& (base_amount_raw.is_none()
|| quote_amount_raw.is_none()
|| price_quote_per_base.is_none())
{
let resolution_result = crate::trade_amount_resolution::apply_pump_fun_amount_fallback(
input,
&mut base_amount_raw,
&mut quote_amount_raw,
&mut price_quote_per_base,
);
if let Err(error) = resolution_result {
return Err(error);
}
}
if (input.decoded_event.event_kind.starts_with("raydium_cpmm.")
|| input.decoded_event.event_kind.starts_with("raydium_clmm."))
&& (base_amount_raw.is_none()
|| quote_amount_raw.is_none()
|| price_quote_per_base.is_none())
{
let resolution_result =
crate::trade_amount_resolution::apply_raydium_instruction_amount_fallback(
input,
&mut base_amount_raw,
&mut quote_amount_raw,
&mut price_quote_per_base,
)
.await;
if let Err(error) = resolution_result {
return Err(error);
}
}
if input.decoded_event.event_kind.starts_with("raydium_cpmm.")
&& (base_amount_raw.is_none() || quote_amount_raw.is_none())
{
let resolution_result = crate::trade_amount_resolution::apply_vault_balance_delta_fallback(
input,
input.base_vault_address,
input.quote_vault_address,
&mut base_amount_raw,
&mut quote_amount_raw,
&mut price_quote_per_base,
);
if let Err(error) = resolution_result {
return Err(error);
}
}
if input.decoded_event.event_kind.starts_with("raydium_clmm.")
&& (base_amount_raw.is_none() || quote_amount_raw.is_none())
{
let resolution_result = crate::trade_amount_resolution::apply_vault_balance_delta_fallback(
input,
input.base_vault_address,
input.quote_vault_address,
&mut base_amount_raw,
&mut quote_amount_raw,
&mut price_quote_per_base,
);
if let Err(error) = resolution_result {
return Err(error);
}
}
if price_quote_per_base.is_none() {
price_quote_per_base =
crate::trade_metric_update::compute_price_quote_per_base_from_raw_amounts_with_decimals(
base_amount_raw.as_deref(),
quote_amount_raw.as_deref(),
input.base_token_decimals,
input.quote_token_decimals,
);
}
if price_quote_per_base.is_none() {
price_quote_per_base =
crate::trade_solana_amounts::compute_price_quote_per_base_with_decimals(
input.transaction.meta_json.as_deref(),
input.transaction.transaction_json.as_str(),
input.base_vault_address,
input.quote_vault_address,
);
}
if price_quote_per_base.is_none() {
price_quote_per_base =
crate::trade_metric_update::compute_price_quote_per_base_from_raw_amounts(
base_amount_raw.as_deref(),
quote_amount_raw.as_deref(),
);
}
return Ok(crate::trade_amount_resolution::TradeAmountResolution {
base_amount_raw,
quote_amount_raw,
price_quote_per_base,
});
}
async fn apply_pump_swap_amount_fallbacks(
input: &crate::trade_amount_resolution::TradeAmountResolutionInput<'_>,
base_amount_raw: &mut std::option::Option<std::string::String>,
quote_amount_raw: &mut std::option::Option<std::string::String>,
price_quote_per_base: &mut std::option::Option<f64>,
) -> Result<(), crate::Error> {
let pool_owner_result = match (input.base_token_mint, input.quote_token_mint) {
(Some(base_mint), Some(quote_mint)) => {
crate::trade_pump_swap_amounts::resolve_pump_swap_trade_amounts_from_pool_balance_deltas(
input.transaction.meta_json.as_deref(),
input.pool_address,
base_mint,
quote_mint,
input.decoded_event.event_kind.as_str(),
input.base_token_decimals,
input.quote_token_decimals,
)
},
_ => Ok(crate::trade_pump_swap_amounts::PumpSwapPoolBalanceDeltaResolution::MissingData),
};
let pool_owner_resolution = match pool_owner_result {
Ok(pool_owner_resolution) => pool_owner_resolution,
Err(error) => return Err(error),
};
let pool_owner_resolution_label = pool_owner_resolution.as_label();
tracing::debug!(
event_kind = %input.decoded_event.event_kind,
pool_account = ?input.decoded_event.pool_account,
decoded_event_id = ?input.decoded_event.id,
transaction_signature = %input.transaction.signature,
base_mint = ?input.base_token_mint,
quote_mint = ?input.quote_token_mint,
pool_owner_resolution = %pool_owner_resolution_label,
"pump_swap pool-owner delta resolution result"
);
match pool_owner_resolution {
crate::trade_pump_swap_amounts::PumpSwapPoolBalanceDeltaResolution::Matched(amounts) => {
*base_amount_raw = Some(amounts.base_amount_raw);
*quote_amount_raw = Some(amounts.quote_amount_raw);
*price_quote_per_base = Some(amounts.price_quote_per_base);
tracing::debug!(
event_kind = %input.decoded_event.event_kind,
pool_account = ?input.decoded_event.pool_account,
decoded_event_id = ?input.decoded_event.id,
base_mint = ?input.base_token_mint,
quote_mint = ?input.quote_token_mint,
base_amount_raw = ?base_amount_raw,
quote_amount_raw = ?quote_amount_raw,
price_quote_per_base = ?price_quote_per_base,
"pump_swap trade amounts recovered from pool-owner token balance deltas"
);
},
crate::trade_pump_swap_amounts::PumpSwapPoolBalanceDeltaResolution::DirectionMismatch => {
tracing::debug!(
event_kind = %input.decoded_event.event_kind,
pool_account = ?input.decoded_event.pool_account,
decoded_event_id = ?input.decoded_event.id,
transaction_signature = %input.transaction.signature,
"pump_swap pool-owner full-transaction delta direction mismatch; continuing with instruction-scoped fallbacks"
);
},
crate::trade_pump_swap_amounts::PumpSwapPoolBalanceDeltaResolution::MissingData => {},
}
let decoded_instruction_index_result =
crate::trade_amount_resolution::load_decoded_instruction_index(
input.database,
input.decoded_event,
)
.await;
let decoded_instruction_index = match decoded_instruction_index_result {
Ok(decoded_instruction_index) => decoded_instruction_index,
Err(error) => return Err(error),
};
let payload_user_base_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["userBaseTokenAccount", "user_base_token_account"],
);
let payload_user_quote_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["userQuoteTokenAccount", "user_quote_token_account"],
);
let payload_pool_base_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["poolBaseTokenAccount", "pool_base_token_account"],
);
let payload_pool_quote_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["poolQuoteTokenAccount", "pool_quote_token_account"],
);
let effective_base_vault_address = match input.base_vault_address {
Some(base_vault_address) => Some(base_vault_address),
None => payload_pool_base_token_account.as_deref(),
};
let effective_quote_vault_address = match input.quote_vault_address {
Some(quote_vault_address) => Some(quote_vault_address),
None => payload_pool_quote_token_account.as_deref(),
};
let (input_vault_address, output_vault_address, input_token_account, output_token_account) =
if input.decoded_event.event_kind.ends_with(".buy") {
(
effective_quote_vault_address,
effective_base_vault_address,
payload_user_quote_token_account.as_deref(),
payload_user_base_token_account.as_deref(),
)
} else if input.decoded_event.event_kind.ends_with(".sell") {
(
effective_base_vault_address,
effective_quote_vault_address,
payload_user_base_token_account.as_deref(),
payload_user_quote_token_account.as_deref(),
)
} else {
(None, None, None, None)
};
let inferred_result =
crate::trade_solana_amounts::extract_trade_amounts_from_instruction_token_transfers(
input.transaction.meta_json.as_deref(),
decoded_instruction_index,
input_vault_address,
output_vault_address,
input_token_account,
output_token_account,
effective_base_vault_address,
effective_quote_vault_address,
);
let inferred = match inferred_result {
Ok(inferred) => inferred,
Err(error) => return Err(error),
};
if base_amount_raw.is_none() {
*base_amount_raw = inferred.0;
}
if quote_amount_raw.is_none() {
*quote_amount_raw = inferred.1;
}
if price_quote_per_base.is_none() {
*price_quote_per_base = inferred.2;
}
if base_amount_raw.is_none() || quote_amount_raw.is_none() {
let fallback_result =
crate::trade_solana_amounts::extract_trade_amounts_from_vault_balance_deltas(
input.transaction.transaction_json.as_str(),
input.transaction.meta_json.as_deref(),
effective_base_vault_address,
effective_quote_vault_address,
);
let fallback = match fallback_result {
Ok(fallback) => fallback,
Err(error) => return Err(error),
};
if base_amount_raw.is_none() {
*base_amount_raw = fallback.0;
}
if quote_amount_raw.is_none() {
*quote_amount_raw = fallback.1;
}
if price_quote_per_base.is_none() {
*price_quote_per_base = fallback.2;
}
}
if base_amount_raw.is_none() || quote_amount_raw.is_none() || price_quote_per_base.is_none() {
let transaction_value_result =
crate::trade_pump_swap_amounts::build_transaction_value_with_meta_json(
input.transaction.transaction_json.as_str(),
input.transaction.meta_json.as_deref(),
);
let transaction_value = match transaction_value_result {
Ok(transaction_value) => transaction_value,
Err(error) => return Err(error),
};
let fallback_amounts = match (input.base_token_mint, input.quote_token_mint) {
(Some(base_mint), Some(quote_mint)) => {
crate::trade_pump_swap_amounts::try_build_pump_swap_trade_amounts_from_token_balance_deltas(
&transaction_value,
base_mint,
quote_mint,
)
},
_ => None,
};
if let Some(fallback_amounts) = fallback_amounts {
if base_amount_raw.is_none() {
*base_amount_raw = crate::trade_pump_swap_amounts::convert_ui_amount_to_raw_string(
fallback_amounts.base_amount,
input.base_token_decimals,
);
}
if quote_amount_raw.is_none() {
*quote_amount_raw = crate::trade_pump_swap_amounts::convert_ui_amount_to_raw_string(
fallback_amounts.quote_amount,
input.quote_token_decimals,
);
}
if price_quote_per_base.is_none() {
*price_quote_per_base = Some(fallback_amounts.price_quote_per_base);
}
tracing::debug!(
event_kind = %input.decoded_event.event_kind,
pool_account = ?input.decoded_event.pool_account,
decoded_event_id = ?input.decoded_event.id,
base_mint = ?input.base_token_mint,
quote_mint = ?input.quote_token_mint,
base_amount_raw = ?base_amount_raw,
quote_amount_raw = ?quote_amount_raw,
price_quote_per_base = ?price_quote_per_base,
"pump_swap trade amounts recovered from token balance deltas"
);
}
}
return Ok(());
}
fn apply_pump_fun_amount_fallback(
input: &crate::trade_amount_resolution::TradeAmountResolutionInput<'_>,
base_amount_raw: &mut std::option::Option<std::string::String>,
quote_amount_raw: &mut std::option::Option<std::string::String>,
price_quote_per_base: &mut std::option::Option<f64>,
) -> Result<(), crate::Error> {
let inferred_result = crate::trade_solana_amounts::extract_pump_fun_amounts_from_transaction(
input.transaction.transaction_json.as_str(),
input.transaction.meta_json.as_deref(),
input.base_vault_address,
input.quote_vault_address,
);
let inferred = match inferred_result {
Ok(inferred) => inferred,
Err(error) => return Err(error),
};
if base_amount_raw.is_none() {
*base_amount_raw = inferred.0;
}
if quote_amount_raw.is_none() {
*quote_amount_raw = inferred.1;
}
if price_quote_per_base.is_none() {
*price_quote_per_base = inferred.2;
}
return Ok(());
}
async fn apply_raydium_instruction_amount_fallback(
input: &crate::trade_amount_resolution::TradeAmountResolutionInput<'_>,
base_amount_raw: &mut std::option::Option<std::string::String>,
quote_amount_raw: &mut std::option::Option<std::string::String>,
price_quote_per_base: &mut std::option::Option<f64>,
) -> Result<(), crate::Error> {
let decoded_instruction_index_result =
crate::trade_amount_resolution::load_decoded_instruction_index(
input.database,
input.decoded_event,
)
.await;
let decoded_instruction_index = match decoded_instruction_index_result {
Ok(decoded_instruction_index) => decoded_instruction_index,
Err(error) => return Err(error),
};
let payload_input_vault_address =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["inputVault", "input_vault"],
);
let payload_output_vault_address =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["outputVault", "output_vault"],
);
let payload_input_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["inputTokenAccount", "input_token_account"],
);
let payload_output_token_account =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["outputTokenAccount", "output_token_account"],
);
let payload_base_vault_address =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["baseVault", "base_vault"],
);
let payload_quote_vault_address =
crate::trade_amount_resolution::extract_string_by_candidate_keys(
input.payload,
&["quoteVault", "quote_vault"],
);
let effective_base_vault_address = match input.base_vault_address {
Some(base_vault_address) => Some(base_vault_address),
None => payload_base_vault_address.as_deref(),
};
let effective_quote_vault_address = match input.quote_vault_address {
Some(quote_vault_address) => Some(quote_vault_address),
None => payload_quote_vault_address.as_deref(),
};
let inferred_result =
crate::trade_solana_amounts::extract_trade_amounts_from_instruction_token_transfers(
input.transaction.meta_json.as_deref(),
decoded_instruction_index,
payload_input_vault_address.as_deref(),
payload_output_vault_address.as_deref(),
payload_input_token_account.as_deref(),
payload_output_token_account.as_deref(),
effective_base_vault_address,
effective_quote_vault_address,
);
let inferred = match inferred_result {
Ok(inferred) => inferred,
Err(error) => return Err(error),
};
if base_amount_raw.is_none() {
*base_amount_raw = inferred.0;
}
if quote_amount_raw.is_none() {
*quote_amount_raw = inferred.1;
}
if price_quote_per_base.is_none() {
*price_quote_per_base = inferred.2;
}
return Ok(());
}
fn apply_vault_balance_delta_fallback(
input: &crate::trade_amount_resolution::TradeAmountResolutionInput<'_>,
base_vault_address: std::option::Option<&str>,
quote_vault_address: std::option::Option<&str>,
base_amount_raw: &mut std::option::Option<std::string::String>,
quote_amount_raw: &mut std::option::Option<std::string::String>,
price_quote_per_base: &mut std::option::Option<f64>,
) -> Result<(), crate::Error> {
let inferred_result =
crate::trade_solana_amounts::extract_trade_amounts_from_vault_balance_deltas(
input.transaction.transaction_json.as_str(),
input.transaction.meta_json.as_deref(),
base_vault_address,
quote_vault_address,
);
let inferred = match inferred_result {
Ok(inferred) => inferred,
Err(error) => return Err(error),
};
if base_amount_raw.is_none() {
*base_amount_raw = inferred.0;
}
if quote_amount_raw.is_none() {
*quote_amount_raw = inferred.1;
}
if price_quote_per_base.is_none() {
*price_quote_per_base = inferred.2;
}
return Ok(());
}
async fn load_decoded_instruction_index(
database: &crate::Database,
decoded_event: &crate::DexDecodedEventDto,
) -> Result<std::option::Option<u32>, crate::Error> {
let instruction_id = match decoded_event.instruction_id {
Some(instruction_id) => instruction_id,
None => return Ok(None),
};
let instruction_result =
crate::query_chain_instructions_get_by_id(database, instruction_id).await;
let instruction_option = match instruction_result {
Ok(instruction_option) => instruction_option,
Err(error) => return Err(error),
};
match instruction_option {
Some(instruction) => return Ok(Some(instruction.instruction_index)),
None => return Ok(None),
}
}
fn extract_amount_string(
payload: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
return crate::trade_amount_resolution::extract_scalar_as_string_by_candidate_keys(
payload,
candidate_keys,
);
}
fn extract_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
let direct_text_option = direct.as_str();
if let Some(direct_text) = direct_text_option {
return Some(direct_text.to_string());
}
}
}
for nested_value in object.values() {
let nested_result = crate::trade_amount_resolution::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result = crate::trade_amount_resolution::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}
fn extract_scalar_as_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
if let Some(text) = direct.as_str() {
return Some(text.to_string());
}
if let Some(number) = direct.as_i64() {
return Some(number.to_string());
}
if let Some(number) = direct.as_u64() {
return Some(number.to_string());
}
if let Some(number) = direct.as_f64() {
return Some(number.to_string());
}
}
}
for nested_value in object.values() {
let nested_result =
crate::trade_amount_resolution::extract_scalar_as_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result =
crate::trade_amount_resolution::extract_scalar_as_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}

230
kb_lib/src/trade_event_materialization.rs Normal file
View File

@@ -0,0 +1,230 @@
// file: kb_lib/src/trade_event_materialization.rs
//! Trade-event materialization.
//!
//! This module persists normalized trade events, updates pair metrics and
//! records the corresponding detection observation/signal.
/// Input required to materialize one normalized trade event.
pub(crate) struct TradeEventMaterializationInput<'a> {
/// Database connection.
pub(crate) database: &'a crate::Database,
/// Detection persistence service used for observations and signals.
pub(crate) persistence: &'a crate::DetectionPersistenceService,
/// Persisted transaction row.
pub(crate) transaction: &'a crate::ChainTransactionDto,
/// Internal transaction id.
pub(crate) transaction_id: i64,
/// Decoded DEX event row.
pub(crate) decoded_event: &'a crate::DexDecodedEventDto,
/// Internal decoded event id.
pub(crate) decoded_event_id: i64,
/// Existing trade event, when the decoded event was already materialized.
pub(crate) existing_trade_event: std::option::Option<crate::TradeEventDto>,
/// Persisted pool row.
pub(crate) pool: &'a crate::PoolDto,
/// Internal pool id.
pub(crate) pool_id: i64,
/// Persisted pair row.
pub(crate) pair: &'a crate::PairDto,
/// Internal pair id.
pub(crate) pair_id: i64,
/// Trade side.
pub(crate) trade_side: crate::SwapTradeSide,
/// Resolved trade amounts.
pub(crate) amount_resolution: &'a crate::trade_amount_resolution::TradeAmountResolution,
}
/// Persists one normalized trade event and updates pair-level metrics.
pub(crate) async fn materialize_trade_event(
input: crate::trade_event_materialization::TradeEventMaterializationInput<'_>,
) -> Result<crate::TradeAggregationResult, crate::Error> {
let base_amount_raw = input.amount_resolution.base_amount_raw.clone();
let quote_amount_raw = input.amount_resolution.quote_amount_raw.clone();
let price_quote_per_base = input.amount_resolution.price_quote_per_base;
let slot_i64 = crate::trade_metric_update::convert_slot_to_i64(input.transaction.slot);
let created_trade_event = input.existing_trade_event.is_none();
let trade_event_dto = crate::TradeEventDto::new(
input.pool.dex_id,
input.pool_id,
input.pair_id,
input.transaction_id,
input.decoded_event_id,
input.transaction.signature.clone(),
slot_i64,
input.trade_side,
input.pair.base_token_id,
input.pair.quote_token_id,
base_amount_raw.clone(),
quote_amount_raw.clone(),
price_quote_per_base,
crate::ObservationSourceKind::Dex,
input.transaction.source_endpoint_name.clone(),
input.decoded_event.payload_json.clone(),
);
tracing::debug!(
event_kind = %input.decoded_event.event_kind,
pool_account = ?input.decoded_event.pool_account,
decoded_event_id = ?input.decoded_event.id,
created_trade_event = created_trade_event,
"trade aggregation candidate"
);
let trade_event_id_result =
crate::query_trade_events_upsert(input.database, &trade_event_dto).await;
let trade_event_id = match trade_event_id_result {
Ok(trade_event_id) => trade_event_id,
Err(error) => return Err(error),
};
let pair_metric_id_result = crate::trade_event_materialization::upsert_pair_metric_for_trade(
input.database,
input.pair_id,
slot_i64,
input.transaction.signature.clone(),
input.trade_side,
base_amount_raw.clone(),
quote_amount_raw.clone(),
price_quote_per_base,
created_trade_event,
)
.await;
let pair_metric_id = match pair_metric_id_result {
Ok(pair_metric_id) => pair_metric_id,
Err(error) => return Err(error),
};
if created_trade_event {
let observation_result =
crate::trade_event_materialization::record_trade_aggregation_observation_and_signal(
input.persistence,
input.transaction,
input.pair_id,
input.pool_id,
trade_event_id,
input.trade_side,
base_amount_raw,
quote_amount_raw,
price_quote_per_base,
)
.await;
if let Err(error) = observation_result {
return Err(error);
}
}
return Ok(crate::TradeAggregationResult {
trade_event_id,
pair_metric_id,
pair_id: input.pair_id,
pool_id: input.pool_id,
created_trade_event,
});
}
async fn upsert_pair_metric_for_trade(
database: &crate::Database,
pair_id: i64,
slot_i64: std::option::Option<i64>,
signature: std::string::String,
trade_side: crate::SwapTradeSide,
base_amount_raw: std::option::Option<std::string::String>,
quote_amount_raw: std::option::Option<std::string::String>,
price_quote_per_base: std::option::Option<f64>,
created_trade_event: bool,
) -> Result<i64, crate::Error> {
let pair_metric_result = crate::query_pair_metrics_get_by_pair_id(database, pair_id).await;
let pair_metric_option = match pair_metric_result {
Ok(pair_metric_option) => pair_metric_option,
Err(error) => return Err(error),
};
if let Some(existing_metric) = pair_metric_option {
let existing_metric_id = match existing_metric.id {
Some(existing_metric_id) => existing_metric_id,
None => {
return Err(crate::Error::InvalidState(
"pair metric has no internal id".to_string(),
));
},
};
if created_trade_event {
let mut updated_metric = existing_metric.clone();
crate::trade_metric_update::apply_trade_to_pair_metric(
&mut updated_metric,
slot_i64,
signature,
trade_side,
base_amount_raw,
quote_amount_raw,
price_quote_per_base,
);
let upsert_result = crate::query_pair_metrics_upsert(database, &updated_metric).await;
if let Err(error) = upsert_result {
return Err(error);
}
}
return Ok(existing_metric_id);
}
let mut new_metric = crate::PairMetricDto::new(pair_id);
crate::trade_metric_update::apply_trade_to_pair_metric(
&mut new_metric,
slot_i64,
signature,
trade_side,
base_amount_raw,
quote_amount_raw,
price_quote_per_base,
);
let upsert_result = crate::query_pair_metrics_upsert(database, &new_metric).await;
match upsert_result {
Ok(pair_metric_id) => return Ok(pair_metric_id),
Err(error) => return Err(error),
}
}
async fn record_trade_aggregation_observation_and_signal(
persistence: &crate::DetectionPersistenceService,
transaction: &crate::ChainTransactionDto,
pair_id: i64,
pool_id: i64,
trade_event_id: i64,
trade_side: crate::SwapTradeSide,
base_amount_raw: std::option::Option<std::string::String>,
quote_amount_raw: std::option::Option<std::string::String>,
price_quote_per_base: std::option::Option<f64>,
) -> Result<(), crate::Error> {
let payload = serde_json::json!({
"pairId": pair_id,
"poolId": pool_id,
"tradeEventId": trade_event_id,
"tradeSide": format!("{:?}", trade_side),
"baseAmountRaw": base_amount_raw,
"quoteAmountRaw": quote_amount_raw,
"priceQuotePerBase": price_quote_per_base,
"transactionSignature": transaction.signature
});
let observation_result = persistence
.record_observation(&crate::DetectionObservationInput::new(
"dex.trade_aggregation".to_string(),
crate::ObservationSourceKind::Dex,
transaction.source_endpoint_name.clone(),
transaction.signature.clone(),
transaction.slot,
payload.clone(),
))
.await;
let observation_id = match observation_result {
Ok(observation_id) => observation_id,
Err(error) => return Err(error),
};
let signal_result = persistence
.record_signal(&crate::DetectionSignalInput::new(
"signal.dex.trade_aggregation.recorded".to_string(),
crate::AnalysisSignalSeverity::Low,
transaction.signature.clone(),
Some(observation_id),
None,
payload,
))
.await;
if let Err(error) = signal_result {
return Err(error);
}
return Ok(());
}

279
kb_lib/src/trade_metric_update.rs Normal file
View File

@@ -0,0 +1,279 @@
// file: kb_lib/src/trade_metric_update.rs
//! Trade metric update and basic trade-pricing helpers.
//!
//! This module contains pure helpers used by trade aggregation:
//! pricing validation, raw amount accumulation and pair metric updates.
/// Returns true when a decoded trade has enough positive values to be persisted.
pub(crate) fn is_priced_trade_event(
base_amount_raw: std::option::Option<&str>,
quote_amount_raw: std::option::Option<&str>,
price_quote_per_base: std::option::Option<f64>,
) -> bool {
let base_amount_raw = match base_amount_raw {
Some(base_amount_raw) => base_amount_raw.trim(),
None => return false,
};
if base_amount_raw.is_empty() {
return false;
}
let base_amount_result = base_amount_raw.parse::<i128>();
let base_amount = match base_amount_result {
Ok(base_amount) => base_amount,
Err(_) => return false,
};
if base_amount <= 0 {
return false;
}
let quote_amount_raw = match quote_amount_raw {
Some(quote_amount_raw) => quote_amount_raw.trim(),
None => return false,
};
if quote_amount_raw.is_empty() {
return false;
}
let quote_amount_result = quote_amount_raw.parse::<i128>();
let quote_amount = match quote_amount_result {
Ok(quote_amount) => quote_amount,
Err(_) => return false,
};
if quote_amount <= 0 {
return false;
}
let price = match price_quote_per_base {
Some(price) => price,
None => return false,
};
if !price.is_finite() {
return false;
}
return price > 0.0;
}
/// Converts an optional Solana slot to an optional signed database slot.
pub(crate) fn convert_slot_to_i64(slot: std::option::Option<u64>) -> std::option::Option<i64> {
match slot {
Some(slot) => match i64::try_from(slot) {
Ok(slot) => return Some(slot),
Err(_) => return None,
},
None => return None,
}
}
/// Applies one newly-created trade event to a pair metric.
pub(crate) fn apply_trade_to_pair_metric(
metric: &mut crate::PairMetricDto,
slot: std::option::Option<i64>,
signature: std::string::String,
trade_side: crate::SwapTradeSide,
base_amount_raw: std::option::Option<std::string::String>,
quote_amount_raw: std::option::Option<std::string::String>,
price_quote_per_base: std::option::Option<f64>,
) {
metric.trade_count += 1;
if trade_side == crate::SwapTradeSide::BuyBase {
metric.buy_count += 1;
}
if trade_side == crate::SwapTradeSide::SellBase {
metric.sell_count += 1;
}
if metric.first_slot.is_none() {
metric.first_slot = slot;
}
if metric.first_signature.is_none() {
metric.first_signature = Some(signature.clone());
}
metric.last_slot = slot;
metric.last_signature = Some(signature);
metric.cumulative_base_amount_raw = crate::trade_metric_update::add_raw_amounts(
metric.cumulative_base_amount_raw.clone(),
base_amount_raw,
);
metric.cumulative_quote_amount_raw = crate::trade_metric_update::add_raw_amounts(
metric.cumulative_quote_amount_raw.clone(),
quote_amount_raw,
);
if price_quote_per_base.is_some() {
metric.last_price_quote_per_base = price_quote_per_base;
}
metric.updated_at = chrono::Utc::now();
}
/// Adds two optional raw integer amount strings.
pub(crate) fn add_raw_amounts(
left: std::option::Option<std::string::String>,
right: std::option::Option<std::string::String>,
) -> std::option::Option<std::string::String> {
match (left, right) {
(None, None) => return None,
(Some(left), None) => return Some(left),
(None, Some(right)) => return Some(right),
(Some(left), Some(right)) => {
let left_value_result = left.parse::<i128>();
let left_value = match left_value_result {
Ok(left_value) => left_value,
Err(_) => return Some(left),
};
let right_value_result = right.parse::<i128>();
let right_value = match right_value_result {
Ok(right_value) => right_value,
Err(_) => return Some(left),
};
return Some((left_value + right_value).to_string());
},
}
}
/// Computes quote/base price from raw amounts and token decimals.
pub(crate) fn compute_price_quote_per_base_from_raw_amounts_with_decimals(
base_amount_raw: std::option::Option<&str>,
quote_amount_raw: std::option::Option<&str>,
base_decimals: std::option::Option<u8>,
quote_decimals: std::option::Option<u8>,
) -> std::option::Option<f64> {
let base_decimals = match base_decimals {
Some(base_decimals) => base_decimals,
None => return None,
};
let quote_decimals = match quote_decimals {
Some(quote_decimals) => quote_decimals,
None => return None,
};
let base_amount_raw = match base_amount_raw {
Some(base_amount_raw) => base_amount_raw.trim(),
None => return None,
};
let quote_amount_raw = match quote_amount_raw {
Some(quote_amount_raw) => quote_amount_raw.trim(),
None => return None,
};
if base_amount_raw.is_empty() || quote_amount_raw.is_empty() {
return None;
}
let base_amount_result = base_amount_raw.parse::<f64>();
let base_amount = match base_amount_result {
Ok(base_amount) => base_amount,
Err(_) => return None,
};
let quote_amount_result = quote_amount_raw.parse::<f64>();
let quote_amount = match quote_amount_result {
Ok(quote_amount) => quote_amount,
Err(_) => return None,
};
if base_amount <= 0.0 || quote_amount <= 0.0 {
return None;
}
let base_scale = 10_f64.powi(i32::from(base_decimals));
let quote_scale = 10_f64.powi(i32::from(quote_decimals));
if base_scale <= 0.0 || quote_scale <= 0.0 {
return None;
}
let base_ui_amount = base_amount / base_scale;
let quote_ui_amount = quote_amount / quote_scale;
if base_ui_amount <= 0.0 || quote_ui_amount <= 0.0 {
return None;
}
return Some(quote_ui_amount / base_ui_amount);
}
/// Computes quote/base price from raw amount strings without decimals.
pub(crate) fn compute_price_quote_per_base_from_raw_amounts(
base_amount_raw: std::option::Option<&str>,
quote_amount_raw: std::option::Option<&str>,
) -> std::option::Option<f64> {
let base_amount_raw = match base_amount_raw {
Some(base_amount_raw) => base_amount_raw.trim(),
None => return None,
};
let quote_amount_raw = match quote_amount_raw {
Some(quote_amount_raw) => quote_amount_raw.trim(),
None => return None,
};
if base_amount_raw.is_empty() || quote_amount_raw.is_empty() {
return None;
}
let base_amount_result = base_amount_raw.parse::<f64>();
let base_amount = match base_amount_result {
Ok(base_amount) => base_amount,
Err(_) => return None,
};
let quote_amount_result = quote_amount_raw.parse::<f64>();
let quote_amount = match quote_amount_result {
Ok(quote_amount) => quote_amount,
Err(_) => return None,
};
if base_amount <= 0.0 {
return None;
}
return Some(quote_amount / base_amount);
}
#[cfg(test)]
mod tests {
#[test]
fn priced_trade_event_rejects_unpriced_values() {
let result = super::is_priced_trade_event(None, Some("2500"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), None, Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("2500"), None);
assert!(!result);
let result = super::is_priced_trade_event(Some("0"), Some("2500"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("0"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("-1"), Some("2500"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("-1"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("abc"), Some("2500"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("abc"), Some(2.5));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("2500"), Some(0.0));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("2500"), Some(f64::NAN));
assert!(!result);
let result = super::is_priced_trade_event(Some("1000"), Some("2500"), Some(2.5));
assert!(result);
}
#[test]
fn raw_amounts_are_added_when_both_are_valid() {
let result = super::add_raw_amounts(Some("1000".to_string()), Some("2500".to_string()));
assert_eq!(result, Some("3500".to_string()));
}
#[test]
fn raw_amount_addition_keeps_left_when_right_is_invalid() {
let result = super::add_raw_amounts(Some("1000".to_string()), Some("abc".to_string()));
assert_eq!(result, Some("1000".to_string()));
}
#[test]
fn price_with_decimals_is_computed() {
let price = super::compute_price_quote_per_base_from_raw_amounts_with_decimals(
Some("1000000"),
Some("2500000000"),
Some(6),
Some(9),
);
assert_eq!(price, Some(2.5));
}
#[test]
fn price_without_decimals_is_computed() {
let price =
super::compute_price_quote_per_base_from_raw_amounts(Some("1000"), Some("2500"));
assert_eq!(price, Some(2.5));
}
#[test]
fn overflowing_slot_is_ignored() {
let slot = super::convert_slot_to_i64(Some(u64::MAX));
assert_eq!(slot, None);
}
}

1233
kb_lib/src/trade_pump_swap_amounts.rs Normal file
View File

File diff suppressed because it is too large Load Diff

118
kb_lib/src/trade_side_resolution.rs Normal file
View File

@@ -0,0 +1,118 @@
// file: kb_lib/src/trade_side_resolution.rs
//! Trade-side resolution helpers.
//!
//! This module resolves a normalized `SwapTradeSide` from a decoded event kind
//! and optional decoded payload metadata.
/// Resolves the normalized trade side from payload metadata and event kind.
pub(crate) fn extract_trade_side(
event_kind: &str,
payload: &serde_json::Value,
) -> crate::SwapTradeSide {
let trade_side_option = crate::trade_side_resolution::extract_string_by_candidate_keys(
payload,
&["tradeSide", "trade_side"],
);
match trade_side_option.as_deref() {
Some("BuyBase") => return crate::SwapTradeSide::BuyBase,
Some("buy") => return crate::SwapTradeSide::BuyBase,
Some("BUY") => return crate::SwapTradeSide::BuyBase,
Some("SellBase") => return crate::SwapTradeSide::SellBase,
Some("sell") => return crate::SwapTradeSide::SellBase,
Some("SELL") => return crate::SwapTradeSide::SellBase,
_ => {},
}
if event_kind.ends_with(".buy") {
return crate::SwapTradeSide::BuyBase;
}
if event_kind.ends_with(".sell") {
return crate::SwapTradeSide::SellBase;
}
return crate::SwapTradeSide::Unknown;
}
fn extract_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
let direct_text_option = direct.as_str();
if let Some(direct_text) = direct_text_option {
return Some(direct_text.to_string());
}
}
}
for nested_value in object.values() {
let nested_result = crate::trade_side_resolution::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result = crate::trade_side_resolution::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}
#[cfg(test)]
mod tests {
#[test]
fn payload_trade_side_wins_over_event_kind() {
let payload = serde_json::json!({
"tradeSide": "SellBase"
});
let side = super::extract_trade_side("pump_swap.buy", &payload);
assert_eq!(side, crate::SwapTradeSide::SellBase);
}
#[test]
fn nested_payload_trade_side_is_resolved() {
let payload = serde_json::json!({
"decoded": {
"meta": {
"trade_side": "buy"
}
}
});
let side = super::extract_trade_side("raydium_cpmm.swap_base_input", &payload);
assert_eq!(side, crate::SwapTradeSide::BuyBase);
}
#[test]
fn buy_suffix_is_resolved_when_payload_has_no_side() {
let payload = serde_json::json!({});
let side = super::extract_trade_side("pump_fun.buy", &payload);
assert_eq!(side, crate::SwapTradeSide::BuyBase);
}
#[test]
fn sell_suffix_is_resolved_when_payload_has_no_side() {
let payload = serde_json::json!({});
let side = super::extract_trade_side("pump_fun.sell", &payload);
assert_eq!(side, crate::SwapTradeSide::SellBase);
}
#[test]
fn unknown_side_is_returned_when_no_hint_exists() {
let payload = serde_json::json!({});
let side = super::extract_trade_side("raydium_cpmm.swap_base_input", &payload);
assert_eq!(side, crate::SwapTradeSide::Unknown);
}
}

913
kb_lib/src/trade_solana_amounts.rs Normal file
View File

@@ -0,0 +1,913 @@
// file: kb_lib/src/trade_solana_amounts.rs
//! Solana transaction/meta trade amount extraction helpers.
//!
//! This module contains generic fallback logic based on transaction JSON,
//! transaction meta, account keys, token balances, native balances and
//! instruction-scoped SPL token transfers.
/// Extracted base/quote amounts and optional quote/base price.
pub(crate) type ExtractedTradeAmounts = (
std::option::Option<std::string::String>,
std::option::Option<std::string::String>,
std::option::Option<f64>,
);
/// Extracts base/quote amounts from token-account vault balance deltas.
pub(crate) fn extract_trade_amounts_from_vault_balance_deltas(
transaction_json: &str,
meta_json: std::option::Option<&str>,
base_vault_address: std::option::Option<&str>,
quote_vault_address: std::option::Option<&str>,
) -> Result<crate::trade_solana_amounts::ExtractedTradeAmounts, crate::Error> {
let meta_json = match meta_json {
Some(meta_json) => meta_json,
None => return Ok((None, None, None)),
};
let transaction_value_result = serde_json::from_str::<serde_json::Value>(transaction_json);
let transaction_value = match transaction_value_result {
Ok(transaction_value) => transaction_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse transaction_json for vault balance amount extraction: {}",
error
)));
},
};
let meta_value_result = serde_json::from_str::<serde_json::Value>(meta_json);
let meta_value = match meta_value_result {
Ok(meta_value) => meta_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse meta_json for vault balance amount extraction: {}",
error
)));
},
};
let account_keys_result =
crate::trade_solana_amounts::extract_transaction_account_keys(&transaction_value);
let account_keys = match account_keys_result {
Ok(account_keys) => account_keys,
Err(error) => return Err(error),
};
let pre_balances_result = crate::trade_solana_amounts::extract_token_balance_map(
&meta_value,
&account_keys,
"preTokenBalances",
);
let pre_balances = match pre_balances_result {
Ok(pre_balances) => pre_balances,
Err(error) => return Err(error),
};
let post_balances_result = crate::trade_solana_amounts::extract_token_balance_map(
&meta_value,
&account_keys,
"postTokenBalances",
);
let post_balances = match post_balances_result {
Ok(post_balances) => post_balances,
Err(error) => return Err(error),
};
let mut base_amount_raw = None;
let mut quote_amount_raw = None;
let mut price_quote_per_base = None;
if let Some(base_vault_address) = base_vault_address {
let base_pre = pre_balances.get(base_vault_address);
let base_post = post_balances.get(base_vault_address);
let base_pre_raw = match base_pre {
Some(value) => Some(value.0.clone()),
None => None,
};
let base_post_raw = match base_post {
Some(value) => Some(value.0.clone()),
None => None,
};
base_amount_raw =
crate::trade_solana_amounts::compute_amount_delta_abs(base_pre_raw, base_post_raw);
let base_pre_ui = match base_pre {
Some(value) => value.1,
None => None,
};
let base_post_ui = match base_post {
Some(value) => value.1,
None => None,
};
let base_delta_ui =
crate::trade_solana_amounts::compute_ui_delta_abs(base_pre_ui, base_post_ui);
if let Some(quote_vault_address) = quote_vault_address {
let quote_pre = pre_balances.get(quote_vault_address);
let quote_post = post_balances.get(quote_vault_address);
let quote_pre_raw = match quote_pre {
Some(value) => Some(value.0.clone()),
None => None,
};
let quote_post_raw = match quote_post {
Some(value) => Some(value.0.clone()),
None => None,
};
quote_amount_raw = crate::trade_solana_amounts::compute_amount_delta_abs(
quote_pre_raw,
quote_post_raw,
);
let quote_pre_ui = match quote_pre {
Some(value) => value.1,
None => None,
};
let quote_post_ui = match quote_post {
Some(value) => value.1,
None => None,
};
let quote_delta_ui =
crate::trade_solana_amounts::compute_ui_delta_abs(quote_pre_ui, quote_post_ui);
if let (Some(base_delta_ui), Some(quote_delta_ui)) = (base_delta_ui, quote_delta_ui) {
if base_delta_ui > 0.0 {
price_quote_per_base = Some(quote_delta_ui / base_delta_ui);
}
}
}
}
return Ok((base_amount_raw, quote_amount_raw, price_quote_per_base));
}
/// Extracts base/quote amounts from instruction-scoped SPL token transfers.
pub(crate) fn extract_trade_amounts_from_instruction_token_transfers(
meta_json: std::option::Option<&str>,
instruction_index: std::option::Option<u32>,
input_vault_address: std::option::Option<&str>,
output_vault_address: std::option::Option<&str>,
input_token_account: std::option::Option<&str>,
output_token_account: std::option::Option<&str>,
base_vault_address: std::option::Option<&str>,
quote_vault_address: std::option::Option<&str>,
) -> Result<crate::trade_solana_amounts::ExtractedTradeAmounts, crate::Error> {
let meta_json = match meta_json {
Some(meta_json) => meta_json,
None => return Ok((None, None, None)),
};
let instruction_index = match instruction_index {
Some(instruction_index) => u64::from(instruction_index),
None => return Ok((None, None, None)),
};
let input_vault_address = match input_vault_address {
Some(input_vault_address) => input_vault_address.trim(),
None => return Ok((None, None, None)),
};
let output_vault_address = match output_vault_address {
Some(output_vault_address) => output_vault_address.trim(),
None => return Ok((None, None, None)),
};
let input_token_account = match input_token_account {
Some(input_token_account) => input_token_account.trim(),
None => return Ok((None, None, None)),
};
let output_token_account = match output_token_account {
Some(output_token_account) => output_token_account.trim(),
None => return Ok((None, None, None)),
};
let base_vault_address = match base_vault_address {
Some(base_vault_address) => base_vault_address.trim(),
None => return Ok((None, None, None)),
};
let quote_vault_address = match quote_vault_address {
Some(quote_vault_address) => quote_vault_address.trim(),
None => return Ok((None, None, None)),
};
if input_vault_address.is_empty()
|| output_vault_address.is_empty()
|| input_token_account.is_empty()
|| output_token_account.is_empty()
|| base_vault_address.is_empty()
|| quote_vault_address.is_empty()
{
return Ok((None, None, None));
}
let meta_value_result = serde_json::from_str::<serde_json::Value>(meta_json);
let meta_value = match meta_value_result {
Ok(meta_value) => meta_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse meta_json for instruction-scoped token transfer amount extraction: {}",
error
)));
},
};
let inner_groups_option =
meta_value.get("innerInstructions").and_then(|value| return value.as_array());
let inner_groups = match inner_groups_option {
Some(inner_groups) => inner_groups,
None => return Ok((None, None, None)),
};
let mut input_amount_raw = None;
let mut output_amount_raw = None;
for inner_group in inner_groups {
let group_index_option = inner_group.get("index").and_then(|value| return value.as_u64());
let group_index = match group_index_option {
Some(group_index) => group_index,
None => continue,
};
if group_index != instruction_index {
continue;
}
let instructions_option =
inner_group.get("instructions").and_then(|value| return value.as_array());
let instructions = match instructions_option {
Some(instructions) => instructions,
None => continue,
};
for instruction in instructions {
if !crate::trade_solana_amounts::is_spl_token_transfer_instruction(instruction) {
continue;
}
let parsed = match instruction.get("parsed") {
Some(parsed) => parsed,
None => continue,
};
let info = match parsed.get("info") {
Some(info) => info,
None => continue,
};
let source_option =
crate::trade_solana_amounts::extract_string_by_candidate_keys(info, &["source"]);
let source = match source_option {
Some(source) => source,
None => continue,
};
let destination_option = crate::trade_solana_amounts::extract_string_by_candidate_keys(
info,
&["destination"],
);
let destination = match destination_option {
Some(destination) => destination,
None => continue,
};
let amount_option =
crate::trade_solana_amounts::extract_scalar_as_string_by_candidate_keys(
info,
&["amount"],
);
let amount = match amount_option {
Some(amount) => amount,
None => continue,
};
if input_amount_raw.is_none()
&& crate::trade_solana_amounts::account_equals(source.as_str(), input_token_account)
&& crate::trade_solana_amounts::account_equals(
destination.as_str(),
input_vault_address,
)
{
input_amount_raw = Some(amount.clone());
continue;
}
if output_amount_raw.is_none()
&& crate::trade_solana_amounts::account_equals(
source.as_str(),
output_vault_address,
)
&& crate::trade_solana_amounts::account_equals(
destination.as_str(),
output_token_account,
)
{
output_amount_raw = Some(amount);
continue;
}
}
}
if input_amount_raw.is_none() && output_amount_raw.is_none() {
return Ok((None, None, None));
}
if crate::trade_solana_amounts::account_equals(input_vault_address, base_vault_address)
&& crate::trade_solana_amounts::account_equals(output_vault_address, quote_vault_address)
{
return Ok((input_amount_raw, output_amount_raw, None));
}
if crate::trade_solana_amounts::account_equals(input_vault_address, quote_vault_address)
&& crate::trade_solana_amounts::account_equals(output_vault_address, base_vault_address)
{
return Ok((output_amount_raw, input_amount_raw, None));
}
return Ok((None, None, None));
}
/// Extracts Pump.fun amounts from transaction token/native balance deltas.
pub(crate) fn extract_pump_fun_amounts_from_transaction(
transaction_json: &str,
meta_json: std::option::Option<&str>,
base_vault_address: std::option::Option<&str>,
quote_native_address: std::option::Option<&str>,
) -> Result<crate::trade_solana_amounts::ExtractedTradeAmounts, crate::Error> {
let meta_json = match meta_json {
Some(meta_json) => meta_json,
None => return Ok((None, None, None)),
};
let transaction_value_result = serde_json::from_str::<serde_json::Value>(transaction_json);
let transaction_value = match transaction_value_result {
Ok(transaction_value) => transaction_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse transaction_json for pump_fun amount extraction: {}",
error
)));
},
};
let meta_value_result = serde_json::from_str::<serde_json::Value>(meta_json);
let meta_value = match meta_value_result {
Ok(meta_value) => meta_value,
Err(error) => {
return Err(crate::Error::Json(format!(
"cannot parse meta_json for pump_fun amount extraction: {}",
error
)));
},
};
let account_keys_result =
crate::trade_solana_amounts::extract_transaction_account_keys(&transaction_value);
let account_keys = match account_keys_result {
Ok(account_keys) => account_keys,
Err(error) => return Err(error),
};
let pre_balances_result = crate::trade_solana_amounts::extract_token_balance_map(
&meta_value,
&account_keys,
"preTokenBalances",
);
let pre_balances = match pre_balances_result {
Ok(pre_balances) => pre_balances,
Err(error) => return Err(error),
};
let post_balances_result = crate::trade_solana_amounts::extract_token_balance_map(
&meta_value,
&account_keys,
"postTokenBalances",
);
let post_balances = match post_balances_result {
Ok(post_balances) => post_balances,
Err(error) => return Err(error),
};
let mut base_amount_raw = None;
let mut quote_amount_raw = None;
let mut price_quote_per_base = None;
let mut base_delta_ui = None;
if let Some(base_vault_address) = base_vault_address {
let base_pre = pre_balances.get(base_vault_address);
let base_post = post_balances.get(base_vault_address);
let base_pre_raw = match base_pre {
Some(value) => Some(value.0.clone()),
None => None,
};
let base_post_raw = match base_post {
Some(value) => Some(value.0.clone()),
None => None,
};
base_amount_raw =
crate::trade_solana_amounts::compute_amount_delta_abs(base_pre_raw, base_post_raw);
let base_pre_ui = match base_pre {
Some(value) => value.1,
None => None,
};
let base_post_ui = match base_post {
Some(value) => value.1,
None => None,
};
base_delta_ui =
crate::trade_solana_amounts::compute_ui_delta_abs(base_pre_ui, base_post_ui);
}
if let Some(quote_native_address) = quote_native_address {
let quote_delta_result =
crate::trade_solana_amounts::extract_native_balance_delta_by_address(
&meta_value,
&account_keys,
quote_native_address,
);
let quote_delta = match quote_delta_result {
Ok(quote_delta) => quote_delta,
Err(error) => return Err(error),
};
if let Some(quote_delta_lamports) = quote_delta {
quote_amount_raw = Some(quote_delta_lamports.to_string());
let quote_delta_ui = quote_delta_lamports as f64 / 1_000_000_000.0;
if let Some(base_delta_ui) = base_delta_ui {
if base_delta_ui > 0.0 {
price_quote_per_base = Some(quote_delta_ui / base_delta_ui);
}
}
}
}
return Ok((base_amount_raw, quote_amount_raw, price_quote_per_base));
}
/// Computes quote/base price from vault balance deltas.
pub(crate) fn compute_price_quote_per_base_with_decimals(
meta_json: std::option::Option<&str>,
transaction_json: &str,
base_vault_address: std::option::Option<&str>,
quote_vault_address: std::option::Option<&str>,
) -> std::option::Option<f64> {
let inferred_result =
crate::trade_solana_amounts::extract_trade_amounts_from_vault_balance_deltas(
transaction_json,
meta_json,
base_vault_address,
quote_vault_address,
);
let inferred = match inferred_result {
Ok(inferred) => inferred,
Err(_) => return None,
};
return inferred.2;
}
fn is_spl_token_transfer_instruction(instruction: &serde_json::Value) -> bool {
let program_id_option = instruction.get("programId").and_then(|value| return value.as_str());
if let Some(program_id) = program_id_option {
let spl_token_program_id = crate::SPL_TOKEN_PROGRAM_ID.to_string();
let spl_token_2022_program_id = crate::SPL_TOKEN_2022_PROGRAM_ID.to_string();
if program_id != spl_token_program_id.as_str()
&& program_id != spl_token_2022_program_id.as_str()
{
return false;
}
}
let parsed_type_option = instruction
.get("parsed")
.and_then(|parsed| return parsed.get("type"))
.and_then(|value| return value.as_str());
match parsed_type_option {
Some("transfer") => return true,
Some("transferChecked") => return true,
_ => return false,
}
}
fn account_equals(left: &str, right: &str) -> bool {
let left = left.trim();
let right = right.trim();
if left.is_empty() || right.is_empty() {
return false;
}
return left == right;
}
fn extract_native_balance_delta_by_address(
meta_value: &serde_json::Value,
account_keys: &[std::string::String],
address: &str,
) -> Result<std::option::Option<u64>, crate::Error> {
let mut account_index = None;
for (index, account_key) in account_keys.iter().enumerate() {
if account_key.as_str() == address {
account_index = Some(index);
break;
}
}
let account_index = match account_index {
Some(account_index) => account_index,
None => return Ok(None),
};
let pre_balances_option =
meta_value.get("preBalances").and_then(|value| return value.as_array());
let post_balances_option =
meta_value.get("postBalances").and_then(|value| return value.as_array());
let pre_balances = match pre_balances_option {
Some(pre_balances) => pre_balances,
None => return Ok(None),
};
let post_balances = match post_balances_option {
Some(post_balances) => post_balances,
None => return Ok(None),
};
if account_index >= pre_balances.len() || account_index >= post_balances.len() {
return Ok(None);
}
let pre_balance = match pre_balances[account_index].as_u64() {
Some(pre_balance) => pre_balance,
None => return Ok(None),
};
let post_balance = match post_balances[account_index].as_u64() {
Some(post_balance) => post_balance,
None => return Ok(None),
};
if post_balance >= pre_balance {
return Ok(Some(post_balance - pre_balance));
}
return Ok(Some(pre_balance - post_balance));
}
fn extract_transaction_account_keys(
transaction_value: &serde_json::Value,
) -> Result<std::vec::Vec<std::string::String>, crate::Error> {
let candidate_arrays = [
transaction_value
.get("message")
.and_then(|value| return value.get("accountKeys")),
transaction_value
.get("transaction")
.and_then(|value| return value.get("message"))
.and_then(|value| return value.get("accountKeys")),
transaction_value
.get("transaction")
.and_then(|value| return value.get("transaction"))
.and_then(|value| return value.get("message"))
.and_then(|value| return value.get("accountKeys")),
transaction_value.get("accountKeys"),
];
for candidate_array_option in candidate_arrays {
let candidate_array = match candidate_array_option {
Some(candidate_array) => candidate_array,
None => continue,
};
let array = match candidate_array.as_array() {
Some(array) => array,
None => continue,
};
let mut account_keys = std::vec::Vec::new();
for item in array {
if let Some(value) = item.as_str() {
account_keys.push(value.to_string());
continue;
}
let pubkey_option = item.get("pubkey").and_then(|value| return value.as_str());
if let Some(pubkey) = pubkey_option {
account_keys.push(pubkey.to_string());
continue;
}
}
if !account_keys.is_empty() {
return Ok(account_keys);
}
}
return Err(crate::Error::Json(
"cannot extract accountKeys from transaction_json".to_string(),
));
}
fn extract_token_balance_map(
meta_value: &serde_json::Value,
account_keys: &[std::string::String],
field_name: &str,
) -> Result<
std::collections::BTreeMap<
std::string::String,
(std::string::String, std::option::Option<f64>),
>,
crate::Error,
> {
let mut result = std::collections::BTreeMap::<
std::string::String,
(std::string::String, std::option::Option<f64>),
>::new();
let balances_option = meta_value.get(field_name).and_then(|value| return value.as_array());
let balances = match balances_option {
Some(balances) => balances,
None => return Ok(result),
};
for balance in balances {
let account_index_option =
balance.get("accountIndex").and_then(|value| return value.as_u64());
let account_index = match account_index_option {
Some(account_index) => account_index as usize,
None => continue,
};
if account_index >= account_keys.len() {
continue;
}
let account_address = account_keys[account_index].clone();
let ui_token_amount = match balance.get("uiTokenAmount") {
Some(ui_token_amount) => ui_token_amount,
None => continue,
};
let raw_amount_option =
ui_token_amount.get("amount").and_then(|value| return value.as_str());
let raw_amount = match raw_amount_option {
Some(raw_amount) => raw_amount.to_string(),
None => continue,
};
let ui_amount_string_option =
ui_token_amount.get("uiAmountString").and_then(|value| return value.as_str());
let ui_amount = match ui_amount_string_option {
Some(ui_amount_string) => {
let parse_result = ui_amount_string.parse::<f64>();
match parse_result {
Ok(ui_amount) => Some(ui_amount),
Err(_) => None,
}
},
None => None,
};
result.insert(account_address, (raw_amount, ui_amount));
}
return Ok(result);
}
fn compute_amount_delta_abs(
pre_amount: std::option::Option<std::string::String>,
post_amount: std::option::Option<std::string::String>,
) -> std::option::Option<std::string::String> {
let pre_amount = match pre_amount {
Some(pre_amount) => pre_amount,
None => "0".to_string(),
};
let post_amount = match post_amount {
Some(post_amount) => post_amount,
None => "0".to_string(),
};
let pre_value_result = pre_amount.parse::<i128>();
let pre_value = match pre_value_result {
Ok(pre_value) => pre_value,
Err(_) => return None,
};
let post_value_result = post_amount.parse::<i128>();
let post_value = match post_value_result {
Ok(post_value) => post_value,
Err(_) => return None,
};
let delta = if post_value >= pre_value {
post_value - pre_value
} else {
pre_value - post_value
};
return Some(delta.to_string());
}
fn compute_ui_delta_abs(
pre_amount: std::option::Option<f64>,
post_amount: std::option::Option<f64>,
) -> std::option::Option<f64> {
let pre_amount = match pre_amount {
Some(pre_amount) => pre_amount,
None => 0.0,
};
let post_amount = match post_amount {
Some(post_amount) => post_amount,
None => 0.0,
};
let delta = if post_amount >= pre_amount {
post_amount - pre_amount
} else {
pre_amount - post_amount
};
return Some(delta);
}
fn extract_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
let direct_text_option = direct.as_str();
if let Some(direct_text) = direct_text_option {
return Some(direct_text.to_string());
}
}
}
for nested_value in object.values() {
let nested_result = crate::trade_solana_amounts::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result = crate::trade_solana_amounts::extract_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}
fn extract_scalar_as_string_by_candidate_keys(
value: &serde_json::Value,
candidate_keys: &[&str],
) -> std::option::Option<std::string::String> {
if let Some(object) = value.as_object() {
for candidate_key in candidate_keys {
let direct_option = object.get(*candidate_key);
if let Some(direct) = direct_option {
if let Some(direct_text) = direct.as_str() {
return Some(direct_text.to_string());
}
if let Some(direct_u64) = direct.as_u64() {
return Some(direct_u64.to_string());
}
if let Some(direct_i64) = direct.as_i64() {
return Some(direct_i64.to_string());
}
}
}
for nested_value in object.values() {
let nested_result =
crate::trade_solana_amounts::extract_scalar_as_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
return None;
}
if let Some(array) = value.as_array() {
for nested_value in array {
let nested_result =
crate::trade_solana_amounts::extract_scalar_as_string_by_candidate_keys(
nested_value,
candidate_keys,
);
if nested_result.is_some() {
return nested_result;
}
}
}
return None;
}
#[cfg(test)]
mod tests {
#[test]
fn vault_balance_deltas_extract_raw_amounts_and_price() {
let transaction_json = serde_json::json!({
"transaction": {
"message": {
"accountKeys": [
"BaseVault111",
"QuoteVault111"
]
}
}
});
let meta_json = serde_json::json!({
"preTokenBalances": [
{
"accountIndex": 0,
"uiTokenAmount": {
"amount": "1000000",
"uiAmountString": "1.0"
}
},
{
"accountIndex": 1,
"uiTokenAmount": {
"amount": "2000000000",
"uiAmountString": "2.0"
}
}
],
"postTokenBalances": [
{
"accountIndex": 0,
"uiTokenAmount": {
"amount": "1500000",
"uiAmountString": "1.5"
}
},
{
"accountIndex": 1,
"uiTokenAmount": {
"amount": "1000000000",
"uiAmountString": "1.0"
}
}
]
});
let transaction_json_text = transaction_json.to_string();
let meta_json_text = meta_json.to_string();
let result = super::extract_trade_amounts_from_vault_balance_deltas(
transaction_json_text.as_str(),
Some(meta_json_text.as_str()),
Some("BaseVault111"),
Some("QuoteVault111"),
);
let amounts = match result {
Ok(amounts) => amounts,
Err(error) => panic!("vault delta extraction should succeed: {}", error),
};
assert_eq!(amounts.0, Some("500000".to_string()));
assert_eq!(amounts.1, Some("1000000000".to_string()));
assert_eq!(amounts.2, Some(2.0));
}
#[test]
fn instruction_transfer_amounts_follow_base_quote_vault_mapping() {
let meta_json = serde_json::json!({
"innerInstructions": [
{
"index": 3,
"instructions": [
{
"programId": crate::SPL_TOKEN_PROGRAM_ID,
"parsed": {
"type": "transfer",
"info": {
"source": "UserQuote111",
"destination": "QuoteVault111",
"amount": "2000000000"
}
}
},
{
"programId": crate::SPL_TOKEN_PROGRAM_ID,
"parsed": {
"type": "transfer",
"info": {
"source": "BaseVault111",
"destination": "UserBase111",
"amount": "500000"
}
}
}
]
}
]
});
let meta_json_text = meta_json.to_string();
let result = super::extract_trade_amounts_from_instruction_token_transfers(
Some(meta_json_text.as_str()),
Some(3),
Some("QuoteVault111"),
Some("BaseVault111"),
Some("UserQuote111"),
Some("UserBase111"),
Some("BaseVault111"),
Some("QuoteVault111"),
);
let amounts = match result {
Ok(amounts) => amounts,
Err(error) => panic!("instruction transfer extraction should succeed: {}", error),
};
assert_eq!(amounts.0, Some("500000".to_string()));
assert_eq!(amounts.1, Some("2000000000".to_string()));
assert_eq!(amounts.2, None);
}
#[test]
fn pump_fun_amounts_extract_token_delta_and_native_delta() {
let transaction_json = serde_json::json!({
"transaction": {
"message": {
"accountKeys": [
"BaseVault111",
"NativeQuote111"
]
}
}
});
let meta_json = serde_json::json!({
"preTokenBalances": [
{
"accountIndex": 0,
"uiTokenAmount": {
"amount": "1000000",
"uiAmountString": "1.0"
}
}
],
"postTokenBalances": [
{
"accountIndex": 0,
"uiTokenAmount": {
"amount": "1500000",
"uiAmountString": "1.5"
}
}
],
"preBalances": [
0,
3000000000u64
],
"postBalances": [
0,
2000000000u64
]
});
let transaction_json_text = transaction_json.to_string();
let meta_json_text = meta_json.to_string();
let result = super::extract_pump_fun_amounts_from_transaction(
transaction_json_text.as_str(),
Some(meta_json_text.as_str()),
Some("BaseVault111"),
Some("NativeQuote111"),
);
let amounts = match result {
Ok(amounts) => amounts,
Err(error) => panic!("pump_fun extraction should succeed: {}", error),
};
assert_eq!(amounts.0, Some("500000".to_string()));
assert_eq!(amounts.1, Some("1000000000".to_string()));
assert_eq!(amounts.2, Some(2.0));
}
}

466
kb_lib/src/transaction_classification.rs Normal file
View File

@@ -0,0 +1,466 @@
// file: kb_lib/src/transaction_classification.rs
//! Transaction classification service.
//!
//! This service classifies projected Solana transactions after transaction
//! projection and optional DEX decoding.
//!
//! The first version is intentionally deterministic and conservative:
//! decoded DEX events win over program-id hints, and unknown transactions are
//! preserved as explicit `unknown_or_unclassified` rows.
/// Service used to classify projected Solana transactions.
#[derive(Debug, Clone)]
pub struct TransactionClassificationService {
database: std::sync::Arc<crate::Database>,
}
impl TransactionClassificationService {
/// Creates a transaction classification service.
pub fn new(database: std::sync::Arc<crate::Database>) -> Self {
return Self { database };
}
/// Classifies one transaction by signature and persists the classification.
pub async fn classify_transaction_by_signature(
&self,
signature: &str,
) -> Result<crate::TransactionClassificationDto, crate::Error> {
let context_result =
load_transaction_classification_context(self.database.as_ref(), signature).await;
let context = match context_result {
Ok(context) => context,
Err(error) => return Err(error),
};
let classification = classify_transaction_context(&context);
let dto = crate::TransactionClassificationDto::new(
context.transaction_id,
context.transaction.signature.clone(),
context.transaction.slot,
classification.kind.to_string(),
classification.primary_protocol,
classification.primary_program_id,
classification.confidence_level,
classification.reason,
classification.evidence_json,
);
let upsert_result =
crate::query_transaction_classifications_upsert(self.database.as_ref(), &dto).await;
if let Err(error) = upsert_result {
return Err(error);
}
let persisted_result = crate::query_transaction_classifications_get_by_transaction_id(
self.database.as_ref(),
context.transaction_id,
)
.await;
let persisted_option = match persisted_result {
Ok(persisted_option) => persisted_option,
Err(error) => return Err(error),
};
let persisted = match persisted_option {
Some(persisted) => persisted,
None => {
return Err(crate::Error::InvalidState(format!(
"transaction classification for '{}' disappeared after upsert",
signature
)));
},
};
let candidate_recording_result =
crate::protocol_candidate_recording::record_protocol_candidates_for_classification(
crate::protocol_candidate_recording::ProtocolCandidateRecordingInput {
database: self.database.as_ref(),
transaction: &context.transaction,
transaction_id: context.transaction_id,
instructions: &context.instructions,
classification_kind: persisted.classification_kind.as_str(),
},
)
.await;
match candidate_recording_result {
Ok(candidate_count) => {
tracing::trace!(
signature = %context.transaction.signature,
classification_kind = %persisted.classification_kind,
protocol_candidate_count = candidate_count,
"transaction protocol candidates recorded"
);
},
Err(error) => return Err(error),
}
return Ok(persisted);
}
}
struct TransactionClassificationContext {
transaction: crate::ChainTransactionDto,
transaction_id: i64,
instructions: std::vec::Vec<crate::ChainInstructionDto>,
decoded_events: std::vec::Vec<crate::DexDecodedEventDto>,
}
struct TransactionClassificationDecision {
kind: &'static str,
primary_protocol: std::option::Option<std::string::String>,
primary_program_id: std::option::Option<std::string::String>,
confidence_level: i16,
reason: std::string::String,
evidence_json: std::string::String,
}
#[derive(Debug, Clone)]
struct KnownDexProgramMatch {
protocol_name: &'static str,
program_id: std::string::String,
instruction_id: std::option::Option<i64>,
instruction_index: u32,
}
async fn load_transaction_classification_context(
database: &crate::Database,
signature: &str,
) -> Result<TransactionClassificationContext, crate::Error> {
let transaction_result =
crate::query_chain_transactions_get_by_signature(database, signature).await;
let transaction_option = match transaction_result {
Ok(transaction_option) => transaction_option,
Err(error) => return Err(error),
};
let transaction = match transaction_option {
Some(transaction) => transaction,
None => {
return Err(crate::Error::InvalidState(format!(
"cannot classify unknown chain transaction '{}'",
signature
)));
},
};
let transaction_id = match transaction.id {
Some(transaction_id) => transaction_id,
None => {
return Err(crate::Error::InvalidState(format!(
"chain transaction '{}' has no internal id",
signature
)));
},
};
let instructions_result =
crate::query_chain_instructions_list_by_transaction_id(database, transaction_id).await;
let instructions = match instructions_result {
Ok(instructions) => instructions,
Err(error) => return Err(error),
};
let decoded_events_result =
crate::query_dex_decoded_events_list_by_transaction_id(database, transaction_id).await;
let decoded_events = match decoded_events_result {
Ok(decoded_events) => decoded_events,
Err(error) => return Err(error),
};
return Ok(TransactionClassificationContext {
transaction,
transaction_id,
instructions,
decoded_events,
});
}
fn classify_transaction_context(
context: &TransactionClassificationContext,
) -> TransactionClassificationDecision {
if !context.decoded_events.is_empty() {
return classify_from_decoded_events(context);
}
let known_program_matches = find_known_dex_program_matches(&context.instructions);
if !known_program_matches.is_empty() {
return classify_from_known_program_matches(context, &known_program_matches);
}
return build_decision(
"unknown_or_unclassified",
None,
None,
25,
"transaction has no decoded DEX event and no known DEX program id".to_string(),
serde_json::json!({
"transactionId": context.transaction_id,
"signature": context.transaction.signature,
"slot": context.transaction.slot,
"instructionCount": context.instructions.len(),
"decodedEventCount": context.decoded_events.len()
}),
);
}
fn classify_from_decoded_events(
context: &TransactionClassificationContext,
) -> TransactionClassificationDecision {
let mut first_protocol = None;
let mut first_program_id = None;
let mut trade_event_count = 0_i64;
let mut non_trade_event_count = 0_i64;
let mut decoded_event_evidence = std::vec::Vec::new();
for decoded_event in &context.decoded_events {
if first_protocol.is_none() {
first_protocol = Some(decoded_event.protocol_name.clone());
}
if first_program_id.is_none() {
first_program_id = Some(decoded_event.program_id.clone());
}
let payload_value_result =
serde_json::from_str::<serde_json::Value>(decoded_event.payload_json.as_str());
let payload_value = match payload_value_result {
Ok(payload_value) => payload_value,
Err(_) => serde_json::Value::Null,
};
let is_trade = crate::is_decoded_event_trade_candidate(
decoded_event.event_kind.as_str(),
&payload_value,
);
if is_trade {
trade_event_count += 1;
} else {
non_trade_event_count += 1;
}
decoded_event_evidence.push(serde_json::json!({
"id": decoded_event.id,
"protocolName": decoded_event.protocol_name,
"programId": decoded_event.program_id,
"eventKind": decoded_event.event_kind,
"poolAccount": decoded_event.pool_account,
"tradeCandidate": is_trade
}));
}
if trade_event_count > 0_i64 {
return build_decision(
"dex_trade",
first_protocol,
first_program_id,
100,
"transaction has at least one decoded DEX trade event".to_string(),
serde_json::json!({
"transactionId": context.transaction_id,
"signature": context.transaction.signature,
"slot": context.transaction.slot,
"decodedEventCount": context.decoded_events.len(),
"tradeEventCount": trade_event_count,
"nonTradeEventCount": non_trade_event_count,
"decodedEvents": decoded_event_evidence
}),
);
}
return build_decision(
"dex_non_trade",
first_protocol,
first_program_id,
95,
"transaction has decoded DEX events but no trade candidate".to_string(),
serde_json::json!({
"transactionId": context.transaction_id,
"signature": context.transaction.signature,
"slot": context.transaction.slot,
"decodedEventCount": context.decoded_events.len(),
"tradeEventCount": trade_event_count,
"nonTradeEventCount": non_trade_event_count,
"decodedEvents": decoded_event_evidence
}),
);
}
fn classify_from_known_program_matches(
context: &TransactionClassificationContext,
known_program_matches: &[KnownDexProgramMatch],
) -> TransactionClassificationDecision {
let first_match = &known_program_matches[0];
let mut evidence_items = std::vec::Vec::new();
for known_program_match in known_program_matches {
evidence_items.push(serde_json::json!({
"protocolName": known_program_match.protocol_name,
"programId": known_program_match.program_id,
"instructionId": known_program_match.instruction_id,
"instructionIndex": known_program_match.instruction_index
}));
}
return build_decision(
"known_dex_program_unclassified",
Some(first_match.protocol_name.to_string()),
Some(first_match.program_id.to_string()),
75,
"transaction has known DEX program instructions but no decoded DEX event".to_string(),
serde_json::json!({
"transactionId": context.transaction_id,
"signature": context.transaction.signature,
"slot": context.transaction.slot,
"instructionCount": context.instructions.len(),
"decodedEventCount": context.decoded_events.len(),
"knownDexProgramMatches": evidence_items
}),
);
}
fn build_decision(
kind: &'static str,
primary_protocol: std::option::Option<std::string::String>,
primary_program_id: std::option::Option<std::string::String>,
confidence_level: i16,
reason: std::string::String,
evidence_value: serde_json::Value,
) -> TransactionClassificationDecision {
let evidence_json_result = serde_json::to_string(&evidence_value);
let evidence_json = match evidence_json_result {
Ok(evidence_json) => evidence_json,
Err(error) => {
return TransactionClassificationDecision {
kind: "unknown_or_unclassified",
primary_protocol: None,
primary_program_id: None,
confidence_level: 0,
reason: format!("cannot serialize classification evidence: {}", error),
evidence_json: "{}".to_string(),
};
},
};
return TransactionClassificationDecision {
kind,
primary_protocol,
primary_program_id,
confidence_level,
reason,
evidence_json,
};
}
fn find_known_dex_program_matches(
instructions: &[crate::ChainInstructionDto],
) -> std::vec::Vec<KnownDexProgramMatch> {
let mut matches = std::vec::Vec::new();
for instruction in instructions {
let program_match = known_dex_program_match(instruction);
let program_match = match program_match {
Some(program_match) => program_match,
None => continue,
};
matches.push(program_match);
}
return matches;
}
fn known_dex_program_match(
instruction: &crate::ChainInstructionDto,
) -> std::option::Option<KnownDexProgramMatch> {
let program_id = match instruction.program_id.as_deref() {
Some(program_id) => program_id,
None => return None,
};
let protocol_name = if program_id == crate::RAYDIUM_AMM_V4_PROGRAM_ID {
"raydium_amm_v4"
} else if program_id == crate::RAYDIUM_CPMM_PROGRAM_ID {
"raydium_cpmm"
} else if program_id == crate::RAYDIUM_CLMM_PROGRAM_ID {
"raydium_clmm"
} else if program_id == crate::RAYDIUM_LAUNCHLAB_PROGRAM_ID {
"raydium_launchlab"
} else if program_id == crate::RAYDIUM_AMM_ROUTING_PROGRAM_ID {
"raydium_router"
} else if program_id == crate::RAYDIUM_STABLE_SWAP_AMM_PROGRAM_ID {
"raydium_stable_swap"
} else if program_id == crate::PUMP_FUN_PROGRAM_ID {
"pump_fun"
} else if program_id == crate::PUMP_SWAP_PROGRAM_ID {
"pump_swap"
} else if program_id == crate::METEORA_DBC_PROGRAM_ID {
"meteora_dbc"
} else if program_id == crate::METEORA_DLMM_PROGRAM_ID {
"meteora_dlmm"
} else if program_id == crate::METEORA_DAMM_V1_PROGRAM_ID {
"meteora_damm_v1"
} else if program_id == crate::METEORA_DAMM_V2_PROGRAM_ID {
"meteora_damm_v2"
} else if program_id == crate::ORCA_WHIRLPOOLS_PROGRAM_ID {
"orca_whirlpools"
} else if program_id == crate::FLUXBEAM_PROGRAM_ID {
"fluxbeam"
} else if program_id == crate::DEXLAB_PROGRAM_ID {
"dexlab"
} else {
return None;
};
return Some(KnownDexProgramMatch {
protocol_name,
program_id: program_id.to_string(),
instruction_id: instruction.id,
instruction_index: instruction.instruction_index,
});
}
#[cfg(test)]
mod tests {
fn test_instruction(
program_id: std::option::Option<std::string::String>,
) -> crate::ChainInstructionDto {
return crate::ChainInstructionDto::new(
1,
None,
0,
None,
program_id,
None,
None,
"[]".to_string(),
None,
None,
Some(serde_json::json!({}).to_string()),
);
}
fn test_transaction() -> crate::ChainTransactionDto {
let mut transaction = crate::ChainTransactionDto::new(
"signature_1".to_string(),
Some(123),
None,
Some("test".to_string()),
None,
None,
None,
serde_json::json!({}).to_string(),
);
transaction.id = Some(1);
return transaction;
}
#[test]
fn known_dex_program_ids_are_matched() {
let instruction = test_instruction(Some(crate::RAYDIUM_CPMM_PROGRAM_ID.to_string()));
let program_match = match super::known_dex_program_match(&instruction) {
Some(program_match) => program_match,
None => {
panic!("expected raydium_cpmm program match");
},
};
assert_eq!(program_match.protocol_name, "raydium_cpmm");
assert_eq!(program_match.program_id, crate::RAYDIUM_CPMM_PROGRAM_ID);
assert_eq!(program_match.instruction_index, 0);
}
#[test]
fn unknown_program_id_is_not_matched() {
let instruction =
test_instruction(Some("UnknownProgram111111111111111111111111111111111".to_string()));
let program_match = super::known_dex_program_match(&instruction);
assert!(program_match.is_none());
}
#[test]
fn unknown_context_is_classified_as_unknown_or_unclassified() {
let transaction = test_transaction();
let context = super::TransactionClassificationContext {
transaction,
transaction_id: 1,
instructions: std::vec::Vec::new(),
decoded_events: std::vec::Vec::new(),
};
let decision = super::classify_transaction_context(&context);
assert_eq!(decision.kind, "unknown_or_unclassified");
assert_eq!(decision.confidence_level, 25);
}
}

17
kb_lib/src/tx_resolution.rs
View File

@@ -109,6 +109,7 @@ pub struct TransactionResolutionService {
wallet_holding_observation_service: crate::WalletHoldingObservationService, wallet_holding_observation_service: crate::WalletHoldingObservationService,
pair_candle_aggregation_service: crate::PairCandleAggregationService, pair_candle_aggregation_service: crate::PairCandleAggregationService,
pair_analytic_signal_service: crate::PairAnalyticSignalService, pair_analytic_signal_service: crate::PairAnalyticSignalService,
transaction_classification_service: crate::TransactionClassificationService,
resolved_signatures: resolved_signatures:
std::sync::Arc<tokio::sync::Mutex<std::collections::HashSet<std::string::String>>>, std::sync::Arc<tokio::sync::Mutex<std::collections::HashSet<std::string::String>>>,
} }
@@ -133,6 +134,8 @@ impl TransactionResolutionService {
let pair_candle_aggregation_service = let pair_candle_aggregation_service =
crate::PairCandleAggregationService::new(database.clone()); crate::PairCandleAggregationService::new(database.clone());
let pair_analytic_signal_service = crate::PairAnalyticSignalService::new(database.clone()); let pair_analytic_signal_service = crate::PairAnalyticSignalService::new(database.clone());
let transaction_classification_service =
crate::TransactionClassificationService::new(database.clone());
return Self { return Self {
http_pool, http_pool,
persistence, persistence,
@@ -147,6 +150,7 @@ impl TransactionResolutionService {
wallet_holding_observation_service, wallet_holding_observation_service,
pair_candle_aggregation_service, pair_candle_aggregation_service,
pair_analytic_signal_service, pair_analytic_signal_service,
transaction_classification_service,
resolved_signatures: std::sync::Arc::new(tokio::sync::Mutex::new( resolved_signatures: std::sync::Arc::new(tokio::sync::Mutex::new(
std::collections::HashSet::new(), std::collections::HashSet::new(),
)), )),
@@ -400,6 +404,17 @@ impl TransactionResolutionService {
Err(error) => return Err(error), Err(error) => return Err(error),
}; };
let pair_analytic_signal_count = pair_analytic_signals.len(); let pair_analytic_signal_count = pair_analytic_signals.len();
let transaction_classification_result = self
.transaction_classification_service
.classify_transaction_by_signature(request.signature.as_str())
.await;
let transaction_classification = match transaction_classification_result {
Ok(transaction_classification) => transaction_classification,
Err(error) => return Err(error),
};
let transaction_classification_id = transaction_classification.id;
let transaction_classification_kind =
transaction_classification.classification_kind.clone();
let payload = serde_json::json!({ let payload = serde_json::json!({
"status": "resolved", "status": "resolved",
"signature": request.signature.clone(), "signature": request.signature.clone(),
@@ -417,6 +432,8 @@ impl TransactionResolutionService {
"tradeEventCount": trade_event_count, "tradeEventCount": trade_event_count,
"pairCandleCount": pair_candle_count, "pairCandleCount": pair_candle_count,
"pairAnalyticSignalCount": pair_analytic_signal_count, "pairAnalyticSignalCount": pair_analytic_signal_count,
"transactionClassificationId": transaction_classification_id,
"transactionClassificationKind": transaction_classification_kind,
"transaction": transaction_value "transaction": transaction_value
}); });
let observation_id_result = self let observation_id_result = self