Skip to content

feat: Enhance Postgres sink with improved query handling and data str… - #39

Merged
MartianGreed merged 3 commits into
mainfrom
introspect/db-store
Mar 19, 2026
Merged

MartianGreed merged 3 commits into
mainfrom
introspect/db-store

Conversation

@bengineer42

Copy link
Copy Markdown
Collaborator

…ucture updates

  • Introduced new queries for inserting tables, columns, and dead fields in the Postgres sink.
  • Refactored the PgTable struct to include methods for generating insert queries.
  • Updated the TableUpgrade struct to track upgraded columns and handle column renaming and type changes more effectively.
  • Implemented async functions for fetching tables, columns, and dead fields from the database.
  • Added a new Executable trait to streamline query execution across different database types.
  • Enhanced the PgTypeDef trait to support more flexible row fetching and mapping.
  • Updated the FlexStr and QueryLike structures to improve query handling.
  • Adjusted example configurations for testing with new schema and batch sizes.

…ucture updates

- Introduced new queries for inserting tables, columns, and dead fields in the Postgres sink.
- Refactored the `PgTable` struct to include methods for generating insert queries.
- Updated the `TableUpgrade` struct to track upgraded columns and handle column renaming and type changes more effectively.
- Implemented async functions for fetching tables, columns, and dead fields from the database.
- Added a new `Executable` trait to streamline query execution across different database types.
- Enhanced the `PgTypeDef` trait to support more flexible row fetching and mapping.
- Updated the `FlexStr` and `QueryLike` structures to improve query handling.
- Adjusted example configurations for testing with new schema and batch sizes.
@github-actions

github-actions Bot commented Mar 19, 2026 •

Copy link
Copy Markdown

📊 Synthetic Benchmark Results

Comparing against baseline: 20260319T121438Z

Performance Summary

Token Type Duration (ms) Blocks Transactions Events Envelopes Blocks/sec Events/sec
erc1155 3475 100 50000 50000 70000 28.78 14388.10
erc20 2887 100 50000 50000 50000 34.64 17318.78
erc721 3238 100 50000 50000 50000 30.88 15439.91

Comparison with Baseline

Token Type Metric Baseline Current Change
erc1155 Throughput (events/sec) 14388.10 14388.10 ➡️ 0.0%
erc1155 Duration (ms) 3475 3475 ➡️ 0.0%
erc20 Throughput (events/sec) 17318.78 17318.78 ➡️ 0.0%
erc20 Duration (ms) 2887 2887 ➡️ 0.0%
erc721 Throughput (events/sec) 15439.91 15439.91 ➡️ 0.0%
erc721 Duration (ms) 3238 3238 ➡️ 0.0%

Legend:

  • ✅ Improvement (faster)
  • ⚠️ Regression (slower)
  • 📈 Throughput increase
  • 📉 Throughput decrease
  • ➡️ No significant change (<5%)

Stage Latency Breakdown

Token Type Extract (ms) Decode (ms) Sink (ms) Commit (ms)
erc1155 20.979572999999995 79.660658 3293.9564950000004 0.025003000000000004
erc20 13.441062000000008 60.703544000000015 2721.1155200000007 0.023424999999999987
erc721 12.279075999999995 51.48155699999999 3103.904044000001 0.037840000000000006

Generated by Synthetic Benchmark CI

@MartianGreed
MartianGreed merged commit 7ef8ab8 into main Mar 19, 2026
8 checks passed
@MartianGreed
MartianGreed deleted the introspect/db-store branch March 19, 2026 12:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants