Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Epic: improve scraper & scraper operations #3357

Open
1 of 7 tasks
tkporter opened this issue Mar 4, 2024 · 0 comments
Open
1 of 7 tasks

Epic: improve scraper & scraper operations #3357

tkporter opened this issue Mar 4, 2024 · 0 comments
Assignees
@tkporter
Labels
agent epic scraper

Comments

@tkporter
Copy link
Collaborator

tkporter commented Mar 4, 2024 •
edited by ameten
Loading

Improvements to the scraper codebase & the operational practices around it. Some of these came out of this comment.

In rough order of priority high -> low

Scraper operations improvements
Preview Give feedback
  1. Adding a new chain to the scraper should not require manual psql tomfoolery #3324
    agent operations scraper
  2. Consolidate scraper domain initialization with chain metadata #3333
    scraper

Scraper improvements
Preview Give feedback
  1. Postgres doesn't support unsigned integers, but scraper DB schema assumes it does #3121
    agent bug scraper
    daniel-savu
  2. Scraper: update the block cursor instead of inserting a new row each time #3335
    scraper
  3. Scraper: the block cursor should have distinct watermarks for different types that are being indexed #3334
    scraper
  4. Scraper: H512 tx hashes for Solana support #3331
    agent scraper solana
    ameten
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@tkporter tkporter

Labels
agent epic scraper
Projects
Hyperlane Tasks
Status: Backlog
Milestone
No milestone
Development

No branches or pull requests
2 participants
@tkporter @avious00