Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Qbeast Spark Refactor: Create Missing Interfaces and Abstractions #418

Closed
JosepSampe opened this issue Sep 16, 2024 · 0 comments
Closed

Qbeast Spark Refactor: Create Missing Interfaces and Abstractions #418

JosepSampe opened this issue Sep 16, 2024 · 0 comments
Assignees
@JosepSampe
Labels
enhancement

Comments

@JosepSampe
Copy link
Member

Review the main Qbeast components, such as StagingDataManager, MetadataManager, PreCommitHook, and RollupDataWriter, etc. Identify and create any missing interfaces and abstractions to improve modularity and maintainability.
@JosepSampe JosepSampe self-assigned this Sep 16, 2024
@JosepSampe JosepSampe mentioned this issue Sep 16, 2024
Merged
6 tasks
cugni pushed a commit that referenced this issue Sep 19, 2024
@JosepSampe
Issue #418: Abstract PreCommitHook and StagingDataManager (#421)
71b98c0 
* Abstract PreCommitHook

* Abstract StagingDataManager

* Propagate stats

* Update MetadataManager

* Update MetadataWriter

* Update MetadataManager

* Add static typing

* Update QbeastFileUtils

* Fix IndexFile stats

* Update DeleteFile logic

* Update qbeast files
@JosepSampe JosepSampe mentioned this issue Sep 19, 2024
Merged
6 tasks
cugni pushed a commit that referenced this issue Sep 19, 2024
@JosepSampe
Issue #418: Abstract RollupDataWriter and QbeastStats (#423)
d604aaa 
* Update rollup data writer

* Abstract QbeastStats

* Update QbeastStats to use Number

* Update QbeastStats to use String

* Update QbeastStats

* Update QbeastStats Delta
JosepSampe added a commit to JosepSampe/qbeast-spark that referenced this issue Oct 24, 2024
@JosepSampe
Issue Qbeast-io#418: Abstract RollupDataWriter and QbeastStats (Qbeas…
566b9cd 
…t-io#423)

* Update rollup data writer

* Abstract QbeastStats

* Update QbeastStats to use Number

* Update QbeastStats to use String

* Update QbeastStats

* Update QbeastStats Delta
osopardo1 pushed a commit that referenced this issue Oct 28, 2024
@JosepSampe
Issue #398: Qbeast Spark Refactor -- Delta abstraction (#446)
de19c2e 
* Issue #417: Abstract the Qbeast Snapshot Module (#411)

* Issue #418: Abstract PreCommitHook and StagingDataManager (#421)

* Issue #418: Abstract RollupDataWriter and QbeastStats  (#423)

* Issue #419: Code Reorganization (#425)

* Issue #420: Create Separate Modules (#427)

* Issue #398: Fix small overhead added during the refactoring (#436)

* Issue #441: Fix data change on optimize (#442)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@JosepSampe JosepSampe

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@JosepSampe