Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

⚡ Refactor thumbnail generation and scanner IO operations (#426) #442

Merged
merged 2 commits into from
Sep 14, 2024

Conversation

aaronleopold
Copy link
Collaborator

This PR updates develop with the latest in experimental. There is only one important commit, so I plan on rebasing experimental afterwards. This will bring in the thumbnail generation and scanner refactors.

I want to reiterate the warning from the original PR:

⚠️ There are potentially breaking changes in this PR from the restructuring of library options. The migration was hand written, and while I tested it on my personal experimental instance without issue I am still weary enough to put out this disclaimer.

I am aiming to merge this in over the weekend, and I'll try to test the update path (nightly -> this) beforehand

* ⚡ Refactor thumbnail generation

* ⚡ Refactor scanner IO operations

Push up last nights changes

* debug: push to new tag for testing

* async-ify IO operations outside scan context

* debug: add tracing

* wip: rename library options and add fields

requires migration still

* wip: rename library options, fix frontend

* handwrite migration 😬

* fix ui

* NON FUNCTIONAL: wip migration

kill me

* debug: stop auto pushing

* fix migration?

* wip: support processing options and restructure

* 🥪 lunch allocator experiment

* super wip: distroless image, zip downgrade

* cleanup docker stuff

* Revert "debug: stop auto pushing"

This reverts commit bd6da98.

* remove missed feature

* fix job upsert after refresh

* cleanup comments and wip review

* Reapply "debug: stop auto pushing"

This reverts commit f43c187.

* cleanup
Copy link

codecov bot commented Sep 13, 2024

Codecov Report

Attention: Patch coverage is 8.51064% with 731 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
core/src/filesystem/scanner/utils.rs 0.00% 316 Missing ⚠️
apps/server/src/routers/api/v1/library.rs 0.00% 75 Missing ⚠️
core/src/filesystem/image/thumbnail/utils.rs 0.00% 56 Missing ⚠️
core/src/filesystem/scanner/library_scan_job.rs 0.00% 41 Missing ⚠️
...e/src/filesystem/image/thumbnail/generation_job.rs 0.00% 39 Missing ⚠️
core/src/filesystem/image/thumbnail/generate.rs 0.00% 32 Missing ⚠️
core/src/filesystem/media/process.rs 27.27% 32 Missing ⚠️
apps/server/src/routers/api/v1/media.rs 0.00% 30 Missing ⚠️
core/src/filesystem/common.rs 0.00% 28 Missing ⚠️
apps/server/src/routers/api/v1/series.rs 0.00% 26 Missing ⚠️
... and 16 more
Files with missing lines Coverage Δ
apps/server/src/routers/utoipa.rs 0.00% <ø> (ø)
core/src/config/stump_config.rs 83.59% <100.00%> (+0.26%) ⬆️
core/src/db/entity/media/entity.rs 0.00% <ø> (ø)
core/src/filesystem/image/mod.rs 100.00% <ø> (+25.00%) ⬆️
core/src/filesystem/media/builder.rs 69.71% <100.00%> (-1.62%) ⬇️
core/src/filesystem/media/format/epub.rs 37.31% <100.00%> (ø)
core/src/filesystem/media/format/pdf.rs 27.31% <100.00%> (ø)
core/src/filesystem/media/format/rar.rs 61.26% <100.00%> (ø)
core/src/filesystem/media/format/zip.rs 75.53% <100.00%> (ø)
core/src/filesystem/media/mod.rs 100.00% <ø> (ø)
... and 29 more

@aaronleopold
Copy link
Collaborator Author

Okay so I've tested the migration, it's definitely "breaking" changes. Running the migration in isolation doesn't really break anything, but when Stump orchestrates the migration for you for whatever reason it removes all of the media from the DB. I assume there is a foreign key constraint when shuffling things around which causes the library series to cascade, and thereby the media cascading. Since scanning is a fairly quick operation, I think this is acceptable TBH, especially with all of the WIP warnings contained in this repo. However, I will not make a habit of allowing this kind of change.

The only feature which I expect there might be some series DB breaks like this would be audiobook support at whatever point that is even feasible to start

@aaronleopold aaronleopold marked this pull request as ready for review September 14, 2024 21:08
@aaronleopold aaronleopold merged commit a05cab6 into develop Sep 14, 2024
10 checks passed
@aaronleopold aaronleopold mentioned this pull request Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant