Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

✨ Smart list filter builder #464

Merged
merged 34 commits into from
Oct 10, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
d56241d
this is awful
aaronleopold Sep 29, 2024
4027d9e
tdd this thing
aaronleopold Sep 29, 2024
70d69e9
more tests
aaronleopold Sep 29, 2024
1ce73e0
hmm
aaronleopold Sep 29, 2024
be3afdc
fix generation, wip conversions
aaronleopold Sep 29, 2024
597c3a2
add grouping
aaronleopold Sep 29, 2024
9630711
starting to have something I think
aaronleopold Oct 3, 2024
4c0d4e7
note future stuff
aaronleopold Oct 3, 2024
feef408
wip: value renderers
aaronleopold Oct 4, 2024
339554f
reintroduce form
aaronleopold Oct 5, 2024
f7ab9b6
cleanup
aaronleopold Oct 5, 2024
c7b9653
Merge remote-tracking branch 'origin/experimental' into al/smart-list-ui
aaronleopold Oct 5, 2024
2975808
ITS ALIVE
aaronleopold Oct 5, 2024
541a62a
tweaks
aaronleopold Oct 5, 2024
e79a75d
wip: locale for smart list form
aaronleopold Oct 5, 2024
a260de0
add locale for smart list form
aaronleopold Oct 5, 2024
ebe6ed7
add tests, add mine param
aaronleopold Oct 5, 2024
b1de50c
add missed locale
aaronleopold Oct 5, 2024
b4493ae
setup settings for sidebar pattern
aaronleopold Oct 5, 2024
2a64630
fix blip to null when deleting group
aaronleopold Oct 5, 2024
6a037b6
rearrange components, simplify enum variant
aaronleopold Oct 6, 2024
67449ca
wip: build out settings pages
aaronleopold Oct 6, 2024
fc5ab4c
update smart list docs
aaronleopold Oct 6, 2024
4fc81b8
fix change detection, fix attribute label, fix misc sdk issues
aaronleopold Oct 6, 2024
a9b0c4b
fix web build
aaronleopold Oct 7, 2024
4f3cbb5
wip: add the filters which you can select in UI
aaronleopold Oct 7, 2024
cc601ea
fix basic number value
aaronleopold Oct 7, 2024
c088265
add perf disclaimer
aaronleopold Oct 7, 2024
eef2c1d
wip: fix numbers and dates
aaronleopold Oct 8, 2024
5745626
Merge remote-tracking branch 'origin/experimental' into al/smart-list-ui
aaronleopold Oct 9, 2024
95d9443
lunch stuff
aaronleopold Oct 9, 2024
3b2f173
fix date parsing, cleanup
aaronleopold Oct 10, 2024
d5218e6
normalize date
aaronleopold Oct 10, 2024
cd6cfbc
fix update preferences
aaronleopold Oct 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
339 changes: 117 additions & 222 deletions Cargo.lock

Large diffs are not rendered by default.

6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ Stump is a free and open source comics, manga and digital book server with OPDS
- [License 📝](#license-)
</details>

> **🚧 Disclaimer 🚧**: Stump is under active development and is an ongoing **WIP**. Anyone is welcome to try it out, but **DO NOT** expect a fully featured or bug-free experience. If you'd like to contribute and help expedite Stump's first release, please review the [developer guide](#developer-guide-).
> **🚧 Disclaimer 🚧**: Stump is under active development and is an ongoing **WIP**. Anyone is welcome to try it out, but **DO NOT** expect a fully featured or bug-free experience. If you'd like to contribute and help expedite feature development, please review the [developer guide](#developer-guide-).

## Roadmap 🗺

The following items are the major targets for Stump's first release:
The following items are the major targets for Stump's first stable release:

- 📃 Full OPDS + OPDS Page Streaming support
- 📕 EPUB, PDF, and CBZ/CBR support
Expand All @@ -59,7 +59,7 @@ The following items are the major targets for Stump's first release:
- 🌏 Language support _(look [here](https://github.com/stumpapp/stump/issues/106))_
- 🌈 And more!

Things you can expect to see after the first release:
Things you can expect to see afterwards:

- 🖥️ Cross-platform desktop app _(Windows, Mac, Linux)_
- 📖 [Tachiyomi](https://github.com/stumpapp/tachiyomi-extensions) support
Expand Down
2 changes: 1 addition & 1 deletion apps/expo/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"nativewind": "^2.0.11",
"react": "18.2.0",
"react-dom": "18.2.0",
"react-hook-form": "^7.50.1",
"react-hook-form": "^7.53.0",
"react-native": "0.73.4",
"react-native-fs": "^2.20.0",
"react-native-gesture-handler": "~2.14.0",
Expand Down
22 changes: 21 additions & 1 deletion apps/server/src/routers/api/v1/smart_list.rs
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,8 @@
#[serde(default)]
all: Option<bool>,
#[serde(default)]
mine: Option<bool>,
#[serde(default)]
search: Option<String>,
}

Expand Down Expand Up @@ -141,11 +143,21 @@
));
}

let mine = params.mine.unwrap_or(false);
if query_all && mine {
return Err(APIError::BadRequest(
"Cannot query all and mine at the same time".to_string(),
));
}

Check warning on line 151 in apps/server/src/routers/api/v1/smart_list.rs

View check run for this annotation

Codecov / codecov/patch

apps/server/src/routers/api/v1/smart_list.rs#L146-L151

Added lines #L146 - L151 were not covered by tests

let where_params = chain_optional_iter(
[],
[
(!query_all)
// If not querying all, and not querying mine, then we need to filter by access
(!query_all && !mine)

Check warning on line 157 in apps/server/src/routers/api/v1/smart_list.rs

View check run for this annotation

Codecov / codecov/patch

apps/server/src/routers/api/v1/smart_list.rs#L156-L157

Added lines #L156 - L157 were not covered by tests
.then(|| smart_list_access_for_user(&user, AccessRole::Reader.value())),
// If querying mine, then we need to filter by the user
mine.then(|| smart_list::creator_id::equals(user.id.clone())),

Check warning on line 160 in apps/server/src/routers/api/v1/smart_list.rs

View check run for this annotation

Codecov / codecov/patch

apps/server/src/routers/api/v1/smart_list.rs#L159-L160

Added lines #L159 - L160 were not covered by tests
params.search.map(|search| {
or![
smart_list::name::contains(search.clone()),
Expand Down Expand Up @@ -174,6 +186,8 @@
pub joiner: Option<FilterJoin>,
#[serde(default)]
pub default_grouping: Option<SmartListItemGrouping>,
#[serde(default)]
pub visibility: Option<EntityVisibility>,
}

#[utoipa::path(
Expand Down Expand Up @@ -217,6 +231,9 @@
input.default_grouping.map(|grouping| {
smart_list::default_grouping::set(grouping.to_string())
}),
input.visibility.map(|visibility| {
smart_list::visibility::set(visibility.to_string())
}),

Check warning on line 236 in apps/server/src/routers/api/v1/smart_list.rs

View check run for this annotation

Codecov / codecov/patch

apps/server/src/routers/api/v1/smart_list.rs#L234-L236

Added lines #L234 - L236 were not covered by tests
],
),
)
Expand Down Expand Up @@ -323,6 +340,9 @@
input.default_grouping.map(|grouping| {
smart_list::default_grouping::set(grouping.to_string())
}),
input.visibility.map(|visibility| {
smart_list::visibility::set(visibility.to_string())
}),

Check warning on line 345 in apps/server/src/routers/api/v1/smart_list.rs

View check run for this annotation

Codecov / codecov/patch

apps/server/src/routers/api/v1/smart_list.rs#L343-L345

Added lines #L343 - L345 were not covered by tests
],
),
)
Expand Down
132 changes: 102 additions & 30 deletions core/src/db/filter/smart_filter.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
use std::{fmt::Display, str::FromStr};

use prisma_client_rust::{and, not, or};
use prisma_client_rust::{
and,
chrono::{DateTime, FixedOffset},
not, or,
};
use serde::{Deserialize, Serialize};
use specta::Type;
use utoipa::ToSchema;
Expand All @@ -15,16 +19,13 @@
// 1. Performance implications. This is mostly because the assumption for each `into_prisma` call is a single param,
// which means for relation filters we will have an `is` call each time. I don't yet know how this actually affects
// performance in real-world scenarios, but it's something to keep in mind.
// 2. Repetition of logic. There is a lot of repetition in the `into_prisma` definitions, and I think there is a way to (maybe)
// consolidate them into a single macro. I'm not sure if this is possible, but it's worth looking into. This will get exponentially
// worse as things like sorting and sorting on relations are added... :weary:

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, ToSchema, Type)]
#[serde(untagged)]
/// A filter for a single value, e.g. `name = "test"`
pub enum Filter<T> {
/// A simple equals filter, e.g. `name = "test"`
Equals(T),
Equals { equals: T },
/// A simple not filter, e.g. `name != "test"`
Not { not: T },
/// A filter for a string that contains a substring, e.g. `name contains "test"`. This should
Expand Down Expand Up @@ -71,7 +72,7 @@
WhereParam: From<prisma_client_rust::Operator<WhereParam>>,
{
match self {
Filter::Equals(value) => equals_fn(value),
Filter::Equals { equals } => equals_fn(equals),

Check warning on line 75 in core/src/db/filter/smart_filter.rs

View check run for this annotation

Codecov / codecov/patch

core/src/db/filter/smart_filter.rs#L75

Added line #L75 was not covered by tests
Filter::Not { not } => not![equals_fn(not)],
Filter::Contains { contains } => contains_fn(contains),
Filter::Excludes { excludes } => not![contains_fn(excludes)],
Expand All @@ -93,7 +94,7 @@
WhereParam: From<prisma_client_rust::Operator<WhereParam>>,
{
match self {
Filter::Equals(value) => equals_fn(Some(value)),
Filter::Equals { equals } => equals_fn(Some(equals)),

Check warning on line 97 in core/src/db/filter/smart_filter.rs

View check run for this annotation

Codecov / codecov/patch

core/src/db/filter/smart_filter.rs#L97

Added line #L97 was not covered by tests
Filter::Not { not } => not![equals_fn(Some(not))],
Filter::Contains { contains } => contains_fn(contains),
Filter::Excludes { excludes } => not![contains_fn(excludes)],
Expand All @@ -102,22 +103,20 @@
_ => unreachable!("Numeric filters should be handled elsewhere"),
}
}
}

impl Filter<i32> {
pub fn into_numeric_params<WhereParam>(
self,
equals_fn: fn(i32) -> WhereParam,
gt_fn: fn(i32) -> WhereParam,
gte_fn: fn(i32) -> WhereParam,
lt_fn: fn(i32) -> WhereParam,
lte_fn: fn(i32) -> WhereParam,
equals_fn: fn(T) -> WhereParam,
gt_fn: fn(T) -> WhereParam,
gte_fn: fn(T) -> WhereParam,
lt_fn: fn(T) -> WhereParam,
lte_fn: fn(T) -> WhereParam,

Check warning on line 113 in core/src/db/filter/smart_filter.rs

View check run for this annotation

Codecov / codecov/patch

core/src/db/filter/smart_filter.rs#L109-L113

Added lines #L109 - L113 were not covered by tests
) -> WhereParam
where
WhereParam: From<prisma_client_rust::Operator<WhereParam>>,
{
match self {
Filter::Equals(value) => equals_fn(value),
Filter::Equals { equals } => equals_fn(equals),

Check warning on line 119 in core/src/db/filter/smart_filter.rs

View check run for this annotation

Codecov / codecov/patch

core/src/db/filter/smart_filter.rs#L119

Added line #L119 was not covered by tests
Filter::Not { not } => not![equals_fn(not)],
Filter::NumericFilter(numeric_filter) => match numeric_filter {
NumericFilter::Gt { gt } => gt_fn(gt),
Expand All @@ -138,17 +137,17 @@

pub fn into_optional_numeric_params<WhereParam>(
self,
equals_fn: fn(Option<i32>) -> WhereParam,
gt_fn: fn(i32) -> WhereParam,
gte_fn: fn(i32) -> WhereParam,
lt_fn: fn(i32) -> WhereParam,
lte_fn: fn(i32) -> WhereParam,
equals_fn: fn(Option<T>) -> WhereParam,
gt_fn: fn(T) -> WhereParam,
gte_fn: fn(T) -> WhereParam,
lt_fn: fn(T) -> WhereParam,
lte_fn: fn(T) -> WhereParam,
) -> WhereParam
where
WhereParam: From<prisma_client_rust::Operator<WhereParam>>,
{
match self {
Filter::Equals(value) => equals_fn(Some(value)),
Filter::Equals { equals } => equals_fn(Some(equals)),

Check warning on line 150 in core/src/db/filter/smart_filter.rs

View check run for this annotation

Codecov / codecov/patch

core/src/db/filter/smart_filter.rs#L150

Added line #L150 was not covered by tests
Filter::Not { not } => not![equals_fn(Some(not))],
Filter::NumericFilter(numeric_filter) => match numeric_filter {
NumericFilter::Gt { gt } => gt_fn(gt),
Expand Down Expand Up @@ -217,11 +216,6 @@
}
}

// pub struct SmartFilterOrder {
// pub direction: Direction,
// pub order_by:
// }

#[derive(Debug, Clone, Serialize, Deserialize, Type, ToSchema)]
#[aliases(SmartFilterSchema = SmartFilter<MediaSmartFilter>)]
pub struct SmartFilter<T> {
Expand All @@ -230,9 +224,6 @@
pub joiner: FilterJoin,
}

// TODO: figure out if perhaps macros can come in with the save here. Continuing down this path
// will be INCREDIBLY verbose..

#[generate_smart_filter]
#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Type, ToSchema)]
#[serde(untagged)]
Expand Down Expand Up @@ -335,7 +326,6 @@
Inkers { inkers: String },
#[is_optional]
Editors { editors: String },
// FIXME: Current implementationm makes it awkward to support numeric filters
#[is_optional]
AgeRating { age_rating: i32 },
#[is_optional]
Expand All @@ -352,8 +342,13 @@
#[prisma_table("media")]
pub enum MediaSmartFilter {
Name { name: String },
Size { size: i64 },
Extension { extension: String },
CreatedAt { created_at: DateTime<FixedOffset> },
UpdatedAt { updated_at: DateTime<FixedOffset> },
Status { status: String },
Path { path: String },
Pages { pages: i32 },
Metadata { metadata: MediaMetadataSmartFilter },
Series { series: SeriesSmartFilter },
}
Expand Down Expand Up @@ -437,6 +432,83 @@
);
}

#[test]
fn it_serializes_number_correctly() {
let filter: FilterGroup<MediaSmartFilter> = FilterGroup::And {
and: vec![MediaSmartFilter::Size {
size: Filter::NumericFilter(NumericFilter::Gte { gte: 3000 }),
}],
};

let json = serde_json::to_string(&filter).unwrap();

assert_eq!(json, r#"{"and":[{"size":{"gte":3000}}]}"#);
}

#[test]
fn it_deserializes_number_correctly() {
let json = r#"{"or":[{"size":{"gte":3000}}]}"#;

let filter: FilterGroup<MediaSmartFilter> = serde_json::from_str(json).unwrap();

assert_eq!(
filter,
FilterGroup::Or {
or: vec![MediaSmartFilter::Size {
size: Filter::NumericFilter(NumericFilter::Gte { gte: 3000 }),
}],
}
);
}

#[test]
fn it_serializes_range_correctly() {
let filter: FilterGroup<MediaSmartFilter> = FilterGroup::And {
and: vec![MediaSmartFilter::Metadata {
metadata: MediaMetadataSmartFilter::AgeRating {
age_rating: Filter::NumericFilter(NumericFilter::Range(
NumericRange {
from: 10,
to: 20,
inclusive: true,
},
)),
},
}],
};

let json = serde_json::to_string(&filter).unwrap();

assert_eq!(
json,
r#"{"and":[{"metadata":{"age_rating":{"from":10,"to":20,"inclusive":true}}}]}"#
);
}

#[test]
fn it_deserializes_range_correctly() {
let json = r#"{"and":[{"metadata":{"age_rating":{"from":10,"to":20,"inclusive":true}}}]}"#;

let filter: FilterGroup<MediaSmartFilter> = serde_json::from_str(json).unwrap();

assert_eq!(
filter,
FilterGroup::And {
and: vec![MediaSmartFilter::Metadata {
metadata: MediaMetadataSmartFilter::AgeRating {
age_rating: Filter::NumericFilter(NumericFilter::Range(
NumericRange {
from: 10,
to: 20,
inclusive: true,
},
)),
},
}],
}
);
}

fn default_book(name: &str) -> media::Data {
media::Data {
id: "test-id".to_string(),
Expand Down
4 changes: 0 additions & 4 deletions docs/pages/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,10 +33,6 @@ The hardware requirements vary and should serve **only as a guide**. Generally s

**This section will be revisited once the first beta release is out.**

## When will Stump be released?

Stump is currently under development. The primary developer is working on it in their free time, mostly on the weekends. There aren't many active contributors, so it's hard to say when it will be release ready. Ideally the first release candidate will be ready by the end of 2024. Please consider contributing to the project if you're interested in expediting the development. You can find more information on ways to contribute in the [contributing](/contributing) guide.

## Can I try it out?

I am working on setting up a public demo instance. If it is available, you will find it at [demo.stumpapp.dev](https://demo.stumpapp.dev). The login credentials are:
Expand Down
33 changes: 16 additions & 17 deletions docs/pages/guides/opds.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,17 +20,16 @@ The general structure of the URL to connect to your Stump server is:

The following clients have been tested with Stump:

| OS | Application | Page Streaming | Issues/Notes |
| ------- | :------------------------------------------------------------------------------------: | -------------: | -------------------------: |
| iOS | [Panels](https://panels.app/) | ✅ | |
| OS | Application | Page Streaming | Issues/Notes |
| ------- | :------------------------------------------------------------------------------------: | -------------: | -------------------------------------------------------------------------------: |
| iOS | [Panels](https://panels.app/) | ✅ | |
| iOS | [Chunky Reader](https://apps.apple.com/us/app/chunky-comic-reader/id663567628) | ✅ | |
| Android | [Moon+ Reader](https://play.google.com/store/apps/details?id=com.flyersoft.moonreader) | ❌ | Users report OK experience |
| Android | [KyBook 3](http://kybook-reader.com/) | ❌ | No testing at this time |
| Android | [Librera](https://play.google.com/store/apps/details?id=com.foobnix.pdf.reader) | ❌ | Supports covers, categories, and downloads, but shows file names not book titles |
| Android | [Cantook by Aldiko](https://play.google.com/store/apps/details?id=com.aldiko.android) | ❌ | No auth support, does not work. Use 2.0 |
| Linux | [Foliate](https://johnfactotum.github.io/foliate/) | ❌ | Loads cover previews, file names and categories |
| Windows | [Thorium 3](https://thorium.edrlab.org/en/) | ❌ | Loads cover previews, file names and categories |

| Android | [Moon+ Reader](https://play.google.com/store/apps/details?id=com.flyersoft.moonreader) | ❌ | Users report OK experience |
| Android | [KyBook 3](http://kybook-reader.com/) | ❌ | No testing at this time |
| Android | [Librera](https://play.google.com/store/apps/details?id=com.foobnix.pdf.reader) | ❌ | Supports covers, categories, and downloads, but shows file names not book titles |
| Android | [Cantook by Aldiko](https://play.google.com/store/apps/details?id=com.aldiko.android) | ❌ | No auth support, does not work. Use 2.0 |
| Linux | [Foliate](https://johnfactotum.github.io/foliate/) | ❌ | Loads cover previews, file names and categories |
| Windows | [Thorium 3](https://thorium.edrlab.org/en/) | ❌ | Loads cover previews, file names and categories |

If you have any experiences, good or bad, using any of these clients or another client not listed here, please consider updating this page with your findings.

Expand All @@ -44,10 +43,10 @@ The general structure of the URL to connect to your Stump server is:

The following clients have been tested with Stump:

| OS | Application | Page Streaming | Issues/Notes |
| --- | :-----------------------------------------------------------------------------------: | -------------: | -----------------------------------------------------------------------------------------------------------------------------------: |
| iOS | [Cantook by Aldiko](https://apps.apple.com/us/app/cantook-by-aldiko/id1476410111) | ❓ | Supports catalog traversal and media downloads. Issues with rendering media covers, however this appears to be an issue on their end |
| Android | [Cantook by Aldiko](https://play.google.com/store/apps/details?id=com.aldiko.android) | ❓ | Supports catalog traversal and media downloads. Issues with rendering media covers, however this appears to be an issue on their end |
| Android | [Librera](https://play.google.com/store/apps/details?id=com.foobnix.pdf.reader) | ❓ | Does not work. Likely lacking OPDS 2.0 support |
| Linux | [Foliate](https://johnfactotum.github.io/foliate/) | ❌ | Loads cover previews, book names, publisher and categories |
| Windows | [Thorium 3](https://thorium.edrlab.org/en/) | ❌ | Loads covers and their previews, book names, publisher and categories |
| OS | Application | Page Streaming | Issues/Notes |
| ------- | :-----------------------------------------------------------------------------------: | -------------: | -----------------------------------------------------------------------------------------------------------------------------------: |
| iOS | [Cantook by Aldiko](https://apps.apple.com/us/app/cantook-by-aldiko/id1476410111) | ❓ | Supports catalog traversal and media downloads. Issues with rendering media covers, however this appears to be an issue on their end |
| Android | [Cantook by Aldiko](https://play.google.com/store/apps/details?id=com.aldiko.android) | ❓ | Supports catalog traversal and media downloads. Issues with rendering media covers, however this appears to be an issue on their end |
| Android | [Librera](https://play.google.com/store/apps/details?id=com.foobnix.pdf.reader) | ❓ | Does not work. Likely lacking OPDS 2.0 support |
| Linux | [Foliate](https://johnfactotum.github.io/foliate/) | ❌ | Loads cover previews, book names, publisher and categories |
| Windows | [Thorium 3](https://thorium.edrlab.org/en/) | ❌ | Loads covers and their previews, book names, publisher and categories |
Loading
Loading