Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support visualizing nD boolean datasets as Line and Heatmap #1499

Merged
merged 5 commits into from
Oct 9, 2023
Merged

Conversation

axelboc
Copy link
Contributor

@axelboc axelboc commented Sep 25, 2023

Proposal implementation for #1498 with some unresolved questions.

EDIT: First part of #1498 (booleans — enums will come later).

I had a couple of options:

  1. Convert the boolean array to a numeric arrays before passing it to the mapped visualization components (current proposal).
  2. Let the boolean array go through and let useDomain handle boolean arrays (to return [0, 1] obviously) – apart from that, it turns out that DataCurve handles boolean array out of the box (though taking advantage of this may not be such a great idea... 🤷 😄)

Option 2 seemed to complexify a bunch lot of code in terms of typing. Option 1 obviously requires mapping through the values array, but seems like a good compromise in terms of complexity (even though it's not trivial either in terms of typing – see below.)

A third option could be to let the data provider convert the boolean values to numbers when visualizing as line/heatmap; or maybe even to request a numeric array straight from the back-end.

Either way, there's a bit of unresolved complexity when it comes to the Value type (i.e. what should Value<Dataset<ArrayShape, NumericType | BooleanType>> resolve to?), since for the matrix vis we want actual booleans (boolean[]), while for the line/heatmap vis, we want either TypedArray | number[] | boolean[] (and not (number | boolean)[]!) for option 1 or TypedArray | number[] for option 2/3/4.

image

image

Other unresolved question: the issue mentions enums, which makes sense since (if I recall) booleans are stored as enums in HDF5. However, we do treat them separately in H5Web. Perhaps that wasn't such a good idea? 🤷

@axelboc axelboc requested a review from loichuder September 25, 2023 14:20
Copy link
Member

@loichuder loichuder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, this PR looks quite all right to me. Sure, it may not seem the prettiest since you need to propagate typing changes down the line, but there is no way around it. The alternatives you listed would have been even more verbose in terms of typings.

Perhaps you can introduce a NumericLikeType to go along with the new guards/assertions ?

@loichuder
Copy link
Member

Also, once there is a real usecase with enums, perhaps we will find a better way to harmonize this. But in the mean time, this is a quite sensible implementation.

@axelboc
Copy link
Contributor Author

axelboc commented Oct 3, 2023

I might just investigate option 3/4 a bit more, because with the current solution, when using H5WasmProvider, the data is retrieved as an array of numbers, converted to an array of booleans, and then back to an array of numbers. Seems like a bit of a waste.

@axelboc axelboc force-pushed the true-false branch 5 times, most recently from b1f7d48 to 12c99a1 Compare October 6, 2023 14:30
@axelboc
Copy link
Contributor Author

axelboc commented Oct 6, 2023

Option 3/4 wasn't as relevant after I realised in #1503 that h5wasm returns booleans, not integers. It does do an integer-to-boolean conversion internally (just like h5py in h5grove, I guess), but I don't think it's worth modifying h5wasm to get this feature through. I'm also reluctant to add more processing work to the providers (and therefore more implicit coupling with the visualizations).

In the end, I went with option 1.5... I convert the value array from boolean to number, but I do so inside MappedLineVis and MappedHeatmapVis, via a memoised utility function.

I also introduce a NumericLikeType as discussed.

.gitattributes Outdated Show resolved Hide resolved
packages/app/src/vis-packs/core/utils.ts Show resolved Hide resolved
return arr.map((val) => (val ? 1 : 0));
}

return arr as NumArray;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scratched my head trying to remove this as, but two problems make this difficult:

  • The value array is separated from the dataset object that describes the type, so narrowing the type of the dataset object does not help narrow the type of the value array...
  • ArrayValue<NumericType | BooleanType> resolves to (number | boolean)[] | TypedArray. I actually found a way to get number[] | boolean[] | TypedArray but it broke a bunch of code related to compound datasets ... and it makes sense for compound value arrays to have type (number | string | ...)[] | TypedArray).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I have the feeling there is always a way to remove as but sometimes it is not worth the complexity.

@@ -30,7 +30,7 @@ interface Props {
aspect?: Aspect;
showGrid?: boolean;
title?: string;
dtype?: NumericType;
dtype?: NumericLikeType;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not a fan of @h5web/lib knowing about NumericLikeType (or any other DType). I'm thinking I might change this to string and move the call to formatNumLikeType into the mapped components at some point.

@axelboc axelboc marked this pull request as ready for review October 6, 2023 14:54
@axelboc axelboc requested a review from loichuder October 6, 2023 14:55
@axelboc
Copy link
Contributor Author

axelboc commented Oct 9, 2023

/approve

Copy link
Member

@loichuder loichuder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well done 🥳

return arr.map((val) => (val ? 1 : 0));
}

return arr as NumArray;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I have the feeling there is always a way to remove as but sometimes it is not worth the complexity.

@axelboc
Copy link
Contributor Author

axelboc commented Oct 9, 2023

/approve

Co-authored-by: axelboc <axelboc@users.noreply.github.com>
@axelboc axelboc merged commit 4dec62d into main Oct 9, 2023
@axelboc axelboc deleted the true-false branch October 9, 2023 08:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants