-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Tutorial data preparation #103
Changes from 2 commits
5f1809e
1962ef8
d379f6d
d88d4e7
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
# Config | ||
Throughout the ParaDigMa toolbox we use configuration objects `Config` to specificy parameters used as input of processes. All configuration classes care defined in `config.py`, and can be imported using `from paradigma.config import X`. The configuration classes frequently use static column names defined in `constants.py` to ensure robustness and consistency. | ||
|
||
## Config classes | ||
The config classes are defined either for sensors (IMU, PPG) or for domains (gait, heart rate, tremor). | ||
|
||
### Sensor configs | ||
There are two sensor config classes: `IMUConfig` and `PPGConfig`. | ||
|
||
### Domain configs | ||
For the latter category, config classes can further be distinguished by the processing steps displayed [here](https://github.com/biomarkersParkinson/paradigma). |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,18 @@ | ||
# Coordinate System | ||
As a prerequisite to reliably estimating gait and arm swing measures, it is important to align the coordinate system of the IMU sensor with the coordinate system of the trained classifiers. For the tremor and heart rate pipelines, differences between coordinate systems do not affect outcomes. | ||
|
||
## Coordinate system used | ||
The coordinate system of the IMU sensor used for training the classifiers can be observed below. The direction of acceleration is indicated by arrows, and the direction of gyroscope rotation can be determined using [Ampère's right-hand grip rule](https://en.wikipedia.org/wiki/Right-hand_rule#Amp%C3%A8re's_right-hand_grip_rule) applied to the accelerometer axes. | ||
|
||
<p align="center"> | ||
<img src="../source/_static/img/directions_axes.png" /> | ||
</p> | ||
|
||
### Accelerometer | ||
The three accelerometer axes are set such that the x-axis is aligned with the arm pointing toward the hand, the y-axis is perpendicular to the arm pointing upward from the top of the sensor, and the z-axis points away from the arm and body. If the arrow representing a specific axis is pointing downward to the ground, parallel to and in the direction of the arrow representing gravity (1G), the acceleration of this specific axis is equal to -1g if the sensor is stable (i.e., no acceleration due to movement). | ||
|
||
### Gyroscope | ||
If the sensor is rotating in the direction of the arrow (deducted using the [Ampère's right-hand grip rule](https://en.wikipedia.org/wiki/Right-hand_rule#Amp%C3%A8re's_right-hand_grip_rule) applied to the accelerometer axes), the gyroscope data will be positive. | ||
|
||
## Lateral differences | ||
Wearing the watch on the left wrist or right wrist influences the relation between movements and the coordinate system. In fact, the x-axis of the accelerometer, and the y-axis and z-axis of the gyroscope, are inverted. For this purpose, we have added `invert_watch_side` to the toolbox (which can be imported using `from paradigma.util import invert_watch_side`). First, ensure the coordinate system aligns with the coordinate system shown above. Do this for all participants wearing the watch on one specific side, for example the left wrist. Then, apply `invert_watch_side` to ensure individuals wearing the watch on the right wrist have the correct coordinate system accounting for differences in sensor orientation. |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,30 +1,34 @@ | ||
```{include} ../README.md | ||
```{toctree} | ||
:maxdepth: 2 | ||
:caption: Tutorials | ||
tutorials/data_preparation.ipynb | ||
``` | ||
|
||
```{toctree} | ||
:maxdepth: 2 | ||
:caption: Example notebooks | ||
:caption: Template Notebooks | ||
notebooks/gait/gait_analysis.ipynb | ||
notebooks/ppg/ppg_analysis.ipynb | ||
``` | ||
|
||
```{toctree} | ||
:maxdepth: 2 | ||
:caption: User Guides | ||
guides/coordinate_system.md | ||
``` | ||
|
||
```{toctree} | ||
:maxdepth: 2 | ||
:caption: API | ||
autoapi/index | ||
``` | ||
|
||
```{toctree} | ||
<!-- ```{toctree} | ||
:maxdepth: 2 | ||
:caption: TSDF schema | ||
tsdf_paradigma_schemas.md | ||
tsdf_paradigma_channels_and_units.md | ||
``` | ||
``` --> | ||
|
||
```{toctree} | ||
:maxdepth: 2 | ||
:caption: Development | ||
changelog.md | ||
contributing.md | ||
conduct.md | ||
``` | ||
:maxdepth: 2 |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -13,7 +13,7 @@ | |
"metadata": {}, | ||
"source": [ | ||
"## Load data\n", | ||
"This example uses data of the Personalized Parkinson Project, which is stored in Time Series Data Format (TSDF). IMU and PPG data are sampled at a different sampling frequency and hence stored separately. " | ||
"This example uses data of the Personalized Parkinson Project, which is stored in Time Series Data Format (TSDF). IMU and PPG data are sampled at a different sampling frequency and therefore stored separately. " | ||
] | ||
}, | ||
{ | ||
|
@@ -54,6 +54,13 @@ | |
"df_ppg.head(5)" | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
"source": [ | ||
"The timestamps in this dataset correspond to delta milliseconds, and the data is not uniformly distributed as can be observed." | ||
] | ||
}, | ||
{ | ||
"cell_type": "markdown", | ||
"metadata": {}, | ||
|
@@ -117,7 +124,8 @@ | |
"metadata": {}, | ||
"source": [ | ||
"#### Set sensor values to the correct units\n", | ||
"First, TSDF stores the data efficiently using scaling factors. We should therefore convert the sensor values back to the true values. " | ||
"TSDF stores the data efficiently using scaling factors. We should therefore convert the sensor values back to the true values. This is only relevant if you use TSDF and scaled the data for storage purposes.\n", | ||
"\n" | ||
] | ||
}, | ||
{ | ||
|
@@ -171,7 +179,7 @@ | |
"metadata": {}, | ||
"source": [ | ||
"#### Account for watch side\n", | ||
"For the Gait & Arm Swing pipeline, it is essential to ensure correct sensor axes orientation. For more information please read [X]. If the sensors are not correctly aligned, you can use `invert_watch_side` to ensure consistency between sensors worn on the left or right wrist." | ||
"For the Gait & Arm Swing pipeline, it is essential to ensure correct sensor axes orientation. For more information please read [Coordinate System](../guides/coordinate_system.md). If the sensors are not correctly aligned, you can use `invert_watch_side` to ensure consistency between sensors worn on the left or right wrist." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. invert_watch_side does not necessarily correctly align the sensors right? It only does when one side is already correctly aligned? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. You're absolutely correct, I'll adjust it accordingly. |
||
] | ||
}, | ||
{ | ||
|
@@ -198,7 +206,7 @@ | |
"metadata": {}, | ||
"source": [ | ||
"#### Change time column\n", | ||
"ParaDigMa expects the data to be in seconds relative to the first row. The toolbox has the built-in function `transform_time_array` to help users transform their time column to the correct format." | ||
"ParaDigMa expects the data to be in seconds relative to the first row, which should be equal to 0. The toolbox has the built-in function `transform_time_array` to help users transform their time column to the correct format." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. first row or first data point? The first is technically correct but I think the latter is more intuitive? |
||
] | ||
}, | ||
{ | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this only relevant if you use TSDF and scaled the data for storage purposes or also when you don't use tsdf but have scaled the data for storage purposes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it's also relevant when using other data formats, but generally data formats such as
parquet
have inherent data compression, meaning that the scaling happens when storing/loading. Therefore I don't expect users to require the scaling. Do you think we should remove this from the tutorial, and perhaps scale it prior to loading the data such that it doesn't distract the user?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not per se I think but I was just wondering if other formats have something like scale factors that you need to apply just like we do with TSDF. But if they all do this like Parquet then we can leave it this way.