Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tutorial data preparation #103

Merged
merged 4 commits into from
Jan 10, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions docs/guides/config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Config
Throughout the ParaDigMa toolbox we use configuration objects `Config` to specificy parameters used as input of processes. All configuration classes care defined in `config.py`, and can be imported using `from paradigma.config import X`. The configuration classes frequently use static column names defined in `constants.py` to ensure robustness and consistency.

## Config classes
The config classes are defined either for sensors (IMU, PPG) or for domains (gait, heart rate, tremor).

### Sensor configs
There are two sensor config classes: `IMUConfig` and `PPGConfig`.

### Domain configs
For the latter category, config classes can further be distinguished by the processing steps displayed [here](https://github.com/biomarkersParkinson/paradigma).
18 changes: 18 additions & 0 deletions docs/guides/coordinate_system.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
# Coordinate System
As a prerequisite to reliably estimating gait and arm swing measures, it is important to align the coordinate system of the IMU sensor with the coordinate system of the trained classifiers. For the tremor and heart rate pipelines, differences between coordinate systems do not affect outcomes.

## Coordinate system used
The coordinate system of the IMU sensor used for training the classifiers can be observed below. The direction of acceleration is indicated by arrows, and the direction of gyroscope rotation can be determined using [Ampère's right-hand grip rule](https://en.wikipedia.org/wiki/Right-hand_rule#Amp%C3%A8re's_right-hand_grip_rule) applied to the accelerometer axes.

<p align="center">
<img src="../source/_static/img/directions_axes.png" />
</p>

### Accelerometer
The three accelerometer axes are set such that the x-axis is aligned with the arm pointing toward the hand, the y-axis is perpendicular to the arm pointing upward from the top of the sensor, and the z-axis points away from the arm and body. If the arrow representing a specific axis is pointing downward to the ground, parallel to and in the direction of the arrow representing gravity (1G), the acceleration of this specific axis is equal to -1g if the sensor is stable (i.e., no acceleration due to movement).

### Gyroscope
If the sensor is rotating in the direction of the arrow (deducted using the [Ampère's right-hand grip rule](https://en.wikipedia.org/wiki/Right-hand_rule#Amp%C3%A8re's_right-hand_grip_rule) applied to the accelerometer axes), the gyroscope data will be positive.

## Lateral differences
Wearing the watch on the left wrist or right wrist influences the relation between movements and the coordinate system. In fact, the x-axis of the accelerometer, and the y-axis and z-axis of the gyroscope, are inverted. For this purpose, we have added `invert_watch_side` to the toolbox (which can be imported using `from paradigma.util import invert_watch_side`). First, ensure the coordinate system aligns with the coordinate system shown above. Do this for all participants wearing the watch on one specific side, for example the left wrist. Then, apply `invert_watch_side` to ensure individuals wearing the watch on the right wrist have the correct coordinate system accounting for differences in sensor orientation.
24 changes: 14 additions & 10 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,34 @@
```{include} ../README.md
```{toctree}
:maxdepth: 2
:caption: Tutorials
tutorials/data_preparation.ipynb
```

```{toctree}
:maxdepth: 2
:caption: Example notebooks
:caption: Template Notebooks
notebooks/gait/gait_analysis.ipynb
notebooks/ppg/ppg_analysis.ipynb
```

```{toctree}
:maxdepth: 2
:caption: User Guides
guides/coordinate_system.md
```

```{toctree}
:maxdepth: 2
:caption: API
autoapi/index
```

```{toctree}
<!-- ```{toctree}
:maxdepth: 2
:caption: TSDF schema
tsdf_paradigma_schemas.md
tsdf_paradigma_channels_and_units.md
```
``` -->

```{toctree}
:maxdepth: 2
:caption: Development
changelog.md
contributing.md
conduct.md
```
:maxdepth: 2
Binary file added docs/source/_static/img/directions_axes.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 12 additions & 4 deletions docs/tutorials/data_preparation.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@
"metadata": {},
"source": [
"## Load data\n",
"This example uses data of the Personalized Parkinson Project, which is stored in Time Series Data Format (TSDF). IMU and PPG data are sampled at a different sampling frequency and hence stored separately. "
"This example uses data of the Personalized Parkinson Project, which is stored in Time Series Data Format (TSDF). IMU and PPG data are sampled at a different sampling frequency and therefore stored separately. "
]
},
{
Expand Down Expand Up @@ -54,6 +54,13 @@
"df_ppg.head(5)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The timestamps in this dataset correspond to delta milliseconds, and the data is not uniformly distributed as can be observed."
]
},
{
"cell_type": "markdown",
"metadata": {},
Expand Down Expand Up @@ -117,7 +124,8 @@
"metadata": {},
"source": [
"#### Set sensor values to the correct units\n",
"First, TSDF stores the data efficiently using scaling factors. We should therefore convert the sensor values back to the true values. "
"TSDF stores the data efficiently using scaling factors. We should therefore convert the sensor values back to the true values. This is only relevant if you use TSDF and scaled the data for storage purposes.\n",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this only relevant if you use TSDF and scaled the data for storage purposes or also when you don't use tsdf but have scaled the data for storage purposes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's also relevant when using other data formats, but generally data formats such as parquet have inherent data compression, meaning that the scaling happens when storing/loading. Therefore I don't expect users to require the scaling. Do you think we should remove this from the tutorial, and perhaps scale it prior to loading the data such that it doesn't distract the user?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not per se I think but I was just wondering if other formats have something like scale factors that you need to apply just like we do with TSDF. But if they all do this like Parquet then we can leave it this way.

"\n"
]
},
{
Expand Down Expand Up @@ -171,7 +179,7 @@
"metadata": {},
"source": [
"#### Account for watch side\n",
"For the Gait & Arm Swing pipeline, it is essential to ensure correct sensor axes orientation. For more information please read [X]. If the sensors are not correctly aligned, you can use `invert_watch_side` to ensure consistency between sensors worn on the left or right wrist."
"For the Gait & Arm Swing pipeline, it is essential to ensure correct sensor axes orientation. For more information please read [Coordinate System](../guides/coordinate_system.md). If the sensors are not correctly aligned, you can use `invert_watch_side` to ensure consistency between sensors worn on the left or right wrist."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

invert_watch_side does not necessarily correctly align the sensors right? It only does when one side is already correctly aligned?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You're absolutely correct, I'll adjust it accordingly.

]
},
{
Expand All @@ -198,7 +206,7 @@
"metadata": {},
"source": [
"#### Change time column\n",
"ParaDigMa expects the data to be in seconds relative to the first row. The toolbox has the built-in function `transform_time_array` to help users transform their time column to the correct format."
"ParaDigMa expects the data to be in seconds relative to the first row, which should be equal to 0. The toolbox has the built-in function `transform_time_array` to help users transform their time column to the correct format."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

first row or first data point? The first is technically correct but I think the latter is more intuitive?

]
},
{
Expand Down
Loading