Skip to content

Commit

Permalink
Improved arepo parser (tardis-sn#1941)
Browse files Browse the repository at this point in the history 
* Removed line profile as option

* Removed mapping to Cartesian grid

* Removed automatic plotting after profile creation

* Fixed typo in arepo plot

* Updated documentation

* [build docs]

* [build docs] Removed outputs from doc-notebook

* Updated tests with new models

* Removed deprecated keyword in snapshot loader

* Replaced wget with requests for testfile download

* Updated file retrieval for testing

* Fixed fixture request

* Added arepo data to refdata helper

* Updated refdata location in docs

* Updated paths in tests

* Update download_reference_data.sh

* Testing areop_parser_tests

* Fixed deleted reference file
  • Loading branch information
@AlexHls @jayantbhakar
AlexHls authored and jayantbhakar committed Jul 29, 2022
1 parent 575478a commit bf03b14
Show file tree
Hide file tree
Showing 8 changed files with 142 additions and 531 deletions.
5 changes: 4 additions & 1 deletion .ci-helpers/download_reference_data.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,12 @@ FILES=('atom_data/kurucz_cd23_chianti_H_He.h5'
'montecarlo_1e5_compare_data.h5'
'packet_unittest.h5'
'sdec_ref.h5'
'unit_test_data.h5')
'unit_test_data.h5'
'arepo_data/arepo_snapshot.hdf5'
'arepo_data/arepo_snapshot.json')

mkdir -p $REF_PATH/atom_data
mkdir -p $REF_PATH/arepo_data

for FILE in "${FILES[@]}"
do
Expand Down
1 change: 0 additions & 1 deletion docs/io/configuration/components/models/converters/arepo_snapshot.json
View file

This file was deleted.

118 changes: 72 additions & 46 deletions docs/io/configuration/components/models/converters/arepo_to_tardis.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"id": "e3b3a256",
"metadata": {},
"source": [
"This parser is intended for loading Arepo output files ('snapshots'), extracting the relevant (line-of-sight dependent) data and exporting it to `csvy` files, which can in turn be used in TARDIS models ([see CSVY model](../index.rst#csvy-model))."
"This parser is intended for loading Arepo output files ('snapshots'), extracting the relevant (line-of-sight dependent) data and exporting it to `csvy` files, which can in turn be used in TARDIS models ([see CSVY model](../index.html#csvy-model))."
]
},
{
Expand Down Expand Up @@ -85,11 +85,28 @@
"In case you have the arepo-snap-util package installed, you can load the data directly from a snapshot:\n",
"```python\n",
"snapshot = arepo.ArepoSnapshot(\n",
" \"arepo_snapshot.hdf5\", [\"ni56\", \"si28\"], \"species55.txt\", resolution=32\n",
" \"arepo_snapshot.hdf5\", [\"ni56\", \"si28\"], \"species55.txt\"\n",
")\n",
"pos, vel, rho, xnuc, time = snapshot.get_grids()\n",
"```\n",
"This will load the necessary from the snapshot. See the [API](../../../../../api/tardis.io.parsers.arepo.rst) documentation for more options on how to load snapshots."
"This will load the necessary from the snapshot. See the [API](../../../../../api/tardis.io.parsers.arepo.html) documentation for more options on how to load snapshots."
]
},
{
"cell_type": "markdown",
"id": "52e061bf",
"metadata": {},
"source": [
"<div class=\"alert alert-info\">\n",
"\n",
"\n",
"\n",
"Note\n",
"\n",
"\n",
"The built in functionality only loads Arepo particle species `0`, i.e. gas particles. For most (if not all) purposes connected to TARDIS, gas particles should be the only species of interest. In case you need to load a different particle species, please manually load the data.\n",
"\n",
"</div>"
]
},
{
Expand Down Expand Up @@ -118,6 +135,16 @@
"</details>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "880b6e59",
"metadata": {},
"outputs": [],
"source": [
"!wget https://github.com/tardis-sn/tardis-refdata/raw/arepo-data/arepo_data/arepo_snapshot.json;"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down Expand Up @@ -159,7 +186,7 @@
"- Nuclear fraction (\"xnuc\") of each element you want to include -> `dict` containing `np.array` \n",
"- Time of the snapshot (\"time\") -> `float`\n",
"\n",
"The data is expected to be mapped to a Carthesian grid and should have the same shape as the one provided by the built-in tool."
"The data should have the same shape as the one provided by the built-in tool (see the given shapes above for an example)."
]
},
{
Expand All @@ -168,9 +195,8 @@
"metadata": {},
"source": [
"### Extracting a profile and converting it to a csvy file\n",
"Now You can create a TARDIS model. There are three possibilities on how to extract the profiles from the snapshot: \n",
"Now You can create a TARDIS model. There are two possibilities on how to extract the profiles from the snapshot: \n",
" \n",
"- **Line profile**: This extracts the data along a straight line (the x-axis) \n",
"- **Cone profile**: This extracts the data within a specified cone \n",
"- **Full profile**: This averages over the whole simulation "
]
Expand All @@ -192,7 +218,6 @@
"source": [
"This loads the data (in this example for a cone profile), which can then be cut to the ranges which you want to include in your TARDIS model. The syntax for the other profiles is similar: \n",
" \n",
"- `arepo.LineProfile(<args>)` \n",
"- `arepo.FullProfile(<args>)` "
]
},
Expand Down Expand Up @@ -231,14 +256,31 @@
"metadata": {},
"outputs": [],
"source": [
"profile.create_profile(opening_angle=40)"
"profile.create_profile(opening_angle=40)\n",
"profile.rebin(100)\n",
"profile.plot_profile()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "bfd37bad",
"metadata": {},
"source": [
"### Plotting the data\n",
"\n",
"Using `profile.plot_profile(save=<filename>, dpi=<dpi>)` you can create a plot of the exported profile. Due to the large number of cells usually used in Arepo simulations, it is recommended to plot the data only **after** rebinning the profiles or exporting them to a `csvy` file (which automatically rebins the data).\n",
"\n",
"### Manually rebinning the data\n",
"Using `profile.rebin(<nshells>, statistic=<statistic>)`, you can manually rebin the data (using [Scipys binned_statistic function](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_statistic.html)), using the mean value of the data in each bin. and use all `<statistic>` keywords accepted by the `scipy.stats.binned_statistic` function."
]
},
{
"cell_type": "markdown",
"id": "d619b5d0",
"metadata": {},
"source": [
"### Selecting a specific region\n",
"In many cases you only want a very specific region from the snapshot, e.g. cutting out the dense, optically thick regions. This can be acchieved using the keywords `inner_radius` and `outer_radius`."
]
},
Expand All @@ -249,15 +291,18 @@
"metadata": {},
"outputs": [],
"source": [
"profile.create_profile(opening_angle=40, inner_radius=1e11, outer_radius=2e11)"
"profile.create_profile(opening_angle=40, inner_radius=1e11, outer_radius=2e11)\n",
"profile.rebin(100)\n",
"profile.plot_profile()\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"id": "dd6310ec",
"metadata": {},
"source": [
"Once you have created a profile of the desired region, you can export the profile to a `csvy` using the commented-out code below, which in turn can be used in a TARDIS model. Here you have to specify how many shells you want to export. The profiles are rebinned using [Scipys binned_statistic function](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_statistic.html), using the mean value of the data in each bin."
"Once you have created a profile of the desired region, you can export the profile to a `csvy` file, which in turn can be used in a TARDIS model. Here you have to specify how many shells you want to export. The profiles are rebinned automatically using [Scipys binned_statistic function](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats.binned_statistic.html), using the mean value of the data in each bin."
]
},
{
Expand All @@ -267,23 +312,7 @@
"metadata": {},
"outputs": [],
"source": [
"profile.export(20, \"snapshot_converted_to_tardis.csvy\", overwrite=True)"
]
},
{
"cell_type": "markdown",
"id": "c478ccec",
"metadata": {
"raw_mimetype": "text/restructuredtext"
},
"source": [
"<div class=\"alert alert-info\">\n",
"\n",
"Note\n",
"\n",
"By default, the `export` method will not overwrite existing files with the same file name. Setting `overwrite=True` allows overwriting -- for example, if `overwrite=True`, if you make changes and rerun the export your exported file will be updated without creaing an additional file.\n",
"\n",
"</div>"
"profile.export(20, \"snapshot_converted_to_tardis.csvy\")"
]
},
{
Expand All @@ -298,25 +327,22 @@
},
{
"cell_type": "markdown",
"id": "5a5ec37b",
"metadata": {},
"source": [
"### Manually rebinning the data\n",
"Using `profile.rebin(<nshells>, statistic=<statistic>)`, you can manually rebin the data and use all `<statistic>` keywords accepted by the `scipy.stats.binned_statistic` function. In this case you should pass the `statistic=None` keyword to the `export` function, so the data does not get rebinned twice."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "15fd5925",
"metadata": {},
"outputs": [],
"id": "c478ccec",
"metadata": {
"raw_mimetype": "text/restructuredtext"
},
"source": [
"profile.create_profile(opening_angle=40, inner_radius=1e11, outer_radius=2e11, show_plot=False)\n",
"profile.rebin(20)\n",
"profile.plot_profile()\n",
"plt.show()\n",
"profile.export(20, \"rebinned_snapshot_converted_to_tardis.csvy\", overwrite=True)"
"<div class=\"alert alert-info\">\n",
"\n",
"\n",
"\n",
"Note\n",
"\n",
"\n",
"The export function will not overwrite existing files, but rather add an incrementing number to the filename.\n",
"\n",
"\n",
"</div>"
]
},
{
Expand All @@ -336,7 +362,7 @@
"\n",
"To export the same profile as in the example above you can run:\n",
"```bash\n",
"python ./<location_of_arepo_parser>/arepo.py snapshot.hdf5 snapshot_converted.csvy -o 40 -n 20 --inner_radius 1e11 --outer_radius 2e11 -e ni56 si28 --save_plot plot.png --resolution 32 --plot_rebinned plot_binned.png\n",
"python ./<location_of_arepo_parser>/arepo.py snapshot.hdf5 snapshot_converted.csvy -o 40 -n 20 --inner_radius 1e11 --outer_radius 2e11 -e ni56 si28 --plot plot.png\n",
"```\n",
"\n",
"This will also save diagnostic plots of both the raw and rebinned profiles. For more information on how to use the command line tool run:\n",
Expand Down
Loading

0 comments on commit bf03b14

Please sign in to comment.