Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Test Modules for Reading and Writing #29

Merged
merged 27 commits into from
Oct 26, 2021

Conversation

kursatyurt
Copy link
Collaborator

This PR adds some basic Ctest modules to check VTK outputs.

There are several tests included

1 - Check the capability of writing scalar data VTK files.
2 - Check the output of scalar data VTK file.
3-4-5-6 are variants of tests 1-2 for 2D vector and 3D vector data.
7 - 8 - 9 are reading tests they read some basic reference files and check reader values.

It also contains some bug fixes in mesh.cpp related to VTK library.

@davidscn
Copy link
Member

davidscn commented Oct 12, 2021

Nice, thank you the tests work on my machine.
However, I get plenty of these warnings when building the prgram

mesh.cpp:115:50: warning: narrowing conversion of ‘cell->vtkCell::GetPointId(0)’ from ‘vtkIdType’ {aka ‘long long int’} to ‘long unsigned int’ [-Wnarrowing]
  115 |       std::array<size_t, 3> elem{cell->GetPointId(0), cell->GetPointId(1), cell->GetPointId(2)};

I guess the arrays you use in several places with size_t need to be vtkIdType

@fsimonis
Copy link
Member

Writing your own test executables doesn't scale, so let's use a testing framework instead.
As ASTE already relies on Boost.Container, we could use Boost.Test for the testing.

Our baseline is boost 1.65.1 (Ubuntu 18.04 LTS). Here are some resources:

  • Documentation Overview
  • Let's use the shared library version. You can #define BOOST_TEST_DYN_LINK using CMake for the entire executable. The testing main should be in a stand-alone file:
    #define BOOST_TEST_MODULE ASTE
    #define BOOST_TEST_DYN_LINK
    #include <boost/test/unit_test.hpp>
  • You should only have to link to Boost::unit_test_framework and Boost::boost (boost container) using CMake.

@davidscn
Copy link
Member

Writing your own test executables doesn't scale, so let's use a testing framework instead.
As ASTE already relies on Boost.Container, we could use Boost.Test for the testing

Scale to what? I don't expect a massive test framework within this small project. We should keep the effort for a port to boost tests in mind. Do you really think it is worth the effort? IMHO, some basic tests as added here should be perfectly fine. Also, upcoming ASTE features like the replay mode and an ASTE tutorial are in my opinion more important then a boost test framework.

@fsimonis
Copy link
Member

Scale to what?

Tests need to be maintained and it is easier to maintain tests written using a well known framework than having to understand a custom framework first.
We also use boost.test in preCICE, so core developers are already familiar with the basics.
Instead of reinventing the wheel, let's reuse the one we already know.

I don't expect a massive test framework within this small project.

Neither do I and relying on an existing framework keeps the tests short, readable and maintainable.

Also, upcoming ASTE features like the replay mode and an ASTE tutorial are in my opinion more important then a boost test framework.

I agree. Migrating to Boost.test is not a lot of work, so let's just do it.

All of this really applies to the logging framework too, but this migration will be more work as we already use logging in many places..

@kursatyurt
Copy link
Collaborator Author

mesh.cpp:115:50: warning: narrowing conversion of ‘cell->vtkCell::GetPointId(0)’ from ‘vtkIdType’ {aka ‘long long int’} to ‘long unsigned int’ [-Wnarrowing]
  115 |       std::array<size_t, 3> elem{cell->GetPointId(0), cell->GetPointId(1), cell->GetPointId(2)};

I guess the arrays you use in several places with size_t need to be vtkIdType

It seems at some point we need a narrowing process. Since ASTE stores them as size_t I kept this convention. preCICE interface accepts int type whether they converted from long long int to unsigned int does not make any sense in the end right? Should I directly apply a static_cast here to convert int?

@davidscn
Copy link
Member

mesh.cpp:115:50: warning: narrowing conversion of ‘cell->vtkCell::GetPointId(0)’ from ‘vtkIdType’ {aka ‘long long int’} to ‘long unsigned int’ [-Wnarrowing]
  115 |       std::array<size_t, 3> elem{cell->GetPointId(0), cell->GetPointId(1), cell->GetPointId(2)};

I guess the arrays you use in several places with size_t need to be vtkIdType

It seems at some point we need a narrowing process. Since ASTE stores them as size_t I kept this convention. preCICE interface accepts int type whether they converted from long long int to unsigned int does not make any sense in the end right? Should I directly apply a static_cast here to convert int?

I see, this is actually a deficiency of preCICE. The problem is that the current state 'looks' a bit like an accident when the compiler warnings pop up. Please use static casts and add a comment in the code why you statically convert the type here.

Copy link
Member

@fsimonis fsimonis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to test the mesh_writing, I would propose to write the test in a way that it first writes a mesh, then reads the vtk and run checks on the points and data.
This also makes it work with both ASCII and Binary vtk files.

@fsimonis
Copy link
Member

I see, this is actually a deficiency of preCICE.

@davidscn There is actually an open issue regarding this precice/precice#887.

@kursatyurt kursatyurt requested a review from fsimonis October 15, 2021 21:36
@davidscn
Copy link
Member

@kursatyurt ynce you decided how to proceed in the discussion, we can update the main branch here and move on to the tutorial. If you do not add a GitHub action for this test, I will do later on.

@kursatyurt
Copy link
Collaborator Author

@kursatyurt once you decided how to proceed in the discussion, we can update the main branch here and move on to the tutorial. If you do not add a GitHub action for this test, I will do later on.

The clearest one is using int arrays instead of size_t arrays to keep it consistent with the preCICE interface. I pushed the changes.

I tried to add one action but it is not working. If you can add an action and close the request it would be nice.

@davidscn
Copy link
Member

I will add one in a later PR. Could you then as a last step add an assertion for the assumption of continuous indices as mentioned above?

@fsimonis
Copy link
Member

I propose a solution to this type juggling in #29 (comment)

@davidscn
Copy link
Member

I don't have enough to argue here, so I will let the discussion up to you.

@davidscn davidscn merged commit d43f1c2 into precice:migrate-vtk Oct 26, 2021
davidscn pushed a commit that referenced this pull request Nov 22, 2021
* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add mistakenly removed assertions

* Change CMake file for Boost Test

* Convert Read Test to A Boost Unit Test

* Add static_cast to PointId's

* Remove md5sum data

* Change CMake to single test file

* Change read/write test to void functions

* Add test to a main test file

* Ignore vscode directory

* Change from size_t arrays to int array to keep it consistent with preCICE interface

* Add new functionality to safely cast from vtkIDtype to preCICE

* Add VID to mesh

* Correct write test cases

* Adapt VID instead of int type arrays

* Add init and final time and corrrect timesteps
davidscn added a commit that referenced this pull request Nov 24, 2021
* Support for reading VTK file format (#26)

* Add VTK library to CMakeList

* Add support for specify data name and add support for reading from VTK files

* Add switch for vector data

* Add warning for VTK version to CMakelist

* Change empty dataname information output from clog to cout

* Change explanation for --vector flag

* Add runtime error if number of components and dimension does not match

* Corrected error message

* Change vector checking to more readable one

* Ensure mesh data vector is empty before fill it

* Throw runtime error if dimensions does not match

* Pass dataname as an argument for clear code

* Fix bad syntax and format document

* Revert pointers instead of smartpointers

* Support for Connectivity Information and VTK format Output (#27)

* Add header files required for VTK support

* Add support for reading connectivity

* Add support for VTK output inc. connectivity

* Make code more readable/clear

* Fix iteration bounds for triangles and edges

* Add quadrilateral support

* Add quad elements to mesh summary

* Add quad element edges to gather_unique_edges method

* Add vector support for preCICE interface (#28)

* Add vector support for preCICE interface

* Add asserttions for reading and writing blocks

* Test Modules for Reading and Writing (#29)

* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add mistakenly removed assertions

* Change CMake file for Boost Test

* Convert Read Test to A Boost Unit Test

* Add static_cast to PointId's

* Remove md5sum data

* Change CMake to single test file

* Change read/write test to void functions

* Add test to a main test file

* Ignore vscode directory

* Change from size_t arrays to int array to keep it consistent with preCICE interface

* Add new functionality to safely cast from vtkIDtype to preCICE

* Add VID to mesh

* Correct write test cases

* Adapt VID instead of int type arrays

* Add init and final time and corrrect timesteps

* Add a GitHub CI in order to test and build aste

* Make tests working from any directorty

* Bug Fix for Vector Data Write/Reads

* Fix reading parallel files

* Fix VTK Cmake configuration for VTK version > 8.90.0 (#43)

VTK version higher than 8.9 requires a different CMake configration

* Fix naming convention (#42)

* Obey VTK naming convention

* Fix naming

* Make output prettier, fix spaces

* Update VTK Calc (#39)

* Upgrade VTK Calculator as Standalone Script

* Add input datatag

* Update arguments

* Make intag argument optional

* Change coordinates variables names

* Support vtu and vtk export. Add guard for output meshname.

* Add relative l2 norm error to stats

* Instead of basename use full filename.

* Add additional log informations

* Change difference mode calculation

* Fix output extension bug

* Fix bug and remove abs

* Add explanation

* Clean unused/old scripts (#44)

* Fix Join Mesh (#41)

* Fix Join Mesh.

Reads `vtu` files and join into a `vtu` or `vtk` file.
If recovery file is given preserves original mesh order and cells else appends partition-wise

* Fix grammar for parser help

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix grammar for explaination

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix grammar for explanation

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Correct language of explanation

* Change help messages

* Add more info to help message

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix Mesh Partitioner  (#40)

* Fix Mesh Partitioner to working with VTK library

Process all PointData. Reduce Recovery file.

* Include size and cell types to recovery info

* Fix bug

* Fix partiton name

* Change Explanations

* Fix readeer types GenericDataObject reader has problem with XML files.

* Fix inserting new tuple to VTK array

* Fix preCICE installation in test

* Fix mesh input explanation

* Change from GenericReader to extension aware reader

* Fix mesh input explanation

* Change from GenericReader to extension aware reader

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>
davidscn added a commit that referenced this pull request Nov 24, 2021
* Support for reading VTK file format (#26)

* Add VTK library to CMakeList

* Add support for specify data name and add support for reading from VTK files

* Add switch for vector data

* Add warning for VTK version to CMakelist

* Change empty dataname information output from clog to cout

* Change explanation for --vector flag

* Add runtime error if number of components and dimension does not match

* Corrected error message

* Change vector checking to more readable one

* Ensure mesh data vector is empty before fill it

* Throw runtime error if dimensions does not match

* Pass dataname as an argument for clear code

* Fix bad syntax and format document

* Revert pointers instead of smartpointers

* Support for Connectivity Information and VTK format Output (#27)

* Add header files required for VTK support

* Add support for reading connectivity

* Add support for VTK output inc. connectivity

* Make code more readable/clear

* Fix iteration bounds for triangles and edges

* Add quadrilateral support

* Add quad elements to mesh summary

* Add quad element edges to gather_unique_edges method

* Add vector support for preCICE interface (#28)

* Add vector support for preCICE interface

* Add asserttions for reading and writing blocks

* Test Modules for Reading and Writing (#29)

* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add vector support for preCICE interface

* Remove old test files

* Add test executables and tests to CMake

* Collect VTK includes and Remove Duplicates

* Bug Fixes

* Add mistakenly removed assertions

* Change CMake file for Boost Test

* Convert Read Test to A Boost Unit Test

* Add static_cast to PointId's

* Remove md5sum data

* Change CMake to single test file

* Change read/write test to void functions

* Add test to a main test file

* Ignore vscode directory

* Change from size_t arrays to int array to keep it consistent with preCICE interface

* Add new functionality to safely cast from vtkIDtype to preCICE

* Add VID to mesh

* Correct write test cases

* Adapt VID instead of int type arrays

* Add init and final time and corrrect timesteps

* Add a GitHub CI in order to test and build aste

* Make tests working from any directorty

* Bug Fix for Vector Data Write/Reads

* Fix reading parallel files

* Fix VTK Cmake configuration for VTK version > 8.90.0 (#43)

VTK version higher than 8.9 requires a different CMake configration

* Fix naming convention (#42)

* Obey VTK naming convention

* Fix naming

* Make output prettier, fix spaces

* Update VTK Calc (#39)

* Upgrade VTK Calculator as Standalone Script

* Add input datatag

* Update arguments

* Make intag argument optional

* Change coordinates variables names

* Support vtu and vtk export. Add guard for output meshname.

* Add relative l2 norm error to stats

* Instead of basename use full filename.

* Add additional log informations

* Change difference mode calculation

* Fix output extension bug

* Fix bug and remove abs

* Add explanation

* Clean unused/old scripts (#44)

* Fix Join Mesh (#41)

* Fix Join Mesh.

Reads `vtu` files and join into a `vtu` or `vtk` file.
If recovery file is given preserves original mesh order and cells else appends partition-wise

* Fix grammar for parser help

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix grammar for explaination

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix grammar for explanation

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Correct language of explanation

* Change help messages

* Add more info to help message

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>

* Fix Mesh Partitioner  (#40)

* Fix Mesh Partitioner to working with VTK library

Process all PointData. Reduce Recovery file.

* Include size and cell types to recovery info

* Fix bug

* Fix partiton name

* Change Explanations

* Fix readeer types GenericDataObject reader has problem with XML files.

* Fix inserting new tuple to VTK array

* Fix preCICE installation in test

* Fix bug intag is not used if it is given

* Fix bug intag is not used if it is given

Co-authored-by: David Schneider <david.schneider@ipvs.uni-stuttgart.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants