Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Data Validation #770

Closed
wants to merge 30 commits into from
Closed

Data Validation #770

wants to merge 30 commits into from

Conversation

greole
Copy link
Collaborator

@greole greole commented May 20, 2021 •
edited
Loading

This PR implements data validation strategies as outlined in #746. The aim is to provide a set of validation functions, to evaluate the validity of matrices for debug builds.

The implementation is done in several stages:

  • Free functions checking:

    • If a given matrix is symmetric.
    • If a given matrix has a non-zero diagonal.
    • If given indices are ordered.
    • If given indices are within given bounds.
    • If given indices are unique.
    • If given values are finite.
    • If differences in indices are below given value.
    • If matrix is triangular.

  • Matrices:

    • Coo: Values are finite, bounded indices (row,column), row pointers are ordered.
    • Csr: Values are finite, indices are within bounds, row_ptr are sorted and unique.
    • Ell: Values are finite, column index is within bounds
    • Diagonal: Values are finite.
    • Dense: Values are finite.
    • Fbcsr: Values are finite, indices are within bounds, row_ptr are sorted and unique.
    • Hybrid: The underlying Coo and Ell matrix are valid.
    • Permutation: indices are unique wrt the permutation.
    • Sellp: Slice offsets are non-descending, slice lengths are consistent with slice offsets.

  • Factorizations:

    • Check if output is valid and has non-zero diagonal
    • IC: check if diagonal entries are >0 and if input matrix is symmetric

  • Preconditioners:

    • Jacobi: The block pointers need to be ascending with gaps smaller than max_block_size
    • Jacobi: The blocks must be invertible/inverted blocks must contain finite values
    • Ic: The system matrix must be symmetric
    • Ilu:
    • Isai: The output must be a valid Csr matrix

  • Solvers:

    • All solvers: system matrix is valid
    • Iterative solvers: preconditioner is valid
    • CG: system matrix is symmetric.
    • Upper/LowerTrs: system matrix needs to be triangular with non-zero diagonal entries.

greole added 13 commits May 8, 2021 11:17
@greole
add file stubs
c5e6805
@greole
Add basic unit tests for validation_helpers
8051f51
@greole
Add implementation of is_row_ordered check
4803049
@greole
add implementation of symmetry validation
1715e0f
@greole
Move macro definition to function scope
9871253
@greole
Add symmetry check for Csr, Coo, and Ell matrices
ffd1fcc
@greole
Add is_finite implementation and tests
b74323f
@greole
Add has_non_zero_diagonal implementation
24d3367
@greole
Add unit tests for has_non_zero_diagonal
69aef9d
@greole
Add is_within_bounds implementation and unit tests
f45f47f
@greole
This commit reworks macro defintions and usage
d44edde 
- reuse CALL_AND_RETURN_IF_CASTABLE macro
- generalise GKO_CALL_FOR_EACH_(NON_COMPLEX)_VALUE_AND_INDEX_TYPE
@greole
Data validation macro generalisation (#1)
36d2357 
* add prefixed macro definition

* Data validation macro generalisation fix windows (#2)

* check if linebreak causes windows build to break

* check if linebreak causes windows build to break

* check if empty macro arg causes windows build to break

* test if removing VA_ARGS resolves msvc error

* test PASS_ON hack

* Add missing PASS_ON

* Add tests for complex values is_finite tests

* Refactor macro expansion macro

* Test if PASS_ON is needed when calling GKO_APPLY_MACRO

* Refactor CALL_AND_RETURN_IF_CASTABLE macro

* Moves GKO_CALL_FOR_... macros to types.hpp
@greole
Add has_unique_idxs implementation and test
a69a538
@greole greole added is:enhancement An improvement of an existing feature. mod:core This is related to the core module. type:solver This is related to the solvers type:preconditioner This is related to the preconditioners type:matrix-format This is related to the Matrix formats 1:ST:WIP This PR is a work in progress. Not ready for review. type:factorization This is related to the Factorizations labels May 20, 2021
@greole greole self-assigned this May 20, 2021
greole added 2 commits May 21, 2021 08:46
@greole
Add is_consecutive implementation and test
277b816
@greole
This commit implements the is_*_triangular helpers
486f552 
- includes unit tests for is_*_triangular
- unit tests for matrix validation function are testing for Coo and Csr
- some minor fixes
@upsj upsj added this to the Ginkgo 1.4.0 milestone May 22, 2021
@greole
This commit includes several changes:
a042982 
1. Unit tests are simplified and are only based on Coo, because the
validation helpers rely on matrix_data and conversion to the specific
matrix data type is tested elsewhere.

2. The predicate for all_of has been corrected by inverting it.

3. The non_zero_diagonal now also tests for missing elements.
@greole greole linked an issue May 25, 2021 that may be closed by this pull request
Data validation and invariants #746
Open
@upsj upsj removed this from the Ginkgo 1.4.0 milestone May 31, 2021
@upsj upsj mentioned this pull request Oct 22, 2021
Merged
@greole greole added 1:ST:ready-to-merge This PR is ready to merge. 1:ST:need-feedback The PR is somewhat ready but feedback on a blocking topic is required before a proper review. 1:ST:WIP This PR is a work in progress. Not ready for review. and removed 1:ST:WIP This PR is a work in progress. Not ready for review. 1:ST:ready-to-merge This PR is ready to merge. 1:ST:need-feedback The PR is somewhat ready but feedback on a blocking topic is required before a proper review. labels Nov 2, 2021
@upsj upsj mentioned this pull request Nov 16, 2021
Merged
9 tasks
@upsj upsj mentioned this pull request Feb 22, 2022
Merged
1 task
@upsj upsj mentioned this pull request Jul 15, 2022
Closed
@MarcelKoch MarcelKoch mentioned this pull request Aug 18, 2022
Open
@tcojean tcojean removed this from the Ginkgo 1.5.0 milestone Oct 20, 2022
@upsj upsj mentioned this pull request Jan 9, 2023
Open
@upsj upsj mentioned this pull request Oct 18, 2024
Draft
2 tasks
@MarcelKoch MarcelKoch added is:help-wanted Need ideas on how to solve this. is:good-first-issue Good for newcomers. labels Feb 13, 2025
@upsj
Copy link
Member

upsj commented Feb 20, 2025

The PR is currently based on a fork and rather outdated, I will start from scratch in a new branch and PR.

@upsj upsj closed this Feb 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@greole greole

Labels
1:ST:WIP This PR is a work in progress. Not ready for review. is:enhancement An improvement of an existing feature. is:good-first-issue Good for newcomers. is:help-wanted Need ideas on how to solve this. mod:core This is related to the core module. type:factorization This is related to the Factorizations type:matrix-format This is related to the Matrix formats type:preconditioner This is related to the preconditioners type:solver This is related to the solvers
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Data validation and invariants
4 participants
@greole @upsj @MarcelKoch @tcojean