Add a fixed coarsening class usable in multigrid and multi-level methods #986

pratikvn · 2022-03-12T08:29:28Z

This PR adds a coarsening class, which allows generation of coarse matrices from user-defined coarsening arrays. The user needs to provide an array with the coarse indices (global indices).

Features

Generation is very cheap compared to other multigrid methods, because the restriction column indices are the coarse indices provided.
Additionally, the coarse matrix itself can be generated without SpGEMM by just creating them as submatrices from the provided coarse indices. This will be added in a future PR after the submatrix from IndexSet feature is complete on all executors.

TODO

~~Discussion on expectation for users when providing sorted or un-sorted inputs for coarse_row vector.~~ We always sort the user provided input. If the user needs to reorder the matrix, an explicit reordering shows the intent better and is better suited.

codecov · 2022-03-12T10:58:48Z

Codecov Report

Merging #986 (3386664) into develop (6df4a68) will decrease coverage by 1.23%.
The diff coverage is 100.00%.

@@             Coverage Diff             @@
##           develop     #986      +/-   ##
===========================================
- Coverage    93.44%   92.21%   -1.24%     
===========================================
  Files          479      484       +5     
  Lines        40210    40614     +404     
===========================================
- Hits         37576    37451     -125     
- Misses        2634     3163     +529

Impacted Files	Coverage Δ
core/multigrid/fixed_coarsening.cpp	`100.00% <100.00%> (ø)`
core/test/multigrid/fixed_coarsening.cpp	`100.00% <100.00%> (ø)`
include/ginkgo/core/multigrid/fixed_coarsening.hpp	`100.00% <100.00%> (ø)`
...erence/test/multigrid/fixed_coarsening_kernels.cpp	`100.00% <100.00%> (ø)`
test/multigrid/fixed_coarsening_kernels.cpp	`100.00% <100.00%> (ø)`
test/matrix/matrix.cpp	`0.00% <0.00%> (-96.02%)`	⬇️
common/unified/matrix/sellp_kernels.cpp	`8.95% <0.00%> (-87.60%)`	⬇️
common/unified/matrix/ell_kernels.cpp	`28.57% <0.00%> (-71.43%)`	⬇️
common/unified/matrix/csr_kernels.cpp	`32.55% <0.00%> (-67.45%)`	⬇️
common/unified/matrix/hybrid_kernels.cpp	`35.71% <0.00%> (-64.29%)`	⬇️
... and 18 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6df4a68...3386664. Read the comment docs.

upsj

LGTM! Nice job building on top of what is already there :)

upsj · 2022-03-17T15:00:09Z

include/ginkgo/core/multigrid/fixed_coarsening.hpp

+    void apply_impl(const LinOp* b, LinOp* x) const override
+    {
+        this->get_composition()->apply(b, x);
+    }
+
+    void apply_impl(const LinOp* alpha, const LinOp* b, const LinOp* beta,
+                    LinOp* x) const override
+    {
+        this->get_composition()->apply(alpha, b, beta, x);
+    }
+
+    explicit FixedCoarsening(std::shared_ptr<const Executor> exec)
+        : EnableLinOp<FixedCoarsening>(std::move(exec))
+    {}
+
+    explicit FixedCoarsening(const Factory* factory,
+                             std::shared_ptr<const LinOp> system_matrix)
+        : EnableLinOp<FixedCoarsening>(factory->get_executor(),
+                                       system_matrix->get_size()),
+          EnableMultigridLevel<ValueType>(system_matrix),
+          parameters_{factory->get_parameters()},
+          system_matrix_{system_matrix}
+    {
+        if (system_matrix_->get_size()[0] != 0) {
+            // generate on the existing matrix
+            this->generate();
+        }
+    }


these can be moved to the source file, since they don't have any template parameters

That is true. But do we want to duplicate the declarations here and in the source file ? I guess there is a case to be made to keep the implementations in the source files to reduce the compilation time, but is there another reason ?

test/multigrid/fixed_coarsening_kernels.cpp

include/ginkgo/core/multigrid/fixed_coarsening.hpp

MarcelKoch · 2022-03-22T16:19:37Z

Just a small note, you say that your implementation would be simpler, if you could use submatrix from index set. I think it would help in that case if you set the target branch of this PR to the submatrix from index set one. That way you can already use that here and won't need a second PR or update this one when the submatrix PR is merged.

upsj · 2022-03-22T17:37:04Z

core/multigrid/fixed_coarsening.cpp

+
+    auto prolong_op = gko::as<csr_type>(share(restrict_op->transpose()));
+
+    // TODO: Can be done with submatrix index_set.


Looks like my initial comment was swallowed by ~~the cookie monster~~ Github's servers. I don't think that is true in the general setting, since index_set has no ordering. The current implementation seems fine to me, does it need to be changed?

Yes, that is one thing I wanted to discuss. Do we want the possibility to have implicit re-ordering when specifying the coarse row indices ?

pratikvn · 2022-03-23T15:05:07Z

@MarcelKoch, I thought it would be better to merge this into first and then work on the submatrix device kernels, as #964 does not yet have device kernels for submatrix creation.

Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

fritzgoebel

LGTM! One thing: I think the user currently needs to specify an index set for the coarse rows and constant jumps are actually not supported, so if I didn't miss something here you could adjust the doc accordingly.

include/ginkgo/core/multigrid/fixed_coarsening.hpp

reference/test/multigrid/fixed_coarsening_kernels.cpp

Co-authored-by: Fritz Göbel <fritz.goebel@kit.edu>

ginkgo-bot · 2022-04-04T12:37:09Z

Note: This PR changes the Ginkgo ABI:

Functions changes summary: 0 Removed, 0 Changed, 928 Added functions
Variables changes summary: 0 Removed, 0 Changed, 0 Added variable

For details check the full ABI diff under Artifacts here

sonarqubecloud · 2022-04-04T18:58:29Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
3 Security Hotspots
30 Code Smells

90.7% Coverage
2.5% Duplication

Advertise release 1.5.0 and last changes + Add changelog, + Update third party libraries + A small fix to a CMake file See PR: #1195 The Ginkgo team is proud to announce the new Ginkgo minor release 1.5.0. This release brings many important new features such as: - MPI-based multi-node support for all matrix formats and most solvers; - full DPC++/SYCL support, - functionality and interface for GPU-resident sparse direct solvers, - an interface for wrapping solvers with scaling and reordering applied, - a new algebraic Multigrid solver/preconditioner, - improved mixed-precision support, - support for device matrix assembly, and much more. If you face an issue, please first check our [known issues page](https://github.com/ginkgo-project/ginkgo/wiki/Known-Issues) and the [open issues list](https://github.com/ginkgo-project/ginkgo/issues) and if you do not find a solution, feel free to [open a new issue](https://github.com/ginkgo-project/ginkgo/issues/new/choose) or ask a question using the [github discussions](https://github.com/ginkgo-project/ginkgo/discussions). Supported systems and requirements: + For all platforms, CMake 3.13+ + C++14 compliant compiler + Linux and macOS + GCC: 5.5+ + clang: 3.9+ + Intel compiler: 2018+ + Apple LLVM: 8.0+ + NVHPC: 22.7+ + Cray Compiler: 14.0.1+ + CUDA module: CUDA 9.2+ or NVHPC 22.7+ + HIP module: ROCm 4.0+ + DPC++ module: Intel OneAPI 2021.3 with oneMKL and oneDPL. Set the CXX compiler to `dpcpp`. + Windows + MinGW and Cygwin: GCC 5.5+ + Microsoft Visual Studio: VS 2019 + CUDA module: CUDA 9.2+, Microsoft Visual Studio + OpenMP module: MinGW or Cygwin. Algorithm and important feature additions: + Add MPI-based multi-node for all matrix formats and solvers (except GMRES and IDR). ([#676](#676), [#908](#908), [#909](#909), [#932](#932), [#951](#951), [#961](#961), [#971](#971), [#976](#976), [#985](#985), [#1007](#1007), [#1030](#1030), [#1054](#1054), [#1100](#1100), [#1148](#1148)) + Porting the remaining algorithms (preconditioners like ISAI, Jacobi, Multigrid, ParILU(T) and ParIC(T)) to DPC++/SYCL, update to SYCL 2020, and improve support and performance ([#896](#896), [#924](#924), [#928](#928), [#929](#929), [#933](#933), [#943](#943), [#960](#960), [#1057](#1057), [#1110](#1110), [#1142](#1142)) + Add a Sparse Direct interface supporting GPU-resident numerical LU factorization, symbolic Cholesky factorization, improved triangular solvers, and more ([#957](#957), [#1058](#1058), [#1072](#1072), [#1082](#1082)) + Add a ScaleReordered interface that can wrap solvers and automatically apply reorderings and scalings ([#1059](#1059)) + Add a Multigrid solver and improve the aggregation based PGM coarsening scheme ([#542](#542), [#913](#913), [#980](#980), [#982](#982), [#986](#986)) + Add infrastructure for unified, lambda-based, backend agnostic, kernels and utilize it for some simple kernels ([#833](#833), [#910](#910), [#926](#926)) + Merge different CUDA, HIP, DPC++ and OpenMP tests under a common interface ([#904](#904), [#973](#973), [#1044](#1044), [#1117](#1117)) + Add a device_matrix_data type for device-side matrix assembly ([#886](#886), [#963](#963), [#965](#965)) + Add support for mixed real/complex BLAS operations ([#864](#864)) + Add a FFT LinOp for all but DPC++/SYCL ([#701](#701)) + Add FBCSR support for NVIDIA and AMD GPUs and CPUs with OpenMP ([#775](#775)) + Add CSR scaling ([#848](#848)) + Add array::const_view and equivalent to create constant matrices from non-const data ([#890](#890)) + Add a RowGatherer LinOp supporting mixed precision to gather dense matrix rows ([#901](#901)) + Add mixed precision SparsityCsr SpMV support ([#970](#970)) + Allow creating CSR submatrix including from (possibly discontinuous) index sets ([#885](#885), [#964](#964)) + Add a scaled identity addition (M <- aI + bM) feature interface and impls for Csr and Dense ([#942](#942)) Deprecations and important changes: + Deprecate AmgxPgm in favor of the new Pgm name. ([#1149](#1149)). + Deprecate specialized residual norm classes in favor of a common `ResidualNorm` class ([#1101](#1101)) + Deprecate CamelCase non-polymorphic types in favor of snake_case versions (like array, machine_topology, uninitialized_array, index_set) ([#1031](#1031), [#1052](#1052)) + Bug fix: restrict gko::share to rvalue references (*possible interface break*) ([#1020](#1020)) + Bug fix: when using cuSPARSE's triangular solvers, specifying the factory parameter `num_rhs` is now required when solving for more than one right-hand side, otherwise an exception is thrown ([#1184](#1184)). + Drop official support for old CUDA < 9.2 ([#887](#887)) Improved performance additions: + Reuse tmp storage in reductions in solvers and add a mutable workspace to all solvers ([#1013](#1013), [#1028](#1028)) + Add HIP unsafe atomic option for AMD ([#1091](#1091)) + Prefer vendor implementations for Dense dot, conj_dot and norm2 when available ([#967](#967)). + Tuned OpenMP SellP, COO, and ELL SpMV kernels for a small number of RHS ([#809](#809)) Fixes: + Fix various compilation warnings ([#1076](#1076), [#1183](#1183), [#1189](#1189)) + Fix issues with hwloc-related tests ([#1074](#1074)) + Fix include headers for GCC 12 ([#1071](#1071)) + Fix for simple-solver-logging example ([#1066](#1066)) + Fix for potential memory leak in Logger ([#1056](#1056)) + Fix logging of mixin classes ([#1037](#1037)) + Improve value semantics for LinOp types, like moved-from state in cross-executor copy/clones ([#753](#753)) + Fix some matrix SpMV and conversion corner cases ([#905](#905), [#978](#978)) + Fix uninitialized data ([#958](#958)) + Fix CUDA version requirement for cusparseSpSM ([#953](#953)) + Fix several issues within bash-script ([#1016](#1016)) + Fixes for `NVHPC` compiler support ([#1194](#1194)) Other additions: + Simplify and properly name GMRES kernels ([#861](#861)) + Improve pkg-config support for non-CMake libraries ([#923](#923), [#1109](#1109)) + Improve gdb pretty printer ([#987](#987), [#1114](#1114)) + Add a logger highlighting inefficient allocation and copy patterns ([#1035](#1035)) + Improved and optimized test random matrix generation ([#954](#954), [#1032](#1032)) + Better CSR strategy defaults ([#969](#969)) + Add `move_from` to `PolymorphicObject` ([#997](#997)) + Remove unnecessary device_guard usage ([#956](#956)) + Improvements to the generic accessor for mixed-precision ([#727](#727)) + Add a naive lower triangular solver implementation for CUDA ([#764](#764)) + Add support for int64 indices from CUDA 11 onward with SpMV and SpGEMM ([#897](#897)) + Add a L1 norm implementation ([#900](#900)) + Add reduce_add for arrays ([#831](#831)) + Add utility to simplify Dense View creation from an existing Dense vector ([#1136](#1136)). + Add a custom transpose implementation for Fbcsr and Csr transpose for unsupported vendor types ([#1123](#1123)) + Make IDR random initilization deterministic ([#1116](#1116)) + Move the algorithm choice for triangular solvers from Csr::strategy_type to a factory parameter ([#1088](#1088)) + Update CUDA archCoresPerSM ([#1175](#1116)) + Add kernels for Csr sparsity pattern lookup ([#994](#994)) + Differentiate between structural and numerical zeros in Ell/Sellp ([#1027](#1027)) + Add a binary IO format for matrix data ([#984](#984)) + Add a tuple zip_iterator implementation ([#966](#966)) + Simplify kernel stubs and declarations ([#888](#888)) + Simplify GKO_REGISTER_OPERATION with lambdas ([#859](#859)) + Simplify copy to device in tests and examples ([#863](#863)) + More verbose output to array assertions ([#858](#858)) + Allow parallel compilation for Jacobi kernels ([#871](#871)) + Change clang-format pointer alignment to left ([#872](#872)) + Various improvements and fixes to the benchmarking framework ([#750](#750), [#759](#759), [#870](#870), [#911](#911), [#1033](#1033), [#1137](#1137)) + Various documentation improvements ([#892](#892), [#921](#921), [#950](#950), [#977](#977), [#1021](#1021), [#1068](#1068), [#1069](#1069), [#1080](#1080), [#1081](#1081), [#1108](#1108), [#1153](#1153), [#1154](#1154)) + Various CI improvements ([#868](#868), [#874](#874), [#884](#884), [#889](#889), [#899](#899), [#903](#903), [#922](#922), [#925](#925), [#930](#930), [#936](#936), [#937](#937), [#958](#958), [#882](#882), [#1011](#1011), [#1015](#1015), [#989](#989), [#1039](#1039), [#1042](#1042), [#1067](#1067), [#1073](#1073), [#1075](#1075), [#1083](#1083), [#1084](#1084), [#1085](#1085), [#1139](#1139), [#1178](#1178), [#1187](#1187))

pratikvn added is:new-feature A request or implementation of a feature that does not exist yet. 1:ST:ready-for-review This PR is ready for review type:multigrid This is related to multigrid labels Mar 12, 2022

pratikvn requested a review from a team March 12, 2022 08:29

pratikvn self-assigned this Mar 12, 2022

ginkgo-bot added mod:core This is related to the core module. mod:reference This is related to the reference module. reg:build This is related to the build system. reg:testing This is related to testing. labels Mar 12, 2022

pratikvn force-pushed the fixed-coarsening branch from 0b41ea2 to 26e5dbe Compare March 12, 2022 08:33

pratikvn mentioned this pull request Mar 12, 2022

Add a uniform coarsening algorithm for coarse grid generation. #979

Closed

2 tasks

upsj self-requested a review March 17, 2022 11:10

upsj approved these changes Mar 17, 2022

View reviewed changes

upsj reviewed Mar 22, 2022

View reviewed changes

pratikvn force-pushed the fixed-coarsening branch from d64e351 to 0de0ccc Compare March 23, 2022 15:02

pratikvn and others added 5 commits April 4, 2022 11:04

Add FixedCoarsening class and core

ffcb8f7

Add reference kernels and tests

32bea54

Add common kernels and tests

960c57a

Simplify core code and remove kernels.

de6e23c

Review update

7d2fa04

Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

pratikvn force-pushed the fixed-coarsening branch from 0de0ccc to 7d2fa04 Compare April 4, 2022 09:04

fritzgoebel approved these changes Apr 4, 2022

View reviewed changes

include/ginkgo/core/multigrid/fixed_coarsening.hpp Outdated Show resolved Hide resolved

reference/test/multigrid/fixed_coarsening_kernels.cpp Outdated Show resolved Hide resolved

Review update.

3386664

Co-authored-by: Fritz Göbel <fritz.goebel@kit.edu>

pratikvn added 1:ST:ready-to-merge This PR is ready to merge. 1:ST:run-full-test and removed 1:ST:ready-for-review This PR is ready for review labels Apr 4, 2022

pratikvn merged commit d096921 into develop Apr 4, 2022

pratikvn deleted the fixed-coarsening branch April 4, 2022 19:34

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a fixed coarsening class usable in multigrid and multi-level methods #986

Add a fixed coarsening class usable in multigrid and multi-level methods #986

pratikvn commented Mar 12, 2022 •

edited

Loading

codecov bot commented Mar 12, 2022 •

edited

Loading

upsj left a comment

upsj Mar 17, 2022

pratikvn Mar 23, 2022

MarcelKoch commented Mar 22, 2022

upsj Mar 22, 2022

pratikvn Mar 23, 2022

pratikvn commented Mar 23, 2022

fritzgoebel left a comment

ginkgo-bot commented Apr 4, 2022

sonarqubecloud bot commented Apr 4, 2022


		auto prolong_op = gko::as<csr_type>(share(restrict_op->transpose()));

		// TODO: Can be done with submatrix index_set.

Add a fixed coarsening class usable in multigrid and multi-level methods #986

Add a fixed coarsening class usable in multigrid and multi-level methods #986

Conversation

pratikvn commented Mar 12, 2022 • edited Loading

Features

TODO

codecov bot commented Mar 12, 2022 • edited Loading

Codecov Report

upsj left a comment

Choose a reason for hiding this comment

upsj Mar 17, 2022

Choose a reason for hiding this comment

pratikvn Mar 23, 2022

Choose a reason for hiding this comment

MarcelKoch commented Mar 22, 2022

upsj Mar 22, 2022

Choose a reason for hiding this comment

pratikvn Mar 23, 2022

Choose a reason for hiding this comment

pratikvn commented Mar 23, 2022

fritzgoebel left a comment

Choose a reason for hiding this comment

ginkgo-bot commented Apr 4, 2022

sonarqubecloud bot commented Apr 4, 2022

pratikvn commented Mar 12, 2022 •

edited

Loading

codecov bot commented Mar 12, 2022 •

edited

Loading