Add chebyshev Iteration #1289

yhmtsai · 2023-02-22T16:12:44Z

It adds the Chebshev iteration in https://en.wikipedia.org/wiki/Chebyshev_iteration
The second-order richardson uses a similar formula but the scalars are constant in all iteration. Chebyshev Iteration update the scalar from the previous one (the scalar are the same if upper/lower eigval does not change)

MarcelKoch · 2023-03-14T13:55:36Z

Do you intend adding an eigenvalue estimation? I think that would be very helpful, because most times users don't have that available. I think PETSc is also doing that.
I briefly searched for that and here are some references that might be interesting for that:

I think this is the GMRES version used by PETSC to compute the estimate: https://petsc.org/release/docs/manualpages/KSP/KSPAGMRES/

yhmtsai · 2023-03-20T11:30:07Z

@MarcelKoch thanks for providing these reference.

I do not think I will put them in this pull request.

From https://doi.org/10.1137/0907057, they introduce the algorithm
adaptive chebyshev iteration
Perform m-step GMRES and use the information from the Hessenberg matrix to get the estimation of eigenvalue (I may be wrong),
and then perform https://link.springer.com/article/10.1007/BF01389971 to get the parameter for chebchev
Repeat these two steps until converge
This will be more like a standalone solver not a smoother for multigrid because it will require several steps on GMRES, which is too expensive IMO.
We can introduce by with_adaptive(GMRES LinOpFactory)

From https://petsc.org/release/docs/manualpages/KSP/KSPCHEBYSHEV/#kspchebyshev,
More precisely, https://petsc.org/release/docs/manualpages/KSP/KSPChebyshevEstEigSet/
They use Lanczo or GMRES to get the maximum estimation to decide the parameter by following
min-eig-boundary = a * min-eigest + b * max-eigest = 0.1 max-eigest (default), and
max-eig-boundary = c * min-eigest + d * max-eigest = 1.1 max-eigest (default)
(because min_eigest is usually inaccurate)
By this, user can use the information by GMRES and generate min/max eig boundary for Chebyshev, so we do not need additional parameter?
It only does once in generation, so this is more usable for Multigrid

For GMRES, we need to get the Hessenberg out and compute eigenvalue on it.
GMRES eigenvalue and Lanczo are related to #135

sonarqubecloud · 2023-03-21T23:48:49Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
8 Code Smells

44.3% Coverage
7.1% Duplication

MarcelKoch

Besides the comments left below, I want to mention the num_keep and num_generated mechanic. That needs some explaination, because right now I don't understand what that is for. It seems like some sort of restart mechanic, but I might be wrong there. Also, exposing this to the user seems confusing to me.

MarcelKoch · 2023-04-06T07:06:22Z

include/ginkgo/core/solver/chebyshev.hpp

+        /**
+         * The number of scalar to keep
+         */
+        int GKO_FACTORY_PARAMETER_SCALAR(num_keep, 2);


It is not clear what this parameter is for.

Perhaps this means, construct Chebyshev polynomials until degree num_keep, and after that just do IR with these polynomial?

no, num_keep is to keep the generated scalar in the storage.
alpha and beta are changed by iteration, but they are fixed by the given bound.
It's used for fixed iteration runs because we do not need to refill these scalars for different apply

But shouldn't we then just keep all scalars?

when using it as normal solver, we may not have the maximum iteration information.
allocating one big dense matrix and then moving them to another one when full may not be a good approach.
we can also add the ability of workspace such that it can handle the std::vector then it should be more flexible for uncertain size.

I can not assume that there's an iteration criterion. It can contain the residual norm or time criterion.
If we assume that, we should provide the standalone iteration parameter.
Users do not need to know the implementation details.
The statement is to keep how many scalars for chebyshev to avoid the refill overhead.
I can introduce this usage later when we have the use case.
I think it should help the situation when kernel launch overhead is noticeable.

You are right, this is very useful to negate the overhead. Still, I think we can assume that there is an iteration criterion somewhere in the combined one and use that. If not, we would just throw.

You could set the default value to unspecified, which you define before. When the solver is generated, you can check if the parameter is unspecified and then try to extract an iteration criterion from the criteria. If that doesn't work, you throw with a message to either pass an iteration criterion or set this parameter to something else.
Or alternatively, just replace the criteria parameter with a iterations parameter and move the class into the preconditioner namespace. I think that is also a fine option, since that would be the main use case anyway.

I have moved it to based on the given iteration from stopping criterion.
It is only increased after creating object.
I also think whether staying a fixed number is enough or not.

Thanks, I think it should be fine the way it is now.

MarcelKoch · 2023-04-06T07:09:33Z

include/ginkgo/core/solver/chebyshev.hpp

+/**
+ * Chebyshev iteration is an iterative method that uses another coarse
+ * method to approximate the error of the current solution via the current
+ * residual. It has another term for the difference of solution. Moreover, this


Suggested change

* residual. It has another term for the difference of solution. Moreover, this

* residual. The solution is then updated using the Chebyshev polynomials. Moreover, this

Is that what the sentence is trying to say? Anyway, I think it should be mentioned somewhere that this uses these polynomials.

the solution x_i is also based on the $x_{i-1} - x_{i-2}$

TBH I don't see the relevance of that. Has that any effect for the user?

no, I only try to explain the algorithm's difference from IR. IR uses $x += \alpha M^{-1}r$, but chebyshev uses $x += \alpha_i M^{-1} r + \beta_i (x_{i-1} - x_{i-2})$ $(x_{i-1} - x_{i-2})$ is the additional term against IR

I still think this has to be rephrased

I try to rephrase it again. Could you take a look?

include/ginkgo/core/solver/chebyshev.hpp

core/solver/chebyshev.cpp

include/ginkgo/core/solver/chebyshev.hpp

reference/test/solver/chebyshev_kernels.cpp

core/test/solver/chebyshev.cpp

vasilisge0

I am fine with merging this PR when corrections in parts of the documentation that are pointed out (comments) take place. I could understand what num_keep variable was used for but perhaps a more descriptive name would be better.

sonarqubecloud · 2023-08-08T22:32:36Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
21 Code Smells

42.6% Coverage
4.4% Duplication

The version of Java (11.0.3) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

MarcelKoch

LGTM, but I would like to resolve some of my earlier issues. Other than that only minor nits.

core/solver/chebyshev.cpp

MarcelKoch · 2023-10-13T10:38:19Z

include/ginkgo/core/solver/chebyshev.hpp

+/**
+ * Chebyshev iteration is an iterative method that uses another coarse
+ * method to approximate the error of the current solution via the current
+ * residual. It has another term for the difference of solution. Moreover, this


I still think this has to be rephrased

include/ginkgo/core/solver/chebyshev.hpp

core/test/solver/chebyshev.cpp

reference/test/solver/chebyshev_kernels.cpp

sonarqubecloud · 2023-10-19T19:36:47Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
1 Security Hotspot
27 Code Smells

59.4% Coverage
5.2% Duplication

The version of Java (11.0.3) you have used to run this analysis is deprecated and we will stop accepting it soon. Please update to at least Java 17.
Read more here

include/ginkgo/core/solver/chebyshev.hpp

core/test/solver/chebyshev.cpp

ginkgo-bot · 2025-01-07T07:37:22Z

Error: The following files need to be formatted:

core/config/config_helper.hpp
core/config/registry.cpp
core/config/solver_config.cpp
core/solver/ir.cpp
core/solver/update_residual.hpp
core/test/config/solver.cpp
core/test/solver/chebyshev.cpp
test/solver/solver.cpp
test/test_install/test_install.cpp

You can find a formatting patch under Artifacts here or run format! if you have write access to Ginkgo

Co-authored-by: Marcel Koch <marcel.koch@kit.edu>

Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

yhmtsai · 2025-02-13T14:57:49Z

@upsj I have changed it to accept the host scalar now, so we do not need to store them in advanced.
I think we can take a look how much difference between passing by device pointer or host value in the future.
do you think it will change a lot in this case?

upsj

I mainly want to discuss the need for mixed precision instantiation here

common/unified/CMakeLists.txt

upsj · 2025-02-14T08:54:01Z

common/unified/solver/chebyshev_kernels.cpp

+namespace chebyshev {
+
+
+template <typename ValueType, typename ScalarType>


Does this actually need a second precision, or can we always use the highest precision available? This is memory-bound after all.

I have the same thought and thinking maybe we just use double or complex directly?

If there is a numerical benefit to using a higher-precision multiplication every time, I would go for it

after rethink about it, I do not have a good way to use double or complex double without any information from the vector or matrix. Maybe still keep the same way as how we handle IR/richardson now. also I need the ScalarType, which is the ValueType from Chebyshev class

currently keep the value type of class but change fosi, alpha, beta type to double/complex
when dpcpp only support float not double, it will cast to float/complex internally to keep the same interface among all backends

Co-authored-by: Tobias Ribizel <mail@ribizel.de>

yhmtsai added the 1:ST:WIP This PR is a work in progress. Not ready for review. label Feb 22, 2023

yhmtsai self-assigned this Feb 22, 2023

ginkgo-bot added reg:build This is related to the build system. reg:testing This is related to testing. mod:core This is related to the core module. mod:reference This is related to the reference module. type:solver This is related to the solvers labels Feb 22, 2023

yhmtsai force-pushed the cheb_iter branch from fa439a5 to 7d38ecb Compare March 21, 2023 12:34

yhmtsai added 1:ST:ready-for-review This PR is ready for review and removed 1:ST:WIP This PR is a work in progress. Not ready for review. labels Mar 21, 2023

MarcelKoch requested changes Apr 6, 2023

View reviewed changes

upsj requested a review from vasilisge0 April 6, 2023 15:51

vasilisge0 reviewed Apr 14, 2023

View reviewed changes

yhmtsai force-pushed the cheb_iter branch 3 times, most recently from c88ea63 to f9d0e28 Compare August 8, 2023 10:06

yhmtsai force-pushed the cheb_iter branch from 58530f8 to a7dcc2a Compare August 9, 2023 07:22

yhmtsai force-pushed the cheb_iter branch from a7dcc2a to edee59a Compare September 12, 2023 09:43

upsj self-requested a review September 18, 2023 12:27

yhmtsai force-pushed the cheb_iter branch from edee59a to 409a2e6 Compare October 4, 2023 15:01

yhmtsai requested review from MarcelKoch and vasilisge0 October 13, 2023 10:07

MarcelKoch requested changes Oct 13, 2023

View reviewed changes

yhmtsai force-pushed the cheb_iter branch from 409a2e6 to aeab966 Compare October 19, 2023 09:37

MarcelKoch reviewed Oct 20, 2023

View reviewed changes

include/ginkgo/core/solver/chebyshev.hpp Outdated Show resolved Hide resolved

include/ginkgo/core/solver/chebyshev.hpp Outdated Show resolved Hide resolved

core/test/solver/chebyshev.cpp Show resolved Hide resolved

yhmtsai force-pushed the cheb_iter branch 3 times, most recently from 3f588f5 to 77a3551 Compare January 7, 2025 07:36

yhmtsai and others added 17 commits February 13, 2025 15:54

init chebyshev

75f83c3

add the check for kept data and remove lend

53d075d

core test

2d05dc0

change to foci

1a289e3

use iteration from stop criterion and update doc

cb64b4c

extract the split residual update and update test

188236e

Co-authored-by: Marcel Koch <marcel.koch@kit.edu>

use preconditioner not inner_solver for chebyshev

2412d54

Co-authored-by: Marcel Koch <marcel.koch@kit.edu>

add chebyshev test and fix the number of generated

5a5439e

use scalar from ws and move some local variables

aa2e704

Co-authored-by: Marcel Koch <marcel.koch@kit.edu>

review update

e70647f

Co-authored-by: Marcel Koch <marcel.koch@kit.edu>

update with new format

494a1f9

update format

d0e9b35

add config for chebyshev

9309aea

use mixed_type in test properly and update documentation

03d1399

Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

add fused kernel for chebyshev

2e8e25b

pass alpha/beta by value to kernel and add test

61b6655

format update

385d784

yhmtsai force-pushed the cheb_iter branch from 77a3551 to 385d784 Compare February 13, 2025 14:55

yhmtsai requested a review from upsj February 13, 2025 14:57

MarcelKoch approved these changes Feb 13, 2025

View reviewed changes

upsj requested changes Feb 14, 2025

View reviewed changes

yhmtsai force-pushed the cheb_iter branch from 7e9d2bd to f8737a8 Compare February 14, 2025 14:57

use double/complex<double> for fosi

37d6ecc

Co-authored-by: Tobias Ribizel <mail@ribizel.de>

yhmtsai force-pushed the cheb_iter branch from f8737a8 to 37d6ecc Compare February 14, 2025 14:58

yhmtsai requested a review from upsj February 14, 2025 14:58

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add chebyshev Iteration #1289

Add chebyshev Iteration #1289

yhmtsai commented Feb 22, 2023

MarcelKoch commented Mar 14, 2023 •

edited

Loading

yhmtsai commented Mar 20, 2023 •

edited

Loading

sonarqubecloud bot commented Mar 21, 2023

MarcelKoch left a comment

MarcelKoch Apr 6, 2023

MarcelKoch Apr 6, 2023

yhmtsai Apr 6, 2023

MarcelKoch Apr 6, 2023

yhmtsai Apr 6, 2023

yhmtsai Apr 6, 2023

MarcelKoch Apr 6, 2023

MarcelKoch May 17, 2023

yhmtsai Aug 4, 2023

MarcelKoch Aug 4, 2023

MarcelKoch Apr 6, 2023

yhmtsai Aug 4, 2023

MarcelKoch Aug 4, 2023

yhmtsai Aug 7, 2023 •

edited

Loading

MarcelKoch Oct 13, 2023

yhmtsai Oct 19, 2023

vasilisge0 left a comment

sonarqubecloud bot commented Aug 8, 2023

MarcelKoch left a comment

MarcelKoch Oct 13, 2023

sonarqubecloud bot commented Oct 19, 2023

ginkgo-bot commented Jan 7, 2025

yhmtsai commented Feb 13, 2025

upsj left a comment

upsj Feb 14, 2025

yhmtsai Feb 14, 2025

upsj Feb 14, 2025

yhmtsai Feb 14, 2025

yhmtsai Feb 14, 2025

	* residual. It has another term for the difference of solution. Moreover, this
	* residual. The solution is then updated using the Chebyshev polynomials. Moreover, this

		namespace chebyshev {


		template <typename ValueType, typename ScalarType>

Add chebyshev Iteration #1289

Are you sure you want to change the base?

Add chebyshev Iteration #1289

Conversation

yhmtsai commented Feb 22, 2023

MarcelKoch commented Mar 14, 2023 • edited Loading

yhmtsai commented Mar 20, 2023 • edited Loading

sonarqubecloud bot commented Mar 21, 2023

MarcelKoch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yhmtsai Aug 7, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vasilisge0 left a comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Aug 8, 2023

MarcelKoch left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sonarqubecloud bot commented Oct 19, 2023

ginkgo-bot commented Jan 7, 2025

yhmtsai commented Feb 13, 2025

upsj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MarcelKoch commented Mar 14, 2023 •

edited

Loading

yhmtsai commented Mar 20, 2023 •

edited

Loading

yhmtsai Aug 7, 2023 •

edited

Loading