ginkgo own ILU and IC #1684

yhmtsai · 2024-09-26T09:22:44Z

This pr enables the ginkgo's implementation ILU through the current LU implementation.
To enable that, we need to allow LU can ignore the missing fill-in entries.
It is controlled by checked_lookup parameter, and user needs to pass the same sparsityCsr as the original one as the symbolic factor to get ILU.
Moreover, this pr also adds a parameter for algorithm selection. It will forward the information to LU and unpack them directly to return the same structure as the original sparselib.
Note. LU by default gives combined matrix but ILU gives the composition.

The first commit shows that the hashmap lookup leads the hang issue and bitmap lookup leads illegal memory access (incorrect result with -1 index return).
The second commit shows the results are wrong if we do not ignore missing fill-in entries update after fixing infinite loop of hashmap lookup

After this pr, we provide another way to factorize ILU without relying on the sparse library.

TODO

cholesky (in this pr, too)

nbeams

I have some questions and comments, but overall I am excited about this! Thanks @yhmtsai

common/cuda_hip/factorization/lu_kernels.cpp

nbeams · 2024-09-26T18:42:26Z

core/factorization/ilu.cpp

+                    local_system_matrix->get_const_row_ptrs())));
+        ilu =
+            gko::experimental::factorization::Lu<ValueType, IndexType>::build()
+                .with_checked_lookup(true)


Same here, would prefer with_safe_lookup or something like that, personally.

IMO this should rather be a new type, LU and ILU(...) have very different use cases. That would also allow us to build a new, improved ILU preconditioner interface based on the Factorization type instead of the slightly hacky Composition-like type ILU is currently.

core/factorization/ilu.cpp

nbeams · 2024-09-26T19:18:08Z

include/ginkgo/core/factorization/ilu.hpp

+ * syncfree is Ginkgo's implementation through the Lu factorization with given
+ * sparsity.
+ */
+enum class factorize_algorithm { sparselib, syncfree };


Since this is specific to incomplete factorizations only (at least for now), would it be better to use ilu_algorithm or something like that? Since it's only in the factorization namespace, nothing specific about ILU there.

Also, nit: cuda and hip aren't capitalized here, but they are in the comment for the factory parameter. I think we should be consistent: either Cuda/Hip (like the Executors) or CUDA/HIP (correct names for the actual software).

I will also use it to IC, so it will be for incomplete factorize algorithm at least.
maybe incomplete_factorize_algorithm if the original one is too general?

That makes sense. I'm mostly thinking about code readability/logicality on the user side. I do think it would be good to specify that this is only for incomplete somehow, though incomplete_factorize_algorithm is pretty long. What if we had the same enum for both ilu and ic but with different names (ilu_algorithm and ic_algorithm)? But we can also go with the longer name. Maybe @upsj has thoughts on this?

include/ginkgo/core/factorization/lu.hpp

core/matrix/csr_lookup.hpp

upsj

Thanks Mike, I'm happy to see how simple this was to extend to ILU. Some design and naming suggestions I'd like to see discussed, and maybe we should build a roadmap for the future ILU preconditioner interface.

upsj · 2024-09-30T12:14:51Z

core/factorization/factorization.cpp

@@ -31,10 +31,25 @@ GKO_REGISTER_OPERATION(initialize_l, factorization::initialize_l);

 template <typename ValueType, typename IndexType>
 std::unique_ptr<Factorization<ValueType, IndexType>>
-Factorization<ValueType, IndexType>::unpack() const
+Factorization<ValueType, IndexType>::unpack(


I think we want to improve the strategy type with 2.0 (or earlier), and the strategies have no impact on triangular solvers, so I think we should avoid adding new strategy-related functionality.

I do not think 2.0 happens soon
if we do not want to introduce here, I can only unpack them by original code path not the unpack them.
@nbeams it should be fine, right? the unpacking codes are the same.

You mean revert back to before 57c0051? It should be fine, as far as I know. I tested it before this commit, anyway.

common/cuda_hip/factorization/lu_kernels.cpp

nbeams

A few more minor comments, but most of my questions/issues have been addressed.

nbeams · 2024-10-29T21:21:07Z

include/ginkgo/core/factorization/lu.hpp

+         * false in order to avoid the possibility of hanging or illegal memory
+         * accesses during the factorization process. When this is true, the
+         * symbolic factorization must still contain the non-zero locations in
+         * the original matrix, at minimum.
+         */


Comment needs another update after the logic switch -- now "When this is true, the symbolic factorization must still contain the non-zero locations in the original matrix, at minimum" sounds like it's saying this is a requirement when the parameter is true, while it's the opposite. Maybe we can make it even more explicit, like "When not using all fill-in, ..."

the later part is wrong. it tries to describe the information when the setting is true.
I have updated and reorganize it.

nbeams · 2024-10-29T21:25:42Z

core/factorization/lu.cpp

@@ -69,6 +69,9 @@ Lu<ValueType, IndexType>::parse(const config::pnode& config,
    if (auto& obj = config.get("skip_sorting")) {
        params.with_skip_sorting(config::get_value<bool>(obj));
    }
+    if (auto& obj = config.get("has_all_fillin")) {
+        params.with_has_all_fillin(config::get_value<bool>(obj));


I don't like has_all_fillin because of this -- that we end up with with_has_all_fillin which sounds very clunky to me. What about full_fillin as the param name so we get with_full_fillin?

sounds good to me.
Do you think it will be a bit misleading for the case we were discussing, the factorization does not contains all values from the matrices?

In what scenario are you envisioning it could be misleading? I.e., what use case? If the factorization does not contain all values from the original matrix, then full_fillin would be false, as it should be, no?

Also, based on the updated comment in lu.hpp (the above coversation), a factorization that doesn't contain all the locations from the original matrix currently isn't allowed, anyway, right?

right. The current version can not work with the factorization misses some entries from the original matrix. I just wonder whether should consider the situation when the factorization does not contains the nonzeros from the original matrix. To do so, we also need to adapt it in the initialize function. When we allow thissituation, we definitely do not want to introduce another option to enable it.

Oh, I see -- just looking to the future. I don't think it's a problem, personally -- I think "fill-in" is pretty well-understood to mean "all the additional locations of the factorization vs original matrix" so I don't think there'd be any confusion that it's meant to differentiate between incomplete factorization with or without all the original matrix locations.

- bitmap: lead wrong answer or segfault - hashmap: infinite loop

…pposite behavior). Co-authored-by: Tobias Ribizel <ribizel@kit.edu> Co-authored-by: Natalie Beams <246972+nbeams@users.noreply.github.com>

…rategy" to avoid new strategy input

upsj · 2024-11-06T20:23:31Z

include/ginkgo/core/factorization/cholesky.hpp

+
+        /**
+         * If the user provides the symbolic factorization, it should contain
+         * the fill-in for the matrix. i.e., When this is true, the symbolic
+         * factorization must contain the non-zero locations in the original
+         * matrix and the corresponding fill-in locations during factorization.
+         * If it does not have full fill-in, as in Ilu, this parameter must be
+         * set to false in order to avoid the possibility of hanging or illegal
+         * memory accesses during the factorization process.
+         */
+        bool GKO_FACTORY_PARAMETER_SCALAR(full_fillin, true);


Why this parameter? This goes against the name LU, and could as well be handled by ILU, if we hand in the sparsity pattern there as a new parameter.

does it go against LU? I try to avoid copying half of functions from lu to ilu.
We do not need the sparsity pattern in ilu currently because we use the same sparsity of the matrix

In its current form they use the same algorithm, but adding this parameter prevents future evolution taking advantage of structural properties of an LU that an ILU doesn't have.

could you elaborate more?
@nbeams I assume you use the ILU on your side now not the LU interface, right?
I will copy some function into ILU.
It should still work with ILU-only interface for the case we discussed above in the future

have copied

upsj · 2024-11-06T20:24:00Z

include/ginkgo/core/factorization/lu.hpp

+
+        /**
+         * If the user provides the symbolic factorization, it should contain
+         * the fill-in for the matrix. i.e., When this is true, the symbolic
+         * factorization must contain the non-zero locations in the original
+         * matrix and the corresponding fill-in locations during factorization.
+         * If it does not have full fill-in, as in Ilu, this parameter must be
+         * set to false in order to avoid the possibility of hanging or illegal
+         * memory accesses during the factorization process. Also, the symbolic
+         * factorization still needs to contain all entries from the original
+         * matrix.
+         */
+        bool GKO_FACTORY_PARAMETER_SCALAR(full_fillin, true);


Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

yhmtsai added the 1:ST:ready-for-review This PR is ready for review label Sep 26, 2024

yhmtsai requested review from nbeams, upsj and a team September 26, 2024 09:22

yhmtsai self-assigned this Sep 26, 2024

ginkgo-bot added reg:build This is related to the build system. reg:testing This is related to testing. type:matrix-format This is related to the Matrix formats type:factorization This is related to the Factorizations mod:all This touches all Ginkgo modules. labels Sep 26, 2024

yhmtsai marked this pull request as ready for review September 26, 2024 09:23

nbeams reviewed Sep 26, 2024

View reviewed changes

upsj requested changes Sep 26, 2024

View reviewed changes

upsj reviewed Sep 30, 2024

View reviewed changes

MarcelKoch reviewed Oct 1, 2024

View reviewed changes

common/cuda_hip/factorization/lu_kernels.cpp Outdated Show resolved Hide resolved

yhmtsai force-pushed the gko_ilu branch from abac7cc to a97eb6a Compare October 7, 2024 20:14

MarcelKoch added this to the Ginkgo 1.9.0 milestone Oct 17, 2024

yhmtsai force-pushed the gko_ilu branch from a97eb6a to 452150d Compare October 29, 2024 09:25

yhmtsai requested review from nbeams, upsj and MarcelKoch October 29, 2024 09:25

nbeams reviewed Oct 29, 2024

View reviewed changes

yhmtsai and others added 8 commits November 6, 2024 15:57

add failed test when given symbolic without fillin.

d176cd1

- bitmap: lead wrong answer or segfault - hashmap: infinite loop

fix infinite loop of lookup_hash_unsafe and add test in reference

cde4bbb

add checked_lookup into LU

90e9f06

add ilu syncfree through lu implementation

db70838

update the documentation, change checked_lookup -> has_full_fillin (o…

670f3ae

…pposite behavior). Co-authored-by: Tobias Ribizel <ribizel@kit.edu> Co-authored-by: Natalie Beams <246972+nbeams@users.noreply.github.com>

use unpack directly from factorization and add unpack with strategy

2cb9365

remove the duplicated initialization

8add2d6

Revert "use unpack directly from factorization and add unpack with st…

ec33e55

…rategy" to avoid new strategy input

yhmtsai added 2 commits November 6, 2024 15:57

update documentation

3e5c076

cholesky failed test

4eb5165

yhmtsai force-pushed the gko_ilu branch from b3f3eb0 to 4eb5165 Compare November 6, 2024 14:57

yhmtsai added 2 commits November 6, 2024 16:06

move the algorithm enum to another header

0b1c40f

cholesky with safe lookup, wrap it into Ic, and add some missing tests

1eed743

upsj reviewed Nov 6, 2024

View reviewed changes

yhmtsai changed the title ~~ginkgo ILU~~ ginkgo own ILU and IC Nov 7, 2024

copy the call from lu/cholesky to ilu/ic and delete full_fillin

2468247

Co-authored-by: Tobias Ribizel <ribizel@kit.edu>

yhmtsai requested review from upsj and nbeams November 12, 2024 18:49

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ginkgo own ILU and IC #1684

ginkgo own ILU and IC #1684

yhmtsai commented Sep 26, 2024 •

edited

Loading

nbeams left a comment

nbeams Sep 26, 2024

upsj Sep 26, 2024

nbeams Sep 26, 2024

yhmtsai Sep 30, 2024

nbeams Sep 30, 2024

upsj left a comment

upsj Sep 30, 2024

yhmtsai Oct 7, 2024

nbeams Oct 8, 2024

yhmtsai Oct 14, 2024

nbeams left a comment

nbeams Oct 29, 2024

yhmtsai Nov 6, 2024

nbeams Oct 29, 2024

yhmtsai Nov 6, 2024

nbeams Nov 6, 2024

nbeams Nov 6, 2024 •

edited

Loading

yhmtsai Nov 6, 2024

nbeams Nov 6, 2024

upsj Nov 6, 2024

yhmtsai Nov 7, 2024

upsj Nov 7, 2024

yhmtsai Nov 7, 2024

yhmtsai Nov 12, 2024

upsj Nov 6, 2024

ginkgo own ILU and IC #1684

Are you sure you want to change the base?

ginkgo own ILU and IC #1684

Conversation

yhmtsai commented Sep 26, 2024 • edited Loading

nbeams left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

upsj left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbeams left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nbeams Nov 6, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

yhmtsai commented Sep 26, 2024 •

edited

Loading

nbeams Nov 6, 2024 •

edited

Loading