ML keras #4172

bikagit · 2024-08-22T12:57:36Z

Integrating Keras capabilities to OPM.
Draft pull request to test/discuss implications of the changes in OPM.
This enables the straightforward and adaptable integration of neural networks into OPM scripts. These models are initially trained using the Keras library in Python, stored in a format readable for the OPM framework and subsequently deployed within OPM.
When the user initializes and loads a stored Keras model inside an OPM script, an automated deployment process handles all the translation. This process works by operating a series of steps handling model interpretation, layer conversion, optimization, and code generation steps to adapt the Keras model to a native OPM function.

totto82 · 2024-08-22T13:00:14Z

jenkins build this please

daavid00 · 2024-08-22T13:20:40Z

jenkins build this please

atgeirr · 2024-08-22T13:56:05Z

This does not add a dependency on a third party library, so then I assume it instead embeds Keras in some way? Or have I misunderstood the purpose of this PR?

bska · 2024-08-22T15:01:20Z

Is there a reason this is added to opm-common instead of being a separate repository? Do we, somehow, need to make (selected) objects in this repository, or any of its downstream repositories for that matter, "aware" of Keras?

bikagit · 2024-08-22T15:31:11Z

This does not add a dependency on a third party library, so then I assume it instead embeds Keras in some way? Or have I misunderstood the purpose of this PR?

We use Keras for the training process (it doesnt need to be done in OPM). The generated models are subsequently embedded and run in OPM. I have updated the description to provide some context.

totto82 · 2024-08-26T13:35:36Z

jenkins build this please

daavid00 · 2024-08-28T07:11:41Z

jenkins build this please

bska · 2024-08-28T10:53:10Z

Maybe I'm missing something, but as far as I can tell no-one have answered my question from last week

Is there a reason this is added to opm-common instead of being a separate repository? Do we, somehow, need to make (selected) objects in this repository, or any of its downstream repositories for that matter, "aware" of Keras?

I would really like an answer to this before I consider the details of the PR.

totto82 · 2024-08-28T12:42:05Z

I would really like an answer to this before I consider the details of the PR.

Sorry for not answering earlier. The idea is to apply the ML-Keras inside OPM for different tasks. This is only the first PR to add the Keras ML model. The applications will follow. For an example of a ML near well model using ML-Keras check out https://github.com/cssr-tools/ML_near_well. Since the ML-Keras model framework is general. We hope it would be useful for the OPM community and therefore suggest to add it to opm-common

bska · 2024-08-28T12:51:38Z

The idea is to apply the ML-Keras inside OPM for different tasks [...] Since the ML-Keras model framework is general, we hope it would be useful for the OPM community and therefore suggest to add it to opm-common

Okay, utility/convenience is clearly one reason for adding it here. Would it be impossible to make [your/certain use cases] work if it were located elsewhere? Do you, for instance, need access to the internals/private data members or member functions of Well or Connection objects or similar in your use cases?

totto82 · 2024-08-28T13:06:18Z

jenkins build this please

bikagit · 2024-09-01T05:45:28Z

The idea is to apply the ML-Keras inside OPM for different tasks [...] Since the ML-Keras model framework is general, we hope it would be useful for the OPM community and therefore suggest to add it to opm-common

Okay, utility/convenience is clearly one reason for adding it here. Would it be impossible to make [your/certain use cases] work if it were located elsewhere? Do you, for instance, need access to the internals/private data members or member functions of Well or Connection objects or similar in your use cases?

Exact! For instance, we need access to the automatic differentiation tools within OPM.

totto82 · 2024-09-06T13:11:49Z

jenkins build this please

atgeirr

There are a lot of changes needed here. I have only looked at the C++ code, and I probably missed some. I have not really checked that the activation functions or the layers do what a user of Keras would expect. I have not looked at any of the Python code, someone else must do that.

I have requested many changes, but I hope it provides a useful learning experience. Feel free to ask about anything that is unclear!

atgeirr · 2024-09-24T09:16:37Z

opm/ml/ml_tools/__init__.py

Is it necessary to add this empty file?

Yes.:

The __init__.py files are required to make Python treat directories containing the file as packages (unless using a namespace package, a relatively advanced feature). This prevents directories with a common name, such as string, from unintentionally hiding valid modules that occur later on the module search path. In the simplest case, __init__.py can just be an empty file, but it can also execute initialization code for the package or set the __all__ variable, described later.

While an empty file is fine, you might want to export some functions here, primarily the function export_model.

atgeirr · 2024-09-24T09:20:58Z

opm/ml/keras_model.hpp

+ Copyright (c) 2024 NORCE
+ This file is part of the Open Porous Media project (OPM).
+
+ OPM is free software: you can redistribute it and/or modify


As the MIT license is compatible with GPLv3, it is fine to include this and use it with OPM Flow. But I would like to clarify: what is the license of this file? Is it MIT or GPL?

If this file is mostly the work of Rose and Maevskikh, then it may be fair to make the entire file MIT licensed. However it is clearly allowed to license it under GPLv3 instead, especially if the NORCE contribution here is substantial.

atgeirr · 2024-09-24T09:22:52Z

opm/ml/keras_model.hpp

+
+#include <algorithm>
+#include <chrono>
+#include <math.h>


Prefer including <cmath> instead of the legacy C headers. (And adapt to functions being in std namespace.)

atgeirr · 2024-09-24T09:24:24Z

opm/ml/keras_model.hpp

+
+#define KASSERT(x, ...) \
+ if (!(x)) { \
+ printf("KASSERT: %s(%d): ", __FILE__, __LINE__); \


This requires you to include the requisite header. Prefer <cstdio>, and use std::printf.

atgeirr · 2024-09-24T09:27:51Z

opm/ml/keras_model.hpp

+#define KDEBUG(x, ...) ;
+#endif
+
+template<class Foo>


The template parameter name should give a clue about what concept it should follow. Foo is not meaningful. For completely generic parameters (i.e. the template can be instantiated with anything from int to MyReallyComplicatedClass) the usual name is T. If it is meant to be a scalar variable, Scalar is a good name. Please clarify and rename.

atgeirr · 2024-09-24T12:33:12Z

opm/ml/keras_model.cpp

+
+
+template<class Evaluation>
+bool KerasModel<Evaluation>::LoadModel(const std::string& filename) {


I assume this file format is defined by Keras? Please include in a comment a reference.

atgeirr · 2024-09-24T12:34:56Z

opm/ml/keras_model.cpp

+ layer = new KerasLayerDense<Evaluation>();
+ break;
+ case kActivation:
+ layer = new KerasLayerActivation<Evaluation>();


Never use new or delete! It has not been necessary or useful except for low-level library implementers since the previous millenium... Instead, use std::make_unique.

atgeirr · 2024-09-24T12:36:12Z

opm/ml/keras_model.cpp

+ KASSERT(layers_[i]->Apply(&temp_in, &temp_out),
+ "Failed to apply layer %d", i);
+
+ temp_in = temp_out;


Implement an O(1) swap() function instead, and avoid the copying in the layer apply() functions.

atgeirr · 2024-09-24T12:38:01Z

opm/ml/keras_model.cpp

+ KASSERT(out, "Invalid output");
+ KASSERT(in->dims_.size() <= 2, "Invalid input dimensions");
+
+ if (in->dims_.size() == 2) {


The dimension things here require an explanation. Why can we not apply a dense layer to a 1-tensor, and why is there a special treatment for 2-tensors here?

atgeirr · 2024-09-24T12:44:05Z

opm/ml/keras_model.cpp

+
+ bool result = layer->LoadLayer(&file);
+ if (!result) {
+ printf("Failed to load layer %d", i);


We will need to replace lots of printing with proper logging. Using OpmLog::error() for things like this, and fmt::format() to do the formatting is what is done elsewhere, and should be done here as well.

For now though, you should concentrate on the other changes requested before you do this.

kjetilly

Some comments on the python bits. I think a major point is that the folder opm/ml_tools, which contains only python code, should probably be moved to say the python folder or similar. And since these scripts use external libraries (tf, numpy, keras) I would really like to see a requirements.txt file specifying the versions used. Especially tensorflow is known to be problematic her.

kjetilly · 2024-09-25T08:37:51Z

opm/ml/ml_tools/kerasify.py

+# * Copyright (c) 2018 Paul Maevskikh
+# *
+# * MIT License, see LICENSE.MIT file.
+# */ 


Why the C style comment here? A normal python style comment would suffice. The copyright statement should probably be on one line, together with the NORCE statement. (and consider Atgeirr's comment about license compatibility)

kjetilly · 2024-09-25T09:38:29Z

opm/ml/ml_tools/kerasify.py

+ else:
+ assert False, "Unsupported activation type: %s" % activation
+
+ model_layers = [l for l in model.layers if type(l).__name__ not in ['Dropout']]


why the indirection through type(l).__name__ here? Wouldn't isinstance(l, Dropout) be clearer?

kjetilly · 2024-09-25T09:48:14Z

opm/ml/ml_tools/kerasify.py

@@ -0,0 +1,161 @@
+# /*


Why are these under the opm folder? I think it would make more sense to put them under the python folder somewhere?

kjetilly · 2024-09-25T09:51:02Z

opm/ml/ml_tools/kerasify.py

+ elif activation == 'hard_sigmoid':
+ f.write(struct.pack('I', ACTIVATION_HARD_SIGMOID))
+ else:
+ assert False, "Unsupported activation type: %s" % activation


Consider using an f string, which is already being used in this file

kjetilly · 2024-09-25T09:51:27Z

opm/ml/ml_tools/kerasify.py

+ else:
+ assert False, "Unsupported activation type: %s" % activation
+
+ model_layers = [l for l in model.layers if type(l).__name__ not in ['Dropout']]


again probably easier to have isinstance(l, Dropout)

kjetilly · 2024-09-25T09:51:47Z

opm/ml/ml_tools/kerasify.py

+ write_activation(activation)
+
+ else:
+ assert False, "Unsupported layer type: %s" % layer_type


Consider using an f string, which is already being used in this file

kjetilly · 2024-09-25T11:38:12Z

opm/ml/ml_tools/__init__.py

While an empty file is fine, you might want to export some functions here, primarily the function export_model.

kjetilly · 2024-09-25T11:39:14Z

opm/ml/ml_tools/scaler_layers.py

+import numpy as np
+import tensorflow as tf
+from numpy.typing import ArrayLike
+from tensorflow import keras


When using external libraries I think it would be a good idea to supply a requirements.txt file to fix versions, so that a future user will be able to run these scripts.

bikagit · 2024-09-27T14:37:55Z

Thanks all for the valuables comments and suggestions.
We have started adding most of the modifications.

fractalmanifold added 5 commits August 22, 2024 14:27

$@fractalmanifold$

initial commit in new repository

0b4b474

$@fractalmanifold$

Add missing files

ded9251

$@fractalmanifold$

adding basic scaling layers

fe515cc

$@fractalmanifold$

scaling test

5aedd89

$@fractalmanifold$

Licensing

7ffe97c

bikagit force-pushed the mlKeras branch from aa1dceb to 1edf188 Compare August 22, 2024 13:16

bikagit force-pushed the mlKeras branch from 1edf188 to 230460c Compare August 22, 2024 14:47

bikagit force-pushed the mlKeras branch from 230460c to 0eb5e3d Compare August 26, 2024 13:34

bikagit force-pushed the mlKeras branch from fdc8a27 to 6e135b9 Compare August 27, 2024 16:33

$@fractalmanifold$

adding MITLicensing

e2f0a97

bikagit force-pushed the mlKeras branch from 6e135b9 to e2f0a97 Compare August 28, 2024 11:12

bikagit marked this pull request as ready for review August 28, 2024 16:43

bikagit marked this pull request as draft August 29, 2024 10:38

bikagit marked this pull request as ready for review September 6, 2024 13:41

atgeirr requested changes Sep 24, 2024

View reviewed changes

kjetilly reviewed Sep 25, 2024

View reviewed changes



		template<class Evaluation>
		bool KerasModel<Evaluation>::LoadModel(const std::string& filename) {

ML keras #4172

Are you sure you want to change the base?

ML keras #4172

Conversation

bikagit commented Aug 22, 2024 • edited Loading

totto82 commented Aug 22, 2024

daavid00 commented Aug 22, 2024

atgeirr commented Aug 22, 2024

bska commented Aug 22, 2024

bikagit commented Aug 22, 2024

totto82 commented Aug 26, 2024

daavid00 commented Aug 28, 2024

bska commented Aug 28, 2024

totto82 commented Aug 28, 2024

bska commented Aug 28, 2024

totto82 commented Aug 28, 2024

bikagit commented Sep 1, 2024

totto82 commented Sep 6, 2024

atgeirr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kjetilly left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

bikagit commented Sep 27, 2024

bikagit commented Aug 22, 2024 •

edited

Loading