Fix LearningShapelets continuous integration test error on Linux #427

YannCabanes · 2022-10-09T15:57:42Z

This PR is related to the following issue: #426 which is related to a test error in the continuous integration tests of the main branch.
This bug was first noticed in continuous integration tests of the PR #411 (which is now merged), but this bug seems unrelated to the PR.
The continuous integration tests are failing with Linux but they pass with Windows and MacOS.
I use Linux and Python 3.8, and the tests pass on my local computer.
The failing test is related to the test the class tslearn.shapelets.shapelets.LearningShapelets by functions:
test_all_estimators (tslearn/tests/test_estimators.py) --> check_estimator (tslearn/tests/test_estimators.py) --> check_pipeline_consistency (tslearn/tests/sklearn_patches.py).

YannCabanes · 2022-10-09T16:00:29Z

I have added some print functions in the function check_pipeline_consistency which is called during the failing test to investigate on the failing test.

YannCabanes · 2022-10-09T16:29:49Z

This problem could be related to:

this issue which investigates on the results of the same random state for different OS versions and different Numba versions: Random_state produces different results on different operating systems lmcinnes/umap#153
this issue which investigates on the results of the same random state for Numpy alone or when Numpy interacts with TensorFlow: Faulty numpy randomness when using GPU tensorflow/tensorflow#18874
The latest issue suggests trying different versions of TensorFlow (versions 1.12 or 1.13) to solve this problem.

rtavenar · 2022-10-18T15:29:27Z

Hi,

In keras, they have a function that is dedicated to setting all seeds, maybe using it at fit time in the LearingShapelets model would do the trick (instead of our current manual seed setting)?

https://www.tensorflow.org/api_docs/python/tf/keras/utils/set_random_seed

EDIT: I'm just not sure in which version of TF it was introduced...

…elets fit

YannCabanes · 2022-10-24T13:52:05Z

Now both Linux and MacOs continuous integration tests are failing.

YannCabanes · 2022-10-24T13:52:41Z

For MacOs, the error message is the following:

`tslearn/clustering/kshape.py::tslearn.clustering.kshape.KShape Terminating: Nested parallel kernel launch detected, the workqueue threading layer does not supported nested parallelism. Try the TBB threading layer.
Fatal Python error: Aborted

Current thread 0x0000700008f8d000 (most recent call first):
Terminating: Nested parallel kernel launch detected, the workqueue threading layer does not supported nested parallelism. Try the TBB threading layer.

Thread 0x0000700009010000 (most recent call first):
/Users/runner/work/_temp/be9b2263-6107-441b-86a4-2f73ad4262be.sh: line 3: 2584 Abort trap: 6 python -m pytest -v tslearn/ --doctest-modules -k 'not test_all_estimators'
##[error]Bash exited with code '134'.`

YannCabanes · 2022-10-24T14:05:44Z

For Linux, the error message is the following:

=================================== FAILURES ===================================
___________ test_all_estimators[LearningShapelets-LearningShapelets] ___________

name = 'LearningShapelets'
Estimator = <class 'tslearn.shapelets.shapelets.LearningShapelets'>

@pytest.mark.parametrize('name, Estimator', get_estimators('all'))
def test_all_estimators(name, Estimator):
    """Test all the estimators in tslearn."""
    allow_nan = (hasattr(checks, 'ALLOW_NAN') and
                 Estimator().get_tags()["allow_nan"])
    if allow_nan:
        checks.ALLOW_NAN.append(name)
    if name in ["GlobalAlignmentKernelKMeans", "ShapeletModel",
                "SerializableShapeletModel"]:
        # Deprecated models
        return

  check_estimator(Estimator)

tslearn/tests/test_estimators.py:215:

tslearn/tests/test_estimators.py:197: in check_estimator
check(estimator)
tslearn/tests/sklearn_patches.py:295: in check_classifiers_classes
check_classifiers_predictions(X, y_, name, classifier_orig)
/opt/hostedtoolcache/Python/3.8.14/x64/lib/python3.8/site-packages/sklearn/utils/_testing.py:320: in wrapper
return fn(*args, **kwargs)

X = array([[-0.84118762, -1.37894673, -1.88806533, ..., 1.1945725 ,
1.36157375, 1.97347041],
[-0.40654 ...259, -0.29293702],
[ 2.01259303, 1.54825042, 1.19081344, ..., -1.59204839,
-1.56189296, -1.11334695]])
y = array(['three', 'two', 'one', 'two', 'three', 'one', 'two', 'two', 'one',
'two', 'two', 'two', 'one', 'three', ...'two', 'three', 'one',
'three', 'two', 'two', 'one', 'three', 'two', 'three', 'two',
'one'], dtype='<U5')
name = 'LearningShapelets'
classifier_orig = LearningShapelets(max_iter=1000, total_lengths=1)
@ignore_warnings
def check_classifiers_predictions(X, y, name, classifier_orig):
classes = np.unique(y)
classifier = clone(classifier_orig)
dec_exp,
y_pred,
err_msg=(
"decision_function does not match "
"classifier for %r: expected '%s', got '%s'"
)
% (
classifier,
", ".join(map(str, dec_exp)),
", ".join(map(str, y_pred)),
),
)
elif getattr(classifier, "decision_function_shape", "ovr") == "ovr":
decision_y = np.argmax(decision, axis=1).astype(int)
y_exp = classifier.classes_[decision_y]
assert_array_equal(
y_exp,
y_pred,
err_msg=(
"decision_function does not match "
"classifier for %r: expected '%s', got '%s'"
)
% (classifier, ", ".join(map(str, y_exp)), ", ".join(map(str, y_pred))),
)
# training set performance
if name != "ComplementNB":
# This is a pathological data set for ComplementNB.
# For some specific cases 'ComplementNB' predicts less classes
[0.37832805 0.37553697 0.2165663 ]
[0.29635883 0.26970334 0.40072633]
[1.09227915 1.30157721 1.16456915]
[1.29398923 1.20421847 1.13723576]
[0.35030782 0.24016326 0.2449771 ]
[1.18805294 1.21934367 1.17398435]
[0.27079372 0.29311523 0.16652041]
[1.34538163 1.30186523 1.10167461]
[1.25248076 1.29813217 1.2619507 ]
[0.32066084 0.3417426 0.18108248]
[0.28866641 0.40470689 0.23478316]
[0.28660575 0.16988941 0. ]
[0.2365806 0.4085769 0.40223486]
[1.21149155 1.13001945 1.33304802]
[1.22414373 1.26091552 1.138784 ]
[0.33140275 0.26746648 0.2996853 ]
[0.43170422 0.2953147 0.35317278]
[0.48227444 0.10986241 0.25987483]
[0.4793883 0.44205478 0.15757119]
[1.28554617 1.19186677 1.21902486]
[1.4041242 1.4448879 1.37317694]
[0.08467196 0.45037652 0.20433376]
[1.08267072 1.2730416 1.21512089]
[1.0939092 1.23402495 1.16575233]
[1.36923905 1.1318164 1.29553315]
[0.05721933 0.22050777 0.27093388]
[1.2373065 1.14822372 1.36074415]
[1.18681797 1.16821927 1.19741402]
[0.22506871 0.15044369 0.11329719]]

y
[1 0 0 1 1 0 1 0 1 1 0 0 0 0 1 1 0 0 0 0 1 1 0 1 1 1 0 1 1 0]

estimator
LearningShapelets(max_iter=100, total_lengths=1)

scale False
random_state 0

pipeline
Pipeline(steps=[('learningshapelets',
LearningShapelets(max_iter=100, random_state=0,
total_lengths=1))])

learningshapelets inside pipeline
LearningShapelets(max_iter=100, random_state=0, total_lengths=1)
pipeline variables before fit
n_shapelets_per_size None
max_iter 100
batch_size 256
verbose 0
optimizer sgd
weight_regularizer 0.0
shapelet_length 0.15
total_lengths 1
max_size None
scale False
random_state 0
ESTIMATOR AND PIPELINE ARE EQUAL BEFORE FIT
True

estimator after fit
LearningShapelets(max_iter=100, random_state=0, total_lengths=1)
estimator variables after fit
n_shapelets_per_size None
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100

pipeline after fit
Pipeline(steps=[('learningshapelets',
LearningShapelets(max_iter=100, random_state=0,
total_lengths=1))])
pipeline variables after fit
n_shapelets_per_size None
max_iter 100
batch_size 256
verbose 0
optimizer sgd
weight_regularizer 0.0
shapelet_length 0.15
total_lengths 1
max_size None
scale False
random_state 0
min_sz_fit 3
X_fit_dims (30, 3, 1)
model <keras.engine.functional.Functional object at 0x7f3db47ca280>
transformer_model <keras.engine.functional.Functional object at 0x7f3db481e9d0>
locator_model_ <keras.engine.functional.Functional object at 0x7f3db4680040>
d_ 1
classes_ [0 1]
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100
ESTIMATOR AND PIPELINE ARE EQUAL AFTER FIT
True

func_name
score

classes_ [0 1]
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100

pipeline variables before func_pipeline
n_shapelets_per_size None
max_iter 100
batch_size 256
verbose 0
optimizer sgd
weight_regularizer 0.0
shapelet_length 0.15
total_lengths 1
max_size None
scale False
random_state 0
min_sz_fit 3
X_fit_dims (30, 3, 1)
model <keras.engine.functional.Functional object at 0x7f3db47ca280>
transformer_model <keras.engine.functional.Functional object at 0x7f3db481e9d0>
locator_model_ <keras.engine.functional.Functional object at 0x7f3db4680040>
d_ 1
classes_ [0 1]
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100

ESTIMATOR AND PIPELINE ARE EQUAL BEFORE FUNC
True

scale False
random_state 0
min_sz_fit 3
X_fit_dims (30, 3, 1)
model <keras.engine.functional.Functional object at 0x7f3db47ca280>
transformer_model <keras.engine.functional.Functional object at 0x7f3db481e9d0>
locator_model_ <keras.engine.functional.Functional object at 0x7f3db4680040>
d_ 1
classes_ [0 1]
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100

ESTIMATOR AND PIPELINE ARE EQUAL BEFORE FUNC
True

estimator variables after func
n_shapelets_per_size None
max_iter 100
batch_size 256
verbose 0
optimizer sgd
weight_regularizer 0.0
shapelet_length 0.15

pipeline variables after func_pipeline
n_shapelets_per_size None
max_iter 100
batch_size 256
verbose 0
optimizer sgd
weight_regularizer 0.0
shapelet_length 0.15
total_lengths 1
max_size None
scale False
random_state 0
min_sz_fit 3
X_fit_dims (30, 3, 1)
model <keras.engine.functional.Functional object at 0x7f3db44401f0>
transformer_model <keras.engine.functional.Functional object at 0x7f3db4551190>
locator_model_ <keras.engine.functional.Functional object at 0x7f3db43eafa0>
d_ 1
classes_ [0 1]
label_to_ind_ {0: 0, 1: 1}
n_shapelets_per_size_ {1: 1}
history_ {'loss': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998], 'binary_accuracy': [0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5, 0.5333333611488342, 0.5333333611488342, 0.5333333611488342, 0.6000000238418579, 0.6000000238418579, 0.6000000238418579, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.6333333253860474, 0.699999988079071, 0.699999988079071, 0.7666666507720947, 0.7666666507720947, 0.7666666507720947, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.800000011920929, 0.8333333134651184, 0.8333333134651184, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526, 0.8666666746139526], 'binary_crossentropy': [0.6975758671760559, 0.6969660520553589, 0.6963578462600708, 0.6957510709762573, 0.6951456665992737, 0.6945418119430542, 0.6939391493797302, 0.6933377981185913, 0.6927376985549927, 0.6921387910842896, 0.6915410757064819, 0.6909444332122803, 0.6903488636016846, 0.6897544264793396, 0.6891607642173767, 0.6885682344436646, 0.687976598739624, 0.6873857975006104, 0.6867958903312683, 0.6862066984176636, 0.6856182813644409, 0.6850306391716003, 0.6844435930252075, 0.6838572025299072, 0.6832715272903442, 0.6826862692832947, 0.6821017861366272, 0.6815175414085388, 0.6809338927268982, 0.680350661277771, 0.6797678470611572, 0.6791852712631226, 0.6786031723022461, 0.6780213713645935, 0.67743980884552, 0.6768584847450256, 0.676277220249176, 0.6756963133811951, 0.6751154661178589, 0.6745347380638123, 0.6739540696144104, 0.6733734011650085, 0.6727929711341858, 0.6722123026847839, 0.6716316342353821, 0.671051025390625, 0.6704701781272888, 0.6698892712593079, 0.669308066368103, 0.6687269806861877, 0.668145477771759, 0.667563796043396, 0.6669818162918091, 0.6663996577262878, 0.665817141532898, 0.6652342677116394, 0.6646509766578674, 0.6640674471855164, 0.6634834408760071, 0.6628989577293396, 0.6623140573501587, 0.6617286801338196, 0.6611427068710327, 0.660556435585022, 0.6599694490432739, 0.6593819856643677, 0.6587938666343689, 0.6582050919532776, 0.6576157808303833, 0.6570257544517517, 0.6564351320266724, 0.6558437347412109, 0.6552517414093018, 0.6546589732170105, 0.6540653705596924, 0.6534711122512817, 0.6528760194778442, 0.6522800922393799, 0.6516834497451782, 0.6510858535766602, 0.65048748254776, 0.6498881578445435, 0.6492879986763, 0.6486870050430298, 0.6480849385261536, 0.6474820971488953, 0.6468782424926758, 0.6462734341621399, 0.6456676721572876, 0.6450608372688293, 0.6444530487060547, 0.6438443064689636, 0.6432344317436218, 0.6426236629486084, 0.6420116424560547, 0.6413987278938293, 0.6407846212387085, 0.6401695013046265, 0.6395533084869385, 0.6389359831809998]}
n_iter_ 100

ESTIMATOR AND PIPELINE ARE EQUAL AFTER FUNC
True

result
[[0.07146144]
[0.3780318 ]
[0.35099065]
[0.00982237]
[0.02075458]
[0.41327304]
[0.03269339]
[0.49007803]
[0.01177311]
[0.06724167]
[0.4243589 ]
[0.34629 ]
[0.49923438]
[0.3417502 ]
[0.0187273 ]
[0.02120304]
[0.43793714]
[0.3152449 ]
[0.26101524]
[0.26397258]
[0.03947997]
[0.14440465]
[0.29462582]
[0.00801015]
[0.01014829]
[0.0192225 ]
[0.5216264 ]
[0.02404118]
[0.03064203]
[0.5899811 ]]

result pipe
[[0.07146144]
[0.3780318 ]
[0.35099065]
[0.00982237]
[0.02075458]
[0.41327304]
[0.03269339]
[0.49007803]
[0.01177311]
[0.06724167]
[0.4243589 ]
[0.34629 ]
[0.49923438]
[0.3417502 ]
[0.0187273 ]
[0.02120304]
[0.43793714]
[0.3152449 ]
[0.26101524]
[0.26397258]
[0.03947997]
[0.14440465]
[0.29462582]
[0.00801015]
[0.01014829]
[0.0192225 ]
[0.5216264 ]
[0.02404118]
[0.03064203]
[0.5899811 ]]

----------------------------- Captured stderr call -----------------------------
WARNING:tensorflow:6 out of the last 8 calls to <function Model.make_predict_function..predict_function at 0x7f3dcabe1700> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.
------------------------------ Captured log call -------------------------------
WARNING tensorflow:def_function.py:148 6 out of the last 8 calls to <function Model.make_predict_function..predict_function at 0x7f3dcabe1700> triggered tf.function retracing. Tracing is expensive and the excessive number of tracings could be due to (1) creating @tf.function repeatedly in a loop, (2) passing tensors with different shapes, (3) passing Python objects instead of tensors. For (1), please define your @tf.function outside of the loop. For (2), @tf.function has reduce_retracing=True option that can avoid unnecessary retracing. For (3), please refer to https://www.tensorflow.org/guide/function#controlling_retracing and https://www.tensorflow.org/api_docs/python/tf/function for more details.

YannCabanes · 2022-10-24T14:21:00Z

As expected, we obtain the exact same failing tests after "Empty commit".

YannCabanes · 2022-10-24T14:36:30Z

We obtain the exact same failing tests with or without scikit-learn==1.0 in the file azure-pipelines.yml.

YannCabanes · 2022-10-24T15:04:55Z

With tensorflow==2.0, we obtain the following error message:

+ python -m pip install tensorflow==2.0
ERROR: Could not find a version that satisfies the requirement tensorflow==2.0 (from versions: 2.2.0, 2.2.1, 2.2.2, 2.2.3, 2.3.0, 2.3.1, 2.3.2, 2.3.3, 2.3.4, 2.4.0, 2.4.1, 2.4.2, 2.4.3, 2.4.4, 2.5.0, 2.5.1, 2.5.2, 2.5.3, 2.6.0rc0, 2.6.0rc1, 2.6.0rc2, 2.6.0, 2.6.1, 2.6.2, 2.6.3, 2.6.4, 2.6.5, 2.7.0rc0, 2.7.0rc1, 2.7.0, 2.7.1, 2.7.2, 2.7.3, 2.7.4, 2.8.0rc0, 2.8.0rc1, 2.8.0, 2.8.1, 2.8.2, 2.8.3, 2.9.0rc0, 2.9.0rc1, 2.9.0rc2, 2.9.0, 2.9.1, 2.9.2, 2.10.0rc0, 2.10.0rc1, 2.10.0rc2, 2.10.0rc3, 2.10.0, 2.11.0rc0, 2.11.0rc1)
ERROR: No matching distribution found for tensorflow==2.0

YannCabanes · 2022-10-24T15:16:31Z

With tensorflow==2.8.0, some continuous integration tests are failing Linux, MacOS and Windows.
On Windows, they fail for Python 3.7 and 3.9, not for Python 3.8.

YannCabanes · 2022-10-24T15:17:31Z

The error message for Linux is the following:

==================================== ERRORS ====================================
________________ ERROR collecting tslearn/shapelets/init.py ________________
tslearn/shapelets/init.py:10: in
from .shapelets import LearningShapelets, ShapeletModel,
tslearn/shapelets/shapelets.py:1: in
from tensorflow.keras.models import Model, model_from_json
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/init.py:37: in
from tensorflow.python.tools import module_util as _module_util
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/init.py:37: in
from tensorflow.python.eager import context
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/eager/context.py:29: in
from tensorflow.core.framework import function_pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py:16: in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py:16: in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py:16: in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in
from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: in
_descriptor.FieldDescriptor(
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in new
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

------------------------------- Captured stderr --------------------------------
2022-10-24 15:03:10.900256: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-10-24 15:03:10.900299: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
_______________ ERROR collecting tslearn/shapelets/shapelets.py ________________
tslearn/shapelets/shapelets.py:1: in
from tensorflow.keras.models import Model, model_from_json
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/init.py:37: in
from tensorflow.python.tools import module_util as _module_util
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/init.py:37: in
from tensorflow.python.eager import context
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/eager/context.py:29: in
from tensorflow.core.framework import function_pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py:16: in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py:16: in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py:16: in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in
from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: in
_descriptor.FieldDescriptor(
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in new
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

______________ ERROR collecting tslearn/tests/test_estimators.py _______________
tslearn/tests/test_estimators.py:43: in
from tslearn.shapelets import LearningShapelets, SerializableShapeletModel
tslearn/shapelets/init.py:10: in
from .shapelets import LearningShapelets, ShapeletModel,
tslearn/shapelets/shapelets.py:1: in
from tensorflow.keras.models import Model, model_from_json
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/init.py:37: in
from tensorflow.python.tools import module_util as _module_util
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/init.py:37: in
from tensorflow.python.eager import context
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/eager/context.py:29: in
from tensorflow.core.framework import function_pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py:16: in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py:16: in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py:16: in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in
from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: in
_descriptor.FieldDescriptor(
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in new
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

___________ ERROR collecting tslearn/tests/test_serialize_models.py ____________
tslearn/tests/test_serialize_models.py:11: in
from tslearn.shapelets import LearningShapelets, SerializableShapeletModel
tslearn/shapelets/init.py:10: in
from .shapelets import LearningShapelets, ShapeletModel,
tslearn/shapelets/shapelets.py:1: in
from tensorflow.keras.models import Model, model_from_json
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/init.py:37: in
from tensorflow.python.tools import module_util as _module_util
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/init.py:37: in
from tensorflow.python.eager import context
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/python/eager/context.py:29: in
from tensorflow.core.framework import function_pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/function_pb2.py:16: in
from tensorflow.core.framework import attr_value_pb2 as tensorflow_dot_core_dot_framework_dot_attr__value__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/attr_value_pb2.py:16: in
from tensorflow.core.framework import tensor_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_pb2.py:16: in
from tensorflow.core.framework import resource_handle_pb2 as tensorflow_dot_core_dot_framework_dot_resource__handle__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/resource_handle_pb2.py:16: in
from tensorflow.core.framework import tensor_shape_pb2 as tensorflow_dot_core_dot_framework_dot_tensor__shape__pb2
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/tensorflow/core/framework/tensor_shape_pb2.py:36: in
_descriptor.FieldDescriptor(
/opt/hostedtoolcache/Python/3.9.15/x64/lib/python3.9/site-packages/google/protobuf/descriptor.py:560: in new
_message.Message._CheckCalledFromGeneratedFile()
E TypeError: Descriptors cannot not be created directly.
E If this call came from a _pb2.py file, your generated code is out of date and must be regenerated with protoc >= 3.19.0.
E If you cannot immediately regenerate your protos, some other possible workarounds are:
E 1. Downgrade the protobuf package to 3.20.x or lower.
E 2. Set PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python (but this will use pure-Python parsing and will be much slower).
E
E More information: https://developers.google.com/protocol-buffers/docs/news/2022-05-06#python-updates

-------- generated Nunit xml file: /home/vsts/work/1/s/test-output.xml ---------
=========================== short test summary info ============================
ERROR tslearn/shapelets/init.py - TypeError: Descriptors cannot not be cr...
ERROR tslearn/shapelets/shapelets.py - TypeError: Descriptors cannot not be c...
ERROR tslearn/tests/test_estimators.py - TypeError: Descriptors cannot not be...
ERROR tslearn/tests/test_estimators.py - TypeError: Descriptors cannot not be...
ERROR tslearn/tests/test_serialize_models.py - TypeError: Descriptors cannot ...
ERROR tslearn/tests/test_serialize_models.py - TypeError: Descriptors cannot ...
!!!!!!!!!!!!!!!!!!! Interrupted: 6 errors during collection !!!!!!!!!!!!!!!!!!!!
============================== 6 errors in 2.11s ===============================
##[error]Bash exited with code '2'.

YannCabanes · 2022-10-24T15:46:22Z

Using tensorflow==2.9.0 for Linux in azure-pipelines.yml, the tests pass for Linux and Windows.
The error message is exactly the same for MacOS.

codecov-commenter · 2022-10-24T15:47:26Z

Codecov Report

Base: 94.60% // Head: 94.67% // Increases project coverage by +0.07% 🎉

Coverage data is based on head (2338aa0) compared to base (3596109).
Patch coverage: 96.28% of modified lines in pull request are covered.

❗ Current head 2338aa0 differs from pull request most recent head 9fddcd2. Consider uploading reports for the commit 9fddcd2 to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #427      +/-   ##
==========================================
+ Coverage   94.60%   94.67%   +0.07%     
==========================================
  Files          59       62       +3     
  Lines        4538     4824     +286     
==========================================
+ Hits         4293     4567     +274     
- Misses        245      257      +12

Impacted Files	Coverage Δ
tslearn/metrics/__init__.py	`100.00% <ø> (ø)`
tslearn/metrics/cycc.py	`81.53% <81.53%> (ø)`
tslearn/metrics/cysax.py	`100.00% <100.00%> (ø)`
tslearn/metrics/soft_dtw_fast.py	`100.00% <100.00%> (ø)`
tslearn/metrics/softdtw_variants.py	`97.84% <100.00%> (ø)`
tslearn/shapelets/shapelets.py	`95.93% <100.00%> (ø)`
tslearn/tests/sklearn_patches.py	`93.53% <100.00%> (+1.47%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

YannCabanes · 2022-10-24T17:47:33Z

Using tensorflow==2.10.0 for Linux in azure-pipelines.yml, the tests are failing on Linux and MacOS.

…apelets.py

…ts.py

YannCabanes · 2022-10-24T20:44:02Z

There is the same failing tests for MacOS with and without using tf.keras.utils.set_random_seed(seed=self.random_state) in the file shapelets.py.
Therefore using tf.keras.utils.set_random_seed(seed=self.random_state) does not seem to be related to the MacOS failing tests.

Add print functions in check_pipeline_consistency

0f90e11

YannCabanes added 3 commits October 18, 2022 11:45

Use tf.keras.utils.set_random_seed to set random seed in LearningShap…

27a2daf

…elets fit

Remove scikit-learn==1.0 from azure-pipelines.yml

546e6ba

Remove commented line scikit-learn==1.0 from azure-pipelines.yml

cbb8719

Empty commit

e0d6901

Add previously removed scikit-learn==1.0 in azure-pipelines.yml

d4ad01c

YannCabanes added 2 commits October 24, 2022 10:43

Add tensorflow==2.0 for Linux in azure-pipelines.yml

f23ac09

Add tensorflow==2.8.0 for Linux in azure-pipelines.yml

739cd07

YannCabanes added 2 commits October 24, 2022 11:21

Use tensorflow==2.9.0 for Linux in azure-pipelines.yml

42ce118

Use tensorflow==2.10.0 for Linux in azure-pipelines.yml

51adbe7

YannCabanes added 4 commits October 24, 2022 13:47

Use tensorflow==2.9.0 for Linux and MacOS in azure-pipelines.yml

2338aa0

Remove tensorflow==2.9.0 for MacOS in azure-pipelines.yml

9581c33

Remove tf.keras.utils.set_random_seed(seed=self.random_state) from sh…

44dc7d3

…apelets.py

Use tf.keras.utils.set_random_seed(seed=self.random_state) in shapele…

9ac5988

…ts.py

Remove print functions from sklearn_patches.py

9fddcd2

rtavenar merged commit 1f80e22 into tslearn-team:main Oct 27, 2022

YannCabanes mentioned this pull request Dec 22, 2022

Fix Import Error sklearn pairwise_estimator_convert_X #435

Merged

YannCabanes mentioned this pull request Jun 26, 2023

Remove support for Python 3.7 #457

Merged

YannCabanes deleted the fix-learning-shapelets-continuous-integration-test-error branch July 13, 2023 07:52

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix LearningShapelets continuous integration test error on Linux #427

Fix LearningShapelets continuous integration test error on Linux #427

YannCabanes commented Oct 9, 2022

YannCabanes commented Oct 9, 2022

YannCabanes commented Oct 9, 2022

rtavenar commented Oct 18, 2022 •

edited

Loading

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

codecov-commenter commented Oct 24, 2022 •

edited

Loading

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

Fix LearningShapelets continuous integration test error on Linux #427

Fix LearningShapelets continuous integration test error on Linux #427

Conversation

YannCabanes commented Oct 9, 2022

YannCabanes commented Oct 9, 2022

YannCabanes commented Oct 9, 2022

rtavenar commented Oct 18, 2022 • edited Loading

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

codecov-commenter commented Oct 24, 2022 • edited Loading

Codecov Report

YannCabanes commented Oct 24, 2022

YannCabanes commented Oct 24, 2022

rtavenar commented Oct 18, 2022 •

edited

Loading

codecov-commenter commented Oct 24, 2022 •

edited

Loading