Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI is broken #137

Closed
astrojuanlu opened this issue Jun 26, 2023 · 3 comments
Closed

CI is broken #137

astrojuanlu opened this issue Jun 26, 2023 · 3 comments
Labels
bug Something isn't working

Comments

@astrojuanlu
Copy link
Member

astrojuanlu commented Jun 26, 2023

#136 made a one character change in the README but there were some test failures.

https://app.circleci.com/pipelines/github/kedro-org/kedro-starters/618/workflows/687e92b1-d6d7-41e1-9348-dc8566bb9c39/jobs/6650

Failing scenarios:
  features/run.feature:26  Run a Kedro project created from pyspark-iris

0 features passed, 1 failed, 0 skipped
4 scenarios passed, 1 failed, 0 skipped
24 steps passed, 1 failed, 0 skipped, 0 undefined
Took 3m56.385s

Exited with code exit status 1

CircleCI received exit code 1

This is the error:

DatasetError: An exception occurred when parsing config for dataset 
'example_classifier':
Dataset type 'kedro.io.memory_dataset.MemoryDataSet' is invalid: all data set 
types must extend 'AbstractDataSet'.
@astrojuanlu
Copy link
Member Author

Root cause:

# This is an example how to use `MemoryDataSet` with Spark objects that aren't `DataFrame`'s.
# In particular, the `assign` copy mode ensures that the `MemoryDataSet` will be assigned
# the Spark object itself, not a deepcopy version of it, since deepcopy doesn't work with
# Spark object generally.
example_classifier:
type: MemoryDataSet
copy_mode: assign

and

https://github.com/kedro-org/kedro/blob/fd8162d5bf384ef666c01ef2c529d01fd9fa8354/kedro/io/core.py#L427-L431

@astrojuanlu
Copy link
Member Author

In progress at kedro-org/kedro#2724

@astrojuanlu
Copy link
Member Author

Test: gh-138

astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 27, 2023
Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 27, 2023
Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@astrojuanlu astrojuanlu moved this from To Do to In Review in Kedro Framework Jun 28, 2023
astrojuanlu pushed a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
@github-project-automation github-project-automation bot moved this from In Review to Done in Kedro Framework Jun 28, 2023
debugger24 pushed a commit to debugger24/kedro that referenced this issue Jun 28, 2023
* Complete build requirements

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add timeout for e2e process waiting

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove DeprecatedClassMeta in favor of getattr

Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Ignore deprecation warnings in our own code

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: debugger24 <rahulcomp24@gmail.com>
astrojuanlu added a commit to kedro-org/kedro that referenced this issue Jun 28, 2023
* FIX: Typo in cli help

Signed-off-by: debugger24 <rahulcomp24@gmail.com>

* Make GitPod works (#2688)

* update base image

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update gitpod image and apt-get

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>

* update docker

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
---------

Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: debugger24 <rahulcomp24@gmail.com>

* Remove `DeprecatedClassMeta` in favor of `getattr` (#2724)

* Complete build requirements

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Add timeout for e2e process waiting

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Remove DeprecatedClassMeta in favor of getattr

Fix kedro-org/kedro-starters#137

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

* Ignore deprecation warnings in our own code

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>

---------

Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Signed-off-by: debugger24 <rahulcomp24@gmail.com>

---------

Signed-off-by: debugger24 <rahulcomp24@gmail.com>
Signed-off-by: Nok <nok.lam.chan@quantumblack.com>
Signed-off-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Co-authored-by: Nok Lam Chan 陳諾林 <nok.lam.chan@quantumblack.com>
Co-authored-by: Deepyaman Datta <deepyaman.datta@utexas.edu>
Co-authored-by: Juan Luis Cano Rodríguez <juan_luis_cano@mckinsey.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

No branches or pull requests

1 participant