Fully enable Python 3.8 tests #1960
Comments
natekupp
pushed a commit
that referenced
this issue
Dec 6, 2019
Summary: This enables Python 3.8 tests for most things in Dagster. See: #1960 for notes on what's not currently covered. Test Plan: unit Reviewers: schrockn, alangenfeld, max, prha Reviewed By: max Differential Revision: https://dagster.phacility.com/D1560
Closed
mgasner
pushed a commit
that referenced
this issue
Jan 10, 2020
Summary: This enables Python 3.8 tests for most things in Dagster. See: #1960 for notes on what's not currently covered. Test Plan: unit Reviewers: schrockn, alangenfeld, max, prha Reviewed By: max Differential Revision: https://dagster.phacility.com/D1560
|
This is almost ready to go: https://dagster.phacility.com/D2261
|
sd2k
added a commit
to sd2k/dagster
that referenced
this issue
Jun 5, 2020
…ks tests on buildkite See dagster-io#1960.
sd2k
added a commit
to sd2k/dagster
that referenced
this issue
Jun 8, 2020
…ks tests on buildkite See dagster-io#1960.
sd2k
added a commit
to sd2k/dagster
that referenced
this issue
Jun 8, 2020
…ks tests on buildkite See dagster-io#1960.
sryza
pushed a commit
that referenced
this issue
Jun 9, 2020
* Add dagster-databricks package This package is closely modeled off the dagster_aws.emr subpackage and provides the databricks_pyspark_step_launcher resource and the DatabricksRunJobSolidDefinition solid for running Databricks jobs. * Reference Databricks docs in dagster-databricks configs module * Move build_pyspark_zip into dagster_pyspark utils module * Fix style/minor issues in dagster-databricks Specifically: - triple single quotes instead of triple double quotes for docstrings - single quotes instead of double quotes everywhere else - oneline docstrings where possible; start on same line everywhere else - rename 'is_terminal' to 'has_terminated' - use 'databricks_run_id' instead of 'run_id' for clarity - make DatabricksJobRunner.client a property - remove unnecessary blank lines * Add references to Databricks storage docs in 'main' script * Add comment explaining global vars in databricks_step_main.py * Fix Python 2 issues in dagster-databricks * Check invariants when setting up storage in Databricks job * Fix dependencies in dagster-databricks/tox.ini * Move 'secret_scope' field into inner credentials object to simplify Databricks storage * isort dagster-databricks * Add pylint to tox.ini for dagster_databricks * Install dagster-databricks in 'make install_dev_python_modules' * Reference GitHub issue for better storage setup in databricks_step_main.py * Uncomment dagster-azure related config * Replace assert_called_once with call_count for Python3.5 compat * Fix lint errors in databricks.py * Improve handling of libraries by including required libs by default * Fix version to match other dagster libraries * Specify supported_pythons to exclude Python 3.8 from dagster-databricks tests on buildkite See #1960. * Add README for dagster-databricks * Install dagster-databricks in dagster-examples tox.ini * Update snapshot test for dagster example using databricks * Add API docs for dagster_databricks * Add coveragerc for dagster-databricks
|
Spark 3 is out: https://spark.apache.org/news/spark-3-0-0-released.html |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
With https://dagster.phacility.com/D1560 we are adding tests for Python 3.8.
However, any test that requires
pyarroworpysparkwon't work right now, so I haven't addedSupportedPython.V3_8toSupportedPythons.Once we can support 3.8 fully, we should add 3.8 to supported Pythons and clean up special casing. I'll leave comments in the code linking to this issue where that is necessary.
The text was updated successfully, but these errors were encountered: