Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add temporary fix for ray import pandas race condition #1174

Merged
merged 2 commits into from
Apr 14, 2020

Conversation

simon-mo
Copy link
Collaborator

@simon-mo simon-mo commented Apr 12, 2020

What do these changes do?

Attempt to address issues raised in ray-project/ray#7879

Signed-off-by: simon-mo <xmo@berkeley.edu>
@codecov
Copy link

codecov bot commented Apr 12, 2020

Codecov Report

Merging #1174 into master will decrease coverage by 0.51%.
The diff coverage is 100.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1174      +/-   ##
==========================================
- Coverage   86.95%   86.43%   -0.52%     
==========================================
  Files          62       62              
  Lines        6315     6318       +3     
==========================================
- Hits         5491     5461      -30     
- Misses        824      857      +33     
Impacted Files Coverage Δ
modin/pandas/__init__.py 80.00% <100.00%> (-2.09%) ⬇️
...gines/ray/pandas_on_ray/frame/partition_manager.py 64.44% <0.00%> (-26.67%) ⬇️
modin/backends/pandas/parsers.py 59.13% <0.00%> (-10.76%) ⬇️
modin/pandas/groupby.py 87.76% <0.00%> (-1.80%) ⬇️
modin/engines/ray/pandas_on_ray/frame/partition.py 90.80% <0.00%> (-1.15%) ⬇️
modin/backends/pandas/query_compiler.py 94.45% <0.00%> (-0.60%) ⬇️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c7d9863...ccf374f. Read the comment docs.

@robertnishihara
Copy link
Contributor

robertnishihara commented Apr 13, 2020

@simon-mo @devin-petersohn does this appear to work around the issue?

@gshimansky
Copy link
Collaborator

@simon-mo @devin-petersohn does this appear to work around the issue?

I tried running tests on Ray several times and didn't have any fails even with 100 pytest workers. So I think the issue is indeed fixed. TeamCity confit is probably wrong and that's the reason why CI testing failed. I restarted it with modified config.

@modin-bot
Copy link

modin-bot commented Apr 13, 2020

🤖 TeamCity test results bot 🤖

Ray Tests
+ python3 -m pytest modin/experimental/pandas/test/test_io_exp.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
collected 2 items

modin/experimental/pandas/test/test_io_exp.py ..                         [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     95    53%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     65    30%
modin/backends/pandas/query_compiler.py                              505     28    94%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     22    83%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20      0   100%
modin/data_management/functions/foldfunction.py                        6      0   100%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      0   100%
modin/data_management/functions/mapreducefunction.py                   8      0   100%
modin/data_management/functions/reductionfunction.py                   6      0   100%
modin/data_management/utils.py                                        32      0   100%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     10    78%
modin/engines/base/frame/data.py                                     425     25    94%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179     20    89%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37     26    30%
modin/engines/base/io/column_stores/feather_reader.py                  9      5    44%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30     25    17%
modin/engines/base/io/file_reader.py                                  76     60    21%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43      6    86%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122    114     7%
modin/engines/base/io/text/json_reader.py                             45     38    16%
modin/engines/base/io/text/text_file_reader.py                        16      9    44%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63      4    94%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      5    58%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      6    77%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      3    77%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87     27    69%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     27    40%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      0   100%
modin/engines/ray/utils.py                                            11      9    18%
modin/error_message.py                                                22      3    86%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     21    50%
modin/experimental/engines/pandas_on_ray/sql.py                       70     49    30%
modin/experimental/pandas/__init__.py                                  6      0   100%
modin/experimental/pandas/io_exp.py                                    6      0   100%
modin/pandas/__init__.py                                              70     14    80%
modin/pandas/base.py                                                1028     51    95%
modin/pandas/concat.py                                                44      7    84%
modin/pandas/dataframe.py                                            736     88    88%
modin/pandas/datetimes.py                                              7      0   100%
modin/pandas/general.py                                               41      1    98%
modin/pandas/groupby.py                                              278     34    88%
modin/pandas/indexing.py                                             183     30    84%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17      0   100%
modin/pandas/reshape.py                                               30      0   100%
modin/pandas/series.py                                               757     55    93%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318   1199    81%


======================== 2 passed, 6 warnings in 3.64s =========================
+ python3 -m pytest -n=48 modin/pandas/test/test_series.py modin/pandas/test/test_dataframe.py modin/pandas/test/test_concat.py modin/pandas/test/test_groupby.py modin/pandas/test/test_reshape.py modin/pandas/test/test_general.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [12619] / gw1 [12619] / gw2 [12619] / gw3 [12619] / gw4 [12619] / gw5 [12619] / gw6 [12619] / gw7 [12619] / gw8 [12619] / gw9 [12619] / gw10 [12619] / gw11 [12619] / gw12 [12619] / gw13 [12619] / gw14 [12619] / gw15 [12619] / gw16 [12619] / gw17 [12619] / gw18 [12619] / gw19 [12619] / gw20 [12619] / gw21 [12619] / gw22 [12619] / gw23 [12619] / gw24 [12619] / gw25 [12619] / gw26 [12619] / gw27 [12619] / gw28 [12619] / gw29 [12619] / gw30 [12619] / gw31 [12619] / gw32 [12619] / gw33 [12619] / gw34 [12619] / gw35 [12619] / gw36 [12619] / gw37 [12619] / gw38 [12619] / gw39 [12619] / gw40 [12619] / gw41 [12619] / gw42 [12619] / gw43 [12619] / gw44 [12619] / gw45 [12619] / gw46 [12619] / gw47 [12619]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  2%]
......................................................................... [  3%]
........................................................................ [  4%]
...............s........................................................ [  4%]
........................................................................ [  5%]
........................................................................ [  5%]
.....................s.................................................. [  6%]
.....................s.....s............................................ [  6%]
........................................................................ [  7%]
........................................................................ [  7%]
........................................................................ [  8%]
...............................................................s........ [  9%]
........................................................................ [  9%]
........................................................................ [ 10%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 11%]
........................................................................ [ 12%]
.......................................................................... [ 13%]
............................................................................ [ 13%]
........................................................................ [ 14%]
.................s......................................................... [ 14%]
......................................................................... [ 15%]
........................................................................ [ 16%]
........................................................................ [ 16%]
........................................................................ [ 17%]
.......................................................................... [ 17%]
........................................................................ [ 18%]
........................................................................ [ 18%]
......................................................................... [ 19%]
......................................................................... [ 20%]
........................................................................ [ 20%]
..............................................s......................... [ 21%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 24%]
...............ssssss................................................... [ 25%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 26%]
........................................................................ [ 27%]
......................................................................... [ 28%]
........................................................................ [ 28%]
........................................................................ [ 29%]
........................................................................ [ 29%]
......................................................................... [ 30%]
........................................................................ [ 30%]
......................................................................... [ 31%]
........................................................................ [ 32%]
........................................................................ [ 32%]
........................................................................ [ 33%]
.......................................................................... [ 33%]
......................................................................... [ 34%]
......................................................................... [ 34%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 38%]
........................................................................ [ 39%]
......................................................................... [ 40%]
......................................sss.ssssss........................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 45%]
........................................................................ [ 46%]
........................................................................ [ 46%]
........................................................................ [ 47%]
........................................................................ [ 48%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 49%]
........................................................................ [ 50%]
........................................................................ [ 50%]
.......................................................................... [ 51%]
........................................................................ [ 52%]
........................................................................ [ 52%]
........................................................................ [ 53%]
......................................................................... [ 53%]
........................................................................ [ 54%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 56%]
......................................................................... [ 56%]
......................................................................... [ 57%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 60%]
......................................................................... [ 60%]
........................................................................ [ 61%]
.......................................................................... [ 61%]
......................................................................... [ 62%]
........................................................................ [ 63%]
........................................................................ [ 63%]
........................................................................ [ 64%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 72%]
........................................................................ [ 72%]
........................................................................ [ 73%]
......................................................................... [ 73%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 75%]
........................................................................ [ 76%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 78%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 80%]
......................................................................... [ 81%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 82%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 85%]
......................................................................... [ 86%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 89%]
........................................................................ [ 90%]
......................................................................... [ 90%]
........................................................................ [ 91%]
........................................................................ [ 92%]
...s...s.s.ss........................................................... [ 92%]
............................................................s........... [ 93%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 96%]
........................................................................ [ 97%]
......................................................................... [ 97%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........................................................................ [ 99%]
......................................................                   [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     95    53%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     64    31%
modin/backends/pandas/query_compiler.py                              505     28    94%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     22    83%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20      0   100%
modin/data_management/functions/foldfunction.py                        6      0   100%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      0   100%
modin/data_management/functions/mapreducefunction.py                   8      0   100%
modin/data_management/functions/reductionfunction.py                   6      0   100%
modin/data_management/utils.py                                        32      0   100%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     10    78%
modin/engines/base/frame/data.py                                     425     25    94%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179     20    89%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37     26    30%
modin/engines/base/io/column_stores/feather_reader.py                  9      5    44%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30     25    17%
modin/engines/base/io/file_reader.py                                  76     41    46%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43      6    86%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122     41    66%
modin/engines/base/io/text/json_reader.py                             45     38    16%
modin/engines/base/io/text/text_file_reader.py                        16      1    94%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63      4    94%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      0   100%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      0   100%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      3    77%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87      8    91%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     16    64%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      0   100%
modin/engines/ray/utils.py                                            11      2    82%
modin/error_message.py                                                22      3    86%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     21    50%
modin/experimental/engines/pandas_on_ray/sql.py                       70     49    30%
modin/experimental/pandas/__init__.py                                  6      0   100%
modin/experimental/pandas/io_exp.py                                    6      0   100%
modin/pandas/__init__.py                                              70     14    80%
modin/pandas/base.py                                                1028     51    95%
modin/pandas/concat.py                                                44      7    84%
modin/pandas/dataframe.py                                            736     88    88%
modin/pandas/datetimes.py                                              7      0   100%
modin/pandas/general.py                                               41      1    98%
modin/pandas/groupby.py                                              278     34    88%
modin/pandas/indexing.py                                             183     30    84%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17      0   100%
modin/pandas/reshape.py                                               30      0   100%
modin/pandas/series.py                                               757     55    93%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318   1050    83%

======== 12591 passed, 28 skipped, 19622 warnings in 776.89s (0:12:56) =========

Dask Tests
+ python3 -m pytest modin/pandas/test/test_io.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
collected 68 items

modin/pandas/test/test_io.py ..........s.........s...................... [ 63%]
....s.s.X......s.........                                                [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     95    53%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     38    59%
modin/backends/pandas/query_compiler.py                              505     28    94%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     22    83%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20      0   100%
modin/data_management/functions/foldfunction.py                        6      0   100%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      0   100%
modin/data_management/functions/mapreducefunction.py                   8      0   100%
modin/data_management/functions/reductionfunction.py                   6      0   100%
modin/data_management/utils.py                                        32      0   100%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     10    78%
modin/engines/base/frame/data.py                                     425     25    94%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179     20    89%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37      0   100%
modin/engines/base/io/column_stores/feather_reader.py                  9      0   100%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30      1    97%
modin/engines/base/io/file_reader.py                                  76      8    89%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43      2    95%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122      3    98%
modin/engines/base/io/text/json_reader.py                             45      2    96%
modin/engines/base/io/text/text_file_reader.py                        16      0   100%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63      4    94%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      0   100%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      0   100%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      3    77%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87      8    91%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     16    64%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      0   100%
modin/engines/ray/utils.py                                            11      2    82%
modin/error_message.py                                                22      3    86%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     21    50%
modin/experimental/engines/pandas_on_ray/sql.py                       70     49    30%
modin/experimental/pandas/__init__.py                                  6      0   100%
modin/experimental/pandas/io_exp.py                                    6      0   100%
modin/pandas/__init__.py                                              70     14    80%
modin/pandas/base.py                                                1028     51    95%
modin/pandas/concat.py                                                44      7    84%
modin/pandas/dataframe.py                                            736     88    88%
modin/pandas/datetimes.py                                              7      0   100%
modin/pandas/general.py                                               41      1    98%
modin/pandas/groupby.py                                              278     34    88%
modin/pandas/indexing.py                                             183     30    84%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17      0   100%
modin/pandas/reshape.py                                               30      0   100%
modin/pandas/series.py                                               757     55    93%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318    857    86%


============ 62 passed, 5 skipped, 1 xpassed, 73 warnings in 49.77s ============
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.38438.851873': JSONDecodeError: Expecting ',' delimiter: line 1 column 34053 (char 34052)
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.38710.745928': JSONDecodeError: Expecting ',' delimiter: line 1 column 8577 (char 8576)
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.39346.650032': JSONDecodeError: Expecting ',' delimiter: line 1 column 6263 (char 6262)
Closing remaining open files:test_write_pandas.hdf...donetest_write_modin.hdf...done
+ python3 -m pytest -n=48 modin/pandas/test/test_series.py modin/pandas/test/test_dataframe.py modin/pandas/test/test_concat.py modin/pandas/test/test_groupby.py modin/pandas/test/test_reshape.py modin/pandas/test/test_general.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [12619] / gw1 [12619] / gw2 [12619] / gw3 [12619] / gw4 [12619] / gw5 [12619] / gw6 [12619] / gw7 [12619] / gw8 [12619] / gw9 [12619] / gw10 [12619] / gw11 [12619] / gw12 [12619] / gw13 [12619] / gw14 [12619] / gw15 [12619] / gw16 [12619] / gw17 [12619] / gw18 [12619] / gw19 [12619] / gw20 [12619] / gw21 [12619] / gw22 [12619] / gw23 [12619] / gw24 [12619] / gw25 [12619] / gw26 [12619] / gw27 [12619] / gw28 [12619] / gw29 [12619] / gw30 [12619] / gw31 [12619] / gw32 [12619] / gw33 [12619] / gw34 [12619] / gw35 [12619] / gw36 [12619] / gw37 [12619] / gw38 [12619] / gw39 [12619] / gw40 [12619] / gw41 [12619] / gw42 [12619] / gw43 [12619] / gw44 [12619] / gw45 [12619] / gw46 [12619] / gw47 [12619]

........................................................................ [  0%]
........................................................................ [  1%]
......................................................................... [  1%]
........................................................................ [  2%]
........................................................................ [  2%]
........................................................................... [  3%]
........................................................................ [  4%]
.......................................................................s [  4%]
..................................................................s...... [  5%]
........................................................................ [  5%]
........................................................................ [  6%]
.......................................s................................ [  6%]
.........................................s............................... [  7%]
........................................................................ [  8%]
...............................s........................................ [  8%]
........................................................................ [  9%]
........................................................................ [  9%]
........................................................................ [ 10%]
........................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 12%]
........................................................................ [ 12%]
..................s..................................................... [ 13%]
........................................................................ [ 13%]
........................................................................ [ 14%]
........................................................................ [ 14%]
........................................................................ [ 15%]
........................................................................ [ 16%]
......................................................................... [ 16%]
........................................................................ [ 17%]
........................................................................ [ 17%]
........................................................................ [ 18%]
........................................................................ [ 18%]
........................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 20%]
........................................................................ [ 21%]
..........................................s............................. [ 21%]
........................................................................ [ 22%]
.........................ss.sssssssssssss............................... [ 22%]
........................................................................ [ 23%]
......................................................................... [ 24%]
........................................................................ [ 24%]
........................................................................ [ 25%]
........................................................................ [ 25%]
........................................................................ [ 26%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 28%]
........................................................................ [ 28%]
........................................................................ [ 29%]
.......................................................................... [ 29%]
........................................................................ [ 30%]
........................................................................ [ 30%]
........................................................................ [ 31%]
........................................................................ [ 32%]
........................................................................ [ 32%]
........................................................................ [ 33%]
........................................................................ [ 33%]
........................................................................ [ 34%]
......................................................................... [ 34%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 36%]
........................................................................ [ 37%]
........................................................................ [ 37%]
........................................................................ [ 38%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 40%]
........................................................................ [ 41%]
........................................................................ [ 41%]
........................................................................ [ 42%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 44%]
........................................................................ [ 44%]
........................................................................ [ 45%]
........................................................................ [ 45%]
........................................................................ [ 46%]
........................................................................ [ 46%]
........................................................................ [ 47%]
......................................................................... [ 48%]
........................................................................ [ 48%]
........................................................................ [ 49%]
........................................................................ [ 49%]
......................................................................... [ 50%]
......................................................................... [ 50%]
......................................................................... [ 51%]
........................................................................ [ 52%]
........................................................................ [ 52%]
........................................................................ [ 53%]
......................................................................... [ 53%]
........................................................................ [ 54%]
........................................................................ [ 54%]
........................................................................ [ 55%]
........................................................................ [ 56%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 57%]
......................................................................... [ 58%]
........................................................................ [ 58%]
........................................................................ [ 59%]
......................................................................... [ 60%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 61%]
........................................................................ [ 62%]
........................................................................ [ 62%]
......................................................................... [ 63%]
......................................................................... [ 64%]
........................................................................ [ 64%]
........................................................................ [ 65%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 69%]
........................................................................ [ 70%]
......................................................................... [ 70%]
......................................................................... [ 71%]
........................................................................ [ 72%]
........................................................................ [ 72%]
........................................................................ [ 73%]
......................................................................... [ 73%]
........................................................................ [ 74%]
........................................................................ [ 74%]
........................................................................ [ 75%]
........................................................................ [ 76%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 77%]
........................................................................ [ 78%]
........................................................................ [ 78%]
........................................................................ [ 79%]
........................................................................ [ 80%]
........................................................................ [ 80%]
......................................................................... [ 81%]
........................................................................ [ 81%]
........................................................................... [ 82%]
........................................................................ [ 82%]
......................................................................... [ 83%]
........................................................................ [ 84%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 88%]
........................................................................ [ 89%]
........................................................................ [ 89%]
........................................................................ [ 90%]
........................................................................ [ 90%]
........................................................................ [ 91%]
...................................................sssss................ [ 92%]
........................................................................... [ 92%]
..................s..................................................... [ 93%]
........................................................................ [ 93%]
........................................................................ [ 94%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 96%]
........................................................................ [ 97%]
........................................................................ [ 97%]
........................................................................ [ 98%]
........................................................................ [ 98%]
........................................................................ [ 99%]
............................................................             [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     95    53%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     38    59%
modin/backends/pandas/query_compiler.py                              505     28    94%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     22    83%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20      0   100%
modin/data_management/functions/foldfunction.py                        6      0   100%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      0   100%
modin/data_management/functions/mapreducefunction.py                   8      0   100%
modin/data_management/functions/reductionfunction.py                   6      0   100%
modin/data_management/utils.py                                        32      0   100%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     10    78%
modin/engines/base/frame/data.py                                     425     25    94%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179     20    89%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37      0   100%
modin/engines/base/io/column_stores/feather_reader.py                  9      0   100%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30      1    97%
modin/engines/base/io/file_reader.py                                  76      8    89%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43      2    95%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122      3    98%
modin/engines/base/io/text/json_reader.py                             45      2    96%
modin/engines/base/io/text/text_file_reader.py                        16      0   100%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63      4    94%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      0   100%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      0   100%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      3    77%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87      8    91%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     16    64%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      0   100%
modin/engines/ray/utils.py                                            11      2    82%
modin/error_message.py                                                22      3    86%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     21    50%
modin/experimental/engines/pandas_on_ray/sql.py                       70     49    30%
modin/experimental/pandas/__init__.py                                  6      0   100%
modin/experimental/pandas/io_exp.py                                    6      0   100%
modin/pandas/__init__.py                                              70     14    80%
modin/pandas/base.py                                                1028     51    95%
modin/pandas/concat.py                                                44      7    84%
modin/pandas/dataframe.py                                            736     88    88%
modin/pandas/datetimes.py                                              7      0   100%
modin/pandas/general.py                                               41      1    98%
modin/pandas/groupby.py                                              278     34    88%
modin/pandas/indexing.py                                             183     30    84%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17      0   100%
modin/pandas/reshape.py                                               30      0   100%
modin/pandas/series.py                                               757     55    93%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318    857    86%

======== 12591 passed, 28 skipped, 19621 warnings in 967.83s (0:16:07) =========
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.38438.851873': JSONDecodeError: Expecting ',' delimiter: line 1 column 34053 (char 34052)
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.38710.745928': JSONDecodeError: Expecting ',' delimiter: line 1 column 8577 (char 8576)
Coverage.py warning: Couldn't read data from '/localdisk/tc_agent/work/199b2674cba60ed6/.coverage.antsatel01.39346.650032': JSONDecodeError: Expecting ',' delimiter: line 1 column 6263 (char 6262)

Python Tests
+ python3 -m pytest modin/pandas/test/test_io.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
collected 68 items

modin/pandas/test/test_io.py ..........s.........s...........s.......... [ 63%]
....s.s.X......s.........                                                [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     96    52%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     69    26%
modin/backends/pandas/query_compiler.py                              505    316    37%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     26    80%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20     13    35%
modin/data_management/functions/foldfunction.py                        6      1    83%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      1    83%
modin/data_management/functions/mapreducefunction.py                   8      1    88%
modin/data_management/functions/reductionfunction.py                   6      1    83%
modin/data_management/utils.py                                        32     12    62%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     20    56%
modin/engines/base/frame/data.py                                     425    303    29%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179    115    36%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37     26    30%
modin/engines/base/io/column_stores/feather_reader.py                  9      5    44%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30     25    17%
modin/engines/base/io/file_reader.py                                  76     60    21%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43     35    19%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122    114     7%
modin/engines/base/io/text/json_reader.py                             45     38    16%
modin/engines/base/io/text/text_file_reader.py                        16      9    44%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63     13    79%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      6    50%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      9    65%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      4    69%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87     63    28%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     30    33%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      4    56%
modin/engines/ray/utils.py                                            11      9    18%
modin/error_message.py                                                22      5    77%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     29    31%
modin/experimental/engines/pandas_on_ray/sql.py                       70     70     0%
modin/experimental/pandas/__init__.py                                  6      6     0%
modin/experimental/pandas/io_exp.py                                    6      6     0%
modin/pandas/__init__.py                                              70     44    37%
modin/pandas/base.py                                                1028    793    23%
modin/pandas/concat.py                                                44     38    14%
modin/pandas/dataframe.py                                            736    561    24%
modin/pandas/datetimes.py                                              7      3    57%
modin/pandas/general.py                                               41     27    34%
modin/pandas/groupby.py                                              278    189    32%
modin/pandas/indexing.py                                             183    183     0%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17     11    35%
modin/pandas/reshape.py                                               30     20    33%
modin/pandas/series.py                                               757    517    32%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318   4143    34%


============ 61 passed, 6 skipped, 1 xpassed, 99 warnings in 16.93s ============
Closing remaining open files:test_write_pandas.hdf...donetest_write_modin.hdf...done
+ python3 -m pytest -n=48 modin/pandas/test/test_series.py modin/pandas/test/test_dataframe.py modin/pandas/test/test_concat.py modin/pandas/test/test_groupby.py modin/pandas/test/test_reshape.py modin/pandas/test/test_general.py
============================= test session starts ==============================
platform linux -- Python 3.7.5, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /localdisk/tc_agent/work/199b2674cba60ed6, inifile: pytest.ini
plugins: custom-exit-code-0.3.0, xdist-1.31.0, forked-1.1.3, testmon-1.0.2, cov-2.8.1
gw0 I / gw1 I / gw2 I / gw3 I / gw4 I / gw5 I / gw6 I / gw7 I / gw8 I / gw9 I / gw10 I / gw11 I / gw12 I / gw13 I / gw14 I / gw15 I / gw16 I / gw17 I / gw18 I / gw19 I / gw20 I / gw21 I / gw22 I / gw23 I / gw24 I / gw25 I / gw26 I / gw27 I / gw28 I / gw29 I / gw30 I / gw31 I / gw32 I / gw33 I / gw34 I / gw35 I / gw36 I / gw37 I / gw38 I / gw39 I / gw40 I / gw41 I / gw42 I / gw43 I / gw44 I / gw45 I / gw46 I / gw47 I
gw0 [12619] / gw1 [12619] / gw2 [12619] / gw3 [12619] / gw4 [12619] / gw5 [12619] / gw6 [12619] / gw7 [12619] / gw8 [12619] / gw9 [12619] / gw10 [12619] / gw11 [12619] / gw12 [12619] / gw13 [12619] / gw14 [12619] / gw15 [12619] / gw16 [12619] / gw17 [12619] / gw18 [12619] / gw19 [12619] / gw20 [12619] / gw21 [12619] / gw22 [12619] / gw23 [12619] / gw24 [12619] / gw25 [12619] / gw26 [12619] / gw27 [12619] / gw28 [12619] / gw29 [12619] / gw30 [12619] / gw31 [12619] / gw32 [12619] / gw33 [12619] / gw34 [12619] / gw35 [12619] / gw36 [12619] / gw37 [12619] / gw38 [12619] / gw39 [12619] / gw40 [12619] / gw41 [12619] / gw42 [12619] / gw43 [12619] / gw44 [12619] / gw45 [12619] / gw46 [12619] / gw47 [12619]

........................................................................ [  0%]
........................................................................ [  1%]
........................................................................ [  1%]
........................................................................ [  2%]
........................................................................ [  2%]
........................................................................ [  3%]
........................................................................ [  3%]
.....................................................s.................. [  4%]
......................................................................... [  5%]
......................................................................... [  5%]
................................................................s........ [  6%]
.......................................................................... [  6%]
........s...........................................s................... [  7%]
........................................................................ [  7%]
......................................................................... [  8%]
........................................................................ [  9%]
........................................................................ [  9%]
................................s........................................ [ 10%]
............................................................................ [ 10%]
........................................................................ [ 11%]
........................................................................ [ 11%]
........................................................................ [ 12%]
........................................................................ [ 13%]
........................................................................... [ 13%]
........................................................................ [ 14%]
........................................................................ [ 14%]
........................................................s.....s.......... [ 15%]
........................................................................ [ 16%]
........................................................................ [ 16%]
......................................................................... [ 17%]
......................................................................... [ 17%]
........................................................................ [ 18%]
.......................................................................... [ 18%]
............................................................................ [ 19%]
........................................................................ [ 20%]
........................................................................ [ 20%]
......................................................................... [ 21%]
........................................................................ [ 21%]
........................................................................ [ 22%]
........................................................................ [ 23%]
........................................................................ [ 23%]
........................................................................ [ 24%]
........................................................................ [ 24%]
......................................................................... [ 25%]
......................................................................... [ 25%]
........................................................................ [ 26%]
........................................................................ [ 27%]
........................................................................ [ 27%]
........................................................................ [ 28%]
......................................................................... [ 28%]
.............................................ssss.sssss.sss.s.s.s....... [ 29%]
........................................................................ [ 29%]
........................................................................ [ 30%]
........................................................................ [ 31%]
......................................................................... [ 31%]
........................................................................ [ 32%]
........................................................................ [ 32%]
......................................................................... [ 33%]
........................................................................ [ 33%]
........................................................................ [ 34%]
.......................................................................... [ 35%]
........................................................................ [ 35%]
........................................................................ [ 36%]
........................................................................ [ 36%]
........................................................................ [ 37%]
......................................................................... [ 37%]
........................................................................ [ 38%]
........................................................................ [ 39%]
........................................................................ [ 39%]
........................................................................ [ 40%]
........................................................................ [ 40%]
........................................................................ [ 41%]
......................................................................... [ 41%]
........................................................................ [ 42%]
........................................................................ [ 43%]
........................................................................ [ 43%]
........................................................................ [ 44%]
......................................................................... [ 44%]
........................................................................ [ 45%]
........................................................................ [ 45%]
........................................................................ [ 46%]
.......................................................................... [ 47%]
........................................................................ [ 47%]
........................................................................ [ 48%]
........................................................................ [ 48%]
......................................................................... [ 49%]
......................................................................... [ 49%]
......................................................................... [ 50%]
........................................................................ [ 51%]
......................................................................... [ 51%]
........................................................................ [ 52%]
........................................................................ [ 52%]
........................................................................ [ 53%]
........................................................................ [ 53%]
.......................................................................... [ 54%]
........................................................................ [ 55%]
......................................................................... [ 55%]
........................................................................ [ 56%]
........................................................................ [ 56%]
........................................................................ [ 57%]
........................................................................ [ 57%]
........................................................................ [ 58%]
........................................................................ [ 59%]
........................................................................ [ 59%]
........................................................................ [ 60%]
........................................................................ [ 60%]
........................................................................ [ 61%]
........................................................................ [ 61%]
........................................................................ [ 62%]
......................................................................... [ 63%]
........................................................................ [ 63%]
........................................................................ [ 64%]
......................................................................... [ 64%]
.......................................................................... [ 65%]
........................................................................ [ 65%]
........................................................................ [ 66%]
........................................................................ [ 67%]
........................................................................ [ 67%]
........................................................................ [ 68%]
........................................................................ [ 68%]
........................................................................ [ 69%]
........................................................................ [ 69%]
........................................................................ [ 70%]
........................................................................ [ 71%]
........................................................................ [ 71%]
......................................................................... [ 72%]
........................................................................ [ 72%]
......................................................................... [ 73%]
........................................................................ [ 73%]
......................................................................... [ 74%]
........................................................................ [ 75%]
........................................................................ [ 75%]
........................................................................ [ 76%]
........................................................................ [ 76%]
........................................................................ [ 77%]
........................................................................ [ 77%]
........................................................................ [ 78%]
......................................................................... [ 79%]
........................................................................ [ 79%]
......................................................................... [ 80%]
........................................................................ [ 80%]
........................................................................ [ 81%]
........................................................................ [ 82%]
........................................................................ [ 82%]
......................................................................... [ 83%]
........................................................................ [ 83%]
........................................................................ [ 84%]
........................................................................ [ 84%]
........................................................................ [ 85%]
........................................................................ [ 85%]
........................................................................ [ 86%]
........................................................................ [ 87%]
........................................................................ [ 87%]
........................................................................ [ 88%]
........................................................................ [ 88%]
......................................................................... [ 89%]
......................................................................... [ 90%]
........................................................................ [ 90%]
.......................................................................s [ 91%]
ss...s................................................................... [ 91%]
........................................................................ [ 92%]
............s........................................................... [ 92%]
...s.................................................................... [ 93%]
........................................................................ [ 94%]
........................................................................ [ 94%]
........................................................................ [ 95%]
........................................................................ [ 95%]
........................................................................ [ 96%]
........................................................................ [ 96%]
......................................................................... [ 97%]
........................................................................ [ 98%]
........................................................................ [ 98%]
........................................................................ [ 99%]
........................................................................ [ 99%]
..................................                                       [100%]

----------- coverage: platform linux, python 3.7.5-final-0 -----------
Name                                                               Stmts   Miss  Cover
--------------------------------------------------------------------------------------
modin/__init__.py                                                     39     15    62%
modin/_version.py                                                    276    154    44%
modin/apply_license_header.py                                         19     19     0%
modin/backends/__init__.py                                             0      0   100%
modin/backends/base/__init__.py                                        0      0   100%
modin/backends/base/query_compiler.py                                202     95    53%
modin/backends/pandas/__init__.py                                      0      0   100%
modin/backends/pandas/parsers.py                                      93     65    30%
modin/backends/pandas/query_compiler.py                              505     28    94%
modin/data_management/__init__.py                                      0      0   100%
modin/data_management/factories.py                                   133     26    80%
modin/data_management/functions/__init__.py                            6      0   100%
modin/data_management/functions/binary_function.py                    20      0   100%
modin/data_management/functions/foldfunction.py                        6      0   100%
modin/data_management/functions/function.py                            6      1    83%
modin/data_management/functions/mapfunction.py                         6      0   100%
modin/data_management/functions/mapreducefunction.py                   8      0   100%
modin/data_management/functions/reductionfunction.py                   6      0   100%
modin/data_management/utils.py                                        32      0   100%
modin/engines/__init__.py                                              0      0   100%
modin/engines/base/__init__.py                                         0      0   100%
modin/engines/base/frame/__init__.py                                   0      0   100%
modin/engines/base/frame/axis_partition.py                            45     10    78%
modin/engines/base/frame/data.py                                     425     25    94%
modin/engines/base/frame/partition.py                                  1      0   100%
modin/engines/base/frame/partition_manager.py                        179     20    89%
modin/engines/base/io/__init__.py                                     10      0   100%
modin/engines/base/io/column_stores/__init__.py                        0      0   100%
modin/engines/base/io/column_stores/column_store_reader.py            37     26    30%
modin/engines/base/io/column_stores/feather_reader.py                  9      5    44%
modin/engines/base/io/column_stores/hdf_reader.py                      3      0   100%
modin/engines/base/io/column_stores/parquet_reader.py                 30     25    17%
modin/engines/base/io/file_reader.py                                  76     60    21%
modin/engines/base/io/io.py                                          102      6    94%
modin/engines/base/io/sql/__init__.py                                  0      0   100%
modin/engines/base/io/sql/sql_reader.py                               43     35    19%
modin/engines/base/io/text/__init__.py                                 0      0   100%
modin/engines/base/io/text/csv_reader.py                             122    114     7%
modin/engines/base/io/text/json_reader.py                             45     38    16%
modin/engines/base/io/text/text_file_reader.py                        16      9    44%
modin/engines/base/series/__init__.py                                  0      0   100%
modin/engines/python/__init__.py                                       0      0   100%
modin/engines/python/pandas_on_python/__init__.py                      0      0   100%
modin/engines/python/pandas_on_python/frame/__init__.py                0      0   100%
modin/engines/python/pandas_on_python/frame/axis_partition.py         14      0   100%
modin/engines/python/pandas_on_python/frame/data.py                    4      0   100%
modin/engines/python/pandas_on_python/frame/partition.py              63      4    94%
modin/engines/python/pandas_on_python/frame/partition_manager.py       7      0   100%
modin/engines/python/pandas_on_python/io.py                            6      0   100%
modin/engines/python/pandas_on_python/series/__init__.py               0      0   100%
modin/engines/ray/__init__.py                                          0      0   100%
modin/engines/ray/generic/__init__.py                                  0      0   100%
modin/engines/ray/generic/frame/__init__.py                            0      0   100%
modin/engines/ray/generic/frame/partition_manager.py                  12      6    50%
modin/engines/ray/generic/io.py                                       14     10    29%
modin/engines/ray/generic/series/__init__.py                           0      0   100%
modin/engines/ray/pandas_on_ray/__init__.py                            0      0   100%
modin/engines/ray/pandas_on_ray/frame/__init__.py                      0      0   100%
modin/engines/ray/pandas_on_ray/frame/axis_partition.py               26      9    65%
modin/engines/ray/pandas_on_ray/frame/data.py                         13      4    69%
modin/engines/ray/pandas_on_ray/frame/partition.py                    87     63    28%
modin/engines/ray/pandas_on_ray/frame/partition_manager.py            45     30    33%
modin/engines/ray/pandas_on_ray/io.py                                 16      0   100%
modin/engines/ray/pandas_on_ray/series/__init__.py                     0      0   100%
modin/engines/ray/task_wrapper.py                                      9      4    56%
modin/engines/ray/utils.py                                            11      9    18%
modin/error_message.py                                                22      3    86%
modin/experimental/__init__.py                                         0      0   100%
modin/experimental/engines/__init__.py                                 0      0   100%
modin/experimental/engines/pandas_on_ray/__init__.py                   0      0   100%
modin/experimental/engines/pandas_on_ray/io_exp.py                    42     29    31%
modin/experimental/engines/pandas_on_ray/sql.py                       70     70     0%
modin/experimental/pandas/__init__.py                                  6      6     0%
modin/experimental/pandas/io_exp.py                                    6      6     0%
modin/pandas/__init__.py                                              70     44    37%
modin/pandas/base.py                                                1028     51    95%
modin/pandas/concat.py                                                44      7    84%
modin/pandas/dataframe.py                                            736     88    88%
modin/pandas/datetimes.py                                              7      0   100%
modin/pandas/general.py                                               41      1    98%
modin/pandas/groupby.py                                              278     34    88%
modin/pandas/indexing.py                                             183     30    84%
modin/pandas/io.py                                                   127      9    93%
modin/pandas/iterator.py                                              17      0   100%
modin/pandas/reshape.py                                               30      0   100%
modin/pandas/series.py                                               757     55    93%
modin/pandas/utils.py                                                 27      3    89%
--------------------------------------------------------------------------------------
TOTAL                                                               6318   1351    79%

======== 12591 passed, 28 skipped, 19531 warnings in 469.93s (0:07:49) =========

@simon-mo
Copy link
Collaborator Author

looks like it's passing the ray test? is there anyway to keep re-run the teamcity build multiple time?

@gshimansky
Copy link
Collaborator

looks like it's passing the ray test? is there anyway to keep re-run the teamcity build multiple time?

Yes Ray test passed with 48 workers (TeamCity system has 48 CPU cores). I don't know a way to repeat TC tests other than to restart them manually, but as I wrote before, I ran tests on another system with 100 workers and they all passed to, so I think this race condition is really fixed with this change.

Signed-off-by: Devin Petersohn <devin.petersohn@gmail.com>
Copy link
Collaborator

@devin-petersohn devin-petersohn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @simon-mo!

@devin-petersohn devin-petersohn merged commit 2f21b14 into modin-project:master Apr 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Race condition on imports with Ray
5 participants