Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

build: refactor and new optional dependencies #1108

Merged
merged 7 commits into from
Apr 25, 2024
Merged

build: refactor and new optional dependencies #1108

merged 7 commits into from
Apr 25, 2024

Conversation

sbrunato
Copy link
Collaborator

@sbrunato sbrunato commented Apr 23, 2024 •
edited
Loading

Fixes #461

Add new extras and move some required dependencies to optional dependencies.
This will make basic installation way lighter.

New available extras:

  • eodag[all], including everything that would be needed to run eodag and associated tutorials with all features
  • eodag[all-providers], including dependencies required to have all providers available
  • eodag[aws], including dependencies for plugins using Amazon S3
  • eodag[csw], including dependencies for plugins using CSW
  • eodag[ecmwf], including dependencies for EcmwfApi
  • eodag[usgs], including dependencies for UsgsApi
  • eodag[server], including dependencies for server-mode

@sbrunato sbrunato changed the title build: refactor and add new optional dependencies build: refactor and add optional dependencies Apr 23, 2024
@sbrunato sbrunato changed the title build: refactor and add optional dependencies build: refactor and new optional dependencies Apr 23, 2024
@sbrunato sbrunato marked this pull request as draft April 23, 2024 15:47
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 23, 2024 •
edited
Loading

Test Results

    4 files  ±0      4 suites  ±0   4m 29s ⏱️ - 1m 0s
  507 tests +2    504 ✅ +2   3 💤 ±0  0 ❌ ±0 
2 028 runs  +8  1 940 ✅ +8  88 💤 ±0  0 ❌ ±0 

Results for commit 42c60d1. ± Comparison against base commit e619c88.

This pull request removes 1 and adds 3 tests. Note that renamed tests count towards both. 
tests.test_requirements.TestRequirements ‑ test_requirements

tests.test_requirements.TestRequirements ‑ test_all_requirements
tests.test_requirements.TestRequirements ‑ test_plugins_extras
tests.units.test_core.TestCore ‑ test_prune_providers_list_skipped_plugin

♻️ This comment has been updated with latest results.

@sbrunato sbrunato self-assigned this Apr 24, 2024
@sbrunato sbrunato force-pushed the extra-reqs-refactor branch from 84cd12a to e87fc11 Compare April 24, 2024 13:18
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 24, 2024 •
edited
Loading

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      318      27  91.51%   80-82, 91, 99, 103-105, 178, 190, 391-393, 457-460, 507-508, 517-518, 597, 666-671, 673
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        211      32  84.83%   53-60, 64-66, 170-177, 261-262, 352, 388, 449, 463-466, 479, 503-506, 549-555
api/product/metadata_mapping.py                649      82  87.37%   67-69, 130-132, 233, 265-266, 318-330, 332, 343, 349-361, 402-405, 442, 463-466, 489, 497-498, 571-572, 596-597, 603-606, 621-622, 771, 817, 934-939, 1070, 1084-1104, 1124, 1129, 1239, 1261, 1275, 1288-1307, 1346, 1398, 1436-1440, 1459
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 81      13  83.95%   35-37, 86, 99, 101, 114-117, 165-168
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       261      57  78.16%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       458      94  79.48%   82-88, 122, 196-208, 211, 292-297, 321, 323, 387, 414-416, 426, 432, 434, 458, 497-501, 558, 644-700, 718, 751-760, 786-787, 814, 838, 846-851, 873-874, 881, 942-948, 1003-1004, 1010, 1020, 1086, 1090, 1104-1120
plugins/download/s3rest.py                     117      27  76.92%   55-58, 124, 165, 199, 229-236, 239-241, 245, 258-264, 272-273, 276-280, 303, 324-327
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      22  87.71%   62, 97, 142-143, 149, 160, 290-293, 327, 384-396, 458, 461, 471, 488, 516, 518
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          198      65  67.17%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     548      54  90.15%   89, 361-367, 375-376, 483-489, 544-547, 620-621, 662, 680, 695, 748, 769, 772-773, 782, 793, 802, 825, 885-890, 894-895, 923, 994, 1041, 1117-1121, 1187-1188, 1209, 1234, 1240, 1271-1272, 1282-1288, 1331, 1345, 1365, 1455
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   199      15  92.46%   71-77, 261, 588, 590, 593-595, 667, 674-678
rest/server.py                                 299      54  81.94%   80-81, 107, 130-131, 242-244, 260, 300-301, 313-329, 415-420, 451, 612-619, 651, 694-695, 783-785, 802-807, 837, 839, 843-844, 848-849
rest/stac.py                                   436      95  78.21%   59-61, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 659-660, 840, 905-907, 1126, 1136-1148, 1161-1183, 1197-1242, 1401-1402
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185       9  95.14%   52-55, 231-235, 288, 291, 359
rest/types/stac_queryables.py                   30       2  93.33%   28, 114
rest/types/stac_search.py                      122      10  91.80%   48-51, 167, 182-184, 192, 196
rest/utils/__init__.py                         118      14  88.14%   53, 108-109, 128-130, 183, 193-207, 239
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                               76       6  92.11%   53, 87, 154, 174, 179, 187
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              496      43  91.33%   83, 88, 109-111, 189-190, 199-226, 229, 243, 325-329, 405-409, 430-432, 514, 519, 529, 567-568, 964-967, 975-976, 1017-1018, 1098, 1182, 1200, 1375
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8707    1523  82.51%

Diff against develop

Filename                      Stmts    Miss  Cover
--------------------------  -------  ------  -------
api/core.py                      +4       0  +0.06%
plugins/manager.py               +5       0  +0.41%
plugins/search/qssearch.py      -13     -11  +1.74%
rest/__init__.py                 +4      +2  -40.00%
TOTAL                             0      -9  +0.11%

Results for commit: 42c60d1

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@sbrunato sbrunato force-pushed the extra-reqs-refactor branch from 4869d2e to 42c60d1 Compare April 24, 2024 13:59
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 24, 2024 •
edited
Loading

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         300      47  84.33%   61, 647-686, 788-839, 843
config.py                                      318      28  91.19%   80-82, 91, 99, 103-105, 178, 190, 391-393, 457-460, 507-508, 517-518, 597, 631, 666-671, 673
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        211      32  84.83%   53-60, 64-66, 170-177, 261-262, 352, 388, 449, 463-466, 479, 503-506, 549-555
api/product/metadata_mapping.py                649      83  87.21%   67-69, 130-132, 233, 265-266, 318-330, 332, 343, 349-361, 402-405, 442, 463-466, 489, 497-498, 571-572, 596-597, 603-606, 621-622, 771, 817, 934-939, 1070, 1084-1104, 1124, 1129, 1239, 1261, 1275, 1288-1307, 1346, 1398, 1421, 1436-1440, 1459
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           99      15  84.85%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       103      64  37.86%   39-41, 138-150, 154-172, 180-222, 228-237, 246-286, 291-299, 304-305
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 81      13  83.95%   35-37, 86, 99, 101, 114-117, 165-168
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        491     165  66.40%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 458-459, 465-469, 502, 537, 541, 548, 578-586, 590, 628-636, 643-645, 686-760, 778-839, 850-855, 871-884, 913, 928-930, 933, 943-951, 959-972, 982-1001, 1008-1020, 1061, 1087, 1132-1134, 1354
plugins/download/base.py                       261      59  77.39%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 689, 711, 719
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       458      94  79.48%   82-88, 122, 196-208, 211, 292-297, 321, 323, 387, 414-416, 426, 432, 434, 458, 497-501, 558, 644-700, 718, 751-760, 786-787, 814, 838, 846-851, 873-874, 881, 942-948, 1003-1004, 1010, 1020, 1086, 1090, 1104-1120
plugins/download/s3rest.py                     117      27  76.92%   55-58, 124, 165, 199, 229-236, 239-241, 245, 258-264, 272-273, 276-280, 303, 324-327
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         127       9  92.91%   49-54, 108, 112, 275, 295, 378
plugins/search/build_search_result.py          179      29  83.80%   62, 97, 142-143, 149, 160, 290-293, 327, 384-396, 458, 461, 471, 488, 508-523
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          198      65  67.17%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 280-283, 291-302, 319, 321, 328-329, 331-332, 350-354, 387, 394, 405, 418, 424-436, 441
plugins/search/qssearch.py                     548      78  85.77%   89, 361-367, 375-376, 483-489, 544-547, 620-621, 662, 680, 695, 748, 769, 772-773, 782, 793, 802, 825, 885-890, 894-895, 923, 994, 1041, 1117-1121, 1187-1188, 1209, 1234, 1240, 1271-1272, 1282-1288, 1331, 1345, 1365, 1415-1484
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   199      88  55.78%   71-77, 158, 160, 162, 168-169, 186-194, 200-206, 253-302, 315-342, 492-519, 537, 587-626, 667, 674-678
rest/server.py                                 299     299  0.00%    18-862
rest/stac.py                                   436     154  64.68%   59-61, 214, 229-231, 273, 286-295, 314-320, 365, 402-404, 427, 462-463, 549-594, 639, 647-648, 652-660, 782, 840, 905-907, 924-926, 934-936, 949-951, 965-982, 992-1013, 1023-1045, 1053-1070, 1093-1116, 1126, 1136-1148, 1161-1183, 1197-1242, 1395-1421
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185      18  90.27%   52-55, 231-235, 268-270, 288, 291, 297, 301, 359, 371-374
rest/types/stac_queryables.py                   30       6  80.00%   28, 53-58, 114
rest/types/stac_search.py                      122      12  90.16%   48-51, 167, 182-184, 192, 196, 240, 243
rest/utils/__init__.py                         118      32  72.88%   53, 79-85, 105, 108-109, 128-130, 143, 150, 176-184, 191-212, 239
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       5  83.87%   73-74, 78, 90, 92
types/__init__.py                               76      11  85.53%   53, 87, 127, 154, 164-166, 174, 179, 187, 197
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              496      44  91.13%   83, 88, 109-111, 189-190, 199-226, 229, 243, 325-329, 405-409, 430-432, 514, 519, 529, 567-568, 964-967, 975-976, 1017-1018, 1098, 1181-1182, 1200, 1375
utils/constraints.py                           123      42  65.85%   31, 84-93, 134, 139, 143, 154, 177-178, 189-197, 206, 220-236, 245-256
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         8707    2002  77.01%

Diff against develop

Filename                      Stmts    Miss  Cover
--------------------------  -------  ------  -------
api/core.py                      +4       0  +0.06%
plugins/manager.py               +5       0  +0.41%
plugins/search/qssearch.py      -13     -11  +1.63%
rest/__init__.py                 +4      +2  -40.00%
TOTAL                             0      -9  +0.11%

Results for commit: 42c60d1

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@sbrunato sbrunato marked this pull request as ready for review April 24, 2024 14:17
@sbrunato sbrunato merged commit 3ebcffd into develop Apr 25, 2024
10 checks passed
@sbrunato sbrunato deleted the extra-reqs-refactor branch April 25, 2024 07:41
@sbrunato sbrunato mentioned this pull request Jun 17, 2024
Merged
@sbrunato sbrunato added this to the 3.0.0b1 milestone Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@sbrunato sbrunato

Labels
None yet
Projects
None yet
Milestone
3.0.0b1
Development

Successfully merging this pull request may close these issues.

move external python apis and server to extras_require
1 participant
@sbrunato