Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: allow local constraints files #1105

Merged
merged 11 commits into from
May 23, 2024
Merged

feat: allow local constraints files #1105

merged 11 commits into from
May 23, 2024

Conversation

anesson-cs
Copy link
Collaborator

It is possible to use a constraints file from a local file by setting constraints_file_url parameter of a search plugin of a provider configuration with an absolute path or a path relative to the root of the project (/path/to/workspace/eodag)

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 19, 2024 •
edited
Loading

Test Results

    4 files  ±0      4 suites  ±0   5m 13s ⏱️ +5s
  531 tests +1    528 ✅ +1   3 💤 ±0  0 ❌ ±0 
2 124 runs  +4  2 030 ✅ +4  94 💤 ±0  0 ❌ ±0 

Results for commit 8e7e0a6. ± Comparison against base commit f343411.

♻️ This comment has been updated with latest results.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 19, 2024 •
edited
Loading

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         303      51  83.17%   60, 646-692, 794-845, 849
config.py                                      343      27  92.13%   81-83, 92, 100, 104-106, 179, 191, 446-448, 512-515, 562-563, 572-573, 652, 721-726, 728
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        203      32  84.24%   52-59, 63-65, 169-176, 260-261, 343, 372, 433, 447-450, 463, 487-490, 533-539
api/product/metadata_mapping.py                677      84  87.59%   67-69, 130-132, 233, 265-266, 312-313, 323-335, 337, 348, 354-366, 407-410, 447, 468-471, 494, 502-503, 579-580, 604-605, 611-614, 629-630, 779, 825, 999-1004, 1135, 1149-1169, 1189, 1194, 1304, 1326, 1340, 1353-1372, 1411, 1463, 1501-1505, 1524
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       3  86.96%   25, 48, 55
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           98      15  84.69%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       117      11  90.60%   40-42, 153, 159, 172-173, 184-189
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      10  80.00%   32-34, 51-52, 71, 80-83, 85, 92-95
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      57  78.08%   58-64, 145, 180, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       529     132  75.05%   85-91, 124, 203-213, 215-216, 251-254, 318-321, 323-324, 331-336, 354-369, 386-388, 400, 448, 455-461, 479, 493, 507, 515-517, 533-538, 549, 567, 609-613, 635, 675, 720, 734-740, 776-837, 855, 888-897, 923-924, 951-956, 962, 978, 995-996, 1023-1024, 1031, 1093-1099, 1154-1155, 1161, 1171, 1237, 1241, 1255-1271
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         133      14  89.47%   49-54, 108, 112, 275, 295, 380, 383-391
plugins/search/build_search_result.py          179      24  86.59%   62, 97, 142-143, 149, 160, 296-299, 333, 390-407, 469, 472, 482, 499, 527, 529
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          197      65  67.01%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 278-281, 289-300, 317, 319, 326-327, 329-330, 348-352, 385, 392, 403, 416, 422-434, 439
plugins/search/qssearch.py                     559      58  89.62%   96, 368-374, 382-383, 490-496, 551-554, 627-628, 669, 688, 703, 756, 777, 780-781, 790, 801, 810, 833, 895-900, 904-905, 933, 1004, 1051, 1127-1131, 1197-1198, 1219, 1244-1256, 1263, 1294-1295, 1306-1312, 1355, 1370, 1390, 1492
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   206      16  92.23%   66-72, 261, 345, 605, 607, 610-612, 684, 691-695
rest/server.py                                 299      54  81.94%   80-81, 107, 130-131, 244-246, 262, 302-303, 315-331, 423-428, 460, 627-634, 667, 711-712, 803-805, 822-827, 858, 860, 864-865, 869-870
rest/stac.py                                   436      86  80.28%   60-62, 272, 294, 366, 403-405, 428, 463-464, 550-595, 640, 660-661, 841, 906-908, 1127, 1137-1149, 1162-1184, 1198-1243, 1402-1403
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185       9  95.14%   52-55, 231-235, 288, 291, 359
rest/types/stac_queryables.py                   32       2  93.75%   28, 116
rest/types/stac_search.py                      122      10  91.80%   48-51, 167, 182-184, 192, 196
rest/utils/__init__.py                          96      13  86.46%   51, 108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       3  90.32%   78, 90, 92
types/__init__.py                              111      11  90.09%   53, 70, 129-132, 199, 214-218, 245, 258
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              502      42  91.63%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1102, 1186, 1204, 1380
utils/constraints.py                           132      42  68.18%   33, 93-102, 143, 148, 152, 163, 190-191, 202-210, 219, 233-249, 258-269
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               37      10  72.97%   47, 49, 51, 53, 55, 74, 79-81, 89
utils/rest.py                                   28       1  96.43%   49
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         9002    1554  82.74%

Diff against develop

Filename                  Stmts    Miss  Cover
----------------------  -------  ------  -------
config.py                    +1       0  +0.02%
plugins/search/base.py       +6      +5  -3.44%
types/__init__.py             0      -1  +0.90%
utils/constraints.py         +9       0  +2.33%
utils/requests.py           +37     +10  +72.97%
TOTAL                       +53     +14  -0.05%

Results for commit: 8e7e0a6

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Apr 19, 2024 •
edited
Loading

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                     11       0  100.00%
cli.py                                         303      51  83.17%   60, 646-692, 794-845, 849
config.py                                      343      28  91.84%   81-83, 92, 100, 104-106, 179, 191, 446-448, 512-515, 562-563, 572-573, 652, 686, 721-726, 728
crunch.py                                        6       6  0.00%    18-24
api/__init__.py                                  1       0  100.00%
api/core.py                                    739      76  89.72%   88-97, 370, 597, 641-644, 682, 787, 791-796, 822, 892, 969, 1082, 1172-1184, 1224, 1226, 1230, 1253, 1257-1268, 1281-1287, 1377-1380, 1409-1429, 1486, 1503-1506, 1518-1521, 1889, 1922-1928, 2193, 2197-2200, 2214-2216, 2251
api/search_result.py                            44       6  86.36%   33-35, 70, 79, 86, 100
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          44       5  88.64%   27-29, 79, 129
api/product/_product.py                        203      32  84.24%   52-59, 63-65, 169-176, 260-261, 343, 372, 433, 447-450, 463, 487-490, 533-539
api/product/metadata_mapping.py                677      85  87.44%   67-69, 130-132, 233, 265-266, 312-313, 323-335, 337, 348, 354-366, 407-410, 447, 468-471, 494, 502-503, 579-580, 604-605, 611-614, 629-630, 779, 825, 999-1004, 1135, 1149-1169, 1189, 1194, 1304, 1326, 1340, 1353-1372, 1411, 1463, 1486, 1501-1505, 1524
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      8       2  75.00%   23, 41
plugins/__init__.py                              1       0  100.00%
plugins/base.py                                 23       4  82.61%   25, 48, 55, 68
plugins/manager.py                             132      14  89.39%   49-51, 105-110, 156, 195, 215, 241, 280-281
plugins/apis/__init__.py                         1       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           98      15  84.69%   47-55, 156-158, 205-206, 232-234
plugins/apis/usgs.py                           172      36  79.07%   59-64, 128, 201, 235, 270-272, 277, 303-304, 309, 339-346, 357-362, 384-390, 392-398, 421
plugins/authentication/__init__.py               7       1  85.71%   31
plugins/authentication/aws_auth.py              20       2  90.00%   25-27
plugins/authentication/base.py                  19       3  84.21%   26, 34, 47
plugins/authentication/generic.py               16       3  81.25%   28, 40, 50
plugins/authentication/header.py                17       1  94.12%   27
plugins/authentication/keycloak.py              88      17  80.68%   32-34, 159-160, 190-212, 238-243
plugins/authentication/oauth.py                 15       8  46.67%   25, 32-34, 38-41
plugins/authentication/openid_connect.py       117      11  90.60%   40-42, 153, 159, 172-173, 184-189
plugins/authentication/qsauth.py                36       2  94.44%   32, 83
plugins/authentication/sas_auth.py              49       2  95.92%   32, 76
plugins/authentication/token.py                 90      17  81.11%   35-37, 79, 107, 109, 131-143, 198-201
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       1       0  100.00%
plugins/crunch/base.py                          10       2  80.00%   25, 38
plugins/crunch/filter_date.py                   62      15  75.81%   30, 53-58, 72, 81, 90, 93, 105-107, 116-118, 125
plugins/crunch/filter_latest_intersect.py       50      35  30.00%   32-34, 48-53, 69-114
plugins/crunch/filter_latest_tpl_name.py        33       2  93.94%   28, 86
plugins/crunch/filter_overlap.py                68      17  75.00%   28-30, 33, 82-85, 91, 99, 110-126
plugins/crunch/filter_property.py               33       8  75.76%   29, 60-65, 68-69, 85-89
plugins/download/__init__.py                     1       0  100.00%
plugins/download/aws.py                        492     165  66.46%   77-83, 272, 285, 352-355, 369-373, 419-421, 425, 459-460, 466-470, 503, 538, 542, 549, 579-587, 591, 629-637, 644-646, 687-761, 779-840, 851-856, 872-885, 914, 929-931, 934, 944-952, 960-973, 983-1002, 1009-1021, 1062, 1088, 1133-1135, 1355
plugins/download/base.py                       260      59  77.31%   58-64, 145, 180, 250-252, 319-320, 340-346, 377-381, 387-388, 432, 435-449, 461, 465, 538-542, 572-573, 581-598, 605-613, 615-619, 666, 688, 710, 718
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       529     132  75.05%   85-91, 124, 203-213, 215-216, 251-254, 318-321, 323-324, 331-336, 354-369, 386-388, 400, 448, 455-461, 479, 493, 507, 515-517, 533-538, 549, 567, 609-613, 635, 675, 720, 734-740, 776-837, 855, 888-897, 923-924, 951-956, 962, 978, 995-996, 1023-1024, 1031, 1093-1099, 1154-1155, 1161, 1171, 1237, 1241, 1255-1271
plugins/download/s3rest.py                     118      27  77.12%   55-58, 121, 162, 197, 227-234, 237-239, 243, 256-262, 270-271, 274-278, 301, 322-325
plugins/search/__init__.py                       1       0  100.00%
plugins/search/base.py                         133      15  88.72%   49-54, 108, 112, 275, 295, 380, 383-391, 393
plugins/search/build_search_result.py          179      31  82.68%   62, 97, 142-143, 149, 160, 296-299, 333, 390-407, 469, 472, 482, 499, 519-534
plugins/search/creodias_s3.py                   54       3  94.44%   55, 73, 107
plugins/search/csw.py                          107      83  22.43%   43-45, 57-58, 62-63, 74-122, 128-141, 149-181, 199-240
plugins/search/data_request_search.py          197      65  67.01%   52, 89-92, 108, 120, 124-125, 139, 144, 149, 156, 169-172, 226-227, 231, 241-247, 252, 278-281, 289-300, 317, 319, 326-327, 329-330, 348-352, 385, 392, 403, 416, 422-434, 439
plugins/search/qssearch.py                     559      82  85.33%   96, 368-374, 382-383, 490-496, 551-554, 627-628, 669, 688, 703, 756, 777, 780-781, 790, 801, 810, 833, 895-900, 904-905, 933, 1004, 1051, 1127-1131, 1197-1198, 1219, 1244-1256, 1263, 1294-1295, 1306-1312, 1355, 1370, 1390, 1452-1521
plugins/search/static_stac_search.py            47       3  93.62%   39-40, 82
rest/__init__.py                                 5       2  60.00%   21-22
rest/core.py                                   206      95  53.88%   66-72, 153, 155, 157, 163-164, 185-193, 199-205, 252-306, 319-355, 505-536, 554, 604-643, 684, 691-695
rest/server.py                                 299     299  0.00%    18-883
rest/stac.py                                   436     145  66.74%   60-62, 214, 272, 294, 366, 403-405, 428, 463-464, 550-595, 640, 648-649, 653-661, 783, 841, 906-908, 925-927, 935-937, 950-952, 966-983, 993-1014, 1024-1046, 1054-1071, 1094-1117, 1127, 1137-1149, 1162-1184, 1198-1243, 1396-1422
rest/types/__init__.py                           1       0  100.00%
rest/types/eodag_search.py                     185      18  90.27%   52-55, 231-235, 268-270, 288, 291, 297, 301, 359, 371-374
rest/types/stac_queryables.py                   32       6  81.25%   28, 55-60, 116
rest/types/stac_search.py                      122      12  90.16%   48-51, 167, 182-184, 192, 196, 240, 243
rest/utils/__init__.py                          96      31  67.71%   51, 79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           31       5  83.87%   73-74, 78, 90, 92
types/__init__.py                              111      36  67.57%   53, 66-70, 81-93, 120-122, 129-132, 172, 199, 209-221, 226, 245, 250, 258, 268
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 104
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              502      43  91.43%   84, 89, 110-112, 193-194, 203-230, 233, 247, 329-333, 409-413, 434-436, 518, 533, 571-572, 968-971, 979-980, 1021-1022, 1102, 1185-1186, 1204, 1380
utils/constraints.py                           132      42  68.18%   33, 93-102, 143, 148, 152, 163, 190-191, 202-210, 219, 233-249, 258-269
utils/exceptions.py                             37       2  94.59%   23, 93
utils/import_system.py                          30      20  33.33%   27, 67-81, 93-103
utils/logging.py                                29       1  96.55%   123
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/requests.py                               37      10  72.97%   47, 49, 51, 53, 55, 74, 79-81, 89
utils/rest.py                                   28       1  96.43%   49
utils/stac_reader.py                            91      28  69.23%   55-56, 63-86, 93-95, 99, 141, 155-158
TOTAL                                         9002    2060  77.12%

Diff against develop

Filename                  Stmts    Miss  Cover
----------------------  -------  ------  -------
config.py                    +1       0  +0.03%
plugins/search/base.py       +6      +6  -4.19%
utils/constraints.py         +9       0  +2.33%
utils/requests.py           +37     +10  +72.97%
TOTAL                       +53     +16  -0.04%

Results for commit: 8e7e0a6

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

sbrunato
sbrunato requested changes May 14, 2024
View reviewed changes
Copy link
Collaborator

@sbrunato sbrunato left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • please add a test with a provider configured to use a local constraints file

@sbrunato sbrunato marked this pull request as draft May 14, 2024 09:15
@anesson-cs anesson-cs force-pushed the local-constraints-file branch 2 times, most recently from 1b0c212 to c6d830d Compare May 15, 2024 15:46
@anesson-cs anesson-cs force-pushed the local-constraints-file branch from c6d830d to 635fdd3 Compare May 16, 2024 14:11
@anesson-cs anesson-cs marked this pull request as ready for review May 16, 2024 14:15
@alambare alambare requested a review from sbrunato May 23, 2024 08:23
@sbrunato sbrunato force-pushed the local-constraints-file branch from 20bcf0d to 8e7e0a6 Compare May 23, 2024 12:39
@sbrunato sbrunato merged commit 59ccd8b into develop May 23, 2024
10 checks passed
@sbrunato sbrunato deleted the local-constraints-file branch May 23, 2024 12:45
@sbrunato sbrunato mentioned this pull request May 24, 2024
Merged
@sbrunato sbrunato added this to the 3.0.0b1 milestone Jul 1, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbrunato sbrunato sbrunato requested changes

Assignees

@anesson-cs anesson-cs

Labels
None yet
Projects
None yet
Milestone
3.0.0b1
Development

Successfully merging this pull request may close these issues.
2 participants
@anesson-cs @sbrunato