Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix(plugins): cop_marine - handling of ids with points #1364

Merged
merged 1 commit into from
Oct 31, 2024
Merged

fix(plugins): cop_marine - handling of ids with points #1364

merged 1 commit into from
Oct 31, 2024

Conversation

jlahovnik
Copy link
Collaborator

@jlahovnik jlahovnik commented Oct 23, 2024 •
edited
Loading

This PR fixes a problem with the ids of cop_marine products. If the filename of a product in the cop_marine s3 contained a "." somewhere in the middle (for example ids of product type MO_GLOBAL_MULTIYEAR_BGC_001_033), the id created from the filename was not correct (only the part until the "." was used). This is now fixed.

@jlahovnik jlahovnik self-assigned this Oct 23, 2024
@github-actions GitHub Actions
Copy link
Contributor

Test Results

    4 files  ±0      4 suites  ±0   5m 52s ⏱️ -2s
  571 tests ±0    568 ✅ ±0   3 💤 ±0  0 ❌ ±0 
2 284 runs  ±0  2 188 ✅ ±0  96 💤 ±0  0 ❌ ±0 

Results for commit accf283. ± Comparison against base commit 10923f0.

@github-actions GitHub Actions
Copy link
Contributor

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         314      59  81.21%   657-716, 818-869, 873
config.py                                      425      27  93.65%   83-85, 94, 102, 106-108, 179, 190, 684-686, 799-802, 845-846, 855-856, 961, 1024-1029, 1031
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    767      69  91.00%   374, 664, 708-711, 749, 793, 827, 872-877, 903, 994, 1062, 1200, 1285-1297, 1333, 1335, 1363, 1367-1378, 1391-1397, 1480-1483, 1516-1536, 1588, 1605-1609, 1621-1624, 1980, 2004-2010, 2261, 2265-2269, 2278-2280, 2312
api/search_result.py                            58       4  93.10%   92, 101, 108, 122
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                680      68  90.00%   130-132, 227, 259-260, 306-307, 317-329, 331, 342, 348-360, 405-406, 443, 464-467, 490, 498-499, 575-576, 600-601, 607-610, 625-626, 775, 821, 992-997, 1124, 1138-1158, 1178, 1183, 1312, 1326, 1397, 1449, 1489-1493, 1508
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             172      15  91.28%   116-121, 171, 209, 231, 235, 259, 281, 399-402, 414-415
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   159-161, 208-209, 235-237
plugins/apis/usgs.py                           180      31  82.78%   155, 257, 291, 326-328, 333, 359-360, 365, 395-402, 413-418, 440-446, 448-454, 477
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   43, 56
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              48       4  91.67%   153, 179-184
plugins/authentication/oauth.py                 13       7  46.15%   42-44, 48-51
plugins/authentication/openid_connect.py       187      17  90.91%   118, 132-158, 166, 300-303, 329
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 89      16  82.02%   101, 130, 132, 155-168, 224-228
plugins/authentication/token_exchange.py        36      20  44.44%   72-78, 90-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   52-53, 69, 78-81, 83, 90-93
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       7  76.67%   55-60, 63-64, 80-84
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     163  66.80%   282, 295, 362-365, 379-383, 425-427, 431, 463-464, 470-474, 503, 535, 539, 546, 576-584, 588, 620-628, 639-641, 672-746, 764-824, 835-840, 852-865, 890, 905-907, 910, 920-928, 936-949, 959-990, 997-1009, 1047, 1073, 1118-1120, 1340
plugins/download/base.py                       253      51  79.84%   136, 164, 296-297, 314-320, 351-355, 361-362, 404, 407-421, 433, 437, 501-505, 535-536, 544-561, 568-576, 578-582, 625, 647, 669, 677
plugins/download/creodias_s3.py                 17       9  47.06%   53-67
plugins/download/http.py                       534     120  77.53%   234, 330-333, 335-336, 343-348, 366-381, 398-400, 412, 460, 467-473, 491, 505, 519, 527-529, 545-550, 561, 579, 621-625, 647, 687, 732, 746-752, 781-845, 863, 893-902, 924-925, 954-959, 965, 968, 985, 1002-1003, 1033-1034, 1041, 1102-1108, 1163-1164, 1170, 1180, 1216, 1252, 1270-1283, 1309-1311
plugins/download/s3rest.py                     116      24  79.31%   118, 154, 161, 196, 223-230, 233-235, 239, 250-256, 264-265, 268-272, 295, 316-319
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         132      13  90.15%   100, 104, 115, 274, 294, 350-351, 371, 374-382
plugins/search/build_search_result.py          188      24  87.23%   100, 141-142, 148, 159, 292-295, 324, 359-376, 425, 454, 457, 467, 484, 512, 514
plugins/search/cop_marine.py                   240      47  80.42%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 218, 224, 228, 232, 245, 256-257, 265, 293, 297, 312, 316, 320, 324, 328-332, 338-341, 344-358, 375, 424-428, 433, 445
plugins/search/creodias_s3.py                   55       3  94.55%   58, 76, 110
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   188-191, 207, 218, 222-223, 234, 239, 244, 251, 264-267, 321-322, 326, 336-342, 347, 373-376, 384-395, 412, 414, 421-424, 426-427, 445-449, 482, 492, 503, 516, 522-537, 542
plugins/search/qssearch.py                     762     103  86.48%   444, 502, 516, 520-526, 554-558, 650-662, 706, 722, 732, 751-766, 803-806, 877-878, 926, 945, 952, 964, 1021, 1042, 1045-1046, 1055-1056, 1065-1066, 1075-1076, 1103, 1174-1179, 1183-1192, 1226, 1248, 1308, 1398, 1485, 1488-1489, 1571-1575, 1637, 1640, 1646-1647, 1668, 1695-1707, 1714, 1746-1748, 1758-1764, 1794, 1817, 1822-1823, 1838, 1854, 1939-1942, 1947-1950, 1959, 1979-1980, 1989-1993, 1999
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  25       0  100.00%
rest/constants.py                                6       0  100.00%
rest/core.py                                   251      63  74.90%   216-217, 273, 281, 299-316, 331-369, 460, 502-533, 680, 687-735
rest/errors.py                                  69       5  92.75%   106, 116, 127, 143-144
rest/server.py                                 188      24  87.23%   89, 112-114, 277-282, 310, 495-497, 514-519, 548, 550, 554-555, 559-560
rest/stac.py                                   319      63  80.25%   306, 328, 380-383, 410-437, 468-470, 493, 525-526, 608-648, 670-686, 778-782, 789, 843-844, 905, 995-997
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     176       5  97.16%   225-229, 282, 285, 353
rest/types/queryables.py                        55       1  98.18%   163
rest/types/stac_search.py                      125       7  94.40%   138, 184, 199-201, 209, 213
rest/utils/__init__.py                          93      12  87.10%   108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              114      14  87.72%   65, 82, 141-144, 211, 225-234, 244, 265, 278
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             80       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              499      36  92.79%   85, 189-190, 199-226, 229, 244, 324-328, 401-405, 424-426, 505, 520, 560-561, 925-928, 936-937, 975-976, 1147
utils/constraints.py                           119      38  68.07%   62, 89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             41       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 86, 88, 90, 92, 94, 110, 118-120, 128
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9692    1597  83.52%

Diff against develop

Filename                        Stmts    Miss  Cover
----------------------------  -------  ------  --------
plugins/search/cop_marine.py       +1       0  +0.09%
TOTAL                              +1       0  +100.00%

Results for commit: accf283

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@github-actions GitHub Actions
Copy link
Contributor

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         314      59  81.21%   657-716, 818-869, 873
config.py                                      425      28  93.41%   83-85, 94, 102, 106-108, 179, 190, 684-686, 799-802, 845-846, 855-856, 961, 992, 1024-1029, 1031
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    767      69  91.00%   374, 664, 708-711, 749, 793, 827, 872-877, 903, 994, 1062, 1200, 1285-1297, 1333, 1335, 1363, 1367-1378, 1391-1397, 1480-1483, 1516-1536, 1588, 1605-1609, 1621-1624, 1980, 2004-2010, 2261, 2265-2269, 2278-2280, 2312
api/search_result.py                            58       4  93.10%   92, 101, 108, 122
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                680      68  90.00%   130-132, 227, 259-260, 306-307, 317-329, 331, 342, 348-360, 405-406, 443, 464-467, 490, 498-499, 575-576, 600-601, 607-610, 625-626, 775, 821, 992-997, 1124, 1138-1158, 1178, 1183, 1312, 1326, 1397, 1449, 1489-1493, 1508
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             172      15  91.28%   116-121, 171, 209, 231, 235, 259, 281, 399-402, 414-415
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   159-161, 208-209, 235-237
plugins/apis/usgs.py                           180      31  82.78%   155, 257, 291, 326-328, 333, 359-360, 365, 395-402, 413-418, 440-446, 448-454, 477
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   43, 56
plugins/authentication/generic.py               14       2  85.71%   51, 61
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              48       4  91.67%   153, 179-184
plugins/authentication/oauth.py                 13       7  46.15%   42-44, 48-51
plugins/authentication/openid_connect.py       187      17  90.91%   118, 132-158, 166, 300-303, 329
plugins/authentication/qsauth.py                34       1  97.06%   91
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 89      16  82.02%   101, 130, 132, 155-168, 224-228
plugins/authentication/token_exchange.py        36      20  44.44%   72-78, 90-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   43
plugins/crunch/filter_date.py                   59      14  76.27%   52-57, 69, 78, 87, 90, 102-104, 113-115, 122
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   49-54, 67-112
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 66-69, 76-79, 85, 93, 104-120
plugins/crunch/filter_property.py               30       7  76.67%   55-60, 63-64, 80-84
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        491     163  66.80%   282, 295, 362-365, 379-383, 425-427, 431, 463-464, 470-474, 503, 535, 539, 546, 576-584, 588, 620-628, 639-641, 672-746, 764-824, 835-840, 852-865, 890, 905-907, 910, 920-928, 936-949, 959-990, 997-1009, 1047, 1073, 1118-1120, 1340
plugins/download/base.py                       253      53  79.05%   136, 164, 231-233, 296-297, 314-320, 351-355, 361-362, 404, 407-421, 433, 437, 501-505, 535-536, 544-561, 568-576, 578-582, 625, 647, 669, 677
plugins/download/creodias_s3.py                 17       9  47.06%   53-67
plugins/download/http.py                       534     121  77.34%   234, 330-333, 335-336, 343-348, 366-381, 398-400, 412, 460, 467-473, 491, 505, 519, 527-529, 545-550, 561, 579, 621-625, 647, 687, 732, 746-752, 781-845, 863, 893-902, 924-925, 954-959, 965, 968, 985, 1002-1003, 1016, 1033-1034, 1041, 1102-1108, 1163-1164, 1170, 1180, 1216, 1252, 1270-1283, 1309-1311
plugins/download/s3rest.py                     116      24  79.31%   118, 154, 161, 196, 223-230, 233-235, 239, 250-256, 264-265, 268-272, 295, 316-319
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         132      14  89.39%   100, 104, 115, 274, 294, 350-351, 371, 374-382, 384
plugins/search/build_search_result.py          188      31  83.51%   100, 141-142, 148, 159, 292-295, 324, 359-376, 425, 454, 457, 467, 484, 504-519
plugins/search/cop_marine.py                   240      47  80.42%   56, 64-66, 76-77, 82, 87-88, 104, 106, 109, 175-176, 218, 224, 228, 232, 245, 256-257, 265, 293, 297, 312, 316, 320, 324, 328-332, 338-341, 344-358, 375, 424-428, 433, 445
plugins/search/creodias_s3.py                   55       3  94.55%   58, 76, 110
plugins/search/csw.py                          105      81  22.86%   98-99, 103-104, 112-160, 166-179, 187-219, 237-278
plugins/search/data_request_search.py          202      69  65.84%   188-191, 207, 218, 222-223, 234, 239, 244, 251, 264-267, 321-322, 326, 336-342, 347, 373-376, 384-395, 412, 414, 421-424, 426-427, 445-449, 482, 492, 503, 516, 522-537, 542
plugins/search/qssearch.py                     762     135  82.28%   444, 502, 516, 520-526, 554-558, 650-662, 706, 709, 722, 732, 751-766, 803-806, 877-878, 926, 945, 952, 964, 1021, 1042, 1045-1046, 1055-1056, 1065-1066, 1075-1076, 1103, 1174-1179, 1183-1192, 1226, 1248, 1308, 1398, 1485, 1488-1489, 1571-1575, 1637, 1640, 1646-1647, 1668, 1695-1707, 1714, 1746-1748, 1758-1764, 1794, 1817, 1822-1823, 1838, 1854, 1922-2028
plugins/search/static_stac_search.py            75      13  82.67%   98-124, 140, 153
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  25       1  96.00%   35
rest/constants.py                                6       0  100.00%
rest/core.py                                   251     139  44.62%   162, 164, 166, 169-170, 184-194, 206-207, 209-210, 216-217, 220, 223, 264-318, 331-369, 400-434, 449-465, 481-490, 502-533, 550, 592-641, 680, 687-735
rest/errors.py                                  69      49  28.99%   60, 65-100, 105-108, 115-118, 126-147, 155-160, 175-181
rest/server.py                                 188     188  0.00%    18-573
rest/stac.py                                   319      68  78.68%   240, 306, 328, 380-383, 410-437, 468-470, 493, 525-526, 608-648, 670-686, 713, 778-782, 789, 843-844, 850, 905, 943, 976, 995-997
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     176      16  90.91%   225-229, 262-264, 282, 285, 291, 295, 353, 370-380
rest/types/queryables.py                        55      13  76.36%   50-51, 58-59, 66-67, 93-98, 107-108, 163
rest/types/stac_search.py                      125      11  91.20%   136-138, 184, 199-201, 209, 213, 261, 264
rest/utils/__init__.py                          93      30  67.74%   79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       5  78.26%   43-44, 48, 60, 62
types/__init__.py                              114      39  65.79%   65, 78-82, 93-105, 132-134, 141-144, 184, 211, 221-237, 242, 244, 265, 270, 278, 288
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             80       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              499      36  92.79%   85, 189-190, 199-226, 229, 244, 324-328, 401-405, 424-426, 505, 520, 560-561, 925-928, 936-937, 975-976, 1147
utils/constraints.py                           119      38  68.07%   62, 89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             41       0  100.00%
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 86, 88, 90, 92, 94, 110, 118-120, 128
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9692    2044  78.91%

Diff against develop

Filename                        Stmts    Miss  Cover
----------------------------  -------  ------  --------
plugins/search/cop_marine.py       +1       0  +0.09%
TOTAL                              +1       0  +100.00%

Results for commit: accf283

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@alambare alambare requested a review from sbrunato October 24, 2024 08:23
@alambare alambare added the dedl DEDL related label Oct 24, 2024
@sbrunato sbrunato changed the title fix: cop_marine - handling of ids with points fix(plugins): cop_marine - handling of ids with points Oct 31, 2024
@sbrunato sbrunato merged commit 441cefa into develop Oct 31, 2024
10 checks passed
@sbrunato sbrunato deleted the cop-marine-id-fix branch October 31, 2024 10:21
@sbrunato sbrunato added this to the 3.0.1.dev milestone Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@sbrunato sbrunato sbrunato approved these changes

Assignees

@jlahovnik jlahovnik

Labels
dedl DEDL related
Projects
None yet
Milestone
3.0.1
Development

Successfully merging this pull request may close these issues.
3 participants
@jlahovnik @sbrunato @alambare