Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(server): logs format #1238

Merged
merged 5 commits into from
Aug 9, 2024
Merged

refactor(server): logs format #1238

merged 5 commits into from
Aug 9, 2024

Conversation

alambare
Copy link
Collaborator

@alambare alambare commented Jun 26, 2024

  • keep thread id only in server logs
  • refactor setup_logging to reduce code redundancy

Copy link
Contributor

github-actions bot commented Jun 26, 2024

Test Results

    4 files  ±0      4 suites  ±0   6m 12s ⏱️ -1s
  555 tests ±0    552 ✅ ±0    3 💤 ±0  0 ❌ ±0 
2 220 runs  ±0  2 118 ✅ ±0  102 💤 ±0  0 ❌ ±0 

Results for commit 906803b. ± Comparison against base commit fad1677.

♻️ This comment has been updated with latest results.

Copy link
Contributor

github-actions bot commented Jun 26, 2024

badge

Code Coverage (Ubuntu)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         312      59  81.09%   655-714, 816-867, 871
config.py                                      367      27  92.64%   82-84, 93, 101, 105-107, 175, 186, 452-454, 514-517, 560-561, 570-571, 676, 739-744, 746
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    758      74  90.24%   623, 667-670, 708, 743, 773, 820, 824-829, 855, 946, 1014, 1144, 1227-1239, 1275, 1277, 1305, 1309-1320, 1333-1339, 1422-1425, 1458-1478, 1530, 1547-1550, 1562-1565, 1587-1594, 1908, 1939-1945, 2196, 2200-2203, 2214-2216, 2248
api/search_result.py                            51       4  92.16%   73, 82, 89, 103
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                675      81  88.00%   129-131, 226, 258-259, 305-306, 316-328, 330, 341, 347-359, 404-405, 442, 463-466, 489, 497-498, 574-575, 599-600, 606-609, 624-625, 774, 820, 991-996, 1123, 1137-1157, 1177, 1182, 1289, 1311, 1325, 1338-1357, 1396, 1448, 1480-1484, 1499
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       2  90.48%   48, 55
plugins/manager.py                             130      12  90.77%   104-109, 159, 197, 219, 223, 247, 281-282
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   149-151, 198-199, 225-227
plugins/apis/usgs.py                           180      31  82.78%   132, 234, 268, 303-305, 310, 336-337, 342, 372-379, 390-395, 417-423, 425-431, 454
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   34, 47
plugins/authentication/generic.py               14       2  85.71%   40, 50
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   132, 156-161
plugins/authentication/oauth.py                 13       7  46.15%   32-34, 38-41
plugins/authentication/openid_connect.py       183      17  90.71%   119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                34       1  97.06%   83
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 88      16  81.82%   79, 107, 109, 131-143, 198-202
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   40
plugins/crunch/filter_date.py                   59      14  76.27%   49-54, 66, 75, 84, 87, 99-101, 110-112, 119
plugins/crunch/filter_latest_intersect.py       47       8  82.98%   51-52, 68, 77-80, 82, 89-92
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 68-71, 78-81, 87, 95, 106-122
plugins/crunch/filter_property.py               30       7  76.67%   54-59, 62-63, 79-83
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        489     163  66.67%   265, 278, 345-348, 362-366, 408-410, 414, 446-447, 453-457, 486, 518, 522, 529, 559-567, 571, 603-611, 618-620, 654-728, 746-807, 818-823, 835-848, 873, 888-890, 893, 903-911, 919-932, 942-973, 980-992, 1030, 1056, 1101-1103, 1323
plugins/download/base.py                       253      51  79.84%   136, 164, 296-297, 314-320, 351-355, 361-362, 404, 407-421, 433, 437, 501-505, 535-536, 544-561, 568-576, 578-582, 625, 647, 669, 677
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       534     129  75.84%   195-207, 209-210, 242-245, 306-309, 311-312, 319-324, 342-357, 374-376, 388, 436, 443-449, 467, 481, 495, 503-505, 521-526, 537, 555, 597-601, 623, 663, 708, 722-728, 757-821, 839, 872-881, 903-904, 931-936, 942, 945, 961, 978-979, 1009-1010, 1017, 1078-1084, 1139-1140, 1146, 1156, 1192, 1228, 1246-1262, 1288-1290
plugins/download/s3rest.py                     116      24  79.31%   113, 149, 156, 191, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         128       9  92.97%   102, 106, 117, 255, 275, 331-332, 352, 361
plugins/search/build_search_result.py          181      23  87.29%   95, 136-137, 143, 154, 288-291, 320, 373-390, 450, 453, 463, 480, 508, 510
plugins/search/cop_marine.py                   197      47  76.14%   55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 249, 253, 268, 272, 276, 280, 284-288, 294-297, 300-314, 331, 354, 357, 363
plugins/search/creodias_s3.py                   55       3  94.55%   53, 71, 105
plugins/search/csw.py                          105      81  22.86%   58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          200      67  66.50%   90-93, 109, 120, 124-125, 136, 141, 146, 153, 166-169, 223-224, 228, 238-244, 249, 275-278, 286-297, 314, 316, 323-324, 326-327, 345-349, 382, 392, 403, 416, 422-437, 442
plugins/search/qssearch.py                     666      81  87.84%   388, 392-398, 406-407, 513-519, 565, 581, 591, 618, 620, 662-665, 739-740, 788, 807, 814, 826, 883, 904, 907-908, 917-918, 927-928, 937-938, 965, 1036-1041, 1045-1054, 1088, 1110, 1170, 1219, 1293-1297, 1357, 1360, 1366-1367, 1388, 1415-1427, 1434, 1466-1468, 1478-1484, 1514, 1537, 1552, 1574, 1681-1691
plugins/search/static_stac_search.py            72      10  86.11%   98-125, 141, 154
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33       7  78.79%   35-37, 53-55, 59, 68
rest/config.py                                  26       0  100.00%
rest/constants.py                                7       0  100.00%
rest/core.py                                   234      26  88.89%   269, 355, 454, 670, 677-725
rest/server.py                                 283      53  81.27%   108, 131-133, 246-248, 304-305, 317-333, 425-430, 458, 623-630, 659, 703-704, 727, 798-800, 817-822, 851, 853, 857-858, 862-863
rest/stac.py                                   464     120  74.14%   303, 325, 373-376, 400-426, 431-437, 457-459, 482, 514-515, 543, 556, 597-637, 659-675, 749-762, 769, 823-824, 885, 950-952, 1156, 1164-1176, 1187-1209, 1220-1265, 1424-1425
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     179       5  97.21%   232-236, 289, 292, 360
rest/types/queryables.py                        56       1  98.21%   164
rest/types/stac_search.py                      126       7  94.44%   129, 175, 190-192, 200, 204
rest/utils/__init__.py                          93      12  87.10%   108-109, 128-130, 182, 192-206
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       3  86.96%   48, 60, 62
types/__init__.py                              114      14  87.72%   53, 70, 129-132, 199, 213-222, 232, 253, 266
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              500      37  92.60%   85, 90, 194-195, 204-231, 234, 248, 328-332, 406-410, 429-431, 510, 525, 561-562, 926-929, 937-938, 976-977, 1148
utils/constraints.py                           119      37  68.91%   89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             35       1  97.14%   93
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 88, 90, 92, 94, 96, 112, 120-122, 130
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9561    1633  82.92%

Diff against develop

Filename            Stmts    Miss  Cover
----------------  -------  ------  -------
cli.py                 +9      +9  -2.41%
utils/logging.py       -1       0  -0.12%
TOTAL                  +8      +9  -0.08%

Results for commit: 906803b

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

Copy link
Contributor

github-actions bot commented Jun 26, 2024

badge

Code Coverage (Windows)

Filename                                     Stmts    Miss  Cover    Missing
-----------------------------------------  -------  ------  -------  --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
__init__.py                                      8       0  100.00%
cli.py                                         312      59  81.09%   655-714, 816-867, 871
config.py                                      367      28  92.37%   82-84, 93, 101, 105-107, 175, 186, 452-454, 514-517, 560-561, 570-571, 676, 707, 739-744, 746
crunch.py                                        5       5  0.00%    20-24
api/__init__.py                                  0       0  100.00%
api/core.py                                    758      74  90.24%   623, 667-670, 708, 743, 773, 820, 824-829, 855, 946, 1014, 1144, 1227-1239, 1275, 1277, 1305, 1309-1320, 1333-1339, 1422-1425, 1458-1478, 1530, 1547-1550, 1562-1565, 1587-1594, 1908, 1939-1945, 2196, 2200-2203, 2214-2216, 2248
api/search_result.py                            51       4  92.16%   73, 82, 89, 103
api/product/__init__.py                          6       0  100.00%
api/product/_assets.py                          48       5  89.58%   75, 147, 155, 158-162
api/product/_product.py                        187      20  89.30%   70-72, 237-238, 313, 342, 399, 413-416, 429, 453-456, 499-505
api/product/metadata_mapping.py                675      81  88.00%   129-131, 226, 258-259, 305-306, 316-328, 330, 341, 347-359, 404-405, 442, 463-466, 489, 497-498, 574-575, 599-600, 606-609, 624-625, 774, 820, 991-996, 1123, 1137-1157, 1177, 1182, 1289, 1311, 1325, 1338-1357, 1396, 1448, 1480-1484, 1499
api/product/drivers/__init__.py                  6       0  100.00%
api/product/drivers/base.py                      6       1  83.33%   38
plugins/__init__.py                              0       0  100.00%
plugins/base.py                                 21       3  85.71%   48, 55, 68
plugins/manager.py                             130      12  90.77%   104-109, 159, 197, 219, 223, 247, 281-282
plugins/apis/__init__.py                         0       0  100.00%
plugins/apis/base.py                             4       0  100.00%
plugins/apis/ecmwf.py                           91       8  91.21%   149-151, 198-199, 225-227
plugins/apis/usgs.py                           180      31  82.78%   132, 234, 268, 303-305, 310, 336-337, 342, 372-379, 390-395, 417-423, 425-431, 454
plugins/authentication/__init__.py               6       1  83.33%   31
plugins/authentication/aws_auth.py              19       0  100.00%
plugins/authentication/base.py                  17       2  88.24%   34, 47
plugins/authentication/generic.py               14       2  85.71%   40, 50
plugins/authentication/header.py                19       0  100.00%
plugins/authentication/keycloak.py              46       4  91.30%   132, 156-161
plugins/authentication/oauth.py                 13       7  46.15%   32-34, 38-41
plugins/authentication/openid_connect.py       183      17  90.71%   119, 133-158, 166, 320-323, 347
plugins/authentication/qsauth.py                34       1  97.06%   83
plugins/authentication/sas_auth.py              47       1  97.87%   76
plugins/authentication/token.py                 88      16  81.82%   79, 107, 109, 131-143, 198-202
plugins/authentication/token_exchange.py        35      19  45.71%   74-80, 92-120
plugins/crunch/__init__.py                       0       0  100.00%
plugins/crunch/base.py                          10       1  90.00%   40
plugins/crunch/filter_date.py                   59      14  76.27%   49-54, 66, 75, 84, 87, 99-101, 110-112, 119
plugins/crunch/filter_latest_intersect.py       47      33  29.79%   48-53, 66-111
plugins/crunch/filter_latest_tpl_name.py        31       1  96.77%   83
plugins/crunch/filter_overlap.py                66      18  72.73%   28-30, 68-71, 78-81, 87, 95, 106-122
plugins/crunch/filter_property.py               30       7  76.67%   54-59, 62-63, 79-83
plugins/download/__init__.py                     0       0  100.00%
plugins/download/aws.py                        489     163  66.67%   265, 278, 345-348, 362-366, 408-410, 414, 446-447, 453-457, 486, 518, 522, 529, 559-567, 571, 603-611, 618-620, 654-728, 746-807, 818-823, 835-848, 873, 888-890, 893, 903-911, 919-932, 942-973, 980-992, 1030, 1056, 1101-1103, 1323
plugins/download/base.py                       253      53  79.05%   136, 164, 231-233, 296-297, 314-320, 351-355, 361-362, 404, 407-421, 433, 437, 501-505, 535-536, 544-561, 568-576, 578-582, 625, 647, 669, 677
plugins/download/creodias_s3.py                 17       9  47.06%   44-58
plugins/download/http.py                       534     130  75.66%   195-207, 209-210, 242-245, 306-309, 311-312, 319-324, 342-357, 374-376, 388, 436, 443-449, 467, 481, 495, 503-505, 521-526, 537, 555, 597-601, 623, 663, 708, 722-728, 757-821, 839, 872-881, 903-904, 931-936, 942, 945, 961, 978-979, 992, 1009-1010, 1017, 1078-1084, 1139-1140, 1146, 1156, 1192, 1228, 1246-1262, 1288-1290
plugins/download/s3rest.py                     116      24  79.31%   113, 149, 156, 191, 221-228, 231-233, 237, 248-254, 262-263, 266-270, 293, 314-317
plugins/search/__init__.py                      22       0  100.00%
plugins/search/base.py                         128      14  89.06%   102, 106, 117, 255, 275, 331-332, 352, 355-363, 365
plugins/search/build_search_result.py          181      30  83.43%   95, 136-137, 143, 154, 288-291, 320, 373-390, 450, 453, 463, 480, 500-515
plugins/search/cop_marine.py                   197      47  76.14%   55, 63-65, 71-72, 88, 90, 93, 128-130, 142-143, 183-192, 196, 199, 203, 221, 249, 253, 268, 272, 276, 280, 284-288, 294-297, 300-314, 331, 354, 357, 363
plugins/search/creodias_s3.py                   55       3  94.55%   53, 71, 105
plugins/search/csw.py                          105      81  22.86%   58-59, 63-64, 72-120, 126-139, 147-179, 197-238
plugins/search/data_request_search.py          200      67  66.50%   90-93, 109, 120, 124-125, 136, 141, 146, 153, 166-169, 223-224, 228, 238-244, 249, 275-278, 286-297, 314, 316, 323-324, 326-327, 345-349, 382, 392, 403, 416, 422-437, 442
plugins/search/qssearch.py                     666     109  83.63%   388, 392-398, 406-407, 513-519, 565, 568, 581, 591, 610-625, 662-665, 739-740, 788, 807, 814, 826, 883, 904, 907-908, 917-918, 927-928, 937-938, 965, 1036-1041, 1045-1054, 1088, 1110, 1170, 1219, 1293-1297, 1357, 1360, 1366-1367, 1388, 1415-1427, 1434, 1466-1468, 1478-1484, 1514, 1537, 1552, 1574, 1640-1711
plugins/search/static_stac_search.py            72      10  86.11%   98-125, 141, 154
rest/__init__.py                                 4       2  50.00%   21-22
rest/cache.py                                   33      22  33.33%   35-37, 44-70
rest/config.py                                  26       4  84.62%   34-36, 68
rest/constants.py                                7       0  100.00%
rest/core.py                                   234     149  36.32%   160-241, 260-314, 327-365, 396-428, 443-459, 477-488, 497-533, 550, 592-631, 670, 677-725
rest/server.py                                 283     283  0.00%    18-874
rest/stac.py                                   464     393  15.30%   123-129, 136-161, 187-192, 219, 236-359, 369-438, 453-492, 505-558, 566-589, 597-637, 659-675, 685, 699-702, 718-788, 804-840, 863-888, 895-911, 918-923, 932-959, 968-970, 977-979, 991-993, 1005-1022, 1030-1051, 1059-1081, 1088-1105, 1124-1147, 1156, 1164-1176, 1187-1209, 1220-1265, 1272-1289, 1297-1455
rest/types/__init__.py                           0       0  100.00%
rest/types/collections_search.py                13      13  0.00%    18-44
rest/types/eodag_search.py                     179      15  91.62%   122, 232-236, 269-271, 289, 292, 298, 302, 360, 372-375
rest/types/queryables.py                        56      13  76.79%   51-52, 59-60, 67-68, 94-99, 108-109, 164
rest/types/stac_search.py                      126      18  85.71%   127-129, 153-154, 159-160, 175, 190-192, 200, 204, 251-256
rest/utils/__init__.py                          93      30  67.74%   79-85, 105, 108-109, 128-130, 143, 150, 175-183, 190-211
rest/utils/cql_evaluate.py                      48       5  89.58%   69, 76, 90, 97, 105
rest/utils/rfc3339.py                           23       6  73.91%   40, 43-44, 48, 60, 62
types/__init__.py                              114      39  65.79%   53, 66-70, 81-93, 120-122, 129-132, 172, 199, 209-225, 230, 232, 253, 258, 266, 276
types/bbox.py                                   43      19  55.81%   46-61, 72-74, 85-87, 99-101, 113-115, 123
types/download_args.py                           9       0  100.00%
types/queryables.py                             81       0  100.00%
types/search_args.py                            70      18  74.29%   60-64, 71-88, 103
types/whoosh.py                                 15       0  100.00%
utils/__init__.py                              500      39  92.20%   85, 90, 194-195, 204-231, 234, 248, 328-332, 406-410, 429-431, 510, 525, 561-562, 926-929, 937-938, 976-977, 1006, 1148, 1309
utils/constraints.py                           119      37  68.91%   89-98, 139, 144, 148, 159, 182-184, 194, 208-224, 233-244
utils/exceptions.py                             35       1  97.14%   93
utils/import_system.py                          28      19  32.14%   64-78, 89-99
utils/logging.py                                28       1  96.43%   41
utils/notebook.py                               44      23  47.73%   25-29, 36-41, 58-62, 72-78, 83-87
utils/repr.py                                   30       8  73.33%   36, 38, 42, 76, 94-101
utils/requests.py                               55      11  80.00%   64, 88, 90, 92, 94, 96, 112, 120-122, 130
utils/rest.py                                   36       1  97.22%   55
utils/stac_reader.py                           111      45  59.46%   56-57, 63-85, 95-97, 101, 137, 153-156, 203-212, 222-252
TOTAL                                         9561    2429  74.59%

Diff against develop

Filename            Stmts    Miss  Cover
----------------  -------  ------  -------
cli.py                 +9      +9  -2.41%
utils/logging.py       -1       0  -0.12%
TOTAL                  +8      +9  -0.08%

Results for commit: 906803b

Minimum allowed coverage is 70%

♻️ This comment has been updated with latest results

@alambare alambare requested a review from sbrunato June 26, 2024 07:28
@alambare alambare self-assigned this Jun 26, 2024
@alambare alambare marked this pull request as ready for review June 26, 2024 07:29
@sbrunato sbrunato marked this pull request as draft July 8, 2024 09:10
eodag/cli.py Outdated Show resolved Hide resolved
@alambare alambare marked this pull request as ready for review July 9, 2024 16:08
@alambare alambare requested a review from sbrunato July 10, 2024 13:51
@alambare alambare dismissed sbrunato’s stale review July 17, 2024 20:10

changes completeed

@alambare alambare requested review from dalpasso and removed request for sbrunato July 17, 2024 20:14
@dalpasso dalpasso force-pushed the server-log-format branch from a131da0 to 574927b Compare July 22, 2024 11:46
@dalpasso dalpasso force-pushed the server-log-format branch from 574927b to 6121a51 Compare July 23, 2024 13:20
@alambare alambare requested a review from sbrunato July 24, 2024 14:44
eodag/utils/logging.py Outdated Show resolved Hide resolved
eodag/utils/logging.py Show resolved Hide resolved
@sbrunato sbrunato marked this pull request as draft July 24, 2024 15:56
@alambare alambare force-pushed the server-log-format branch from 3cab954 to 8aed8a9 Compare July 26, 2024 16:32
@alambare alambare dismissed sbrunato’s stale review August 1, 2024 08:34

changes are done

@alambare alambare requested a review from sbrunato August 1, 2024 08:34
@alambare alambare marked this pull request as ready for review August 1, 2024 08:34
@alambare alambare force-pushed the server-log-format branch from 8aed8a9 to 906803b Compare August 5, 2024 10:03
remove obsolet sentinelsat logger

add missing eodag-cube logger
@sbrunato sbrunato changed the title Server log format refactor(server): logs format Aug 9, 2024
@sbrunato sbrunato merged commit c8733ce into develop Aug 9, 2024
10 checks passed
@sbrunato sbrunato deleted the server-log-format branch August 9, 2024 08:09
@sbrunato sbrunato added this to the 3.0.0.dev milestone Sep 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants