Skip to content

Commit d6bf04d

Browse files
wphyojplwphyojpl
authored
Merge pull request #120 from unity-sds/develop
Release Attempt
2 parents 609a70f + 02e7e65 commit d6bf04d

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

50 files changed

+3164
-402
lines changed

‎.gitignore‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -13,4 +13,5 @@ __pycache__
1313
.terraform.lock.hcl
1414
terraform.tf
1515
terraform.tfvars
16-
cumulus_lambda_functions_deployment.zip
16+
cumulus_lambda_functions_deployment.zip
17+
examples/**/*

‎CHANGELOG.md‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,74 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
===================================================================
99

10+
## [1.1.1.dev001100] - 2025-09-24
11+
### Changed
12+
- [#118](https://github.com/unity-sds/mdps-ds-lib/pull/118) feat: dist utils is obsolete. replace with shutil
13+
14+
## [1.1.1.dev001000] - 2025-09-09
15+
### Changed
16+
- [#116](https://github.com/unity-sds/mdps-ds-lib/pull/116) feat: using opensearch instead of elasticsearch
17+
18+
## [1.1.1.dev000900] - 2025-08-21
19+
### Changed
20+
- [#114](https://github.com/unity-sds/mdps-ds-lib/pull/114) feat: stage-out maap use case
21+
22+
## [1.1.1.dev000801] - 2025-06-03
23+
### Fixed
24+
- [#112](https://github.com/unity-sds/mdps-ds-lib/pull/112) chore: unit client updates
25+
26+
## [1.1.1.dev000800] - 2025-05-21
27+
### Changed
28+
- [#110](https://github.com/unity-sds/mdps-ds-lib/pull/110) feat: allow sns to set external roles to publish messages
29+
30+
## [1.1.1.dev000702] - 2025-05-13
31+
### Fixed
32+
- [#108](https://github.com/unity-sds/mdps-ds-lib/pull/108) fix: stage out error
33+
34+
## [1.1.1.dev000701] - 2025-04-24
35+
### Fixed
36+
- [#106](https://github.com/unity-sds/mdps-ds-lib/pull/106) fix: case insensitivity
37+
38+
## [1.1.1.dev000700] - 2025-04-09
39+
### Changed
40+
- [#104](https://github.com/unity-sds/mdps-ds-lib/pull/104) feat: using single collection endpoint to retrieve the collection details
41+
42+
## [1.1.1.dev000600] - 2025-04-07
43+
### Changed
44+
- [#102](https://github.com/unity-sds/mdps-ds-lib/pull/102) feat: item transfomer
45+
46+
## [1.1.1.dev000500] - 2025-03-24
47+
### Changed
48+
- [#100](https://github.com/unity-sds/mdps-ds-lib/pull/100) feat: new client cli methods
49+
50+
## [1.1.1.dev000401] - 2025-03-04
51+
### Fixed
52+
- [#98](https://github.com/unity-sds/mdps-ds-lib/pull/98) fix: client updates
53+
54+
## [1.1.1.dev000400] - 2025-02-10
55+
### Changed
56+
- [#94](https://github.com/unity-sds/mdps-ds-lib/pull/94) feat: ds client library
57+
58+
## [1.1.1.dev000300] - 2025-01-17
59+
### Changed
60+
- [#92](https://github.com/unity-sds/mdps-ds-lib/pull/92) feat: allow es connection with no ssl for demo purposes only
61+
62+
## [1.1.1.dev000202] - 2025-01-09
63+
### Fixed
64+
- [#90](https://github.com/unity-sds/mdps-ds-lib/pull/90) fix: poetry error requires project.name
65+
66+
## [1.1.1.dev000201] - 2025-01-08
67+
### Fixed
68+
- [#88](https://github.com/unity-sds/mdps-ds-lib/pull/88) fix: check bbox in granules stac
69+
70+
## [1.1.1.dev000200] - 2024-12-16
71+
### Changed
72+
- [#86](https://github.com/unity-sds/mdps-ds-lib/pull/86) feat: s3 delete
73+
74+
## [1.1.1.dev000100] - 2024-12-09
75+
### Changed
76+
- [#84](https://github.com/unity-sds/mdps-ds-lib/pull/84) feat: coginto class update
77+
1078
## [1.1.1] - 2024-12-09
1179

1280
- [#81](https://github.com/unity-sds/mdps-ds-lib/pull/81) release attempt 2

‎mdps_ds_lib/ds_client/__init__.py‎

Whitespace-only changes.

‎mdps_ds_lib/ds_client/auth_token/__init__.py‎

Whitespace-only changes.

‎mdps_ds_lib/ds_client/auth_token/token_abstract.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
from abc import ABC, abstractmethod
2+
3+
4+
class TokenAbstract(ABC):
5+
@abstractmethod
6+
def get_token(self):
7+
return ''

‎mdps_ds_lib/ds_client/auth_token/token_cognito.py‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
import os
2+
3+
from mdps_ds_lib.ds_client.auth_token.token_abstract import TokenAbstract
4+
from mdps_ds_lib.lib.cognito_login.cognito_token_retriever import CognitoTokenRetriever
5+
6+
7+
class TokenCognito(TokenAbstract):
8+
def get_token(self):
9+
token_retriever = CognitoTokenRetriever()
10+
token = token_retriever.start()
11+
header = {'Authorization': f'Bearer {token}'}
12+
return token

‎mdps_ds_lib/ds_client/auth_token/token_dummy.py‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
import os
2+
3+
from mdps_ds_lib.ds_client.auth_token.token_abstract import TokenAbstract
4+
5+
6+
class TokenDummy(TokenAbstract):
7+
def get_token(self):
8+
env_token = os.getenv('DS_TOKEN', None)
9+
if env_token is None:
10+
raise ValueError(f'missing env_token DS_TOKEN')
11+
return env_token

‎mdps_ds_lib/ds_client/auth_token/token_factory.py‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
from mdps_ds_lib.lib.utils.factory_abstract import FactoryAbstract
2+
3+
4+
class TokenFactory(FactoryAbstract):
5+
DUMMY = 'DUMMY'
6+
COGNITO = 'COGNITO'
7+
8+
def get_instance_from_env(self, **kwargs):
9+
raise NotImplementedError('not yet')
10+
11+
def get_instance(self, class_type, **kwargs):
12+
ct = class_type.upper()
13+
if ct == self.DUMMY:
14+
from mdps_ds_lib.ds_client.auth_token.token_dummy import TokenDummy
15+
return TokenDummy()
16+
if ct == self.COGNITO:
17+
from mdps_ds_lib.ds_client.auth_token.token_cognito import TokenCognito
18+
return TokenCognito()
19+
raise ModuleNotFoundError(f'cannot find ES class for {ct}')

‎mdps_ds_lib/ds_client/ds_client_admin.py‎

Lines changed: 148 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,148 @@
1+
import json
2+
import logging
3+
4+
import requests
5+
6+
from mdps_ds_lib.ds_client.auth_token.token_abstract import TokenAbstract
7+
from mdps_ds_lib.ds_client.ds_client_base import DsClient
8+
LOGGER = logging.getLogger(__name__)
9+
10+
11+
class DsClientAdmin(DsClient):
12+
13+
14+
def __init__(self, token_retriever: TokenAbstract, ds_url: str, ds_stage: str):
15+
super().__init__(token_retriever, ds_url, ds_stage)
16+
17+
def add_admin_group(self, crud_actions: list, group_name: str):
18+
request_url = f'{self._uds_url}admin/auth'
19+
LOGGER.debug(f'request_url: {request_url}')
20+
collection_complete_name = '.*' if self.collection is None else \
21+
f'{self.collection}.*' if self.collection_venue is None else f'{self.collection}___{self.collection_venue}'
22+
resources = [self.urn, self.org, self.project,
23+
self.tenant if self.tenant is not None else '*',
24+
self.tenant_venue if self.tenant_venue is not None else '*',
25+
collection_complete_name
26+
]
27+
admin_add_body = {
28+
"actions": [k.upper() for k in crud_actions],
29+
"resources": [':'.join(resources)],
30+
"tenant": self.tenant,
31+
"venue": self.tenant_venue,
32+
"group_name": group_name
33+
}
34+
LOGGER.debug(f"admin_add_body: {admin_add_body}")
35+
s = requests.session()
36+
s.trust_env = self._trust_env
37+
response = s.put(url=request_url, headers={
38+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
39+
'Content-Type': 'application/json',
40+
}, verify=self._trust_env, data=json.dumps(admin_add_body))
41+
response.raise_for_status()
42+
return response.text
43+
44+
def delete_admin_group(self, group_name: str):
45+
request_url = f'{self._uds_url}admin/auth'
46+
LOGGER.debug(f'request_url: {request_url}')
47+
admin_delete_body = {
48+
"tenant": self.tenant,
49+
"venue": self.tenant_venue,
50+
"group_name": group_name
51+
}
52+
LOGGER.debug(f"admin_add_body: {admin_delete_body}")
53+
s = requests.session()
54+
s.trust_env = self._trust_env
55+
response = s.delete(url=request_url, headers={
56+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
57+
'Content-Type': 'application/json',
58+
}, verify=self._trust_env, data=json.dumps(admin_delete_body))
59+
response.raise_for_status()
60+
return response.text
61+
62+
def list_admin_group(self, group_names: list=[]):
63+
request_url = f'{self._uds_url}admin/auth'
64+
params = [
65+
f'tenant={self.tenant}',
66+
f'venue={self.tenant_venue}',
67+
]
68+
if len(group_names) < 1:
69+
params.append(f'group_names={",".join(group_names)}')
70+
params = '&'.join(params)
71+
request_url = f'{request_url}?{params}'
72+
LOGGER.debug(f'request_url: {request_url}')
73+
s = requests.session()
74+
s.trust_env = self._trust_env
75+
response = s.get(url=request_url, headers={
76+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
77+
'Content-Type': 'application/json',
78+
}, verify=self._trust_env)
79+
response.raise_for_status()
80+
return json.loads(response.text)
81+
82+
def update_admin_group(self, crud_actions: list, group_name: str):
83+
raise NotImplementedError(f'TO DO')
84+
85+
def setup_database(self):
86+
es_setup_url = f'{self._uds_url}admin/system/es_setup/'
87+
print(f'es_setup_url: {es_setup_url}')
88+
s = requests.session()
89+
s.trust_env = self._trust_env
90+
response = s.put(url=es_setup_url, headers={'Authorization': f'Bearer {self._token_retriever.get_token()}'}, verify=self._trust_env)
91+
response.raise_for_status()
92+
return response.text
93+
94+
def add_tenant_database_index(self, custom_metadata_es_index: dict):
95+
request_url = f'{self._uds_url}admin/custom_metadata/{self.tenant}/'
96+
if self.tenant_venue is not None:
97+
request_url = f'{request_url}?venue={self.tenant_venue}'
98+
print(f'request_url: {request_url}')
99+
s = requests.session()
100+
s.trust_env = self._trust_env
101+
response = s.put(url=request_url, headers={
102+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
103+
'Content-Type': 'application/json',
104+
}, verify=self._trust_env, data=json.dumps(custom_metadata_es_index))
105+
response.raise_for_status()
106+
return response.text
107+
108+
def get_tenant_database_index(self):
109+
request_url = f'{self._uds_url}admin/custom_metadata/{self.tenant}/'
110+
if self.tenant_venue is not None:
111+
request_url = f'{request_url}?venue={self.tenant_venue}'
112+
print(f'request_url: {request_url}')
113+
s = requests.session()
114+
s.trust_env = self._trust_env
115+
response = s.get(url=request_url, headers={
116+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
117+
'Content-Type': 'application/json',
118+
}, verify=self._trust_env)
119+
response.raise_for_status()
120+
return json.loads(response.text)
121+
122+
def delete_tenant_database_index(self):
123+
request_url = f'{self._uds_url}admin/custom_metadata/{self.tenant}/'
124+
if self.tenant_venue is not None:
125+
request_url = f'{request_url}?venue={self.tenant_venue}'
126+
print(f'request_url: {request_url}')
127+
s = requests.session()
128+
s.trust_env = self._trust_env
129+
response = s.delete(url=request_url, headers={
130+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
131+
'Content-Type': 'application/json',
132+
}, verify=self._trust_env)
133+
response.raise_for_status()
134+
return response.text
135+
136+
def destroy_tenant_database_index(self):
137+
request_url = f'{self._uds_url}admin/custom_metadata/{self.tenant}/destroy/'
138+
if self.tenant_venue is not None:
139+
request_url = f'{request_url}?venue={self.tenant_venue}'
140+
print(f'request_url: {request_url}')
141+
s = requests.session()
142+
s.trust_env = self._trust_env
143+
response = s.delete(url=request_url, headers={
144+
'Authorization': f'Bearer {self._token_retriever.get_token()}',
145+
'Content-Type': 'application/json',
146+
}, verify=self._trust_env)
147+
response.raise_for_status()
148+
return response.text

0 commit comments

Comments
 (0)