Skip to content

Dev 325 #116

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 59 commits into from
Nov 29, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
59 commits
Select commit Hold shift + click to select a range
2be3a68
INSIGHTS-12 Initial structure of insight functionality
robert-close Sep 2, 2020
601e367
INSIGHTS-12 Added todo statements
robert-close Sep 2, 2020
ea1741b
INSIGHTS-12 Moved Insights out of client
robert-close Sep 3, 2020
6e53c18
INSIGHTS-12 Adjusted insight methods to reside in the client class. R…
robert-close Sep 3, 2020
6b84190
INSIGHTS-12 Added a todo. Tests fail for unknown reasons at this time
robert-close Sep 3, 2020
0cf1255
INSIGHTS-12 Fixed method call. Added a todo to get url from config if…
robert-close Sep 4, 2020
46ad61c
INSIGHTS-12 Fixed method call.
robert-close Sep 4, 2020
5cc049e
INSIGHTS-12 added json serialization. might not be needed
robert-close Sep 8, 2020
d53f8cc
INSIGHTS-12 commented test temporarily
robert-close Sep 8, 2020
d984e63
INSIGHTS-12 comment updates and json .encode change
robert-close Sep 8, 2020
80fb4c2
INSIGHTS-12 comment update
robert-close Sep 8, 2020
1054514
INSIGHTS-12 changed method signatures to match documentation https://…
robert-close Sep 8, 2020
0d9c005
INSIGHTS-12 Added system property for queue reader url
robert-close Sep 9, 2020
71313fd
INSIGHTS-12 Fixed URL to not be https
robert-close Sep 10, 2020
0f2a377
INSIGHTS-12 minor version update
robert-close Sep 10, 2020
c5324a0
INSIGHTS-12 revert change
robert-close Sep 10, 2020
3884857
INSIGHTS-12 removed todo
robert-close Sep 10, 2020
9b15069
INSIGHTS-12 uncommented test. May start failing again in the pipeline.
robert-close Sep 10, 2020
d429304
INSIGHTS-12 commented test.
robert-close Sep 14, 2020
3e3a3ec
INSIGHTS-12 changed version. Removed unused import.
robert-close Sep 15, 2020
a0637d2
INSIGHTS-12 changed url to include /v1/
robert-close Sep 15, 2020
2e0cc07
Allow listing of non-data:// files on cli
kennydaniel Oct 7, 2020
21165cd
Allow catting non-data:// files on cli
kennydaniel Oct 7, 2020
1b244d1
Fix tests
kennydaniel Oct 7, 2020
fc7b0e2
Merge branch 'master' into ALGO-852
kennydaniel Oct 8, 2020
957d4ad
Merge branch 'ALGO-852' of https://github.com/algorithmiaio/algorithm…
Oct 8, 2020
d59d23e
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
Oct 8, 2020
a6efe29
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
Oct 12, 2020
cd7c919
Merge remote-tracking branch 'origin/master' into master
Oct 19, 2020
ee32a91
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
Oct 19, 2020
f447733
Merge remote-tracking branch 'origin/master'
John-Bragg Nov 3, 2020
4641208
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Nov 3, 2020
dd7af8e
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Nov 3, 2020
74cc6e2
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Nov 3, 2020
47467d9
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Nov 13, 2020
d5ab302
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Dec 1, 2020
20f6e1b
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Dec 1, 2020
e941a4c
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Dec 1, 2020
a7d2cc6
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 6, 2021
7e866bc
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 6, 2021
e0993ab
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 8, 2021
3a1bb48
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 11, 2021
13f145c
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 11, 2021
c7da2ce
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 22, 2021
b1a0178
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 27, 2021
226442f
Merge branch 'master' of https://github.com/algorithmiaio/algorithmia…
John-Bragg Jan 27, 2021
57c6a8c
Merge https://github.com/algorithmiaio/algorithmia-python
John-Bragg Jun 10, 2021
03ab189
Merge https://github.com/algorithmiaio/algorithmia-python
John-Bragg Jun 23, 2021
b2b1d63
Merge https://github.com/algorithmiaio/algorithmia-python
John-Bragg Jun 25, 2021
e6ef6c2
Merge https://github.com/algorithmiaio/algorithmia-python into Dev-325
John-Bragg Jun 30, 2021
3b086de
Merge https://github.com/algorithmiaio/algorithmia-python into Dev-325
John-Bragg Jul 12, 2021
34d5791
adding jwt support to CLI
John-Bragg Jul 12, 2021
30439ba
Merge branch 'develop' of https://github.com/algorithmiaio/algorithmi…
Oct 27, 2021
e2353d7
refactoring bearertoken method
Oct 28, 2021
d525d0c
adding test and simplifying getclient method
Nov 2, 2021
1f55e12
test fixes
Nov 4, 2021
0eb2be9
Merge branch 'develop' of https://github.com/algorithmiaio/algorithmi…
Nov 23, 2021
7d1d63e
adding test
Nov 23, 2021
d6fba83
Merge branch 'develop' into Dev-325
John-Bragg Nov 29, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 29 additions & 11 deletions Algorithmia/CLI.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,7 @@ class CLI:
def __init__(self):
self.client = Algorithmia.client()
# algo auth

def auth(self, apikey, apiaddress, cacert="", profile="default"):
def auth(self, apiaddress, apikey="", cacert="", profile="default", bearer=""):

# store api key in local config file and read from it each time a client needs to be created
key = self.getconfigfile()
Expand All @@ -24,20 +23,17 @@ def auth(self, apikey, apiaddress, cacert="", profile="default"):
config['profiles'][profile]['api_key'] = apikey
config['profiles'][profile]['api_server'] = apiaddress
config['profiles'][profile]['ca_cert'] = cacert
config['profiles'][profile]['bearer_token'] = bearer

else:
config['profiles'][profile] = {'api_key': apikey, 'api_server': apiaddress, 'ca_cert': cacert}
config['profiles'][profile] = {'api_key':apikey,'api_server':apiaddress,'ca_cert':cacert,'bearer_token':bearer}
else:
config['profiles'] = {profile: {'api_key': apikey, 'api_server': apiaddress, 'ca_cert': cacert}}
config['profiles'] = {profile:{'api_key':apikey,'api_server':apiaddress,'ca_cert':cacert,'bearer_token':bearer }}

with open(key, "w") as key:
toml.dump(config, key)
client = Algorithmia.client(
api_key=self.getAPIkey(profile),
api_address=self.getAPIaddress(profile),
ca_cert=self.getCert(profile)
)
self.ls(path=None, client=client)
toml.dump(config,key)

self.ls(path = None,client = CLI().getClient(profile))

# algo run <algo> <args..> run the the specified algo
def runalgo(self, options, client):
Expand Down Expand Up @@ -366,6 +362,7 @@ def getconfigfile(self):
file.write("api_key = ''\n")
file.write("api_server = ''\n")
file.write("ca_cert = ''\n")
file.write("bearer_token = ''\n")

key = keyPath + keyFile

Expand All @@ -383,6 +380,16 @@ def getAPIkey(self, profile):
return config_dict['profiles'][profile]['api_key']
else:
return None

def getBearerToken(self,profile):
key = self.getconfigfile()
config_dict = toml.load(key)
if 'profiles' in config_dict and profile in config_dict['profiles'] and \
config_dict['profiles'][profile]['bearer_token'] != "":
return config_dict['profiles'][profile]['bearer_token']
else:
return None


def getAPIaddress(self, profile):
key = self.getconfigfile()
Expand All @@ -401,3 +408,14 @@ def getCert(self, profile):
return config_dict['profiles'][profile]['ca_cert']
else:
return None

def getClient(self,profile):
apiAddress = self.getAPIaddress(profile)
apiKey = self.getAPIkey(profile)
caCert = self.getCert(profile)
bearer = None

if apiKey is None:
bearer = self.getBearerToken(profile)

return Algorithmia.client(api_key=apiKey,api_address=apiAddress,ca_cert=caCert,bearer_token = bearer)
4 changes: 2 additions & 2 deletions Algorithmia/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,8 +23,8 @@ def file(dataUrl):
def dir(dataUrl):
return getDefaultClient().dir(dataUrl)

def client(api_key=None, api_address=None, ca_cert=None):
return Client(api_key, api_address, ca_cert)
def client(api_key=None, api_address=None, ca_cert=None, bearer_token=None):
return Client(api_key, api_address, ca_cert, bearer_token)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the documented use for this bearer token?


def handler(apply_func, load_func=lambda: None):
return Handler(apply_func, load_func)
Expand Down
32 changes: 16 additions & 16 deletions Algorithmia/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
import six
from Algorithmia.CLI import CLI
import argparse
import re

#bind input to raw input
try:
Expand Down Expand Up @@ -145,27 +146,26 @@ def main():
APIkey = input("enter API key: ")
CACert = input('(optional) enter path to custom CA certificate: ')

if len(APIkey) == 28 and APIkey.startswith("sim"):
if APIaddress == "" or not APIaddress.startswith("https://api."):
APIaddress = "https://api.algorithmia.com"

CLI().auth(apikey=APIkey, apiaddress=APIaddress, cacert=CACert, profile=args.profile)
if APIaddress == "" or not APIaddress.startswith("https://api."):
print("invalid API address")
else:
print("invalid api key")

if len(APIkey) == 28 and APIkey.startswith("sim"):
CLI().auth(apikey=APIkey, apiaddress=APIaddress, cacert=CACert, profile=args.profile)
else:
jwt = re.compile(r"^([a-zA-Z0-9_=]+)\.([a-zA-Z0-9_=]+)\.([a-zA-Z0-9_\-\+\/=]*)")
Bearer = input("enter JWT token: ")
if jwt.match(Bearer):
CLI().auth(apikey=APIkey, bearer=Bearer, apiaddress=APIaddress, cacert=CACert, profile=args.profile)
else:
print("invalid authentication")



if args.cmd == 'help':
parser.parse_args(['-h'])

#create a client with the appropreate api address and key
client = Algorithmia.client()
if len(CLI().getAPIaddress(args.profile)) > 1:
client = Algorithmia.client(CLI().getAPIkey(args.profile), CLI().getAPIaddress(args.profile))
elif len(CLI().getAPIaddress(args.profile)) > 1 and len(CLI().getCert(args.profile)) > 1:
client = Algorithmia.client(CLI().getAPIkey(args.profile), CLI().getAPIaddress(args.profile),CLI().getCert(args.profile))
elif len(CLI().getAPIaddress(args.profile)) < 1 and len(CLI().getCert(args.profile)) > 1:
client = Algorithmia.client(CLI().getAPIkey(args.profile), CLI().getAPIaddress(args.profile),CLI().getCert(args.profile))
else:
client = Algorithmia.client(CLI().getAPIkey(args.profile))
client = CLI().getClient(args.profile)

if args.cmd == 'run':

Expand Down
23 changes: 22 additions & 1 deletion Algorithmia/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,13 +21,20 @@ class Client(object):
apiKey = None
apiAddress = None
requestSession = None
bearerToken = None

def __init__(self, apiKey=None, apiAddress=None, caCert=None):

def __init__(self, apiKey = None, apiAddress = None, caCert = None, bearerToken=None):
# Override apiKey with environment variable
config = None
self.requestSession = requests.Session()
if apiKey is None and 'ALGORITHMIA_API_KEY' in os.environ:
apiKey = os.environ['ALGORITHMIA_API_KEY']
if apiKey is None:
if bearerToken is None and 'ALGORITHMIA_BEARER_TOKEN' in os.environ:
bearerToken = os.environ['ALGORITHMIA_BEARER_TOKEN']
self.bearerToken = bearerToken

self.apiKey = apiKey
if apiAddress is not None:
self.apiAddress = apiAddress
Expand Down Expand Up @@ -217,6 +224,8 @@ def postJsonHelper(self, url, input_object, parse_response_as_json=True, **query
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = "Bearer "+ self.bearerToken

input_json = None
if input_object is None:
Expand Down Expand Up @@ -244,32 +253,42 @@ def getHelper(self, url, **query_parameters):
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
return self.requestSession.get(self.apiAddress + url, headers=headers, params=query_parameters)

def getStreamHelper(self, url, **query_parameters):
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
return self.requestSession.get(self.apiAddress + url, headers=headers, params=query_parameters, stream=True)

def patchHelper(self, url, params):
headers = {'content-type': 'application/json'}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
return self.requestSession.patch(self.apiAddress + url, headers=headers, data=json.dumps(params))

# Used internally to get http head result
def headHelper(self, url):
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
return self.requestSession.head(self.apiAddress + url, headers=headers)

# Used internally to http put a file
def putHelper(self, url, data):
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
if isJson(data):
headers['Content-Type'] = 'application/json'

Expand All @@ -283,6 +302,8 @@ def deleteHelper(self, url):
headers = {}
if self.apiKey is not None:
headers['Authorization'] = self.apiKey
else:
headers['Authorization'] = 'Bearer '+ self.bearerToken
response = self.requestSession.delete(self.apiAddress + url, headers=headers)
if response.reason == "No Content":
return response
Expand Down
52 changes: 49 additions & 3 deletions Test/CLI_test.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ class CLIDummyTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
cls.client = Algorithmia.client(api_address="http://localhost:8080", api_key="simabcd123")
cls.bearerClient = Algorithmia.client(api_address="http://localhost:8080", bearer_token="simabcd123.token.token")

def test_run(self):
name = "util/Echo"
Expand Down Expand Up @@ -52,6 +53,40 @@ def test_run(self):
result = CLI().runalgo(args, self.client)
self.assertEqual(result, inputs)

def test_run_token(self):
name = "util/Echo"
inputs = "test"

parser = argparse.ArgumentParser('CLI for interacting with Algorithmia')

subparsers = parser.add_subparsers(help='sub cmd', dest='subparser_name')
parser_run = subparsers.add_parser('run', help='algo run <algo> [input options] <args..> [output options]')

parser_run.add_argument('algo')
parser_run.add_argument('-d', '--data', action='store', help='detect input type', default=None)
parser_run.add_argument('-t', '--text', action='store', help='treat input as text', default=None)
parser_run.add_argument('-j', '--json', action='store', help='treat input as json data', default=None)
parser_run.add_argument('-b', '--binary', action='store', help='treat input as binary data', default=None)
parser_run.add_argument('-D', '--data-file', action='store', help='specify a path to an input file',
default=None)
parser_run.add_argument('-T', '--text-file', action='store', help='specify a path to a text file',
default=None)
parser_run.add_argument('-J', '--json-file', action='store', help='specify a path to a json file',
default=None)
parser_run.add_argument('-B', '--binary-file', action='store', help='specify a path to a binary file',
default=None)
parser_run.add_argument('--timeout', action='store', type=int, default=300,
help='specify a timeout (seconds)')
parser_run.add_argument('--debug', action='store_true',
help='print the stdout from the algo <this only works for the owner>')
parser_run.add_argument('--profile', action='store', type=str, default='default')
parser_run.add_argument('-o', '--output', action='store', default=None, type=str)

args = parser.parse_args(['run', name, '-d', inputs])

result = CLI().runalgo(args, self.bearerClient)
self.assertEqual(result, inputs)


class CLIMainTest(unittest.TestCase):
def setUp(self):
Expand Down Expand Up @@ -156,7 +191,7 @@ def test_auth(self):
key = os.getenv('ALGORITHMIA_API_KEY')
address = 'https://api.algorithmia.com'
profile = 'default'
CLI().auth(key, address, profile=profile)
CLI().auth(address, key, profile=profile)
resultK = CLI().getAPIkey(profile)
resultA = CLI().getAPIaddress(profile)
self.assertEqual(resultK, key)
Expand All @@ -176,13 +211,24 @@ def test_auth_cert(self):
cacert = localfile
profile = 'test'

CLI().auth(key, address, cacert, profile)
CLI().auth(address, key, cacert=cacert, profile=profile)
resultK = CLI().getAPIkey(profile)
resultA = CLI().getAPIaddress(profile)
resultC = CLI().getCert(profile)
self.assertEqual(resultK, key)
self.assertEqual(resultA, address)
self.assertEqual(resultC, cacert)

def test_auth_token(self):
address = 'https://api.algorithmia.com'
bearer = 'testtokenabcd'
profile = 'test'

CLI().auth(apiaddress=address, bearer=bearer, profile=profile)
resultA = CLI().getAPIaddress(profile)
resultT = CLI().getBearerToken(profile)
self.assertEqual(resultA, address)
self.assertEqual(resultT, bearer)

def test_get_environment(self):
result = CLI().get_environment_by_language("python2", self.client)
Expand Down Expand Up @@ -230,7 +276,7 @@ def test_get_template(self):
def test_api_address_auth(self):
api_key = os.getenv('ALGORITHMIA_TEST_API_KEY')
api_address = "https://api.test.algorithmia.com"
CLI().auth(api_key, api_address)
CLI().auth(api_address, api_key)
profile = "default"

client = Algorithmia.client(CLI().getAPIkey(profile), CLI().getAPIaddress(profile), CLI().getCert(profile))
Expand Down