Skip to content

Cortex local #973

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 102 commits into from
Apr 27, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
e30a2a5
Cortex local in progress
vishalbollu Mar 30, 2020
f7159d0
Add docker logs
deliahu Mar 30, 2020
d5a74ec
Local round #2
vishalbollu Apr 1, 2020
f075695
Round #3
vishalbollu Apr 1, 2020
cfb7c49
CLI profiles
deliahu Apr 1, 2020
dda5715
Update CLI docs
deliahu Apr 1, 2020
5468ff7
Update version.go
deliahu Apr 1, 2020
0e167bf
Update lib_http_client.go
deliahu Apr 2, 2020
f5fac08
Add aws profile not configured prompt
deliahu Apr 2, 2020
8e033a0
Round #4
vishalbollu Apr 2, 2020
213b6e1
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 2, 2020
d4fb510
Local model caching
vishalbollu Apr 3, 2020
c248aab
Nits
vishalbollu Apr 3, 2020
b6c2854
Merge branch 'master' into cortex-local
vishalbollu Apr 6, 2020
7d38887
Fix error in files.DeleteDirIfPresent
vishalbollu Apr 7, 2020
a07aec9
Use new syntax
vishalbollu Apr 7, 2020
105e9c1
Add support for ONNX and Tensorflow
vishalbollu Apr 7, 2020
1a24eea
Make tf_serve_ip configurable
vishalbollu Apr 7, 2020
01b745e
Rename profile to environment
deliahu Apr 7, 2020
50b4e91
Support cx get
vishalbollu Apr 8, 2020
3668bc8
Move provider.go to pkg/types
vishalbollu Apr 8, 2020
e5ebafc
Update env commands
deliahu Apr 8, 2020
59fbe12
Rename env delete command
deliahu Apr 8, 2020
8b31cc9
Update lib_cli_config.go
deliahu Apr 8, 2020
1c9206f
Make env delete command not require environment name
deliahu Apr 8, 2020
1a54768
Add env default command
deliahu Apr 9, 2020
54252ff
Rename configure.go to env.go
deliahu Apr 9, 2020
10a5d09
Update default env
deliahu Apr 9, 2020
511f9f3
Update env messages
deliahu Apr 9, 2020
2fa3817
Update env default behavior
deliahu Apr 9, 2020
6ed4b7b
Show default in env list output
deliahu Apr 9, 2020
d56bbf1
Move readCLIConfig()
deliahu Apr 9, 2020
69b2179
Cache CLI config
deliahu Apr 9, 2020
a1a0059
Improve env usage help messages
deliahu Apr 9, 2020
1c7b5c1
Misc
deliahu Apr 9, 2020
50208ef
Update cli.md
deliahu Apr 9, 2020
f1c75ca
Reorder command flags in help
deliahu Apr 9, 2020
2717ebd
Create generate_cli_md.sh
deliahu Apr 9, 2020
8da89a2
Update env flag and cli.md
deliahu Apr 9, 2020
98add90
Update generate_cli_md.sh
deliahu Apr 9, 2020
daca86d
Add files.CopyDirOverwrite()
deliahu Apr 9, 2020
dd2b33f
Merge local and cluster commands
vishalbollu Apr 10, 2020
d14a027
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 10, 2020
3b48978
Delete local.go
vishalbollu Apr 10, 2020
3462bbf
Bring back CopyRecursiveShell()
deliahu Apr 9, 2020
97d63f1
Automatically convert old CLI config
deliahu Apr 10, 2020
c5f7fc5
Use Flags() instead of LocalFlags()
deliahu Apr 10, 2020
856b268
Add env messages to cluster up
deliahu Apr 10, 2020
501d39f
Add command-specific flag variables
deliahu Apr 13, 2020
2741ac8
Add includeDirObjects param to S3 listers
deliahu Apr 14, 2020
408e8d5
Add anonymous aws client constructors
deliahu Apr 14, 2020
1c85eaa
More local changes
vishalbollu Apr 15, 2020
8083d20
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 15, 2020
ae10505
Fix merge problems from previous merge
vishalbollu Apr 15, 2020
41d145a
Merge branch 'master' into cortex-local
vishalbollu Apr 15, 2020
4c897d4
Add wasEnvFlagProvided()
deliahu Apr 15, 2020
1a77d04
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 15, 2020
db0ef38
Compile operator
vishalbollu Apr 15, 2020
660a645
Merge branch 'master' of github.com:cortexlabs/cortex into cortex-local
deliahu Apr 15, 2020
23f40dd
Merge docker libraries
deliahu Apr 15, 2020
62d6e0a
Use images provided in api spec for cx local
vishalbollu Apr 15, 2020
74a8fba
Remove unnecessary debug
vishalbollu Apr 15, 2020
fca6fea
Use region correctly
vishalbollu Apr 15, 2020
1c0fa06
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 15, 2020
bea914f
Self review PR fixes round #1
vishalbollu Apr 16, 2020
374cb14
Self review round 2
vishalbollu Apr 17, 2020
fa6b242
Self review round 3
vishalbollu Apr 17, 2020
e6af180
Fix lint
vishalbollu Apr 17, 2020
4c3ee2a
Code review
vishalbollu Apr 22, 2020
0173922
Update api.go
vishalbollu Apr 22, 2020
d5d6f1a
Add license to model_cache.go
vishalbollu Apr 22, 2020
3be6211
Code review round 2 and example updates
vishalbollu Apr 24, 2020
e201b45
Merge branch 'master' into cortex-local
vishalbollu Apr 24, 2020
67ef71a
Code review 3
vishalbollu Apr 24, 2020
f37dcec
Remove unnecessary import
vishalbollu Apr 24, 2020
111fdb1
Update validations.go
vishalbollu Apr 24, 2020
0174bcb
Ensure workspace/apis dir is created
vishalbollu Apr 24, 2020
fb237ef
Make compute.cpu a pointer
deliahu Apr 24, 2020
94dcc97
Don't print empty compute
deliahu Apr 24, 2020
9a59a01
Prevent nil pointer in s.Indent()
deliahu Apr 24, 2020
3216330
Update using environment message
deliahu Apr 24, 2020
24c15dd
Update compute printing
deliahu Apr 24, 2020
b6dc53d
Only use _flagClusterEnv in cluster.go
deliahu Apr 24, 2020
991814b
Update generate_cli_md.sh
deliahu Apr 24, 2020
627a5b2
Update cli.md
deliahu Apr 24, 2020
f9c5750
Set GPU to 0 if nvidia runtime is not there
vishalbollu Apr 25, 2020
aad30c5
Authenticate docker client for pulling images from ecr
vishalbollu Apr 25, 2020
3130c8e
Merge branch 'cortex-local' of github.com:cortexlabs/cortex into cort…
vishalbollu Apr 25, 2020
d12778a
Decrease bold cutoff length
deliahu Apr 26, 2020
6cbe578
Update TensorFlow S3 path error
deliahu Apr 26, 2020
a8e09c0
Add ECR region check
deliahu Apr 26, 2020
abbdf2d
Add ECR no auth error message
deliahu Apr 26, 2020
b6c6a4f
Add local aws credentials reason message
deliahu Apr 26, 2020
a467788
Validate AWS credentials in env configure
deliahu Apr 26, 2020
dc18055
Merge branch 'master' into cortex-local
vishalbollu Apr 27, 2020
20221e3
Use docker auth config when pulling images
vishalbollu Apr 27, 2020
b88f193
Don't save to local dir in examples
deliahu Apr 27, 2020
52be29b
Fix bug where api status error when an api is deployed but the contai…
vishalbollu Apr 27, 2020
eff3774
Error when using anonymous aws creds to access ECR image
vishalbollu Apr 27, 2020
32b0e57
Improve docker permissions help
vishalbollu Apr 27, 2020
efa6e0b
Explain how to get cortex operator endpoint
vishalbollu Apr 27, 2020
7387dd1
Code review 4
vishalbollu Apr 27, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
69 changes: 69 additions & 0 deletions cli/cluster/delete.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
/*
Copyright 2020 Cortex Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cluster

import (
"fmt"

"github.com/cortexlabs/cortex/pkg/lib/errors"
"github.com/cortexlabs/cortex/pkg/lib/json"
"github.com/cortexlabs/cortex/pkg/lib/prompt"
s "github.com/cortexlabs/cortex/pkg/lib/strings"
"github.com/cortexlabs/cortex/pkg/operator/schema"
)

func Delete(operatorConfig OperatorConfig, apiName string, keepCache bool, force bool) (schema.DeleteResponse, error) {
if !force {
readyReplicas := getReadyReplicasOrNil(operatorConfig, apiName)
if readyReplicas != nil && *readyReplicas > 2 {
prompt.YesOrExit(fmt.Sprintf("are you sure you want to delete %s (which has %d live replicas)?", apiName, *readyReplicas), "", "")
}
}

params := map[string]string{
"apiName": apiName,
"keepCache": s.Bool(keepCache),
}

httpRes, err := HTTPDelete(operatorConfig, "/delete/"+apiName, params)
if err != nil {
return schema.DeleteResponse{}, err
}

var deleteRes schema.DeleteResponse
err = json.Unmarshal(httpRes, &deleteRes)
if err != nil {
return schema.DeleteResponse{}, errors.Wrap(err, "/delete", string(httpRes))
}

return deleteRes, nil
}

func getReadyReplicasOrNil(operatorConfig OperatorConfig, apiName string) *int32 {
httpRes, err := HTTPGet(operatorConfig, "/get/"+apiName)
if err != nil {
return nil
}

var apiRes schema.GetAPIResponse
if err = json.Unmarshal(httpRes, &apiRes); err != nil {
return nil
}

totalReady := apiRes.Status.Updated.Ready + apiRes.Status.Stale.Ready
return &totalReady
}
46 changes: 46 additions & 0 deletions cli/cluster/deploy.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
/*
Copyright 2020 Cortex Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cluster

import (
"github.com/cortexlabs/cortex/pkg/lib/errors"
"github.com/cortexlabs/cortex/pkg/lib/json"
s "github.com/cortexlabs/cortex/pkg/lib/strings"
"github.com/cortexlabs/cortex/pkg/operator/schema"
)

func Deploy(operatorConfig OperatorConfig, configPath string, deploymentBytesMap map[string][]byte, force bool) (schema.DeployResponse, error) {
params := map[string]string{
"force": s.Bool(force),
"configPath": configPath,
}
uploadInput := &HTTPUploadInput{
Bytes: deploymentBytesMap,
}

response, err := HTTPUpload(operatorConfig, "/deploy", uploadInput, params)
if err != nil {
return schema.DeployResponse{}, err
}

var deployResponse schema.DeployResponse
if err := json.Unmarshal(response, &deployResponse); err != nil {
return schema.DeployResponse{}, errors.Wrap(err, "/deploy", string(response))
}

return deployResponse, nil
}
89 changes: 89 additions & 0 deletions cli/cluster/errors.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
/*
Copyright 2020 Cortex Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cluster

import (
"fmt"
"net/url"
"strings"

"github.com/cortexlabs/cortex/pkg/lib/errors"
"github.com/cortexlabs/cortex/pkg/lib/urls"
)

const (
_errStrCantMakeRequest = "unable to make request"
_errStrRead = "unable to read"
)

func errStrFailedToConnect(u url.URL) string {
return "failed to connect to " + urls.TrimQueryParamsURL(u)
}

const (
ErrFailedToConnectOperator = "cli.failed_to_connect_operator"
ErrOperatorSocketRead = "cli.operator_socket_read"
ErrResponseUnknown = "cli.response_unknown"
ErrOperatorResponseUnknown = "cli.operator_response_unknown"
ErrOperatorStreamResponseUnknown = "cli.operator_stream_response_unknown"
)

func ErrorFailedToConnectOperator(originalError error, envName string, operatorURL string) error {
originalErrMsg := ""
if originalError != nil {
originalErrMsg = urls.TrimQueryParamsStr(errors.Message(originalError)) + "\n"
}

return errors.WithStack(&errors.Error{
Kind: ErrFailedToConnectOperator,
Message: fmt.Sprintf("%sfailed to connect to the operator in the %s environment (operator endpoint: %s); run `cortex env configure %s` if you need to update the operator endpoint, `cortex cluster info` to show your operator endpoint, or `cortex cluster up` to create a new cluster", originalErrMsg, envName, operatorURL, envName),
})
}

func ErrorOperatorSocketRead(err error) error {
return errors.WithStack(&errors.Error{
Kind: ErrOperatorSocketRead,
Message: err.Error(),
NoPrint: true,
})
}

func ErrorResponseUnknown(body string, statusCode int) error {
msg := body
if strings.TrimSpace(body) == "" {
msg = fmt.Sprintf("empty response (status code %d)", statusCode)
}

return errors.WithStack(&errors.Error{
Kind: ErrResponseUnknown,
Message: msg,
})
}

func ErrorOperatorResponseUnknown(body string, statusCode int) error {
return errors.WithStack(&errors.Error{
Kind: ErrOperatorResponseUnknown,
Message: fmt.Sprintf("unexpected response from operator (status code %d): %s", statusCode, body),
})
}

func ErrorOperatorStreamResponseUnknown(body string, statusCode int) error {
return errors.WithStack(&errors.Error{
Kind: ErrOperatorStreamResponseUnknown,
Message: fmt.Sprintf("unexpected response from operator (status code %d): %s", statusCode, body),
})
}
50 changes: 50 additions & 0 deletions cli/cluster/get.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
/*
Copyright 2020 Cortex Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cluster

import (
"github.com/cortexlabs/cortex/pkg/lib/errors"
"github.com/cortexlabs/cortex/pkg/lib/json"
"github.com/cortexlabs/cortex/pkg/operator/schema"
)

func GetAPIs(operatorConfig OperatorConfig) (schema.GetAPIsResponse, error) {
httpRes, err := HTTPGet(operatorConfig, "/get")
if err != nil {
return schema.GetAPIsResponse{}, err
}

var apisRes schema.GetAPIsResponse
if err = json.Unmarshal(httpRes, &apisRes); err != nil {
return schema.GetAPIsResponse{}, errors.Wrap(err, "/get", string(httpRes))
}
return apisRes, nil
}

func GetAPI(operatorConfig OperatorConfig, apiName string) (schema.GetAPIResponse, error) {
httpRes, err := HTTPGet(operatorConfig, "/get/"+apiName)
if err != nil {
return schema.GetAPIResponse{}, err
}

var apiRes schema.GetAPIResponse
if err = json.Unmarshal(httpRes, &apiRes); err != nil {
return schema.GetAPIResponse{}, errors.Wrap(err, "/get/"+apiName, string(httpRes))
}

return apiRes, nil
}
38 changes: 38 additions & 0 deletions cli/cluster/info.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
/*
Copyright 2020 Cortex Labs, Inc.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
*/

package cluster

import (
"github.com/cortexlabs/cortex/pkg/lib/errors"
"github.com/cortexlabs/cortex/pkg/lib/json"
"github.com/cortexlabs/cortex/pkg/operator/schema"
)

func Info(operatorConfig OperatorConfig) (*schema.InfoResponse, error) {
httpResponse, err := HTTPGet(operatorConfig, "/info")
if err != nil {
return nil, errors.Wrap(err, "unable to connect to operator", "/info")
}

var infoResponse schema.InfoResponse
err = json.Unmarshal(httpResponse, &infoResponse)
if err != nil {
return nil, errors.Wrap(err, "/info", string(httpResponse))
}

return &infoResponse, nil
}
Loading