Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Automatically Retry the CLI Command upon Session Unlocking #240

Closed
4 tasks done
CMCDragonkai opened this issue Sep 6, 2021 · 13 comments
Closed
4 tasks done

Automatically Retry the CLI Command upon Session Unlocking #240

CMCDragonkai opened this issue Sep 6, 2021 · 13 comments
Assignees
Labels
development Standard development

Comments

@CMCDragonkai
Copy link
Member

CMCDragonkai commented Sep 6, 2021

Specification

MR: https://gitlab.com/MatrixAI/Engineering/Polykey/js-polykey/-/merge_requests/213

When a user executes a CLI command without having been authenticated previously the command sent to the agent will be rejected as "unauthorised". Instead of throwing an exception here, we can prompt the user for the root password in order to authenticate the client. Provided we know that the user is not a machine (which would classify as unattended usage), we can continue to prompt the user for the root password until the correct password is supplied and they can be authenticated. We will refer to this loop as the CLI Authentication Retry Loop (CARL):
CARL attended grpc call flow

Before we can activate the CARL and begin to prompt the user for the password, we need to ensure that the command is not unattended (since we need to account for both machine and human usage). We can do this by checking if either of the two environment variables PK_PASSWORD (for the root password) or PK_TOKEN (for the session token) are set (since the purpose of setting these is for unattended usage). Before we activate the CARL we also need to check that the first exception we receive back from the agent is ErrorClientAuthMissing (thrown when the call has no authorisation metadata), since we only want to retry the call if the client is not authenticated, rather than if the authentication metadata is invalid or if there is some other error on the agent side. Once we are inside the loop we only want to restart it if the error we receive back from the agent is ErrorClientAuthDenied (thrown when the authorisation metadata is invalid), since this means that the password supplied by the user was invalid and we want to prompt them to try again. The user can always manually exit the loop from the terminal if they have forgotten their password or otherwise wish to cancel the call.

Since we want this loop to be as automated as possible, prompting the user to enter their password should be our last option. As such, the retry function can be optionally called with an initial metadata object that is constructed during parsing of command line options for each call. This metadata is encoded using the first set value from the following in order:

  1. Password File (optional for every CLI call)
  2. PK_PASSWORD environment variable
  3. PK_TOKEN environment variable

In this way, if the password is already supplied it will be made use of. Once we enter the CARL we know that all other options have been exhausted and we have to prompt the user for the password.

Additional context

  • Previous @DrFacepalm had asked how this is supposed to be done in an efficient way. This should be done by separating the phases of the UI/UX loop above procedure functions, and then using a while loop to enable recursion. Note that we don't use function recursion in JavaScript runtimes due to the lack of tail-call optimisation in JS runtimes.

Tasks

  1. - Update the design of the sequence diagrams in the sessions reference article https://github.com/MatrixAI/js-polykey/wiki/Sessions to include the CLI command retry loop
  2. - Break up the loop into separate named phases, and create procedure functions for each phase in the CLI src/bin location
  3. - Add these tests to the tests/bin (these are singular tests, you can pick any CLI command for this):
    • Retry loop when the session token is missing, succeeds in unlocking, succeeds in retrying the call
    • Failure to unlock session when prompted
    • Failure to to do the repeated call because session token is invalid
    • Ensure that if password is supplied, and the token is missing, the root password can be substituted for the session token, and the token is established
  4. - Integrate functions into bin/utils and all relevant commands
@CMCDragonkai
Copy link
Member Author

@emmacasolin please update the MR description with your plan as you proceed in this issue too.

@CMCDragonkai
Copy link
Member Author

Remember to link the MRs/PRs.

@emmacasolin
Copy link
Contributor

I'm currently mocking up a diagram of the retry loop in ASCII flow before I make it in plantuml but I've got a couple questions about the root password input preference system.

I'm assuming "take from the parameter" would be checking for an argument like unlock.arguments('[password]');. Would "read from file" be reading from the password file if it's passed in as an option like passwordFile: true? And what does "optional environment variable" refer to?

@emmacasolin
Copy link
Contributor

Here's a mockup of the CLI command retry loop:

        ┌──────┐                              ┌────────┐                                 ┌───────┐                   ┌───────────────┐
        │ User │                              │ Client │                                 │ Agent │                   │ Password File │
        └───┬──┘                              └───┬────┘                                 └───┬───┘                   └───────┬───────┘
            │                                     │                                          │                               │
            │                                     │Call made with no token                   │                               │
            │                                     ├─────────────────────────────────────────►│                               │
            │                                     │                                          │                               │
            │                                     │            ErrorClientJWTTokenNotProvided│                               │
            │                                     │◄─────────────────────────────────────────┤                               │
            │                                     │                                          │                               │
┌──────┬────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼──────┐
│ loop │    │ [Call made with no password]        │                                          │                               │      │
├──────┘    │                                     │                                          │                               │      │
│           │                                     │                                          │                               │      │
│ ┌─────┬───┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼────┐ │
│ │ alt │   │ [Password supplied as parameter]    │                                          │                               │    │ │
│ ├─────┘   │                                     │Get password from parameter               │                               │    │ │
│ │         │                                     ├───────────────────────────┐              │                               │    │ │
│ │         │                                     │ ◄─────────────────────────┘              │                               │    │ │
│ │         │                                     │                                          │                               │    │ │
│ ├─────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼────┤ │
│ │         │ [Password file provided]            │                                          │                               │    │ │
│ │         │                                     │Request password from file                │                               │    │ │
│ │         │                                     ├──────────────────────────────────────────┼──────────────────────────────►│    │ │
│ │         │                                     │                                          │                               │    │ │
│ │         │                                     │                                          │         Responds with password│    │ │
│ │         │                                     │◄─────────────────────────────────────────┼───────────────────────────────┤    │ │
│ │         │                                     │                                          │                               │    │ │
│ ├─────────┼─────────────────────────────────────┴──────────────────────────────────────────┼───────────────────────────────┼────┤ │
│ │         │ [Password supplied as optional environment variable]                           │                               │    │ │
│ │         │                                     │                                          │                               │    │ │
│ │         │                                     │Get password from env variable            │                               │    │ │
│ │         │                                     ├─────────────────────────────┐            │                               │    │ │
│ │         │                                     │ ◄───────────────────────────┘            │                               │    │ │
│ │         │                                     │                                          │                               │    │ │
│ ├─────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼────┤ │
│ │         │ [else]                              │                                          │                               │    │ │
│ │         │           Request password via STDIN│                                          │                               │    │ │
│ │         │◄────────────────────────────────────┤                                          │                               │    │ │
│ │         │                                     │                                          │                               │    │ │
│ │         │Responds with password               │                                          │                               │    │ │
│ │         ├────────────────────────────────────►│                                          │                               │    │ │
│ │         │                                     │                                          │                               │    │ │
│ └─────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼────┘ │
│           │                                     │                                          │                               │      │
│           │                                     │Retry call with password                  │                               │      │
│           │                                     ├─────────────────────────────────────────►│                               │      │
│           │                                     │                                          │                               │      │
└───────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼──────┘
            │                                     │                                          │                               │
┌─────┬─────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼──────┐
│ alt │     │ [Call success]                      │                                          │                               │      │
├─────┘     │                                     │              Responds with requested data│                               │      │
│           │                                     │◄─────────────────────────────────────────┤                               │      │
│           │                                     │                                          │                               │      │
├───────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼──────┤
│           │ [Call made with invalid token]      │                                          │                               │      │
│           │                                     │                  ErrorSessionTokenInvalid│                               │      │
│           │                                     │◄─────────────────────────────────────────┤                               │      │
│           │                                     │                                          │                               │      │
└───────────┼─────────────────────────────────────┼──────────────────────────────────────────┼───────────────────────────────┼──────┘
            │                                     │                                          │                               │
            │                                     │                                          │                               │
           ─┴─                                   ─┴─                                        ─┴─                             ─┴─

I'm still not sure what the optional environment variable is so I've left it as a self request from the client for now. I was struggling to show that the password request options are tried one after the other in separate loops of the retry loop (as opposed to one after the other in the same loop of the retry loop) so I might try to find a way to make that clearer, but other than these two issues I think this diagram is ready to be remade in plantuml.

@CMCDragonkai
Copy link
Member Author

CMCDragonkai commented Oct 11, 2021

  1. A flow chart can be used to represent the different input options it uses. There is a cascading order in terms of priority. It should be env variable, parameter/parameter file, input prompt in order of rising priority. The higher priority overrides the previous data.
  2. An optional env variable is like PK_PASSWORD. Just another way to set the root password. I think we should have this as an example on the .env.example.

@emmacasolin
Copy link
Contributor

Ah ok, makes sense. How about having something like this:

                           CLI Command Automatic Retry Loop                  
                                                                             
                            ┌──────┐                        ┌─────┐          
                            │Client│                        │Agent│          
                            └──┬───┘                        └──┬──┘          
                               │    Call made with no token    │             
                               │ ──────────────────────────────>             
                               │                               │             
                               │ ErrorClientJWTTokenNotProvided│             
                               │ <─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─              
                               │                               │             
                               │                               │             
         ╔═══════╤═════════════╪═══════════════════════════════╪════════════╗
         ║ LOOP  │  call made with no password                 │            ║
         ╟───────┘             │                               │            ║
         ║      ╔══════════════╧═══════════════╗               │            ║
         ║      ║Request root password        ░║               │            ║
         ║      ║see Root Password Input Flow  ║               │            ║
         ║      ╚══════════════╤═══════════════╝               │            ║
         ║                     │    Retry call with password   │            ║
         ║                     │ ──────────────────────────────>            ║
         ╚═════════════════════╪═══════════════════════════════╪════════════╝
                               │                               │             
                               │                               │             
                  ╔══════╤═════╪═══════════════════════════════╪════════════╗
                  ║ ALT  │  call success                       │            ║
                  ╟──────┘     │                               │            ║
                  ║            │  Responds with requested data │            ║
                  ║            │ <─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─             ║
                  ╠════════════╪═══════════════════════════════╪════════════╣
                  ║ [call made with invalid token]             │            ║
                  ║            │    ErrorSessionTokenInvalid   │            ║
                  ║            │ <─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─ ─             ║
                  ╚════════════╪═══════════════════════════════╪════════════╝
                               │                               │             
                               │                               │             

And then I'll make a separate flow diagram for the input options

@emmacasolin
Copy link
Contributor

Unfortunately plantuml's flow diagram functionality is quite limited so I had to make the input flow diagram using another website in order to get it the way I wanted. The top one is a sequence diagram of the loop and the bottom one is a flow diagram of the root password input preference system.

retry-loop.png

input-flow.png

@emmacasolin
Copy link
Contributor

I've created an MR for this issue here - still need to update the description both there and here (will do first thing Wednesday) and then I'll start planning out what needs to be done.

@joshuakarp
Copy link
Contributor

From a CLI perspective then, am I right in thinking it's something like this:

> pk vaults create ...    // no password file

// pk internally checks env variables. 
// If no password, pk internally checks parameters
// If no password, pk internally checks password file
// If no password, request password from STDIN:

> Enter password.
> ...

@emmacasolin
Copy link
Contributor

emmacasolin commented Oct 11, 2021 via email

@CMCDragonkai
Copy link
Member Author

After a review and merge of sessions retry, please work with @joshuakarp to clean up documentation too as this should be pushed into reference docs. We will need to come up with a proper structure for our reference docs.

@joshuakarp
Copy link
Contributor

joshuakarp commented Nov 16, 2021

I believe this can be closed once the Gitlab MR is merged. There's been some changes to this process there.

@emmacasolin should also look into writing some documentation/tidying up diagrams on this new retry process (as per the latest discussions here https://gitlab.com/MatrixAI/Engineering/Polykey/js-polykey/-/merge_requests/213#note_728972375)

@CMCDragonkai
Copy link
Member Author

Merged. @emmacasolin has documentation pending to be reviewed in MatrixAI/Polykey-Docs#4.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
development Standard development
Development

No branches or pull requests

4 participants