Add support for timer create with data_access (#699)

* Add support for timer create with data_access

Subsequent to the changes below, `globus timer create transfer` is
no longer marked hidden and it is included in the changelog. This
therefore marks the "release" of `globus timer create transfer` as a
command.

The data access check is now followed by a check against the Auth
Consents API if data_access scopes are required. The Consents API
data can be assumed to match a specific shape due to the enforcement
of "contract versions" on CLI scopes.

We do not request or need the view_consents scope at present, as the
Consents API does not require this in order for the CLI to see its own
consents.

In order to use the Consents API, the CLI now pulls the user's ID from
storage, so tests now mock the partial userinfo data which is written
at the end of a login flow.

Client Credentials handling is not included for now.
If client credentials are used to run
`globus timer create transfer` against a collection with `data_access`,
an explicit usage error is thrown for now.
In the future, it should be possible to use client creds to
request a new token which has the required dependency, but this is
explicitly set aside as future work.

'session consent' is enhanced to allow timer-based data_access scopes
to be requested more easily.
Rather than presenting the whole (long, even unsightly) scope string
on the command line, define an aliasing option for this purpose which
makes the `globus session consent` command more approachable. This
puts usage of `compute_timer_scope()` into the internals for
`globus session consent --timer-data-access ...`.

On the one hand, this "dirties" the design of
`globus session consent`, in that the command is no longer fully
generic and now contains service-specific details. On the other hand,
making this comprommise lets us provide users with a more coordinated
experience across various parts of the CLI, in which they don't need
to traffic in as much information. Even if the information is only
ever copy-pasted, exposing it directly to the user makes the commands
more error-prone and less clear to read and reason about.

----

This work was rebased and squashed from prior work. It is possible
that some detail was lost in the process.

Co-authored-by: Kurt McKee <contactme@kurtmckee.org>

* Update src/globus_cli/login_manager/__init__.py

* Resolve typing issue introduced by rebase

`list[str]` does not match `list[str | X]` because `list` is
invariant. Either we use a covariant type in the helper like
`Sequence`, or we expand the type of the scope list to `str |
MutableScope` to match the type of the helper.

---------

Co-authored-by: Kurt McKee <contactme@kurtmckee.org>

changelog.d/20221102_223217_sirosen_timer_prompt.md

@@ -0,0 +1,9 @@
+### Enhancements
+
+* Add `globus timer create transfer` as a new command for creating new timers
+
+** The command prompts for login if data_access consents are detected as a
+   requirement
+
+* `globus session consent` now supports a `--timer-data-access` flag, specifically
+   to help support timer creation

src/globus_cli/commands/session/consent.py

@@ -4,7 +4,7 @@
 from globus_sdk.scopes import MutableScope
 
 from globus_cli import utils
-from globus_cli.login_manager import LoginManager
+from globus_cli.login_manager import LoginManager, compute_timer_scope
 from globus_cli.parsing import command, no_local_server_option
 
 
@@ -14,8 +14,21 @@
     disable_options=["format", "map_http_status"],
 )
 @no_local_server_option
-@click.argument("SCOPES", nargs=-1, required=True)
-def session_consent(scopes: tuple[str], no_local_server: bool) -> None:
+@click.argument("SCOPES", nargs=-1)
+@click.option(
+    "--timer-data-access",
+    multiple=True,
+    help=(
+        "This is a shorthand for specifying a Globus Timer data_access scope, "
+        "a type of consent needed for a timer to access certain collections."
+    ),
+)
+def session_consent(
+    *,
+    scopes: tuple[str, ...],
+    timer_data_access: tuple[str, ...],
+    no_local_server: bool,
+) -> None:
     """
     Update your current CLI auth session by authenticating with a specific scope or set
     of scopes.
@@ -26,6 +39,16 @@ def session_consent(scopes: tuple[str], no_local_server: bool) -> None:
     scope_list: list[str | MutableScope] = [
         utils.unquote_cmdprompt_single_quotes(s) for s in scopes
     ]
+
+    if timer_data_access:
+        scope_list.append(
+            compute_timer_scope(data_access_collection_ids=timer_data_access)
+        )
+    if not scope_list:
+        raise click.UsageError(
+            "You must provide either SCOPES or at least one scope-defining option."
+        )
+
     manager = LoginManager()
 
     manager.run_login_flow(

src/globus_cli/commands/timer/create/__init__.py

@@ -4,7 +4,6 @@
 @group(
     "create",
     short_help="Submit a Timer job",
-    hidden=True,
     lazy_subcommands={"transfer": (".transfer", "transfer_command")},
 )
 def create_command() -> None:

src/globus_cli/commands/timer/create/transfer.py

@@ -9,7 +9,11 @@
 import globus_sdk
 
 from globus_cli.endpointish import Endpointish
-from globus_cli.login_manager import LoginManager
+from globus_cli.login_manager import (
+    LoginManager,
+    is_client_login,
+    read_well_known_config,
+)
 from globus_cli.parsing import (
     ENDPOINT_PLUS_OPTPATH,
     TimedeltaType,
@@ -33,11 +37,6 @@
 else:
     from typing_extensions import Literal
 
-# XXX: this may need to be parametrized over data_access scopes?
-_TRANSFER_AP_SCOPE = (
-    "https://auth.globus.org/scopes/actions.globus.org/transfer/transfer"
-)
-
 
 INTERVAL_HELP = """\
 Interval at which the job should run. Expressed in weeks, days, hours, minutes, and
@@ -96,7 +95,7 @@ def resolve_start_time(start: datetime.datetime | None) -> datetime.datetime:
     type=click.IntRange(min=1),
     help="Stop running the transfer after this number of runs have happened",
 )
-@LoginManager.requires_login("timer", "transfer")
+@LoginManager.requires_login("auth", "timer", "transfer")
 def transfer_command(
     *,
     login_manager: LoginManager,
@@ -130,6 +129,7 @@ def transfer_command(
     """
     from globus_cli.services.transfer import add_batch_to_transfer_data, autoactivate
 
+    auth_client = login_manager.get_auth_client()
     timer_client = login_manager.get_timer_client()
     transfer_client = login_manager.get_transfer_client()
 
@@ -162,30 +162,55 @@ def transfer_command(
 
     # Check endpoint activation, figure out scopes needed.
 
+    # the autoactivate helper may present output and exit in the case of v4 endpoints
+    # which need activation (e.g. OA4MP)
+    autoactivate(transfer_client, source_endpoint, if_expires_in=86400)
+    autoactivate(transfer_client, dest_endpoint, if_expires_in=86400)
+
     # check if either source or dest requires the data_access scope, and if so
-    # FIXME: hard fail for now (in the future, we should pick up on the requirement and
-    # generate a scope string to check against our logins)
+    # prompt the user to go through the requisite login flow
     source_epish = Endpointish(source_endpoint, transfer_client=transfer_client)
     dest_epish = Endpointish(dest_endpoint, transfer_client=transfer_client)
     needs_data_access: list[str] = []
     if source_epish.requires_data_access_scope:
         needs_data_access.append(str(source_endpoint))
     if dest_epish.requires_data_access_scope:
         needs_data_access.append(str(dest_endpoint))
+
+    # this list will only be populated *if* one of the two endpoints requires
+    # data_access, so if it's empty, we can skip any handling
     if needs_data_access:
-        raise click.UsageError(
-            "Unsupported operation. 'globus timer create transfer' does not currently "
-            "support collections which use the data_access scope: "
-            f"{','.join(needs_data_access)}"
-        )
+        # if the user is using client credentials, we cannot support the incremental
+        # auth step in the current implementation
+        #
+        # TODO: think through how we can use the client creds to request the
+        # requisite token in this case; it should be possible
+        if is_client_login():
+            raise click.UsageError(
+                "Unsupported operation. When using client credentials, "
+                "'globus timer create transfer' does not currently support "
+                "collections which use the data_access scope: "
+                f"{','.join(needs_data_access)}"
+            )
 
-    # Note this will provide help text on activating endpoints.
-    autoactivate(transfer_client, source_endpoint, if_expires_in=86400)
-    autoactivate(transfer_client, dest_endpoint, if_expires_in=86400)
+        request_data_access = _derive_needed_scopes(auth_client, needs_data_access)
+
+        if request_data_access:
+            scope_request_opts = " ".join(
+                f"--timer-data-access '{target}'" for target in request_data_access
+            )
+            click.echo(
+                f"""\
+A collection you are trying to use in this timer requires you to grant consent
+for the Globus CLI to access it.
+
+Please run
 
-    # XXX: this section needs to be re-evaluated when 'timer create transfer' gets more
-    # capabilities to handle endpoints with scope requirements
-    # this would likely be the place to insert scope related checks
+  globus session consent {scope_request_opts}
+
+to login with the required scopes"""
+            )
+            click.get_current_context().exit(4)
 
     transfer_data = globus_sdk.TransferData(
         source_endpoint=source_endpoint,
@@ -222,7 +247,98 @@ def transfer_command(
             name=name,
             stop_after=stop_after_date,
             stop_after_n=stop_after_runs,
-            scope=_TRANSFER_AP_SCOPE,
+            # the transfer AP scope string (without any dependencies)
+            scope="https://auth.globus.org/scopes/actions.globus.org/transfer/transfer",
         )
     )
     display(response, text_mode=TextMode.text_record, fields=JOB_FORMAT_FIELDS)
+
+
+def _derive_needed_scopes(
+    auth_client: globus_sdk.AuthClient,
+    needs_data_access: list[str],
+) -> list[str]:
+    from globus_sdk.scopes import GCSCollectionScopeBuilder
+
+    # read the identity ID stored from the login flow
+    # we will semi-gracefully handle the case of the data having been damaged/corrupted
+    user_data = read_well_known_config("auth_user_data")
+    if user_data is None:
+        raise RuntimeError(
+            "Identity ID was unexpectedly not visible in storage. "
+            "A new login should fix the issue. "
+            "Consider using `globus login --force`"
+        )
+    user_identity_id = user_data["sub"]
+
+    # get the user's Globus CLI consents
+    consents = auth_client.get(f"/v2/api/identities/{user_identity_id}/consents")[
+        "consents"
+    ]
+
+    # we need to now find the relevant consents which might match the data_access scopes
+    #
+    # this takes the form of a tree traversal
+    # the first parts of the tree should always be present, as they indicate the Timer
+    # consent which the CLI requests statically on login...
+
+    # find the top-level Timer consent
+    for consent in consents:
+        if (
+            consent["scope_name"] == globus_sdk.TimerClient.scopes.timer
+            and len(consent["dependency_path"]) == 1
+        ):
+            timer_consent = consent
+            break
+    else:
+        raise LookupError("could not find timer consent")
+
+    # find the Timer->TransferAP consent
+    first_order_dependencies = {
+        c["scope_name"]: c
+        for c in consents
+        if len(c["dependency_path"]) == 2
+        and timer_consent["id"] in c["dependency_path"]
+    }
+    timer2transferAP_consent = first_order_dependencies[
+        "https://auth.globus.org/scopes/actions.globus.org/transfer/transfer"
+    ]
+
+    # find the Timer->TransferAP->Transfer consent
+    second_order_dependencies = {
+        c["scope_name"]: c
+        for c in consents
+        if len(c["dependency_path"]) == 3
+        and timer_consent["id"] in c["dependency_path"]
+        and timer2transferAP_consent["id"] in c["dependency_path"]
+    }
+    timer2transferAP2transfer_consent = second_order_dependencies[
+        globus_sdk.TransferClient.scopes.all
+    ]
+
+    # find all of the Timer->TransferAP->Transfer->* consents
+    third_order_dependencies = {
+        c["scope_name"]: c
+        for c in consents
+        if len(c["dependency_path"]) == 4
+        and timer_consent["id"] in c["dependency_path"]
+        and timer2transferAP_consent["id"] in c["dependency_path"]
+        and timer2transferAP2transfer_consent["id"] in c["dependency_path"]
+    }
+
+    # in that last step, we reached the leaves of the tree
+    # (Okay, actually, that's a lie. We don't know what other values might exist
+    # further down in the tree. But luckily, it doesn't matter. We only care about the
+    # children of the node we've reached.)
+    # now we need to evaluate those leaves against our requirements
+
+    # check the 'needs_data_access' scope names against the 3rd-order dependencies
+    # of the Timer scope and record the names of the ones which we need to request
+    will_request_data_access: list[str] = []
+    for name in needs_data_access:
+        scope_name = GCSCollectionScopeBuilder(name).data_access
+        if scope_name not in third_order_dependencies:
+            will_request_data_access.append(name)
+
+    # return these ultimately filtered requirements
+    return will_request_data_access

tests/conftest.py

@@ -80,8 +80,14 @@ def mock_login_token_response():
     mock_token_res.by_resource_server = {
         "auth.globus.org": _mock_token_response_data(
             "auth.globus.org",
-            "openid profile email "
-            "urn:globus:auth:scope:auth.globus.org:view_identity_set",
+            " ".join(
+                [
+                    "openid",
+                    "profile",
+                    "email",
+                    "urn:globus:auth:scope:auth.globus.org:view_identity_set",
+                ]
+            ),
         ),
         "transfer.api.globus.org": _mock_token_response_data(
             "transfer.api.globus.org",
@@ -134,6 +140,12 @@ def test_token_storage(mock_login_token_response):
         "auth_client_data",
         {"client_id": "fakeClientIDString", "client_secret": "fakeClientSecret"},
     )
+    # NB: this carefully matches the ID provided by our "foo_user_info" data fixture
+    # in the future, this should be moved to a dedicated fixture providing the current
+    # user's identity ID
+    mockstore.store_config(
+        "auth_user_data", {"sub": "25de0aed-aa83-4600-a1be-a62a910af116"}
+    )
     mockstore.store(mock_login_token_response)
     return mockstore