Skip to content

Commit 40afbb4

Browse files
committed
gdrive: add User Guide page
1 parent ef74b9d commit 40afbb4

File tree

6 files changed

+132
-49
lines changed

6 files changed

+132
-49
lines changed

public/static/docs/command-reference/remote/add.md

Lines changed: 7 additions & 44 deletions
Original file line numberDiff line numberDiff line change
@@ -200,52 +200,15 @@ $ dvc remote add myremote "azure://"
200200

201201
### Click for Google Drive
202202

203-
Since Google Drive has tight API usage quotas, creation and configuration of
204-
your own `Google Project` is required:
205-
206-
1. Log into the [Google Cloud Platform](https://console.developers.google.com)
207-
account.
208-
2. Create `New Project` or select available one.
209-
3. Click `ENABLE APIS AND SERVICES` and search for `drive` to enable
210-
`Google Drive API` from search results.
211-
4. Navigate to
212-
[All Credentials](https://console.developers.google.com/apis/credentials)
213-
page and click `Create Credentials` to select `OAuth client ID`. It might
214-
ask you to setup a product name on the consent screen.
215-
5. Select `Other` for `Application type` and click `Create` to proceed with
216-
default `Name`.
217-
6. `client id` and `client secret` should be showed to you. Use them for
218-
further DVC's configuration.
203+
Please check out
204+
[Setup a Google Drive DVC Remote](/doc/user-guide/setup-google-drive-remote) for
205+
a full guide on configuring Google Drives for use as DVC remote storage,
206+
including obtaining the necessary credentials, and how to form `gdrive://` URLs.
219207

220208
```dvc
221-
$ dvc remote add myremote gdrive://root/my-dvc-root
222-
$ dvc remote modify myremote gdrive_client_id my_gdrive_client_id
223-
$ dvc remote modify myremote gdrive_client_secret gdrive_client_secret
224-
```
225-
226-
> See `dvc remote modify` for a full list of GDrive parameters.
227-
228-
On first usage of the remote you will be prompted to visit an access token
229-
generation URL via browser. It will ask you to log into the Google account
230-
associated with the Google Drive you want to use as remote. The login process
231-
will guide you through the granting of the required access permissions.
232-
233-
On successful access token generation, the token data will be cached in a
234-
Git-ignored directory (located in `.dvc/tmp/gdrive-user-credentials.json`).
235-
236-
> 💡 Do not share the token data with anyone else to prevent unauthorized access
237-
> to your Google Drive.
238-
239-
**Support for shared drives**
240-
241-
For this, you need to obtain the directory ID and use it as part of URL passed
242-
to DVC. This ID can be found in your web browser address bar when the shared
243-
drive is opened. For example, for the URL
244-
`https://drive.google.com/drive/folders/0AIac4JZqHhKmUk9PDA`, use
245-
`0AIac4JZqHhKmUk9PDA` as ID:
246-
247-
```dvc
248-
$ dvc remote add myremote gdrive://0AIac4JZqHhKmUk9PDA/my-dvc-root
209+
$ dvc remote add -d myremote gdrive://root/path/to/folder
210+
$ dvc remote modify myremote gdrive_client_id <client ID>
211+
$ dvc remote modify myremote gdrive_client_secret <client secret>
249212
```
250213

251214
Note that GDrive remotes are not "trusted" by default. This means that the

public/static/docs/command-reference/remote/modify.md

Lines changed: 10 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -248,22 +248,27 @@ For more information on configuring Azure Storage connection strings, visit
248248

249249
### Click for Google Drive
250250

251+
Please check out
252+
[Setup a Google Drive DVC Remote](/doc/user-guide/setup-google-drive-remote) for
253+
a full guide on configuring Google Drives for use as DVC remote storage,
254+
including obtaining the necessary credentials, and how to form `gdrive://` URLs.
255+
251256
- `url` - remote location URL.
252257

253258
```dvc
254-
$ dvc remote modify myremote url "gdrive://root/my-dvc-root"
259+
$ dvc remote modify myremote url gdrive://root/path/to/folder
255260
```
256261

257-
- `gdrive_client_id` - Google Project's OAuth 2.0 client id.
262+
- `gdrive_client_id` - Google Project's OAuth 2.0 **client ID**.
258263

259264
```dvc
260-
$ dvc remote modify myremote gdrive_client_id my_gdrive_client_id
265+
$ dvc remote modify myremote gdrive_client_id <client ID>
261266
```
262267

263-
- `gdrive_client_secret` - Google Project's OAuth 2.0 client secret.
268+
- `gdrive_client_secret` - Google Project's OAuth 2.0 **client secret**.
264269

265270
```dvc
266-
$ dvc remote modify myremote gdrive_client_secret gdrive_client_secret
271+
$ dvc remote modify myremote gdrive_client_secret <client secret>
267272
```
268273

269274
</details>

public/static/docs/sidebar.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,7 @@
130130
}
131131
},
132132
"updating-tracked-files",
133+
"setup-google-drive-remote",
133134
"large-dataset-optimization",
134135
"external-dependencies",
135136
{
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# Setup a Google Drive DVC Remote
2+
3+
Follow this guide to setup Google Drive as your <abbr>DVC project</abbr>'s
4+
[remote storage](/doc/command-reference/remote).
5+
6+
## Configure a Google Cloud project
7+
8+
DVC uses the [Google Drive API](https://developers.google.com/drive) to connect
9+
to your Google Drive. This requires a Google Cloud _project_ that allows Drive
10+
API connections, and its
11+
[OAuth](https://developers.google.com/identity/protocols/OAuth2) credentials
12+
(**client ID** and **client secret**).
13+
14+
1. Sign into the [Google API Console](https://console.developers.google.com).
15+
16+
> Double check you're using the intended Google account (upper-right corner).
17+
18+
2. Select or
19+
[Create](https://cloud.google.com/resource-manager/docs/creating-managing-projects#creating_a_project)
20+
a project for DVC remote connections.
21+
22+
3. [Enable the Drive API](https://developers.google.com/drive/api/v2/about-sdk)
23+
from the **APIs & Services** **Dashboard** (left sidebar), click on **+
24+
ENABLE APIS AND SERVICES**. Find and select the "Google Drive API" in the API
25+
Library, and click on the **ENABLE** button.
26+
27+
![](/static/img/gdrive-enable-apis-and-services.png)
28+
29+
4. Go back to **APIs & Services** in the left sidebar, and select **OAuth
30+
consent screen**. Chose a **User Type** and click **CREATE**. On the next
31+
screen, enter an **Application name** e.g. "DVC remote storage", and click
32+
the **Save** (scroll to bottom).
33+
34+
5. From the left sidebar, select **Credentials**, and click the **Create
35+
credentials** dropdown to select **OAuth client ID**. Chose **Other** and
36+
click **Create** to proceed with a default client name.
37+
38+
![](/static/img/gdrive-create-credentials.png)
39+
40+
6. The newly generated **client ID** and **client secret** should be shown to
41+
you now, and you can always come back to **Credentials** to fetch them.
42+
43+
⚠️ It should be safe to share **client ID** and **client secret** among your
44+
team. These credentials are only used to generate the
45+
[authorization](#authorization) DVC will later need in order to connect to the
46+
Google Drive.
47+
48+
> Note that Google Drive API has usage limits/quotas per credentials in a
49+
> _project_ (which you can review in the
50+
> [OAuth consent screen](https://console.developers.google.com/apis/credentials/consent)).
51+
> Please keep this in mind when sharing them, or you may
52+
> [exceed the limits](https://developers.google.com/drive/api/v2/handle-errors?hl=ro#resolve_a_403_error_usage_limit_exceeded).
53+
54+
## URL format for Google Drive remotes
55+
56+
A GDrive DVC remote URLs is needed for the `dvc remote add` command. It can be
57+
constructed with a _base_, and an **optional** _path_ to a folder i.e.
58+
`gdrive://<base>/path/to/folder`. The base can be one of:
59+
60+
1. `root` - indicates your topmost Google Drive directory.
61+
62+
```dvc
63+
$ dvc remote add -d mygdroot gdrive://root
64+
$ dvc remote add mygdfolder gdrive://root/path/to/folder
65+
```
66+
67+
2. Google Drive _Folder ID_
68+
69+
To obtain the unique _Folder ID_ in question, navigate into that folder in your
70+
web browser, and find it in the address bar. For example, for
71+
`https://drive.google.com/drive/folders/0AIac4JZqHhKmUk9PDA`:
72+
73+
```dvc
74+
$ dvc remote add mygdfolder gdrive://0AIac4JZqHhKmUk9PDA
75+
$ dvc remote add mygdsubfolder \
76+
gdrive://0AIac4JZqHhKmUk9PDA/sub/path
77+
```
78+
79+
[Shared drives](https://developers.google.com/drive/api/v2/about-shareddrives)
80+
should be referenced by _Folder ID_ in the same way.
81+
82+
3. `appDataFolder` - special
83+
[application-specific data](https://developers.google.com/drive/api/v2/appdata)
84+
folder only accessible by your application and hidden from the user.
85+
86+
```dvc
87+
$ dvc remote add mygdappata gdrive://appDataFolder
88+
```
89+
90+
## Configure the GDrive remote
91+
92+
Use the `dvc remote modify` command to set the credentials for each `gdrive://`
93+
remote, for example:
94+
95+
```dvc
96+
$ dvc remote add -d mygdfolder gdrive://root/path/to/folder
97+
$ dvc remote modify mygdfolder gdrive_client_id <client ID>
98+
$ dvc remote modify mygdfolder gdrive_client_secret <client secret>
99+
```
100+
101+
## Authorization
102+
103+
On the first usage of a GDrive [DVC remote](/doc/command-reference/remote),
104+
you'll be prompted to visit a web page to generate an access token, which will
105+
require you to sign into the corresponding Google account. The
106+
[sign-in process](https://developers.google.com/drive/api/v2/about-auth) will
107+
guide you through granting the necessary access permissions. On success, the
108+
token data will be cached in a Git-ignored directory located in
109+
`.dvc/tmp/gdrive-user-credentials.json`.
110+
111+
⚠️ In order to prevent unauthenticated access to your Google Drive, **do not
112+
share access token data with others**. Each team member should have
113+
[separate access](https://support.google.com/drive/answer/7166529) to the
114+
appropriate Google Drive resources, and generate their own tokens.
51.7 KB
Loading
23.2 KB
Loading

0 commit comments

Comments
 (0)