gsheet

Table of Contents

• Introduction
  • Why?
• Installation
• Authentication and Authorization
  • What about OAuth authentication?
• CLI Usage
  • Sheet commands
  • Drive commands
  • Ranges
  • Finding document and parent IDs
  • Environment Variables
• Use as Golang Package
• Hack

Introduction

gsheet is a cli tool for sending and receiving csv data to and from Google Sheets.

With gsheet you can:

• Pipe csv data from stdin to a Google Sheet range
  cat data.csv | gsheet csv --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range 'Sheet1'

• Pipe csv data from a Google Sheet range to stdout
  gsheet csv --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range 'Sheet!A1:D20' > data.csv

• Clear a Google Sheet range
  gsheet clear --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --range Sheet2

• Create and delete sheets of a Spreadsheet document
  gsheet newSheet --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --name NewSheet

• Sort a sheet by a column
  gsheet sort --id 1o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ --name Sheet1 -c 2

Google Drive operations (with special handling for .csv):

• Upload .csv files to Google Drive as Sheet documents
  gsheet upload --parent PARENT_ID ./path/to/data.csv

• Download Google Sheet documents from Google Drive as .csv files
  gsheet download 2o88FhvAXg8Q_ZMFudQLuZ1ShsigbAgJ > data.csv

• Create a new empty folder
  gsheet createFolder --parent PARENT_ID 'New Folder Name'

You can also upload/delete and get info about arbitrary files on Google Drive, but gsheet isn’t trying to be a general purpose gdrive interface. If you need a full Google Drive command-line client, check out odeke-em/drive, or something that provides a filesystem interface like rclone.

For more on how to use gsheet see CLI Usage below.

gsheet supports authenticating with Google using service accounts, which is a simple way to allow scripts to access Google Sheets (see Authentication and Authorization below).

Why?

I often write scripts and cron jobs for clients that create tabular reports, and I’ve found that Google Sheets is a convenient way to expose them on the web (easy to use, built-in access control). In addition, I’ve found that setting up a Sheet with a few fields that my script can read is a convenient way to allow clients to configure apps without needing to build a web interface just to get a few runtime config values.

My typical workflow is to create a Sheet using Google’s web interface, set up the formatting and any formulas, and then use gsheet to update the data from scripts.

Installation

gsheet is not yet packaged for any package manager yet (let me know if you can help with that!), but you can download binaries for major platforms from the Releases page.

Otherwise, if you have Go installed you can:

# Install with go
go install github.com/cristoper/gsheet/cmd/gsheet@latest

# or build/install from git repo
git clone https://github.com/cristoper/gsheet.git
cd gsheet
go install ./cmd/gsheet

However, building from source will pull in the build dependencies (Google’s API SDKs) which are big (200MB+) so can be slow to download.

Authentication and Authorization

The hardest part about getting started with gsheet is creating the API credentials so that the program can access Google Drive, but it’s not so bad and you only have to do it once. In order for gsheet to read and update Sheets documents, it must 1) be provided credentials to authenticate with Google and 2) be granted access to whichever Google Drive folders/documents it should be able to read/write:

1. Create a service account and download the credentials .json file to the computer you will use gsheet on. Set the GOOGLE_APPLICATION_CREDENTIALS envvar to the absolute path to that file before running gsheet.

   • From the Google Cloud API Dashboard create a new project and enable both the Google Drive API and the Google Sheets API on it. See Google’s Getting Started Guide for creating a project and enabling APIs.

   • From your project’s dashboard Click on "IAM & Admin" > "Service Accounts". Then click "Create Service Account". For the service account’s role I recommend "Basic>Editor".

   • Once the service account is created, click on it to manage its details. From the "Keys" tab click "Add Key" to create credentials in a .json file for the service account that gsheet will use to authenticate as the user.

     • The .json file containing the credentials should download automatically. KEEP THIS FILE SECRET (do not check it into source control). Anyone with the credentials can edit any files you grant the service account access to.

     • Google’s documentation is available as Creating and Managing Service Accounts and Creating and Managing Service Account Keys

   • Set the GOOGLE_APPLICATION_CREDENTIALS environment variable to the path to the .json file. This is how gsheet finds the credentials when it runs. NOTE: GOOGLE_APPLICATION_CREDENTIALS must contain the absolute path to the .json file. (See Google’s Authentication Overview.)

2. In Google Drive grant share files and folders with the service account (the email address generated when you created the service account above) as if it were any other user. Note that if you only want to store files in the service account’s own drive (not viewable from the Google Drive web interface), then you don’t actually have to grant it access to any of your folders or documents.

What about OAuth authentication?

Currently gsheet only supports service accounts which, despite the convoluted instructions above, are simple to generate and use once you figure it out (you just need a single .json file with credentials to grant a script access to Google services).

However, if an OAuth workflow (where you can authenticate gsheet using your own Google account) would be useful to someone, I’m open to implementing it. Feel free to create an issue. In the mean time check out simon3z/gsheetcsv which is another simple cli tool in Go which allows interacting with Google Sheets and uses OAuth for authentication.

CLI Usage

To get an overview of all the commands provided by gsheet run:

gsheet help

NAME:
   gsheet - upload and download Google Sheet data from the cli

USAGE:
   gsheet [global options] command [command options] [arguments...]

COMMANDS:
   help, h  Shows a list of commands or help for one command
   Files:
     createFolder  Creates a new folder
     delete        Delete file(s) from drive (careful, does not trash them!)
     list          List file names and ids
     upload        Upload a file to Google Drive.
     download      Download a file from google drive and send it to stdout
     info          Dump all file's metadata as json to stdout
   Sheets:
     csv          Pipe csv data to range or read it from range
     title        Get the title of a sheet by its id
     sheetInfo    Dump info about the spreadsheet as json
     clear        Clear all values from given range
     newSheet     Create a new sheet
     deleteSheet  Delete the named sheet
     sort         Sort a sheet by column(s)

GLOBAL OPTIONS:
   --help, -h  show help

You can also run gsheet help CMD to get help for each command. Below are some further usage hints.

Remember that for any of the commands to work you must have the GOOGLE_APPLICATION_CREDENTIALS environment variable set to a path containing the service account credentials in a .json file.

Sheet commands

csv and clear

The csv command is the heart of gsheet. If you pipe csv data to it on std input, it sends the data to the specified range of the Sheets document identified by the --id flag. If you pass the --append flag, data will be appended to the last row of data found in range.

If you don’t connect stdin to a pipe, then it will read the specified range and output it to stdout in csv format. To force gsheet to read a range even if stdin is not connected to a tty, you can pass the --read flag.

Note	csv does not clear the range before updating data in a Sheets document. If the piped data is smaller (fewer rows or columns) than the specified range, then any pre-existing data in the spreadsheet will remain after the update. Use gsheet clear to clear a range.

# Replace an entire sheet of a Spreadsheet doc with the contents of data.csv
gsheet --id SHEETS_DOC_ID clear --range Sheet1
cat data.csv | gsheet --id SHEETS_DOC_ID --range Sheet1

# Append the contents of data.csv after the lat line of existing data in Sheet1
cat data.csv | gsheet --id SHEETS_DOC_ID --range Sheet1 --append

# Read a specific range of a sheet to output.csv
# (You can always single quote sheet names and include the exclamation point in
# the single quotes so that the shell doesn't try to interpret it.)
gsheet --id SHEETS_DOC_ID --range 'Sheet1!'A2:C5 > output.csv

sort

An existing sheet can be sorted by any (single) column in either descending (default) or ascending order:

# Sort sheet by B column in ascending order
sort --id SHEET_NAME -name Sheet1 --column=1 --asc

newSheet and deleteSheet

These commands simply create and delete sheets from a spreadsheet document. The new sheets appear after all other visible sheets.

Note	sheets are deleted by name (the title of the sheet) and not by id; this is a bit fragile because if a user changes the title of a sheet in Google Docs then a script depending on gsheet deleteSheet may break. For a convenient way to look up a sheet’s title by its id, see the gsheet title command.

# After running this you should see a new sheet called "SHEET_NAME" in the
# spreadsheet with id "SHEETS_DOC_ID"
gsheet newSheet --id SHEETS_DOC_ID --name SHEET_NAME

# After running this it should be gone again
gsheet deleteSheet --id SHEETS_DOC_ID --name SHEET_NAME

Drive commands

upload and download

The upload and download commands can be used to upload and download arbitrary files to Google Drive. They provide special handling for .csv files: uploading a .csv file will import it to Google Drive as a Sheets document, and downloading a Sheets document will export the first visible sheet as a .csv file.

Downloading any other Google Workspace document types will attempt to export them as plain text files.

Not that using upload without giving it a parent id with --parent (or setting the GSHEET_PARENT envar) will cause it to upload the file to the service account’s root folder where it is not accessible to humans via Google Drive.

# Upload data.csv as a Sheets document in the service account's root directory
gsheet upload --parent root data.csv

# Download an image from drive
# Note that download takes a single positional argument: the id of the google
# drive file to download, and it sends its output to stdout.
gsheet download DRIVE_DOC_ID > image.png

delete

The delete command can be used to delete one or more files by id (list each id as a positional argument). Outputs a confirmation as each file is deleted.

Note	delete immediately deletes a file and does not move it to the trash.

list

# List all files and their ids that are in the service account's root folder
gsheet list --parent root

createFolder

Sometimes it is nice if a script can create a new folder to keep all of its own files in. The output of the createFolder command includes the id of the created folder.

# Create a foler in service account's root (specify --parent to use a different
# folder)
$ gsheet createFolder FOLDER_NAME
Created directory named FOLDER_NAME with id 1ApMOHtZtTVM_UU7HyUCvMIIa3R5fDf6N

Ranges

The csv and other commands make use of ranges in A1 notation. Examples of A1 notation can be found in the Google documentation here:

https://developers.google.com/sheets/api/guides/concepts

Finding document and parent IDs

Many of the commands operate on the Google Drive ID of a document or a "parent" folder. A convenient way to get these IDs is to just use a web browser and open a file or folder on https://drive.google.com/ to see the ID in the URL. But you can also use gsheet list to list all of the files and folders the service account knows about along with their IDs.

Environment Variables

GOOGLE_APPLICATION_CREDENTIALS

Must be set to the absolute path of a .json file containing credentials for a service account

GSHEET_ID

Can be set instead of setting the --id flag on any command that accepts that flag to identify a spreadsheet document to operate on

GSHEET_PARENT

Can be set instead of setting the --parent flag on any command that accepts that flag to identify a drive folder to operate on

Use as Golang Package

In addition to the cli tool, gsheet can be used as a Golang package to simplify access to Google Sheets and Google Drive from Go.

All of the Sheets related functions are in the gsheets package (gsheets/sheets.go), and all of the Drive related functions are in the gdrive pacakge (gdrive/files.go).

Online godoc documentation for the packages can be found here:

• https://pkg.go.dev/github.com/cristoper/gsheet/gdrive

• https://pkg.go.dev/github.com/cristoper/gsheet/gsheets

For a quick-and-dirty example of how to use the packages look at the integration_test.go file included in each package.

Hack

To run tests:

make test

To build:

make build

To build binaries for various platforms in build/:

make xbuild

To release:

tbd