Skip to content

Commit

Permalink
Make all tests pass on Windows.
Browse files Browse the repository at this point in the history
- Remove any explicit references to executables or paths that are not available on a standard Windows installation
- Run all tests in copies of the testdata dir; do not modify the main testdata dir
- Clean paths in tests to make Unix-style and Windows-style paths comparable
- Move some logic from the Makefile into the Go test driver, allowing tests to be run without Make
- Remove the Test_snippetCreate_Global_Editor test, which seems unlikely to work on Windows and of dubious value
- Update CONTRIBUTING.md to reflect reduced reliance on make
  • Loading branch information
matthewdale committed Jan 1, 2019
1 parent 78c18e0 commit d05ff83
Show file tree
Hide file tree
Showing 30 changed files with 361 additions and 180 deletions.
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -13,3 +13,4 @@ _test

coverage*
vendor
testdata-*
30 changes: 19 additions & 11 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@

## Project Goals

lab is fundamentally a workflow tool; we don't add features just to cover the API.
*lab* is fundamentally a workflow tool; we don't add features just to cover the API.
Instead, we add them to support and improve our cli workflows, which we want to flow seamlessly and feel intuitive and natural.

## Overview of Tests

lab runs integration tests in addition to unit tests. The integration tests run against gitlab.com. We are willing to trade some test autonomy and speed in exchange for 100% guarantees that features work against a live GitLab API. Integration tests are largely identified as tests which execute the `./lab_bin` binary. There are two primary projects used for these integration tests: [zaquestion/test](https://gitlab.com/zaquestion/test) and [lab-testing/test](https://gitlab.com/lab-testing/test).
*lab* runs integration tests in addition to unit tests. The integration tests run against [gitlab.com](https://gitlab.com). We are willing to trade some test autonomy and speed in exchange for 100% guarantees that features work against a live GitLab API. Integration tests are largely identified as tests which execute the `./lab.test` binary. There are two primary projects used for these integration tests: [zaquestion/test](https://gitlab.com/zaquestion/test) and [lab-testing/test](https://gitlab.com/lab-testing/test).

## Setup and Prerequestites

**New to Go?** Check out the Go docs on [How to Write Go Code](https://golang.org/doc/code.html) for some background on Go coding conventions, many of which are used by lab.
**New to Go?** Check out the Go docs on [How to Write Go Code](https://golang.org/doc/code.html) for some background on Go coding conventions, many of which are used by *lab*.

To run the lab tests, you will need:
1. A gitlab.com account configured with an [SSH key](https://docs.gitlab.com/ce/ssh/README.html#adding-an-ssh-key-to-your-gitlab-account). If you can push and pull from gitlab.com remotes, you're probably all set.
2. The `GOPATH` environment variable needs to be explicitly set. (eg `export GOPATH=$(go env GOPATH)`)
3. Add `$GOPATH/bin` to your `$PATH`.
3. The `GO111MODULE` environment variable needs to be set to `on`. (eg `export GO111MODULE=on`)
4. The tests assume that the lab source repo is located in `$GOPATH/src/github.com/zaquestion/lab`
4. It should go without saying, but you'll also need `go`, `git` and `make` installed.
To run the *lab* tests, you will need:
1. `go` and `git` must be installed (optionally `make`)
2. A gitlab.com account configured with an [SSH key](https://docs.gitlab.com/ce/ssh/README.html#adding-an-ssh-key-to-your-gitlab-account). If you can push and pull from gitlab.com remotes, you're probably all set.
3. The `GOPATH` environment variable needs to be explicitly set. (eg `export GOPATH=$(go env GOPATH)`)
4. Add `$GOPATH/bin` to your `$PATH`.
5. The `GO111MODULE` environment variable needs to be set to `on`. (eg `export GO111MODULE=on`)
6. The tests assume that the lab source repo is located in `$GOPATH/src/github.com/zaquestion/lab`

## Running Tests
In order to setup the integration test data, the tests must be run via `make test`:
Tests can be run via `make test`:

```sh
$ cd $GOPATH/src/github.com/zaquestion/lab
Expand All @@ -32,3 +32,11 @@ $ make test
# run only tests matching "pattern"
$ make test run=pattern
```

or with `go test`:

```sh
$ cd $GOPATH/src/github.com/zaquestion/lab

$ GO111MODULE=on go test ./cmd ./internal/...
```
4 changes: 1 addition & 3 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -12,10 +12,8 @@ test:

internal-test:
rm coverage-* 2>&1 > /dev/null || true
mv testdata/test.git testdata/.git
GO111MODULE=on go test -coverprofile=coverage-main.out -covermode=count -coverpkg ./... -run=$(run) $(GOURL)/cmd $(GOURL)/internal/...
mv testdata/.git testdata/test.git
go get github.com/wadey/gocovmerge
go get -u github.com/wadey/gocovmerge
gocovmerge coverage-*.out > coverage.txt && rm coverage-*.out

.PHONY: deps install test internal-test
2 changes: 1 addition & 1 deletion cmd/ci_lint_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
func Test_ciLint(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "ci", "lint")
cmd := exec.Command(labBinaryPath, "ci", "lint")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
2 changes: 1 addition & 1 deletion cmd/ci_run_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
func Test_ciRun(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "ci", "run")
cmd := exec.Command(labBinaryPath, "ci", "run")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
8 changes: 4 additions & 4 deletions cmd/ci_trace_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,21 +10,21 @@ import (
func Test_ciTrace(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "fetch", "origin")
cmd := exec.Command(labBinaryPath, "fetch", "origin")
cmd.Dir = repo
if b, err := cmd.CombinedOutput(); err != nil {
t.Log(string(b))
t.Fatal(err)
}

cmd = exec.Command("../lab_bin", "checkout", "origin/ci_test_pipeline")
cmd = exec.Command(labBinaryPath, "checkout", "origin/ci_test_pipeline")
cmd.Dir = repo
if b, err := cmd.CombinedOutput(); err != nil {
t.Log(string(b))
t.Fatal(err)
}

cmd = exec.Command("../lab_bin", "checkout", "-b", "ci_test_pipeline")
cmd = exec.Command(labBinaryPath, "checkout", "-b", "ci_test_pipeline")
cmd.Dir = repo
if b, err := cmd.CombinedOutput(); err != nil {
t.Log(string(b))
Expand Down Expand Up @@ -74,7 +74,7 @@ func Test_ciTrace(t *testing.T) {
test := test
t.Run(test.desc, func(t *testing.T) {
t.Parallel()
cmd = exec.Command("../lab_bin", append([]string{"ci", "trace"}, test.args...)...)
cmd = exec.Command(labBinaryPath, append([]string{"ci", "trace"}, test.args...)...)
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
2 changes: 1 addition & 1 deletion cmd/clone_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
func Test_clone(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "clone", "test")
cmd := exec.Command(labBinaryPath, "clone", "test")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
2 changes: 1 addition & 1 deletion cmd/fork_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ func Test_fork(t *testing.T) {
}

t.Run("do_fork", func(t *testing.T) {
cmd = exec.Command("../lab_bin", "fork")
cmd = exec.Command(labBinaryPath, "fork")
cmd.Dir = repo
b, err := cmd.CombinedOutput()
if err != nil {
Expand Down
2 changes: 1 addition & 1 deletion cmd/issue_create_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (

func Test_issueCreate(t *testing.T) {
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "create", "lab-testing",
cmd := exec.Command(labBinaryPath, "issue", "create", "lab-testing",
"-m", "issue title")
cmd.Dir = repo

Expand Down
12 changes: 6 additions & 6 deletions cmd/issue_edit_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ import (

// issueEditCmdTestCreateIssue creates an issue and returns the issue number
func issueEditCmdTestCreateIssue(t *testing.T, dir string) string {
cmd := exec.Command("../lab_bin", "issue", "create", "lab-testing",
cmd := exec.Command(labBinaryPath, "issue", "create", "lab-testing",
"-m", "issue title", "-l", "bug")
cmd.Dir = dir

Expand All @@ -29,7 +29,7 @@ func issueEditCmdTestCreateIssue(t *testing.T, dir string) string {

// issueEditCmdTestShowIssue returns the `lab issue show` output for the given issue
func issueEditCmdTestShowIssue(t *testing.T, dir string, issueNum string) string {
cmd := exec.Command("../lab_bin", "issue", "show", "lab-testing", issueNum)
cmd := exec.Command(labBinaryPath, "issue", "show", "lab-testing", issueNum)
cmd.Dir = dir

b, err := cmd.CombinedOutput()
Expand All @@ -46,7 +46,7 @@ func Test_issueEditCmd(t *testing.T) {
issueNum := issueEditCmdTestCreateIssue(t, repo)

// update the issue
cmd := exec.Command("../lab_bin", "issue", "edit", "lab-testing", issueNum,
cmd := exec.Command(labBinaryPath, "issue", "edit", "lab-testing", issueNum,
"-m", "new title")
cmd.Dir = repo

Expand All @@ -70,7 +70,7 @@ func Test_issueEditLabels(t *testing.T) {
issueNum := issueEditCmdTestCreateIssue(t, repo)

// update the issue
cmd := exec.Command("../lab_bin", "issue", "edit", "lab-testing", issueNum,
cmd := exec.Command(labBinaryPath, "issue", "edit", "lab-testing", issueNum,
"-l", "critical", "--unlabel", "bug")
cmd.Dir = repo

Expand All @@ -94,7 +94,7 @@ func Test_issueEditAssignees(t *testing.T) {
issueNum := issueEditCmdTestCreateIssue(t, repo)

// add an assignee
cmd := exec.Command("../lab_bin", "issue", "edit", "lab-testing", issueNum,
cmd := exec.Command(labBinaryPath, "issue", "edit", "lab-testing", issueNum,
"-a", "lab-testing")
cmd.Dir = repo

Expand All @@ -111,7 +111,7 @@ func Test_issueEditAssignees(t *testing.T) {
require.Contains(t, issueShowOuput, "Assignees: lab-testing")

// now remove the assignee
cmd = exec.Command("../lab_bin", "issue", "edit", "lab-testing", issueNum,
cmd = exec.Command(labBinaryPath, "issue", "edit", "lab-testing", issueNum,
"--unassign", "lab-testing")
cmd.Dir = repo

Expand Down
8 changes: 4 additions & 4 deletions cmd/issue_list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import (
func Test_issueList(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "list")
cmd := exec.Command(labBinaryPath, "issue", "list")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -27,7 +27,7 @@ func Test_issueList(t *testing.T) {
func Test_issueListFlagLabel(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "list", "-l", "enhancement")
cmd := exec.Command(labBinaryPath, "issue", "list", "-l", "enhancement")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -43,7 +43,7 @@ func Test_issueListFlagLabel(t *testing.T) {
func Test_issueListStateClosed(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "list", "-s", "closed")
cmd := exec.Command(labBinaryPath, "issue", "list", "-s", "closed")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -59,7 +59,7 @@ func Test_issueListStateClosed(t *testing.T) {
func Test_issueListSearch(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "list", "filter labels")
cmd := exec.Command(labBinaryPath, "issue", "list", "filter labels")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
2 changes: 1 addition & 1 deletion cmd/issue_note_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (

func Test_issueCreateNote(t *testing.T) {
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "note", "lab-testing", "1",
cmd := exec.Command(labBinaryPath, "issue", "note", "lab-testing", "1",
"-m", "note text")
cmd.Dir = repo

Expand Down
2 changes: 1 addition & 1 deletion cmd/issue_show_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import (
func Test_issueShow(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "show", "1", "--comments")
cmd := exec.Command(labBinaryPath, "issue", "show", "1", "--comments")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
8 changes: 4 additions & 4 deletions cmd/issue_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ func Test_issueCmd(t *testing.T) {
var issueID string
t.Run("create", func(t *testing.T) {
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "create", "lab-testing",
cmd := exec.Command(labBinaryPath, "issue", "create", "lab-testing",
"-m", "issue title",
"-m", "issue description",
"-l", "bug",
Expand All @@ -39,7 +39,7 @@ func Test_issueCmd(t *testing.T) {
t.Skip("issueID is empty, create likely failed")
}
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "show", "lab-testing", issueID)
cmd := exec.Command(labBinaryPath, "issue", "show", "lab-testing", issueID)
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -61,7 +61,7 @@ func Test_issueCmd(t *testing.T) {
t.Skip("issueID is empty, create likely failed")
}
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue", "lab-testing", "-d", issueID)
cmd := exec.Command(labBinaryPath, "issue", "lab-testing", "-d", issueID)
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -75,7 +75,7 @@ func Test_issueCmd(t *testing.T) {

func Test_issueCmd_noArgs(t *testing.T) {
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "issue")
cmd := exec.Command(labBinaryPath, "issue")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
6 changes: 3 additions & 3 deletions cmd/mr_checkout_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ func Test_mrCheckoutCmdRun(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)

cmd := exec.Command("../lab_bin", "mr", "checkout", "1")
cmd := exec.Command(labBinaryPath, "mr", "checkout", "1")
cmd.Dir = repo
b, err := cmd.CombinedOutput()
if err != nil {
Expand Down Expand Up @@ -44,7 +44,7 @@ func Test_mrCheckoutCmd_track(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)

cmd := exec.Command("../lab_bin", "mr", "checkout", "1", "-t", "-b", "mrtest_track")
cmd := exec.Command(labBinaryPath, "mr", "checkout", "1", "-t", "-b", "mrtest_track")
cmd.Dir = repo
b, err := cmd.CombinedOutput()
if err != nil {
Expand Down Expand Up @@ -86,7 +86,7 @@ func Test_mrCheckoutCmdRunWithDifferentName(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)

cmd := exec.Command("../lab_bin", "mr", "checkout", "1", "-b", "mrtest_custom_name")
cmd := exec.Command(labBinaryPath, "mr", "checkout", "1", "-b", "mrtest_custom_name")
cmd.Dir = repo
b, err := cmd.CombinedOutput()
if err != nil {
Expand Down
18 changes: 10 additions & 8 deletions cmd/mr_list_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,14 @@ import (
"strings"
"testing"

"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)

func Test_mrList(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list")
cmd := exec.Command(labBinaryPath, "mr", "list")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -27,7 +28,7 @@ func Test_mrList(t *testing.T) {
func Test_mrListFlagLabel(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-l", "confirmed")
cmd := exec.Command(labBinaryPath, "mr", "list", "-l", "confirmed")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -43,7 +44,7 @@ func Test_mrListFlagLabel(t *testing.T) {
func Test_mrListStateMerged(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-s", "merged")
cmd := exec.Command(labBinaryPath, "mr", "list", "-s", "merged")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -59,7 +60,7 @@ func Test_mrListStateMerged(t *testing.T) {
func Test_mrListStateClosed(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-s", "closed")
cmd := exec.Command(labBinaryPath, "mr", "list", "-s", "closed")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -76,7 +77,7 @@ func Test_mrListStateClosed(t *testing.T) {
func Test_mrListFivePerPage(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-n", "5")
cmd := exec.Command(labBinaryPath, "mr", "list", "-n", "5")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -92,7 +93,7 @@ func Test_mrListFivePerPage(t *testing.T) {
func Test_mrFilterByTargetBranch(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-t", "non-existing")
cmd := exec.Command(labBinaryPath, "mr", "list", "-t", "non-existing")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand All @@ -101,13 +102,14 @@ func Test_mrFilterByTargetBranch(t *testing.T) {
}

mrs := strings.Split(string(b), "\n")
require.Equal(t, 3, len(mrs))
mrs = truncAppOutput(mrs)
assert.Empty(t, mrs, "Expected to find no MRs for non-existent branch")
}

func Test_mrListByTargetBranch(t *testing.T) {
t.Parallel()
repo := copyTestRepo(t)
cmd := exec.Command("../lab_bin", "mr", "list", "-t", "master")
cmd := exec.Command(labBinaryPath, "mr", "list", "-t", "master")
cmd.Dir = repo

b, err := cmd.CombinedOutput()
Expand Down
Loading

0 comments on commit d05ff83

Please sign in to comment.