os/exec: execution of batch-files (.cmd/.bat) is vulnerable in go-lang for windows / insufficient escape

[sebres]

Execution of batch-files using os/exec with arguments containing some special meta-characters is vulnerable and may be used to execute foreign data/code.

What version of Go are you using (go version)?

Latest stable build: go1.10.3 windows/386

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

Windows / x86

set GOARCH=386
set GOBIN=
set GOCACHE=%USERPROFILE%\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=386
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=%USERPROFILE%\go
set GORACE=
set GOROOT=C:\dev\go
set GOTMPDIR=
set GOTOOLDIR=C:\dev\go\pkg\tool\windows_386
set GCCGO=gccgo
set GO386=sse2
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m32 -mthreads -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=C:\Temp\go-build080181794=/tmp/go-build -gno-record-gcc-switches

What did you do?

Execution of batch-file using os/exec with arguments containing some special meta-characters.

A recipe for reproducing the error as well as more extensive PoC with additional info (and more lang's affected also) - github/sebres/PoC/SB-0D-001-win-exec
A complete runnable program - test-dump-part.go:

Content of test-dump-part.go

package main 

import (
  "os"
  "os/exec"
)

func main() {
  args := os.Args[2:]
  cmd := exec.Command(os.Args[1], args...)

  cmd.Stdout = os.Stdout
  cmd.Stderr = os.Stderr

  err := cmd.Start()
  if err != nil {
    os.Exit(1)
  }
}

An example:

# invoke exe-file:
go run test-dump-part.go test-dump.exe "test&whoami"
+    `test-dump.exe´ `test&whoami´
# invoke cmd-file:
go run test-dump-part.go test-dump.CMD "test&whoami"
-    `test-dump.exe´ `test´my_domain\sebres

For more "broken" cases see the result of my test-suite:
https://github.com/sebres/PoC/blob/master/SB-0D-001-win-exec/results/go.diff

What did you expect to see?

Arguments are escaped/quoted properly.

What did you see instead?

Arguments are insufficient escaped/quoted, so it is vulnerable currently.

Solution:

For possible solution see the algorithm description resp. how it was fixed in TCL (see the function BuildCommandLine)

Possible similar issues:

#17149, #3752

[ianlancetaylor]

What is interpreting the special characters? It's not immediately clear to me that this should be the responsibility of the os/exec package. But I don't know much about things work on Windows.

[sebres]

What is interpreting the special characters?

What do you mean exactly?

In case of execution of the process (no matter exe-file of bat- or cmd-file) with arguments (so as safe method with argument-list, not using shell-syntax command line) - the special meta-chars should not cause any behavior, allowing to produce an injection, that can cause execution of something else as the process self.
Anything else would be a nonsense!

It's not immediately clear to me that this should be the responsibility of the os/exec package.

So whose responsibility is should be in your opinion?
The developer using go-lang for win, I assume... Well, I thought that "go" is a platform-independend language... If not - just close the issue and forget it.

[sebres]

BTW as regards the "Go1.12 milestone"...
In my opinion it is a 0-day vulni, that should be fixed asap in all still supported versions.
But you are the boss...

[FMNSSun]

Could you point to relevant msdocs? Generally users should be aware that if they execute a file windows will look up the assoc and ftype to know how to "execute" that file and special restrictions might apply depending on the file type you execute? If you execute test.myfile you must escape arguments properly for whatever interpreter/exe actually executes test.myfile. But I guess it's fair to say that .bat and .cmd are just inherently part of windows as their ftype suggests it's hardcoded somewhere much deeper because those don't redirect directly to cmd.exe:

C:\WINDOWS\system32>ftype batfile
batfile="%1" %*

There's this: https://msdn.microsoft.com/en-us/library/a1y7w461.aspx but it seems that cmd.exe has it's own argument parse algorithm indeed.

[ALTree]

In my opinion it is a 0-day vuln, that should be fixed asap in all still supported versions.

FWIW (suspected) security vulnerabilities should be disclosed privately (see https://golang.org/security).

[sebres]

Could you point to relevant msdocs?

http://msdn.microsoft.com/en-us/library/17w5ykft.aspx
or search http://msdn.microsoft.com for "Parsing C++ Command-Line Arguments" resp. how arguments in escaped and unescaped quotes will be processed.

Generally users should be aware that if they execute a file windows will look up the assoc and ftype to know how to "execute" that file...

Generally any program can do with supplied arguments what it wants. But this is not a reason to disregard the common rules how arguments will be unescaped back inside the windows process.
As you can possibly see, my algorithm works the same way by execution of exe-file and cmd-file.
So it is universal for both... Current algorithm of go-lang does it not (so disregarded the rule applied by recall via comspec).

... and special restrictions might apply depending on the file type you execute?

Yes. It is still worse in windows, because the whole command-line will be build together, before the child-process invoked, and takes splitting (parsed back) into argv by the CRT-library or CommandLineToArgv, etc inside the child-process (whatever it is).
But the common escape-rules under windows must nevertheless be taken into account.

Consider this scenario. You can register how windows should execute your files using assoc and ftype ...

You don't really want to compare the execution from go-lang with execution within command-processor (inside the batch-file)? Because this way the args you supplied as command will be processed inside the COMSPEC (still BEFORE the execution occurs there).
Don't try to test it there (otherwise you will experience still more surprises ;).

What happens if foo.exe does some special parsing of the command line

It does no matter.

So again, I told about common escape rules (exe-file, batch-file) and not how one can define it's own handling in the child-process.
One could write the executable, that will split and join the command-line/args several times, but this changes nothing as regards the initial way how this args reached the executable from parent process or command processor "recall".

[FMNSSun]

@sebres

I mean https://msdn.microsoft.com/en-us/library/17w5ykft.aspx doesn't mention that "test&whoami" requires special escaping. (Btw: I'm not part of the go project at all). I was just curious if there's documentation available for how this process exactly works because there must be something inherently different in how the microsoft crt parses arguments than to how they are parsed for the command interpreter so that one could fix this manually in vulnerable projects/code/dependencies one uses. You propose an algorithm as a solution but how can one verify that your solution is actually correct and covers everything without having documentation to verify this?

[sebres]

FWIW (suspected) vulnerabilities should be disclosed privately

So could you call me the person, which I could reach "privately"? :)

But I'm agree, if it would be absolutely 100% reproducible vulnerability, where for example some service is affected by supplying of some concrete arguments - that is the right way.
Here I wrote a PoC for potential "vulnerability" of many lang's for windows (not the concrete implementation where it's used), in this case it is better that the people having potentially vulnerable places using exec's like this, could check their services self and do something to avoid potential injuries, until I have convinced the community (we'll clear the responsibility question), the maintainer of language accepted/confirmed the issue, developer of lang get fixed it, and the next release will reach the "affected" service a year later.

Savvy?

[sebres]

I was just curious if there's documentation available for how this process exactly works...

There is no documentation as one place (as well as no procedure in windows like ArgvToCommandLine as opposite for CommandLineToArgv), one should discover multiple pices of docu and analyze several results to come to the same conclusion or/and algorithm.

And one really wonders that MS has no (or rather few) docu for something? This is normal case unfortunatelly for people developing for this platform.

You propose an algorithm as a solution...

As I wrote "my algo makes it correct", it was just the answer to "assoc and ftype" question, so I meant just the algorithm works proper for both cases equally (exe and batch).
And the "solution" resp. the algorithm description is just a explanation how I fixed this for another lang (having almost the same issue) without necessity to understand all internal structures and API of this language.

If someone meant that the writing an expensive PoC, several test cases and analysis, creating an issue with description is still not enough, he can try to do it better...

but how can one verify that you're solution is actually correct and covers everything without having documentation to verify this?

For example if one write more test-cases resp. extend they to reach better coverage.
If you don't have degradation on the another (already available) test-cases of the test-suite, it is good indicator that all works proper or (in case of windows rather) better as previous.
Or if one will cumulate the same knowledge like me (even possibly with "rummage" through the docu, fighting with reverse engineering, trying, etc), to be able to assess that it is fine.
I don't see any hindrance for it.

[magical]

I looked into this and it does appear to be a real issue.

Background

On Windows, command line arguments are not passed to programs as an array like they are on UNIX, but as a single, unprocessesed string.

In principle, this means that any program is free to parse the command line string any way it wants, using any quoting rules it wants.

In practice, most programs use the CommandLineToArgv function to parse command line arguments, so Go's os/exec package quotes the argument array in such a way that CommandLineToArgv will yield the same arguments.

The problem

Most windows programs use CommandLineToArgv to parse their arguments, but there is an important exception: cmd.exe. Also, .BAT and .CMD files are executed on windows by running CMD.EXE /C. If you run a .BAT file with arguments, as in e.g.,

echo.bat hi&whoami

then windows ends up running

C:\WINDOWS\system32\cmd.exe /c echo.bat hi&whoami

The part after /c is interpreted as a command, which may contain metacharacters like <>&. cmd.exe also has different quoting rules: in particular, backslash escapes like \" are not recognized and the caret ^ is treated as an escape character.

All together, this means that if a Go program ever executes a .bat file with os/exec, it is likely that the command line string will be misinterpreted. Since CMD.EXE can execute arbitrary commands, this leads directly to arbitrary code execution.

---

@sebres does that about sum it up?

Incidentally, it looks like this problem was already recognized and reported two years ago (#15566, maybe #17149) and a fix proposed (https://golang.org/cl/30947) but never merged.

[magical]

Minimal reproducer:

exec.go

package main

import (
	"log"
	"os"
	"os/exec"
)

func main() {
	cmd := exec.Command(`echo.bat`, "&whoami")
	cmd.Stdin = os.Stdin
	cmd.Stdout = os.Stdout
	cmd.Stderr = os.Stderr

	err := cmd.Run()
	if err != nil {
		log.Fatal(err)
	}
}

echo.bat

@echo hi

output

C:\Users\Andrew\Go\src>go run exec.go
hi
kenshin\andrew

expected output

C:\Users\Andrew\Go\src>go run exec.go
hi

[agnivade]

@alexbrainman

[alexbrainman]

looks like this problem was already recognized and reported two years ago (#15566, maybe #17149) and a fix proposed (https://golang.org/cl/30947) but never merged.

This already been discussed in other issues. There is no point repeating all that here too.

Alex

[sebres]

Thus closed as "unlikely to be implemented" resp. as "duplicate".

_{There are two categories of people: one of that says "the world is unfair" (MS must die, etc), and another that try to repair this "broken" world.}

Luckily there is many other lang's, on which one could build, regardless the platform they are running.

[gopherbot]

Change https://golang.org/cl/132695 mentions this issue: os/exec: document how to do special args quoting on Windows

[alexbrainman]

There are two categories of people: one of that says "the world is unfair" (MS must die, etc), and another that try to repair this "broken" world.

I am happy to repair this "broken" world, but I do not have free time to spend on this issue. If you have proposal to fix cmd.exe command line building, this https://golang.org/doc/contribute.html is how to contribute your solution. I will help you in whatever way I can.

Alex

[as]

arguments containing some special meta-characters is vulnerable and may be used to execute foreign data/code.

We established that

• exec.Command("notepad.exe", "&calc.exe") doesn't run the calculator
• exec.Command("test.bat", "&calc.exe") does
• exec.Command("test.bat", "&calc.exe") is equivalent to exec.Command("cmd", "/c", "test.bat", "&calc.exe").

Go is a platform independent language. Why should it be aware of a platform specific plumbing rule in Windows that launches a broken argument parser? This point was already brought up, however:

How often does the adversary control a command line argument but not the executable name (which would have to be a batch file)? In what case would executing such ambiguous arguments result in a change in scope that gets the adversary more access than they already have on the system?

The linked issue page ranks several languages and uses a subjective scoring system to emphasize priority. But there are established scoring systems for vulnerabilities (e.g., https://www.first.org/cvss/calculator/3.0) that are more useful.

The priority of this issue appears inflated with respect to its actual surface area. Are there any data that demonstrate examples of this issue in practical Go programs?