Final changes for release based on feedback (#151)

README.md

@@ -19,21 +19,19 @@ glslangValidator and spirv-tools to additionally target SPIR-V.
 * [Introduction: my shader is being weird, now what?](docs/glsl-reduce-intro.md)
 * [glsl-reduce walkthrough](docs/glsl-reduce-walkthrough.md)
 * [glsl-reduce manual](docs/glsl-reduce.md)
-* [Developer documentation](docs/glsl-fuzz-develop.md)
 
 ### glsl-fuzz
 
 * [How it works (high-level): metamorphic testing using glsl-fuzz](docs/glsl-fuzz-intro.md)
 * [glsl-fuzz walkthrough](docs/glsl-fuzz-walkthrough.md)
 * [glsl-generate manual](docs/glsl-fuzz-generate.md)
 * [glsl-reduce manual (for reducing fuzzed shaders)](docs/glsl-fuzz-reduce.md)
-* [Developer documentation](docs/glsl-fuzz-develop.md)
 
 ## Get the GraphicsFuzz tools
 
 
 * **Pre-built binaries** are available on the [GitHub releases page](docs/glsl-fuzz-releases.md)
-* [**Build instructions**](docs/glsl-fuzz-develop.md)
+* [**Developer documentation with build instructions**](docs/glsl-fuzz-develop.md)
 
 
 ## Contribute

build/travis/release.py

@@ -29,6 +29,10 @@ def go():
         print("Skipping release because this is not the master branch.")
         sys.exit(0)
 
+    if "TRAVIS_TAG" not in os.environ:
+        print("Skipping release because this is not a tag build")
+        sys.exit(0)
+
     if "TRAVIS_PULL_REQUEST" in os.environ and os.environ["TRAVIS_PULL_REQUEST"] != "false":
         print("Skipping release because this is a pull request.")
         sys.exit(0)
@@ -37,6 +41,7 @@ def go():
         print("Please define TRAVIS_COMMIT")
         sys.exit(1)
     commit_hash = os.environ["TRAVIS_COMMIT"]
+    tag_name = os.environ["TRAVIS_TAG"]
 
     if not os.path.isdir("out"):
         print("Failing release because 'out' directory was not found.")
@@ -51,7 +56,7 @@ def go():
     subprocess.check_call([
         "github-release",
         repo_name,
-        "v-" + commit_hash,
+        tag_name,
         commit_hash,
         description + "\n" + git_log,
         "out/*"])

docs/glsl-fuzz-develop.md

@@ -14,8 +14,13 @@ page](glsl-fuzz-releases.md).
 * [JDK 1.8+](http://www.oracle.com/technetwork/java/javase/downloads/index.html)
 * [Maven](https://maven.apache.org/)
 * [Python](https://www.python.org/)
-* For Android workers: [Android SDK & NDK](https://developer.android.com/studio/#command-tools)
-* For Vulkan worker: [Vulkan SDK](https://vulkan.lunarg.com/sdk/home)
+
+* For our Android workers: [Android SDK & NDK](https://developer.android.com/studio/#command-tools)
+* For the Vulkan desktop worker: [Vulkan SDK](https://vulkan.lunarg.com/sdk/home)
+
+> Our *workers* are applications that run on the device you wish to test;
+> they communicate with the `glsl-server` application that is typically run on
+> a more powerful x86 machine.
 
 ### Get a local copy of this repository
 
@@ -24,17 +29,14 @@ To clone this repository:
 ```sh
 git clone https://github.com/google/graphicsfuzz.git
 
-# Or: git clone git@github.com:google/graphicsfuzz.git
-
 # Change into the cloned directory:
 cd graphicsfuzz
 ```
 
-The Vulkan worker build also requires git submodules to be cloned:
+The Vulkan worker build requires git submodules to be initialized and updated:
 
 ```sh
-git submodule init
-git submodule update
+git submodule update --init
 ```
 
 ### Build the GraphicsFuzz package
@@ -204,11 +206,10 @@ git clone https://github.com/google/graphicsfuzz.git
 cd graphicsfuzz
 ```
 
-The Vulkan worker build also requires git submodules to be cloned:
+The Vulkan worker build requires git submodules to be initialized and updated:
 
 ```sh
-git submodule init
-git submodule update
+git submodule update --init
 ```
 
 ### Opening GraphicsFuzz in IntelliJ
@@ -342,7 +343,7 @@ manager.queueCommand(
     "shaderfamilies/shaderfamily1/variants/variant_1.json",
     "IDENTICAL",
     "--server", "http://localhost:8080",
-    "--woker", "my-laptop"
+    "--worker", "my-laptop"
    ],
    "my-laptop",
    "processing/my-laptop/shaderfamily1/reductions/variant_1/command.log")

docs/glsl-fuzz-generate.md

@@ -4,8 +4,31 @@
 
 `glsl-generate [OPTIONS] DONORS REFERENCES NUM_VARIANTS GLSL_VERSION PREFIX OUTPUT_FOLDER`
 
-glsl-generate mutates some reference GLSL shaders to produce families of variant
-shaders.
+
+## Description
+
+glsl-generate takes two directories of GLSL shaders, `DONORS` and `REFERENCES`,
+and mutates the shaders in `REFERENCES` to produce families of variant
+shaders to `OUTPUT_FOLDER`.
+
+Each shader is provided (and output)
+as a **shader job** (a `.json` file)
+that represents all shaders and metadata needed to
+render an image.
+However, the common use-case
+is for each input shader to be a fragment shader, e.g. `shader.frag`,
+with a corresponding JSON file, `shader.json`, alongside.
+The JSON file can either contain `{}` or a dictionary of uniform values.
+See below for more details on shader jobs.
+
+`DONORS` is a corpus of shaders; code from these is injected into the `REFERENCES`.
+
+Each output shader family will have `NUM_VARIANTS` variant shaders, plus
+the original reference shader from which they are derived.
+
+`GLSL_VERSION` specifies the version of GLSL, such as `100` or `"310 es"`.
+
+Each shader family will be output to a directory starting with `OUTPUT_FOLDER/PREFIX_*`.
 
 ## Options
 
@@ -62,38 +85,116 @@ optional arguments:
 
 ## Example
 
-From the graphicsfuzz-1.0 directory:
+Assuming you have extracted the `graphicsfuzz-1.0.zip` file to get `graphicsfuzz-1.0/`:
 
-```shell
-mkdir generated
-glsl-generate shaders/src/main/glsl/samples/donors shaders/src/main/glsl/samples/310es 2 "310 es" my_prefix generated
+```sh
+# Copy the sample shaders into the current directory:
+cp -r graphicsfuzz-1.0/shaders/samples samples
+
+# Create a work directory to store our generated shader families.
+# The directory structure allows glsl-server
+# to find the shaders later.
+mkdir -p work/shaderfamilies
+
+# Generate several shader families from the set of sample shaders.
+# Synopsis:
+# glsl-generate [options] donors references num_variants glsl_version prefix output_folder
+
+# Generate some GLSL version 300 es shaders.
+glsl-generate --max_bytes 500000 --multi_pass samples/donors samples/300es 10 "300 es" family_300es work/shaderfamilies
+
+# Generate some GLSL version 100 shaders.
+glsl-generate --max_bytes 500000 --multi_pass samples/donors samples/100 10 "100" family_100 work/shaderfamilies
+
+# Generate some "Vulkan-compatible" GLSL version 300 es shaders that can be translated to SPIR-V for Vulkan testing.
+glsl-generate --max_bytes 500000 --multi_pass --generate_uniform_bindings --max_uniforms 10 samples/donors samples/310es 10 "310 es" family_vulkan work/shaderfamilies
+
+# The lines above will take approx. 1-2 minutes each, and will generate a shader family for every
+# shader in samples/300es or samples/100:
+ls work/shaderfamilies
+
+# Output:
+
+# family_100_bubblesort_flag
+# family_100_mandelbrot_blurry
+# family_100_squares
+# family_100_colorgrid_modulo
+# family_100_prefix_sum
+
+# family_300es_bubblesort_flag
+# family_300es_mandelbrot_blurry
+# family_300es_squares
+# family_300es_colorgrid_modulo
+# family_300es_prefix_sum
+
+# family_vulkan_bubblesort_flag
+# family_vulkan_mandelbrot_blurry
+# family_vulkan_squares
+# family_vulkan_colorgrid_modulo
+# family_vulkan_prefix_sum
 ```
 
-Note that the output directory must already exists.
+For this example, each shader family will contain 11 shaders;
+1 for the reference shader, and 10 for the variant shaders:
 
-The above command will produce:
+```sh
+ls work/shaderfamilies/family_100_bubblesort_flag/
 
-```shell
-generated/
-├── my_prefix_bubblesort_flag
-│   ├── infolog.json
-│   ├── reference.frag
-│   ├── reference.json
-│   ├── variant_000.frag
-│   ├── variant_000.json
-│   ├── variant_000.prob
-│   ├── variant_001.frag
-│   ├── variant_001.json
-│   └── variant_001.prob
-├── my_prefix_colorgrid_modulo
-│   ├── infolog.json
-│   ├── reference.frag
-│   ├── reference.json
-│   ├── variant_000.frag
-│   ├── variant_000.json
-│   ├── variant_000.prob
-│   ├── variant_001.frag
-│   ├── variant_001.json
-│   └── variant_001.prob
-[ ... etc ... ]
+# Output:
+
+# infolog.json      variant_001.json  variant_004.json  variant_007.json
+# reference.frag    variant_002.frag  variant_005.frag  variant_008.frag
+# reference.json    variant_002.json  variant_005.json  variant_008.json
+# variant_000.frag  variant_003.frag  variant_006.frag  variant_009.frag
+# variant_000.json  variant_003.json  variant_006.json  variant_009.json
+# variant_001.frag  variant_004.frag  variant_007.frag
 ```
+
+## Shader jobs
+
+Each input is, in fact,
+a JSON file that we refer to as a
+**shader job** that
+represents all shaders and metadata needed to
+render an image.
+However,
+our well-tested use-case is
+a shader job that contains a fragment shader
+and a set of uniform values.
+Our worker applications that are used to render a frame
+can only set uniforms
+(they cannot set textures, etc.).
+
+For example, from our release zip:
+
+`graphicsfuzz-1.0/samples/300es/squares.json`:
+
+```json
+{
+  "time": {
+    "func": "glUniform1f",
+    "args": [
+      0.0
+    ]
+  },
+  "mouse": {
+    "func": "glUniform2f",
+    "args": [
+      0.0,
+      0.0
+    ]
+  },
+  "resolution": {
+    "func": "glUniform2f",
+    "args": [
+      256.0,
+      256.0
+    ]
+  }
+}
+```
+
+The fragment shader file for this shader job
+must have the same name and be alongside the shader job file
+with a `.frag` extension;
+in this case, at `graphicsfuzz-1.0/samples/300es/squares.frag`.

docs/glsl-fuzz-intro.md

@@ -4,7 +4,9 @@
 
 ## The problem
 
-A graphics driver takes a *shader program* as input and executes it on a GPU (graphics processing unit) to render an image.
+A graphics driver takes one or more *shaders* as input and executes them on a GPU (graphics processing unit) to render an image.
+Shaders are typically written in a C-like language,
+such as GLSL.
 
 ![shader program, to GPU, to image](images/shader-gpu-image.png)
 
@@ -40,12 +42,12 @@ Finding bugs is not the end of the story: a variant shader that exposes a bug is
 
 Fortunately, our reducer is able to selectively reverse those transformations that are not relevant to the bug. After reduction, we obtain a small difference sufficient to expose the driver issue.
 
-![The same source code, the majority of which is highlighted in yellow and striked out, but parts of one statement remain.](images/variant-reduced.png)
+![The same source code, the majority of which is highlighted in yellow and struck out, but parts of one statement remain.](images/variant-reduced.png)
 
 The reduced variant *still exposes the bug*, and differs from the reference only slightly: this is a great starting point to isolate the root cause of the bug in the graphics driver.
 
 ## Summary
 
 GraphicsFuzz finds bugs in graphics drivers by rendering families of semantically equivalent shaders, and looking for output discrepancies. This approach is known as *metamorphic testing*. For each bug, the reducer saves a lot of debugging time by producing a simpler *minimal-difference test case* that still exposes the bug.
 
-To learn more, checkout the [walkthrough](glsl-fuzz-walkthrough.md), [test genrator manual](glsl-fuzz-generate.md) and [test reducer manual](glsl-fuzz-reduce.md).
+To learn more, check out the [walkthrough](glsl-fuzz-walkthrough.md).

docs/glsl-fuzz-reduce.md

@@ -16,7 +16,7 @@ See below for common examples.
 glsl-reduce takes a *shader job* `SHADER_JOB` (a .json file) as an argument
 as well as further arguments or options to specify the interestingness test.
 glsl-reduce will try to simplify the given shader job to a smaller,
-simpler shader job for that is still deemed "interesting".
+simpler shader job that is still deemed "interesting".
 
 `SHADER_JOB` should be a `.json` shader job file that represents all shaders and metadata needed to
 render an image. If the shader job is named `foo.json`,
@@ -40,7 +40,7 @@ a shader job is interesting if it does *not* produce an
 image and, optionally, the run log from running the shader job includes the
 regular expression `ERROR`.
 This reduction kind is typically used **without** `--preserve-semantics`,
-so the the reducer can change the semantics of the shader;
+so the reducer can change the semantics of the shader;
 we are usually reducing a crash or incorrect shader compilation error,
 so changing the semantics of the shaders is fine, as they should still compile and run.
 Note that the reducer always produces valid shaders that should compile.
@@ -69,7 +69,7 @@ by glsl-generate.
 
 Options:
 
-```sh
+```
 
 Reduce GLSL shaders, driven by a criterion of interest.