Skip to content

Commit

Permalink
feat(all): fix reconditioning, implement OpLoopMerge, update README
Browse files Browse the repository at this point in the history
  • Loading branch information
rayanht committed May 21, 2022
1 parent 363c08c commit bee68dc
Show file tree
Hide file tree
Showing 16 changed files with 452 additions and 279 deletions.
97 changes: 90 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@
[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)
[![DOI](https://zenodo.org/badge/429076310.svg)](https://zenodo.org/badge/latestdoi/429076310)

## Introduction
## SPIRVSmith

SPIRVSmith is a differential testing tool that leverages structured fuzzing techniques to find bugs in producers and consumers of SPIRV shaders.

Expand All @@ -20,6 +20,59 @@ SPIRVSmith attempts to find bugs in the following projects:
- [MoltenVK](https://github.com/KhronosGroup/MoltenVK)
- ...and more to come!

## Table of Contents
- [Installation](#installation)
- [Usage](#usage)
- [How does it work?](#how-does-it-work)
- [Differential testing](#differential-testing)
- [Program Reconditioning](#program-reconditioning)
- [SPIR-V Language Coverage](#spirv-language-coverage)
- [Contributing](#contributing)
- [Authors](#authors)
- [License](#license)


## Installation

SPIRVSmith uses `poetry` to manage Python dependencies and ships with scripts to install external dependencies.

1. Follow the poetry [installation instructions](https://python-poetry.org/docs/#installation) for your platform.

1. Grab a local copy of SPIRVSmith:

```bash
$ git clone https://github.com/rayanht/SPIRVSmith.git && cd SPIRVSmith
```

3. Install Python dependencies using `poetry` and start a `poetry` shell:

```bash
$ poetry install && poetry shell
```

4. Install the external dependencies:

```bash
$ mkdir bin
$ sh scripts/get_spirv_tools.sh <platform>
```

Replace `<platform>` by either `linux` or `macos` (No support for Windows :sob:)


## Usage

The first step is to run `SPIRVSmith` is to head to `config.py`. That file contains the various parameters that are used to dictate the behaviour of the fuzzer.

`SPIRVSmith` is highly parametrizable, you can choose to disable certain features of the SPIR-V language (e.g. do not emit any control flow operation), limit the number of global constants generated, favour the generation of certain kinds of instructions etc.

Once you are happy with your parametrization, make sure you are in a `poetry` virtual environment (`$ poetry shell`) and run `SPIRVSmith`:

```bash
$ sh scripts/run.sh
```

SPIR-V assembly files will be saved as they are generated to the `out/` directory and the fuzzer can be stopped at any time by pressing `Ctrl+C`.

## How does it work?

Expand Down Expand Up @@ -78,11 +131,6 @@ void main () {
}
```

### SPIRVSmith in the cloud

![SPIRVSmith drawio](https://user-images.githubusercontent.com/42040895/162422826-c65dfd6f-1f32-4dcc-be8c-8c7e6117b42b.png)


## SPIRV Language Coverage

<details>
Expand Down Expand Up @@ -247,7 +295,7 @@ void main () {
| OpStore |:white_check_mark: |
| OpCopyMemory | :red_circle: |
| OpCopyMemorySized | :red_circle: |
| OpAccessChain | :red_circle: |
| OpAccessChain | :white_check_mark: |
| OpInBoundsAccessChain | :red_circle: |
| OpPtrAccessChain | :red_circle: |
| OpPtrEqual | :red_circle: |
Expand Down Expand Up @@ -476,7 +524,42 @@ void main () {

</details>

#### Control Flow

<details>

<summary>Expand</summary>


|OpCode| Status |
|--|--|
| OpPhi | :red_circle: |
| OpLoopMerge | :white_check_mark: |
| OpSelectionMerge | :white_check_mark: |
| OpLabel |:white_check_mark: |
| OpBranch | :white_check_mark: |
| OpBranchConditional | :white_check_mark: |
| OpSwitch | :red_circle: |
| OpReturn | :red_circle: |
| OpReturnValue | :red_circle: |

</details>

</details>

## Contributing

Encountered a bug? Have an idea for a new feature? This project is open to all
sorts of contribution! Feel free to head to the `Issues` tab and describe your
request!

## Authors

* **Rayan Hatout** - [GitHub](https://github.com/rayanht)
| [Twitter](https://twitter.com/rayanhtt)
| [LinkedIn](https://www.linkedin.com/in/rayan-hatout/)

## License
This project is licensed under the Apache 2.0 license.

[![FOSSA Status](https://app.fossa.com/api/projects/custom%2B22322%2Fgithub.com%2Frayanht%2FSPIRVSmith.svg?type=large)](https://app.fossa.com/projects/custom%2B22322%2Fgithub.com%2Frayanht%2FSPIRVSmith?ref=badge_large)
168 changes: 168 additions & 0 deletions config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
import random
from dataclasses import dataclass

from src.utils import get_spirvsmith_version

rng = random.SystemRandom()


@dataclass
class BinariesConfig:
ASSEMBLER_PATH: str = "bin/spirv-as"
VALIDATOR_PATH: str = "bin/spirv-val"
OPTIMISER_PATH: str = "bin/spirv-opt"
CROSS_PATH: str = "bin/spirv-cross"
AMBER_PATH: str = "bin/amber"


@dataclass
class MutationsConfig:
# Operations
w_memory_operation: tuple[int, int] = (1, 2)
w_logical_operation: tuple[int, int] = (2, 6)
w_arithmetic_operation: tuple[int, int] = (2, 6)
w_control_flow_operation: tuple[int, int] = (0, 2)
w_function_operation: tuple[int, int] = (1, 1)
w_bitwise_operation: tuple[int, int] = (2, 6)
w_conversion_operation: tuple[int, int] = (2, 6)
w_composite_operation: tuple[int, int] = (2, 6)

# Types
w_scalar_type: tuple[int, int] = (1, 3)
w_container_type: tuple[int, int] = (1, 3)

# Constants
w_composite_constant: tuple[int, int] = (1, 3)
w_scalar_constant: tuple[int, int] = (2, 4)


@dataclass
class LimitsConfig:
# How many types should the fuzzer aim to generate.
n_types: int = 20
# How many constants should the fuzzer aim to generate.
n_constants: int = 50
# How many functions should the fuzzer aim to generate.
n_functions: int = 1
# How deep can the shader go when generating shaders (in terms of control flow).
# e.g. with a depth of 2, the fuzzer will never generate a triply-nested loop.
max_depth: int = 3


@dataclass
class FuzzingStrategyConfig:
mutations_config: MutationsConfig = MutationsConfig()

# If True, SPIRVSmith will include instructions from
# the GLSL extension in generated shaders
enable_ext_glsl_std_450: bool = True

# The following parameters are used to determine
# what instructions will be generated by SPIRVSmith.
#
# These are interpreted by the fuzzer as weights rather than probabilities.
# i.e. if you set:
# w_logical_operation = 1
# w_arithmetic_operation = 2
# then the fuzzer will generate twice as many arithmetic operations.
#
# Classification of operations:
# memory_operations -> OpLoad, OpStore, OpAccessChain etc.
# logical_operations -> OpLogicalNot, OpLogicalAnd, OpIEqual etc.
# arithmetic_operations -> OpIAdd, OpSMod, OpUDiv etc.
# (/!\ all instructions from the GLSL extensions are considered arithmetic /!\)
# control_flow_operations -> OpSelectionMerge, OpLoopMerge etc.
# function_operations -> OpFunctionCall, OpReturn etc.
# bitwise_operations -> OpBitwiseAnd, OpBitwiseOr etc.
# conversion_operations -> OpConvertFToU, OpConvertFToS etc.
# composite_operations -> OpVectorExtractDynamic, OpCompositeConstruct etc.
#
# If you want to prevent SPIRVSmith from generating any of these operations,
# simply set the corresponding weight to 0.
w_memory_operation: int = rng.randint(*mutations_config.w_memory_operation)
w_logical_operation: int = rng.randint(*mutations_config.w_logical_operation)
w_arithmetic_operation: int = rng.randint(*mutations_config.w_arithmetic_operation)
w_control_flow_operation: int = rng.randint(
*mutations_config.w_control_flow_operation
)
w_function_operation: int = rng.randint(*mutations_config.w_function_operation)
w_bitwise_operation: int = rng.randint(*mutations_config.w_bitwise_operation)
w_conversion_operation: int = rng.randint(*mutations_config.w_conversion_operation)
w_composite_operation: int = rng.randint(*mutations_config.w_composite_operation)

# The following parameters are used to determine
# what types will be generated by SPIRVSmith
#
# They are similarly interpreted as weight rather than probabilities.
#
# Classification of types:
# scalar_types -> OpTypeInt, OpTypeFloat, OpTypeBool
# container_types -> OpTypeVector, OpTypeMatrix, OpTypeArray etc.
#
# In practice, this is only useful when you want to prevent SPIRVSmith
# from generating container types as there are only a few scalar types
# that can be generated until we exhaust them all and SPIRVSmith is forced
# to generate container types to fulfill the quota specified in the LimitsConfig.
w_scalar_type: int = rng.randint(*mutations_config.w_scalar_type)
w_container_type: int = rng.randint(*mutations_config.w_container_type)

# The following parameters are used to determine
# what constants will be generated by SPIRVSmith
#
# They are similarly interpreted as weight rather than probabilities.
#
# Classification of types:
# scalar_constant -> OpConstant with underlying type OpTypeInt, OpTypeFloat, OpTypeBool
# composite_constant -> OpConstantComposite with underlying type OpTypeVector, OpTypeMatrix etc.
w_composite_constant: int = rng.randint(*mutations_config.w_composite_constant)
w_scalar_constant: int = rng.randint(*mutations_config.w_scalar_constant)

# P(generating a statement at step t + 1 | a statement was generated at step t)
p_statement: float = 0.995

# The following parameter is used to determine how much the fuzzer should
# favour statements rather than constants when looking for operands.
#
# When SPIRVSmith has to pick operands for an operation (say an OpIAdd),
# it has the choice between:
# - Constants -> a global OpConstant or OpConstantComposite
# - Statements -> the result of a previous operation in scope
#
# Setting a higher probability will favour statements over constants.
p_picking_statement_operand: float = 0.5

# The following parameter is used to determine how often a fuzzer mutation
# should be trigger.
#
# When a mutation is triggered, one random parameter from FuzzingStrategyConfig
# is chosen and slightly altered, within the bounds specified in the MutationsConfig.
#
# This is only useful in practice in the context of looking for bugs in
# consumers of SPIR-V since it increases variance.
mutation_rate: float = 0.05


@dataclass
class MiscConfig:
# The following parameters are only useful when running SPIRVSmith in
# a distributed fashion. Enabling these will almost definitely crash SPIRVSmith
# unless you have deployed the associated infrastructure and have a credentials file.
start_web_server: bool = False
broadcast_generated_shaders: bool = False
upload_logs: bool = False
version: str = get_spirvsmith_version()


@dataclass
class SPIRVSmithConfig:
# Binaries
binaries: BinariesConfig = BinariesConfig()

# Limits
limits: LimitsConfig = LimitsConfig()

# Fuzzing strategy
strategy: FuzzingStrategyConfig = FuzzingStrategyConfig()

# Misc
misc: MiscConfig = MiscConfig()
3 changes: 2 additions & 1 deletion interestingness.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@


def is_interesting(spv_file_path: str, n_reports: int) -> int:
if not validate_spv_file(spv_file_path):
if validate_spv_file(spv_file_path).exit_code != 0:
return 1
with NamedTemporaryFile(suffix=".spasm") as disassembled_spasm_file:
if not disassemble_spv_file(spv_file_path, disassembled_spasm_file.name):
Expand All @@ -38,6 +38,7 @@ def is_interesting(spv_file_path: str, n_reports: int) -> int:
if len(buffer_dumps) >= n_reports:
break
time.sleep(1)
print(buffer_dumps)
BQ_delete_shader(parsed_shader.id)
if not all(x == buffer_dumps[0] for x in buffer_dumps):
return 0
Expand Down
1 change: 0 additions & 1 deletion reducer_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,6 @@ def reduce(cls, target: ReductionTarget) -> ReductionResult:
capture_output=False,
)
if reduce_process.returncode != 0:
print(reduce_process.stderr.decode("utf-8"))
return ReductionResult(False, shader)
with NamedTemporaryFile(
suffix=".spasm"
Expand Down
Loading

0 comments on commit bee68dc

Please sign in to comment.