Skip to content
This repository has been archived by the owner on Nov 1, 2023. It is now read-only.

Commit

Permalink
Aligning feature branch with main (#1389)
Browse files Browse the repository at this point in the history 
* Bump reqwest from 0.11.4 to 0.11.5 in /src/proxy-manager (#1336)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/reqwest@v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.4 to 0.11.5 in /src/agent (#1335)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.4 to 0.11.5.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/reqwest@v0.11.4...v0.11.5)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Work around for newly-upgraded pip breaking pip-licenses (#1346)

* Work around for newly upgrdaded pip breaking pip-licenses  (can be reverted once raimon49/pip-licenses#113 is fixed)

* Update .github/workflows/ci.yml

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Bump iced-x86 from 1.14.0 to 1.15.0 in /src/agent (#1337)

Bumps [iced-x86](https://github.com/icedland/iced) from 1.14.0 to 1.15.0.
- [Release notes](https://github.com/icedland/iced/releases)
- [Commits](icedland/iced@v1.14.0...v1.15.0)

---
updated-dependencies:
- dependency-name: iced-x86
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* revert pip breaking pip-licenses workaround (#1348)

Co-authored-by: stas <statis@microsoft.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/proxy-manager (#1341)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump thiserror from 1.0.29 to 1.0.30 in /src/agent (#1342)

Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.29 to 1.0.30.
- [Release notes](https://github.com/dtolnay/thiserror/releases)
- [Commits](dtolnay/thiserror@1.0.29...1.0.30)

---
updated-dependencies:
- dependency-name: thiserror
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump strum from 0.21.0 to 0.22.0 in /src/agent (#1343)

Bumps [strum](https://github.com/Peternator7/strum) from 0.21.0 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump azure cli to 2.27.2 (#1355)

* Bump azure cli to 2.27.2

* fixing up add-corpus-storage-account script

Co-authored-by: stas <statis@microsoft.com>

* Bump azure-identity to 1.6.1 (#1356)

Co-authored-by: stas <statis@microsoft.com>

* Bump strum_macros from 0.21.1 to 0.22.0 in /src/agent (#1344)

Bumps [strum_macros](https://github.com/Peternator7/strum) from 0.21.1 to 0.22.0.
- [Release notes](https://github.com/Peternator7/strum/releases)
- [Changelog](https://github.com/Peternator7/strum/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Peternator7/strum/commits)

---
updated-dependencies:
- dependency-name: strum_macros
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump sysinfo from 0.20.4 to 0.20.5 in /src/agent (#1353)

Bumps [sysinfo](https://github.com/GuillaumeGomez/sysinfo) from 0.20.4 to 0.20.5.
- [Release notes](https://github.com/GuillaumeGomez/sysinfo/releases)
- [Changelog](https://github.com/GuillaumeGomez/sysinfo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/GuillaumeGomez/sysinfo/commits)

---
updated-dependencies:
- dependency-name: sysinfo
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Release 3.2.0 (#1361)

* Release 3.2.0

* Added python dependencies

* Update CHANGELOG.md

Co-authored-by: Cheick Keita <kcheick@gmail.com>

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Update grammar

* Update CHANGELOG.md

Co-authored-by: Joe Ranweiler <joe@lemma.co>

Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Azure DevOps notifications not appearing (#1370)

Co-authored-by: stas <statis@microsoft.com>

* Bump procfs from 0.10.1 to 0.11.0 in /src/agent (#1360)

Bumps [procfs](https://github.com/eminence/procfs) from 0.10.1 to 0.11.0.
- [Release notes](https://github.com/eminence/procfs/releases)
- [Commits](eminence/procfs@v0.10.1...v0.11.0)

---
updated-dependencies:
- dependency-name: procfs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Bump structopt from 0.3.23 to 0.3.25 in /src/agent (#1364)

Bumps [structopt](https://github.com/TeXitoi/structopt) from 0.3.23 to 0.3.25.
- [Release notes](https://github.com/TeXitoi/structopt/releases)
- [Changelog](https://github.com/TeXitoi/structopt/blob/master/CHANGELOG.md)
- [Commits](TeXitoi/structopt@v0.3.23...v0.3.25)

---
updated-dependencies:
- dependency-name: structopt
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.5 to 0.11.6 in /src/proxy-manager (#1367)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/reqwest@v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump reqwest from 0.11.5 to 0.11.6 in /src/agent (#1368)

Bumps [reqwest](https://github.com/seanmonstar/reqwest) from 0.11.5 to 0.11.6.
- [Release notes](https://github.com/seanmonstar/reqwest/releases)
- [Changelog](https://github.com/seanmonstar/reqwest/blob/master/CHANGELOG.md)
- [Commits](seanmonstar/reqwest@v0.11.5...v0.11.6)

---
updated-dependencies:
- dependency-name: reqwest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Bump crossterm from 0.21.0 to 0.22.1 in /src/agent (#1369)

Bumps [crossterm](https://github.com/crossterm-rs/crossterm) from 0.21.0 to 0.22.1.
- [Release notes](https://github.com/crossterm-rs/crossterm/releases)
- [Changelog](https://github.com/crossterm-rs/crossterm/blob/master/CHANGELOG.md)
- [Commits](https://github.com/crossterm-rs/crossterm/commits)

---
updated-dependencies:
- dependency-name: crossterm
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Fix validation of `target_exe` blob name (#1371)

* NSG Updated After CLI Update to Instance_Config (#1375)

* Creating InstanceConfig Attributes for NSG Refactor (#1331)

* Updating instance_config

* Updating attribute names.

* Updating list factory.

* Updating config attributes.

Co-authored-by: nharper285 <nharper285@gmail.com>

* NSG deployment on a creation of new debug/repro proxy. (#1340)

Co-authored-by: stas <statis@microsoft.com>

* Code for updating NSGs when instance_config updated.

* Updating argument to set_allowed_rules

* Temporarily ignore non-actionable `cargo audit` errors (#1365)

* Updating model to no longer be optional.

* Fixing args for set_allowed_rules

* trying to fix calls to get_nsg

* Updating calls to nsg lib

* Fixing imports.

* Updating calls to set_allowed and creating constructor for NSGConfig type.

* Removing constructor and manually setting default ip

* Fixing models.

* Hopefully fixing docs.

* Fix set_allowed call

* Adding error handling for update config.

* Changing to error check.

* Fixing error call.

* Fixing imports.

* Updating instanceconfig retrieval.

* Fixing imports.

* Adding empty() function on request.

* Fixing name of function.

* Removing empty function.

Co-authored-by: nharper285 <nharper285@gmail.com>
Co-authored-by: Stas <stishkin@live.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>

* Revert "NSG Updated After CLI Update to Instance_Config (#1375)" (#1384)

This reverts commit 357bc4f.

* Bump backtrace from 0.3.61 to 0.3.62 in /src/agent (#1382)

Bumps [backtrace](https://github.com/rust-lang/backtrace-rs) from 0.3.61 to 0.3.62.
- [Release notes](https://github.com/rust-lang/backtrace-rs/releases)
- [Commits](rust-lang/backtrace-rs@0.3.61...0.3.62)

---
updated-dependencies:
- dependency-name: backtrace
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marc Greisen <marc@greisen.org>

* Set compiler env vars to effect Win10 SDK downgrade (#1388)

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: stas <statis@microsoft.com>
Co-authored-by: Joe Ranweiler <joe@lemma.co>
Co-authored-by: Marc Greisen <marc@greisen.org>
Co-authored-by: Cheick Keita <kcheick@gmail.com>
Co-authored-by: Joe Ranweiler <joranwei@microsoft.com>
Co-authored-by: Noah McGregor Harper <74685766+nharper285@users.noreply.github.com>
Co-authored-by: nharper285 <nharper285@gmail.com>
  • Loading branch information
@stishkin @dependabot
@ranweiler @mgreisen @chkeita @nharper285 @nharper32
9 people authored Oct 22, 2021
1 parent 1d2f28a commit 80d7e92
Show file tree
Hide file tree
Showing 5 changed files with 160 additions and 23 deletions.
29 changes: 28 additions & 1 deletion .github/workflows/ci.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -421,7 +421,34 @@ jobs:
choco install llvm
choco install make
$env:Path += ";C:\Program Files\LLVM\bin;C:\ProgramData\chocolatey\bin"
# WORKAROUND: effectively downgrade the default Windows 10 SDK version.
#
# This ensures we link against a version of the SDK which won't trigger a
# startup bug in the LLVM-shipped ASAN runtime.
# Assume a default MSVC 2019 install path.
$MsvcDir = 'C:/Program Files (x86)/Microsoft Visual Studio/2019/Enterprise/VC/Tools/MSVC'
# Assume that `$MsvcDir` only contains version-named subdirectories.
$MsvcVersion = ((Get-ChildItem $MsvcDir).name | Sort-Object -Descending)[0]
$MsvcLib = "${MsvcDir}/${MsvcVersion}/lib/x64"
# Known "good" (non-bug-surfacing) version.
$WindowsSdkVersion = '10.0.18362.0'
# Assume default install path.
$WindowsSdkDir = 'C:/Program Files (x86)/Windows Kits/10'
$WindowsSdkLib = "${WindowsSdkDir}/Lib/${WindowsSdkVersion}"
$WindowsSdkInclude = "${WindowsSdkDir}/Include/${WindowsSdkVersion}"
# Used by `clang.exe`.
$env:CPATH = $WindowsSdkInclude
$env:LIBRARY_PATH = "${MsvcLib};${WindowsSdkLib}/ucrt/x64;${WindowsSdkLib}/um/x64"
# Used by `link.exe`.
$env:LIB = $env:LIBRARY_PATH
cd src/integration-tests
mkdir artifacts/windows-libfuzzer
Expand Down
31 changes: 20 additions & 11 deletions src/agent/Cargo.lock
View file

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion src/agent/onefuzz-agent/Cargo.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ atexit = { path = "../atexit" }
backoff = { version = "0.3", features = ["tokio"] }
clap = "2.33"
coverage = { path = "../coverage" }
crossterm = "0.21"
crossterm = "0.22"
env_logger = "0.9"
flume = "0.10"
futures = "0.3"
Expand Down
64 changes: 54 additions & 10 deletions src/api-service/__app__/onefuzzlib/tasks/config.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@
# Licensed under the MIT License.

import logging
import ntpath
import os
import posixpath
import pathlib
from typing import Dict, List, Optional
from uuid import UUID

Expand Down Expand Up @@ -105,14 +104,12 @@ def check_target_exe(config: TaskConfig, definition: TaskDefinition) -> None:

return

# Azure Blob Store uses virtualized directory structures. As such, we need
# the paths to already be canonicalized. As an example, accessing the blob
# store path "./foo" generates an exception, but "foo" and "foo/bar" do
# not.
if (
posixpath.relpath(config.task.target_exe) != config.task.target_exe
or ntpath.relpath(config.task.target_exe) != config.task.target_exe
):
# User-submitted paths must be relative to the setup directory that contains them.
# They also must be normalized, and exclude special filesystem path elements.
#
# For example, accessing the blob store path "./foo" generates an exception, but
# "foo" and "foo/bar" do not.
if not is_valid_blob_name(config.task.target_exe):
raise TaskConfigError("target_exe must be a canonicalized relative path")

container = [x for x in config.containers if x.type == ContainerType.setup][0]
Expand All @@ -124,6 +121,53 @@ def check_target_exe(config: TaskConfig, definition: TaskDefinition) -> None:
LOGGER.warning(err)


# Azure Blob Storage uses a flat scheme, and has no true directory hierarchy. Forward
# slashes are used to delimit a _virtual_ directory structure.
def is_valid_blob_name(blob_name: str) -> bool:
# https://docs.microsoft.com/en-us/rest/api/storageservices/naming-and-referencing-containers--blobs--and-metadata#blob-names
MIN_LENGTH = 1
MAX_LENGTH = 1024 # Inclusive
MAX_PATH_SEGMENTS = 254

length = len(blob_name)

# No leading/trailing whitespace.
if blob_name != blob_name.strip():
return False

if length < MIN_LENGTH:
return False

if length > MAX_LENGTH:
return False

path = pathlib.PurePosixPath(blob_name)

if len(path.parts) > MAX_PATH_SEGMENTS:
return False

# No path segment should end with a dot (`.`).
for part in path.parts:
if part.endswith("."):
return False

# Reject absolute paths to avoid confusion.
if path.is_absolute():
return False

# Reject paths with special relative filesystem entries.
if "." in path.parts:
return False

if ".." in path.parts:
return False

# Will not have a leading `.`, even if `blob_name` does.
normalized = path.as_posix()

return blob_name == normalized


def target_uses_input(config: TaskConfig) -> bool:
if config.task.target_options is not None:
for option in config.task.target_options:
Expand Down
57 changes: 57 additions & 0 deletions src/api-service/tests/test_task_config.py
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
#!/usr/bin/env python
#
# Copyright (c) Microsoft Corporation.
# Licensed under the MIT License.

from typing import Tuple

import pytest

from __app__.onefuzzlib.tasks.config import is_valid_blob_name

BlobNameTestCase = Tuple[str, bool]


BLOB_NAME_TEST_CASES = [
# Valid
("fuzz.exe", True),
("bin/fuzz.exe", True),
("/".join("a" * 254), True),
("a" * 1024, True),
# Invalid (absolute)
("/fuzz.exe", False),
("/bin/fuzz.exe", False),
# Invalid (special dirs)
("./fuzz.exe", False),
("././fuzz.exe", False),
("./bin/fuzz.exe", False),
("./bin/./fuzz.exe", False),
("../fuzz.exe", False),
("../bin/fuzz.exe", False),
(".././fuzz.exe", False),
("../bin/./fuzz.exe", False),
# Out of Azure size bounds
("", False),
(" ", False),
("/".join("a" * 255), False),
("a" * 1025, False),
# Paths with invalid segments.
("a.", False),
("a..", False),
("a./b", False),
("a/b./c", False),
("a./", False),
("a../", False),
("a./b/", False),
("a/b./c/", False),
("a//", False),
]


@pytest.mark.parametrize("blob_name_test_case", BLOB_NAME_TEST_CASES)
def test_is_valid_blob_name(blob_name_test_case: BlobNameTestCase) -> None:
blob_name, expected = blob_name_test_case

is_valid = is_valid_blob_name(blob_name)

assert is_valid == expected

0 comments on commit 80d7e92

Please sign in to comment.