Skip to content

Commit

Permalink
Blob encryption scope sas api view (#24526)
Browse files Browse the repository at this point in the history 
* ADT ownership transitioning (#24404)

* Remove SchemaRegistryClient caching (#24380)

* Remove builder caching references.

* Remove caching from SchemaRegistryAsyncClient.
Make methods public for Response.

* Remove cached tests.

* Adding service annotation.

* [Amqp-core, EH]: Prepending namespace|entitypath consistenty in log, first untrack processor subscriber then notify and adding retry to EventHubConsumer[Receiver]Client (#24417)

* Added encryption scope blob sas

* Hide HttpHeaders.toMultiMap API (#24428)

* . (#24440)

* mgmt, bug fix, container group without ports (#24418)

* Update TRC in Azure Core (#24436)

* Sync eng/common directory with azure-sdk-tools for PR 2046 (#24431)

* Pass package name from calling pipeline to uniquely identify pull request review

* Update log summary

* Update eng/common/scripts/Detect-Api-Changes.ps1

Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
Co-authored-by: praveenkuttappan <55455725+praveenkuttappan@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>

* Enable API change detection in PR pipeline (#24234)

* Enable API change detection in PR pipeline

* Communication: Add TokenCredentialAddHostHeaderPolicy for TokenCredential Requests (#24442)

* Communication: Add TokenCredentialAdditionalHeaderPolicy for CallingServerClientBuilder

* Add TokenCredentialAddHostHeaderPolicyTests

* Fixing comment

* Use URL class to get hostname

* Fix style errors

Co-authored-by: Melissa Neubert <mneubert@microsoft.com>

* Add Compliance stage with policheck (#24276)

* Add Compliance stage with policheck

* Add vmImage pool

* Fix issues flagged by PoliCheck

* Move credscan into the compliance stage

* update readme for storage libraries to include BOM information. (#22858)

update readme for storage libraries

* Enable dependency validation of a single library (#24241)

* Enable validation of a library via it's POM file.

* Incorporate feedback and remove unused code.

* PR feedback

* Move Form Recognizer beta to public (#24453)

* Increment version for appconfiguration releases (#24450)

* Rename certificates-refresh-interval to certificates-refresh-interval--in-ms in keyvault jca (#24339)

* Prepare to release azure-spring-bom and azure-spring-cloud-dependencies. (#24425)

* mgmt, support validateMoveResources (#24465)

* Updated `KeyVaultCredentialPolicy` to extend `BearerTokenAuthenticationPolicy` in Key Vault clients. (#24199)

* Replaced all uses of KeyVaultCredentialPolicy with BearerTokenAuthenticationPolicy in Key Vault clients. Removed the KeyVaultCredentialPolicy and ScopeTokeCache classes from all Track 2 Key Vault libraries.

* We now pass the appropriate scope to BearerTokenAuthenticationPolicy creating a new instance in client builders, tests and samples.

* Added tests and recordings for KEK tests on MHSM. Fixed and cleaned up tests.

* Removed unused imports.

* Renamed MHSM_SCOPE to MANAGED_HSM_SCOPE in all client builders.

* Reintroduced KeyVaultCredentialPolicy and modified it to extend from BearerTokenAuthenticationPolicy while extracting the scope provided in bearer challenges returned by the Key Vault service.

* Fixed CvheckStyle errors.

* Made changes to KeyVaultCredentialPolicy so we don't set the body of a request as null, but an empty String instead.

* Removed scope constants from Key vault client builders.

* Attempted to fix flaky live tests.

* Removed verify test for HSM as the FromSource test already verifies the build's code coverage and running in parallel against the same HSM can cause problems for some tests.

* Reverted KeyVaultCredentialPolicy in all libraries to set the request body to null instead of an empty string when sending the first unauthenticated  request to get a bearer challenge. Also stored the value of the "Content-Length" header in the pipeline context for use in a subsequent request.

* Fixed KV Administration client live tests that failed due to the authentication policy changes. Also fixed some flaky live tests.

* Fixed CheckStyle issues.

* Fixed another CheckStyle issue.

* Fixed issue that caused an NPE in KeyVaultCredentialPolicy if the content of the request being originally sent were null from the beginning.

* Updated KeyVaultCredentialPolicy in all other libraries.

* Made an attempt at fixing the backup async live tests.

* Added sleep timer when running against service for restore operations.

* Applied PR feedback.

* [Storage] Try GMavenPlus to unblock Java 17 adoption. (#24471)

* lets try.

* fix java8

* fix java8 again:/

* track 1

* Update Form recognizer readme (#24476)

* [Storage] Bump Groovy version to 3 that works with Java 17. (#24477)

* bump groovy version

* Revert "bump groovy version"

This reverts commit a80c805.

* use different spocks depending on java version.

* rename.

* Use New Javadoc Codesnippet Tooling to Support Java 17 (#24475)

Use New Javadoc Snippet Tooling to Support Java 17

* Delete unused tests pipeline for track 1 blob package (#24488)

* [Storage] Fix track 1 tests. (#24490)

* Fix track 1 tests.

* revert that.

* Use Different Dummy Javadoc Option (#24491)

* Update Jackson, Netty, and Reactor Versions (#24312)

* Adding additional logging to ReactorDispatcher and ReactorExecutor. Adding closing logic (#24457)

* Closing ReactorExecutor if it has never been run.

* Adding documentation to ReactorDispatcher.

* Updating ReactorExecutor to schedule close work when reactor has not started or scheduler is closed.

* Adding tests.

* In method invocations, adding catch for RejectedExecutionException in the case that the scheduler is disposed.

* Adding assertion for ReactorExecutorTest that an onError is also called.

* Adding documentatioln to reactor connection and timeout to closing execturo.

* Splitting try/catch conditions.

* Add documentation to RequestResponseChannel.

* Using testPublisher for AmqpChannelProcessorTest. Using Flux.never().

* Adding Andy to the IoT CODEOWNERS (#24438)

Adding Andy to the IoT CODEOWNERS

* [Form Recognizer] Update to latest swagger (#24494)

* Update docker-start-proxy.ps1 (#24495)

Update to the latest version of the container

Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>

* Added support for Key Rotation. (#24452)

* Added support for Key Rotation.

* Added tests and updated recordings where necessary.

* Added code snippets for Key Rotation. Updated client documentation and existing code snippets.

* Applied PR feedback.

* Removed unused import.

* Updated releaseKey test for MHSM.

* Fixed tests after merge from main.

* Renamed Mixed Audio models ( Addressed comments in apiview ) (#24481)

* Suggestions after apiview review

* Changes for the comments of  API Review

* tests added back

* Added StartRecordingOptions class

* Annotation added for new class

* Setters return type changed

* Renaming enum names ( feedback on APIView review )

Co-authored-by: Ninika Sharma <ninsharm@microsoft.com>

* Prepare Azure Core Libraries for October 2021 Release (#24498)

Prepare Azure Core Libraries for October 2021 Release

* [EventGrid] Regenerate code using the latest rest commit sha (#24482)

* Add Storage Live Test Run to Core Live Test Run (#24499)

Add Storage Live Test Run to Core Live Test Run

* [Storage] Run CI and live tests on Java 17 (#24492)

* does this work?

* try this.

* Revert "try this."

This reverts commit f157e60.

* does this help ?

* hmm?

* hungry?

* use java 17 in ci.

* fix at least nio.

* move it.

* fixes.

* disable these tests on java 17. CGLib doesn't work

* fix that.

* add support for setting throughput on database creation (#24456)

* add support for setting throughput on database creation

* added section to readme

* removed locale from links

* fix checkstyle issues

* do not overwrite cosmosTemplate

* Increment version for core releases (#24504)

Increment package version after release of Core libraries

* fix(*): use library RedirectPolicy now that it is available (#24502)

* Fix azure-core-http-jdk-httpclient Tests (#24511)

Fix azure-core-http-jdk-httpclient Tests

* Add Form recognizer migration guide (#24472)

* Fixed some test build failures

Co-authored-by: David R. Williamson <drwill@microsoft.com>
Co-authored-by: Connie Yau <conniey@microsoft.com>
Co-authored-by: Anu Thomas Chandy <anuamd@hotmail.com>
Co-authored-by: Alan Zimmer <48699787+alzimmermsft@users.noreply.github.com>
Co-authored-by: Soyoung Eom <soeom@microsoft.com>
Co-authored-by: Weidong Xu <weidxu@microsoft.com>
Co-authored-by: Vinay Gera <vigera@microsoft.com>
Co-authored-by: Azure SDK Bot <53356347+azure-sdk@users.noreply.github.com>
Co-authored-by: praveenkuttappan <prmarott@microsoft.com>
Co-authored-by: praveenkuttappan <55455725+praveenkuttappan@users.noreply.github.com>
Co-authored-by: Wes Haggard <weshaggard@users.noreply.github.com>
Co-authored-by: Melissa Neubert <melissa.neubert1@gmail.com>
Co-authored-by: Melissa Neubert <mneubert@microsoft.com>
Co-authored-by: Chidozie Ononiwu (His Righteousness) <31145988+chidozieononiwu@users.noreply.github.com>
Co-authored-by: Pallavi Taneja <pallavit@users.noreply.github.com>
Co-authored-by: Sameeksha Vaity <savaity@microsoft.com>
Co-authored-by: liuzhicheng <70368631+zhichengliu12581@users.noreply.github.com>
Co-authored-by: Rujun Chen <Rujun.Chen@microsoft.com>
Co-authored-by: vcolin7 <vicolina@microsoft.com>
Co-authored-by: Kamil Sobol <61715331+kasobol-msft@users.noreply.github.com>
Co-authored-by: jamdavi <73593426+jamdavi@users.noreply.github.com>
Co-authored-by: Sean Kane <68240067+seankane-msft@users.noreply.github.com>
Co-authored-by: ninikasharma <67986119+ninikasharma@users.noreply.github.com>
Co-authored-by: Ninika Sharma <ninsharm@microsoft.com>
Co-authored-by: Shawn Fang <45607042+mssfang@users.noreply.github.com>
Co-authored-by: Blackbaud-MikeLueders <Blackbaud-MikeLueders@users.noreply.github.com>
Co-authored-by: Christian Whitehead (MSFT) <35080559+chrwhit@users.noreply.github.com>
  • Loading branch information
@rickle-msft @conniey
@anuchandy @alzimmermsft @soeom @weidongxu-microsoft @g2vinay @azure-sdk @praveenkuttappan @weshaggard @melneubert @chidozieononiwu @pallavit @samvaity @zhichengliu12581 @vcolin7 @kasobol-msft @jamdavi @seankane-msft @ninikasharma @ninsharm @mssfang @Blackbaud-MikeLueders @chrwhit
28 people authored Oct 4, 2021
1 parent 728261d commit 844e780
Show file tree
Hide file tree
Showing 1,515 changed files with 54,163 additions and 66,959 deletions.
4 changes: 2 additions & 2 deletions .github/CODEOWNERS
View file
Original file line number Diff line number Diff line change
Expand Up @@ -56,10 +56,10 @@
/sdk/cosmos/azure-spring-data-cosmos-test/ @kushagraThapar @mbhaskar @saragluna @yiliuTo @chenrujun @backwind1233 @stliu @ZhuXiaoBing-cn @zhichengliu12581 @moarychan @michaelqi793

# PRLabel: %Digital Twins
/sdk/digitaltwins/ @drwill-ms @timtay-microsoft @abhipsaMisra @azabbasi
/sdk/digitaltwins/ @drwill-ms @azabbasi @johngallardo

# PRLabel: %Models Repository
/sdk/modelsrepository @drwill-ms @timtay-microsoft @abhipsaMisra @azabbasi @digimaun
/sdk/modelsrepository @drwill-ms @timtay-microsoft @abhipsaMisra @azabbasi @digimaun @andykwong-ms

# PRLabel: %Event Grid
/sdk/eventgrid/ @srnagar @lmolkova @mssfang
Expand Down
8 changes: 4 additions & 4 deletions common/perf-test-core/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -55,12 +55,12 @@
<rules>
<bannedDependencies>
<includes>
<include>com.fasterxml.jackson.core:jackson-databind:[2.12.4]</include> <!-- {x-include-update;com.fasterxml.jackson.core:jackson-databind;external_dependency} -->
<include>com.fasterxml.jackson.core:jackson-databind:[2.12.5]</include> <!-- {x-include-update;com.fasterxml.jackson.core:jackson-databind;external_dependency} -->

<!-- special allowance for perf-test-core as it is not a shipping library: -->
<include>com.beust:jcommander:[1.78]</include> <!-- {x-include-update;com.beust:jcommander;external_dependency} -->

<include>io.projectreactor:reactor-core:[3.4.9]</include> <!-- {x-include-update;io.projectreactor:reactor-core;external_dependency} -->
<include>io.projectreactor:reactor-core:[3.4.10]</include> <!-- {x-include-update;io.projectreactor:reactor-core;external_dependency} -->
</includes>
</bannedDependencies>
</rules>
Expand All @@ -73,7 +73,7 @@
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.12.4</version> <!-- {x-version-update;com.fasterxml.jackson.core:jackson-databind;external_dependency} -->
<version>2.12.5</version> <!-- {x-version-update;com.fasterxml.jackson.core:jackson-databind;external_dependency} -->
</dependency>
<dependency>
<groupId>com.beust</groupId>
Expand All @@ -83,7 +83,7 @@
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-http-netty</artifactId>
<version>1.11.0</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
<version>1.11.1</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
</dependency>
</dependencies>
</project>
8 changes: 4 additions & 4 deletions common/smoke-tests/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,25 +88,25 @@
<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core</artifactId>
<version>1.20.0</version> <!-- {x-version-update;com.azure:azure-core;dependency} -->
<version>1.21.0</version> <!-- {x-version-update;com.azure:azure-core;dependency} -->
</dependency>

<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-http-netty</artifactId>
<version>1.11.0</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
<version>1.11.1</version> <!-- {x-version-update;com.azure:azure-core-http-netty;dependency} -->
</dependency>

<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-tracing-opentelemetry</artifactId>
<version>1.0.0-beta.14</version> <!-- {x-version-update;com.azure:azure-core-tracing-opentelemetry;dependency} -->
<version>1.0.0-beta.15</version> <!-- {x-version-update;com.azure:azure-core-tracing-opentelemetry;dependency} -->
</dependency>

<dependency>
<groupId>com.azure</groupId>
<artifactId>azure-core-http-okhttp</artifactId>
<version>1.7.3</version> <!-- {x-version-update;com.azure:azure-core-http-okhttp;dependency} -->
<version>1.7.4</version> <!-- {x-version-update;com.azure:azure-core-http-okhttp;dependency} -->
</dependency>

<dependency>
Expand Down
5 changes: 0 additions & 5 deletions eng/bomgenerator/generateAzureSDKBOM.cmd
View file

This file was deleted.

28 changes: 28 additions & 0 deletions eng/bomgenerator/generateAzureSDKBOM.ps1
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
$repoRoot = Resolve-Path "${PSScriptRoot}..\..\.."
$inputDir = Join-Path ${PSScriptRoot} "inputDir"
$outputDir = Join-Path ${PSScriptRoot} "outputDir"
$versionClientFileName = "version_client.txt"
$pomFileName = "pom.xml"
$defaultVersionClientFilePath = Join-Path $inputDir $versionClientFileName
$defaultPomFilePath = Join-Path $inputDir $pomFileName
$versionClientFilePath = Join-Path $repoRoot "eng" "versioning" $versionClientFileName
$bomPomFilePath = Join-Path $repoRoot "sdk" "containerregistry" "azure-containers-containerregistry" $pomFileName

if(! (Test-Path $inputDir)) {
New-Item -Path $PSScriptRoot -Name "inputDir" -ItemType "directory"
}

if(! (Test-Path $defaultVersionClientFilePath)) {
Copy-Item $versionClientFilePath -Destination $inputDir
}

if(! (Test-Path $defaultPomFilePath)) {
Copy-Item $bomPomFilePath -Destination $inputDir
}


"mvn exec:java -Dexec.args='-inputDir=$inputDir -outputDir=$outputDir -mode=analyze'"
if($LASTEXITCODE -ne 0) {
LogError "Failed to generate the BOM."
exit 1
}
11 changes: 7 additions & 4 deletions eng/bomgenerator/pom.xml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version> <!-- {x-version-update;org.slf4j:slf4j-log4j12;external_dependency} -->
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.dataformat</groupId>
<artifactId>jackson-dataformat-xml</artifactId>
<version>2.12.4</version> <!-- {x-version-update;com.fasterxml.jackson.dataformat:jackson-dataformat-xml;external_dependency} -->
</dependency>
</dependencies>
<build>
<resources>
Expand Down Expand Up @@ -69,10 +74,8 @@
<workingDirectory>${project.basedir}</workingDirectory>
<mainClass>com.azure.tools.bomgenerator.Main</mainClass>
<arguments>
<argument>-inputFile=${project.basedir}/../versioning/version_client.txt</argument>
<argument>-outputFile=${project.basedir}/target/newbom.xml</argument>
<argument>-pomFile=${project.basedir}/../../sdk/boms/azure-sdk-bom/pom.xml</argument>
<argument>-reportFile=${project.basedir}/target/dependency_conflictlist.html</argument>
<argument>-inputdir=${project.basedir}/inputdir</argument>
<argument>-outputdir=${project.basedir}/outputdir</argument>
</arguments>
<cleanupDaemonThreads>false</cleanupDaemonThreads>
</configuration>
Expand Down
15 changes: 13 additions & 2 deletions eng/bomgenerator/readme.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,18 @@ This tool generates a BOM from a given set of input dependencies.
This tool has 2 modes
1. Generate mode - When run in this mode the tool generates the BOM. This is also the default mode.
Arguments to run the tool in this mode.
-inputFile=<location_of_versionClient.txt> -outputFile=<location_of_new_bomFile> -pomFile=<location_of_current_bomFile> -inputdependenciesfile=<optional_file_to_override_input_dependencies>
-inputDir=<location_of_input_directory>
This includes the following files.
1. Version_client.txt file
2. The BOM file which is the initial POM file.
3. InputDependencies file to override the inputs from the version_client.txt. This is an optional file.

-outputDir=<location_of_outputput_directory>
This generates the followibg files in the location.
1. Report log.
2. Newly created BOM file

2. Analyze mode - When run in this mode the tool validates the BOM.
Arguments to run the tool in this mode.
-mode=analyze -pomFile=<location_of_bomFile>
-mode=analyze -inputDir=<location_to_inputDir)>
This location includes the pom file for the BOM or any library.
104 changes: 50 additions & 54 deletions eng/bomgenerator/src/main/java/com/azure/tools/bomgenerator/BomGenerator.java
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,12 @@
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
import java.util.ArrayList;
import java.util.Collection;
Expand Down Expand Up @@ -48,84 +50,77 @@ public class BomGenerator {
private String overriddenInputDependenciesFileName;
private String reportFileName;
private String mode;
private String outputDirectory;
private String inputDirectory;

private static Logger logger = LoggerFactory.getLogger(BomGenerator.class);

BomGenerator() {
this.mode = GENERATE_MODE;
}

public String getInputFileName() {
return this.inputFileName;
}

public void setInputFileName(String inputFileName) {
this.inputFileName = inputFileName;
}

public String getOverriddenInputDependenciesFileName() {
return this.overriddenInputDependenciesFileName;
}
BomGenerator(String inputDirectory, String outputDirectory, String mode) throws FileNotFoundException {
validateNotNullOrEmpty(inputDirectory, "inputDirectory");
validateNotNullOrEmpty(outputDirectory, "outputDirectory");

public void setOverriddenInputDependenciesFileName(String overriddenInputDependenciesFileName) {
this.overriddenInputDependenciesFileName = overriddenInputDependenciesFileName;
}

public String getOutputFileName() {
return this.outputFileName;
}
this.inputDirectory = inputDirectory;
this.outputDirectory = outputDirectory;
this.mode = (mode == null ? GENERATE_MODE : mode);

public void setOutputFileName(String outputFileName) {
this.outputFileName = outputFileName;
}
parseInputs();
validateInputs();

public String getPomFileName() {
return this.pomFileName;
Path outputDirPath = Paths.get(outputDirectory);
outputDirPath.toFile().mkdirs();
}

public void setPomFileName(String pomFileName) {
this.pomFileName = pomFileName;
}

public String getReportFileName() {
return this.reportFileName;
}
private void parseInputs() throws FileNotFoundException {
this.outputFileName = Paths.get(outputDirectory, "pom.xml").toString();
this.reportFileName = Paths.get(outputDirectory, "dependency_conflictlist.html").toString();
this.inputFileName = Paths.get(inputDirectory, "version_client.txt").toString();
this.pomFileName = Paths.get(inputDirectory, "pom.xml").toString();
this.overriddenInputDependenciesFileName = Paths.get(inputDirectory, "dependencies.txt").toString();
}

public void setReportFileName(String reportFileName) {
this.reportFileName = reportFileName;
}
private void validateInputs() throws FileNotFoundException {
switch (this.mode) {
case ANALYZE_MODE:
validateFilePath(this.pomFileName);
break;

public String getMode() {
return this.mode;
case GENERATE_MODE:
// In generate mode, we should have the inputFile, outputFile and the pomFile.
validateFilePath(this.pomFileName);
validateFilePath(this.inputFileName);
break;
}
}

public void setMode(String mode) {
this.mode = mode;
}
private void validateFilePath(String filePath) throws FileNotFoundException {
if(Files.notExists(Paths.get(filePath))) {
throw new FileNotFoundException(String.format("%s not found.", filePath));
}
}

public void run() {
public boolean run() {
switch (mode) {
case ANALYZE_MODE:
validate();
break;
return validate();

case GENERATE_MODE:
generate();
break;
return generate();

default:
logger.error("Unknown value for mode: {}", mode);
break;
}

return false;
}

private void validate() {
private boolean validate() {
var inputDependencies = parsePomFileContent(this.pomFileName);
DependencyAnalyzer analyzer = new DependencyAnalyzer(inputDependencies, null, this.reportFileName);
analyzer.validate();
return !analyzer.validate();
}

private void generate() {
private boolean generate() {
List<BomDependency> inputDependencies = scan();
List<BomDependency> externalDependencies = resolveExternalDependencies();

Expand All @@ -141,14 +136,15 @@ private void generate() {
outputDependencies = analyzer.getBomEligibleDependencies();

// 4. Create the new BOM file.
if(!validationFailed) {
if (!validationFailed) {
// Rewrite the existing BOM to have the dependencies in the order in which we insert them, making the diff PR easier to review.
rewriteExistingBomFile();
writeBom(outputDependencies);
return true;
}
else {
logger.trace("Validation for the BOM failed. Exiting...");
}

logger.trace("Validation for the BOM failed. Exiting...");
return false;
}

private List<BomDependency> scanVersioningClientFileDependencies() {
Expand Down
Loading

0 comments on commit 844e780

Please sign in to comment.