Skip to content

Commit

Permalink
Merge pull request #2257 from Azure/dev
Browse files Browse the repository at this point in the history
  • Loading branch information
davidmrdavid authored Aug 25, 2022
2 parents bd448b0 + ec4c49b commit ab353f0
Show file tree
Hide file tree
Showing 76 changed files with 1,898 additions and 169 deletions.
536 changes: 536 additions & 0 deletions .github/fabricbot.json

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions .github/workflows/smoketest-java8-v4.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,20 @@ jobs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
- name: Set up JDK 8
uses: actions/setup-java@v2
with:
java-version: '8'
distribution: 'temurin'
- name: Build azure functions sample
uses: gradle/gradle-build-action@bc3340afc5e3cc44f2321809ac090d731c13c514
with:
gradle-version: 6.5
arguments: azureFunctionsPackage -p test/SmokeTests/OOProcSmokeTests/durableJava/
continue-on-error: true
- name: Download azure functions java library # TODO: Remove this step once gradle plugin is updated
run: |
wget -P test/SmokeTests/OOProcSmokeTests/durableJava/build/azure-functions/durableJava/lib/ "https://repo.maven.apache.org/maven2/com/microsoft/azure/functions/azure-functions-java-library/2.0.1/azure-functions-java-library-2.0.1.jar" --show-progress
- name: Run V4 Java 8 Smoke Test
run: test/SmokeTests/e2e-test.ps1 -DockerfilePath test/SmokeTests/OOProcSmokeTests/durableJava/Dockerfile -HttpStartPath api/StartOrchestration
shell: pwsh
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,8 +36,8 @@ Durable Functions supports a subset of languages supported by Azure Functions:
| C# | Generally available - [get started](https://docs.microsoft.com/azure/azure-functions/durable/durable-functions-create-first-csharp) | [Azure/azure-functions-durable-extension](https://github.com/Azure/azure-functions-durable-extension) |
| JavaScript | Generally available - [get started](https://docs.microsoft.com/azure/azure-functions/durable/quickstart-js-vscode) | [Azure/azure-functions-durable-js](https://github.com/Azure/azure-functions-durable-js) |
| Python | Generally available - [get started](https://docs.microsoft.com/azure/azure-functions/durable/quickstart-python-vscode) | [Azure/azure-functions-durable-python](https://github.com/Azure/azure-functions-durable-python) |
| PowerShell | Public preview - [get started](https://docs.microsoft.com/en-us/azure/azure-functions/durable/quickstart-powershell-vscode) | [Azure/azure-functions-powershell-worker](https://github.com/Azure/azure-functions-powershell-worker) |
| Java | Under consideration - [give feedback](https://github.com/Azure/azure-functions-java-worker/issues/213) | |
| PowerShell | Generally available - [get started](https://docs.microsoft.com/en-us/azure/azure-functions/durable/quickstart-powershell-vscode) | [Azure/azure-functions-powershell-worker](https://github.com/Azure/azure-functions-powershell-worker) |
| Java | Public Preview | [Microsoft/durabletask-java](https://github.com/microsoft/durabletask-java) |

Each language has its own language-specific SDK and programming model. Regardless of which language you use, the extension in this repo must be installed to enable the Durable Functions triggers.

Expand All @@ -46,7 +46,7 @@ Each language has its own language-specific SDK and programming model. Regardles
The Durable Functions NuGet package can be referenced directly in a Visual Studio project or can be installed using the [Azure Functions Core Tools](https://docs.microsoft.com/azure/azure-functions/functions-run-local) command-line:

```bash
func extensions install -p Microsoft.Azure.WebJobs.Extensions.DurableTask -v 2.3.0
func extensions install -p Microsoft.Azure.WebJobs.Extensions.DurableTask -v <latest version on Nuget.org>
```

Durable Functions is also available in supported [extension bundles](https://docs.microsoft.com/azure/azure-functions/functions-bindings-register#extension-bundles). Note that extension bundles are only supported for non-.NET languages.
Expand All @@ -60,7 +60,7 @@ Many features of Durable Functions have been voluntarily contributed by the comm
Durable Functions is developed in collaboration with Microsoft Research. As a result, the Durable Functions team actively produces research papers and artifacts; these include:

* [Durable Functions: Semantics for Stateful Serverless](https://www.microsoft.com/en-us/research/uploads/prod/2021/10/DF-Semantics-Final.pdf) _(OOPSLA'21)_
* [Serverless Workflows with Durable Functions and Netherite](https://arxiv.org/pdf/2103.00033.pdf) _(pre-print)_
* [Netherite: Efficient Execution of Serverless Workflows](https://www.microsoft.com/en-us/research/uploads/prod/2022/07/p1591-burckhardt.pdf) _(VLDB'22)_

## License

Expand Down
148 changes: 148 additions & 0 deletions azure-pipelines-analyzer-release.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
trigger: none
pr: none

pool:
name: '1ES-Hosted-DurableTaskFramework'
demands:
- ImageOverride -equals MMS2022TLS

steps:

# Configure all the .NET SDK versions we need
- task: UseDotNet@2
displayName: 'Use the .NET Core 2.1 SDK (required for build signing)'
inputs:
packageType: 'sdk'
version: '2.1.x'

- task: UseDotNet@2
displayName: 'Use the .NET Core 3.1 SDK'
inputs:
packageType: 'sdk'
version: '3.1.x'

- task: UseDotNet@2
displayName: 'Use the .NET 6 SDK'
inputs:
packageType: 'sdk'
version: '6.0.x'

# Use NuGet
- task: NuGetToolInstaller@1
displayName: 'Use NuGet '

# dotnet restore
- task: DotNetCoreCLI@2
displayName: 'dotnet restore'
inputs:
command: restore
projects: '**/**/*.csproj'
feedsToUse: config
nugetConfigPath: '.nuget/nuget.config'

# Build durable-analyzer
- task: VSBuild@1
displayName: 'Build Durable Analyzer'
inputs:
solution: '**/WebJobs.Extensions.DurableTask.Analyzers.sln'
vsVersion: "16.0"
configuration: Release

# Authenticode sign all the DLLs with the Microsoft certificate.
- task: EsrpCodeSigning@1
inputs:
ConnectedServiceName: 'ESRP Service'
FolderPath: 'src/WebJobs.Extensions.DurableTask.Analyzers/bin/Release'
Pattern: '*DurableTask.Analyzers.dll'
signConfigType: 'inlineSignParams'
inlineOperation: |
[
{
"KeyCode": "CP-230012",
"OperationCode": "SigntoolSign",
"Parameters": {
"OpusName": "Microsoft",
"OpusInfo": "http://www.microsoft.com",
"FileDigest": "/fd \"SHA256\"",
"PageHash": "/NPH",
"TimeStamp": "/tr \"http://rfc3161.gtm.corp.microsoft.com/TSS/HttpTspServer\" /td sha256"
},
"ToolName": "sign",
"ToolVersion": "1.0"
},
{
"KeyCode": "CP-230012",
"OperationCode": "SigntoolVerify",
"Parameters": {},
"ToolName": "sign",
"ToolVersion": "1.0"
}
]
SessionTimeout: '60'
MaxConcurrency: '50'
MaxRetryAttempts: '5'

# SBOM generator task for additional supply chain protection
- task: AzureArtifacts.manifest-generator-task.manifest-generator-task.ManifestGeneratorTask@0
displayName: 'Manifest Generator '
inputs:
BuildDropPath: '$(System.DefaultWorkingDirectory)'

# dotnet pack
# Packaging needs to be a separate step from build.
# This will automatically pick up the signed DLLs.
- task: DotNetCoreCLI@2
displayName: 'dotnet pack WebJobs.Extensions.DurableTask.Analyzers.csproj'
inputs:
command: pack
packagesToPack: 'src/**/WebJobs.Extensions.DurableTask.Analyzers.csproj'
configuration: Release
packDirectory: 'azure-functions-durable-extension'
nobuild: true

# Remove redundant symbol package(s)
- script: |
echo *** Searching for .symbols.nupkg files to delete...
dir /s /b *.symbols.nupkg
echo *** Deleting .symbols.nupkg files...
del /S /Q *.symbols.nupkg
echo *** Listing remaining packages
dir /s /b *.nupkg
displayName: 'Remove Redundant Symbols Package(s)'
continueOnError: true

# Digitally sign all the nuget packages with the Microsoft certificate.
# This appears to be an in-place signing job, which is convenient.
- task: EsrpCodeSigning@1
inputs:
ConnectedServiceName: 'ESRP Service'
FolderPath: '$(System.DefaultWorkingDirectory)'
Pattern: '*.nupkg'
signConfigType: 'inlineSignParams'
inlineOperation: |
[
{
"KeyCode": "CP-401405",
"OperationCode": "NuGetSign",
"Parameters": {},
"ToolName": "sign",
"ToolVersion": "1.0"
},
{
"KeyCode": "CP-401405",
"OperationCode": "NuGetVerify",
"Parameters": {},
"ToolName": "sign",
"ToolVersion": "1.0"
}
]
SessionTimeout: '60'
MaxConcurrency: '50'
MaxRetryAttempts: '5'

# Make the nuget packages available for download in the ADO portal UI
- publish: '$(System.DefaultWorkingDirectory)/azure-functions-durable-extension'
displayName: 'Publish nuget packages to Artifacts'
artifact: PackageOutput
4 changes: 3 additions & 1 deletion release_notes.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,12 @@
## Bug Fixes
- Correctly Serialize HostStoppingEvent in ActivityShim (https://github.com/Azure/azure-functions-durable-extension/pull/2178)
- Fix NotImplementedException for management API calls from Java client (https://github.com/Azure/azure-functions-durable-extension/pull/2193)
- Handle OOM and other exceptions in entity shim by aborting the session (https://github.com/Azure/azure-functions-durable-extension/pull/2234)

## Enhancements
- add optional 'instanceIdPrefix' query parameter to the HTTP API for instance queries

## Dependencies
- DurableTask.Core --> v2.10.*
- DurableTask.AzureStorage --> v1.12.*
- DurableTask.AzureStorage --> v1.12.*
- DurableTask.Analyzers --> 0.5.0
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
<AppDesignerFolder>Properties</AppDesignerFolder>
<RootNamespace>DurableFunctionsAnalyzer.Vsix</RootNamespace>
<AssemblyName>DurableFunctionsAnalyzer</AssemblyName>
<TargetFrameworkVersion>v4.6.1</TargetFrameworkVersion>
<TargetFrameworkVersion>v4.6.2</TargetFrameworkVersion>
<GeneratePkgDefFile>false</GeneratePkgDefFile>
<IncludeAssemblyInVSIXContainer>false</IncludeAssemblyInVSIXContainer>
<IncludeDebugSymbolsInVSIXContainer>false</IncludeDebugSymbolsInVSIXContainer>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ private static bool TryGetInputTypeFromContext(SemanticModel semanticModel, Synt

private static bool TryGetDurableActivityContextExpression(SemanticModel semanticModel, SyntaxNode node, out SyntaxNode durableContextExpression)
{
if (SyntaxNodeUtils.TryGetMethodDeclaration(node, out SyntaxNode methodDeclaration))
if (SyntaxNodeUtils.TryGetMethodDeclaration(node, out MethodDeclarationSyntax methodDeclaration))
{
var memberAccessExpressionList = methodDeclaration.DescendantNodes().Where(x => x.IsKind(SyntaxKind.SimpleMemberAccessExpression));
foreach (var memberAccessExpression in memberAccessExpressionList)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ private void AnalyzeDispatchAndFindMethodDeclarations(SyntaxNodeAnalysisContext
var name = expression.Name;
if (name.ToString().StartsWith("DispatchAsync"))
{
if(SyntaxNodeUtils.TryGetMethodDeclaration(expression, out SyntaxNode methodDeclaration))
if(SyntaxNodeUtils.TryGetMethodDeclaration(expression, out MethodDeclarationSyntax methodDeclaration))
{
methodDeclarations.Add(methodDeclaration);
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,7 @@ public class StaticFunctionAnalyzer

public static void ReportProblems(CompilationAnalysisContext context, SyntaxNode methodDeclaration)
{
var staticKeyword = methodDeclaration.ChildTokens().FirstOrDefault(x => x.IsKind(SyntaxKind.StaticKeyword));
if (staticKeyword == null || staticKeyword.IsKind(SyntaxKind.None))
if (!SyntaxNodeUtils.IsInStaticMethod(methodDeclaration))
{
SemanticModel semanticModel = context.Compilation.GetSemanticModel(methodDeclaration.SyntaxTree);
if (IsInEntityClass(semanticModel, methodDeclaration))
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using System.Collections.Generic;
using System.Linq;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
Expand All @@ -21,10 +22,15 @@ public class BindingAnalyzer

public static readonly DiagnosticDescriptor Rule = new DiagnosticDescriptor(DiagnosticId, Title, MessageFormat, Category, Severity, isEnabledByDefault: true);

public static bool RegisterDiagnostic(CompilationAnalysisContext context, SyntaxNode method)
public static bool RegisterDiagnostic(CompilationAnalysisContext context, SemanticModel semanticModel, SyntaxNode method)
{
var diagnosedIssue = false;

if (!SyntaxNodeUtils.IsInsideFunction(semanticModel, method))
{
return diagnosedIssue;
}

var parameterList = method.ChildNodes().First(x => x.IsKind(SyntaxKind.ParameterList));

foreach (SyntaxNode descendant in parameterList.DescendantNodes())
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
// Copyright (c) .NET Foundation. All rights reserved.
// Licensed under the MIT License. See LICENSE in the project root for license information.

using System.Collections.Generic;
using System.Linq;
using Microsoft.CodeAnalysis;
using Microsoft.CodeAnalysis.CSharp;
using Microsoft.CodeAnalysis.CSharp.Syntax;
using Microsoft.CodeAnalysis.Diagnostics;

namespace Microsoft.Azure.WebJobs.Extensions.DurableTask.Analyzers
{
[DiagnosticAnalyzer(LanguageNames.CSharp)]
public class DependencyInjectionAnalyzer
{
public const string DiagnosticId = "DF0113";

private static readonly LocalizableString Title = new LocalizableResourceString(nameof(Resources.DependencyInjectionAnalyzerTitle), Resources.ResourceManager, typeof(Resources));
private static readonly LocalizableString MessageFormat = new LocalizableResourceString(nameof(Resources.DeterministicAnalyzerMessageFormat), Resources.ResourceManager, typeof(Resources));
private static readonly LocalizableString Description = new LocalizableResourceString(nameof(Resources.DeterministicAnalyzerDescription), Resources.ResourceManager, typeof(Resources));
private const string Category = SupportedCategories.Orchestrator;
public const DiagnosticSeverity Severity = DiagnosticSeverity.Warning;

public static readonly DiagnosticDescriptor Rule = new DiagnosticDescriptor(DiagnosticId, Title, MessageFormat, Category, Severity, isEnabledByDefault: true, description: Description);

public static bool RegisterDiagnostic(CompilationAnalysisContext context, SyntaxNode method)
{
var diagnosedIssue = false;

if (!SyntaxNodeUtils.IsInStaticClass(method))
{
if(TryGetInjectedVariables(method, out List<SyntaxNode> injectedVariables))
{
var methodVariablesUsed = method.DescendantNodes().Where(x => x.IsKind(SyntaxKind.IdentifierName));

var varaiblesToDiagnose = methodVariablesUsed.Where(x => injectedVariables.Exists(y => y.ToString() == x.ToString()));

foreach (var variable in varaiblesToDiagnose)
{
var diagnostic = Diagnostic.Create(Rule, variable.GetLocation(), variable);

if (context.Compilation.ContainsSyntaxTree(method.SyntaxTree))
{
context.ReportDiagnostic(diagnostic);
}

diagnosedIssue = true;
}
}
}

return diagnosedIssue;
}

private static bool TryGetInjectedVariables(SyntaxNode method, out List<SyntaxNode> injectedVariables)
{
injectedVariables = new List<SyntaxNode>();
var addedVariable = false;
if (SyntaxNodeUtils.TryGetConstructor(method, out ConstructorDeclarationSyntax constructor))
{
var parameters = constructor.ParameterList.ChildNodes();
var injectedParameterNames = new List<SyntaxToken>();
foreach (SyntaxNode parameter in parameters)
{
injectedParameterNames.Add(parameter.ChildTokens().FirstOrDefault(x => x.IsKind(SyntaxKind.IdentifierToken)));
}

var assignementExpressions = constructor.DescendantNodes().Where(x => x.IsKind(SyntaxKind.SimpleAssignmentExpression));

foreach (AssignmentExpressionSyntax assignmentExpression in assignementExpressions)
{
var injectedRightSideNode = assignmentExpression.Right.DescendantNodes().Where(x => x.IsKind(SyntaxKind.IdentifierName) && injectedParameterNames.Contains(((IdentifierNameSyntax)x).Identifier));

if (injectedRightSideNode == null)
{
continue;
}

var assignedLeftSideNode = assignmentExpression.Left.DescendantNodes().FirstOrDefault(x => x.IsKind(SyntaxKind.IdentifierName));

if (assignedLeftSideNode != null)
{
injectedVariables.Add(assignedLeftSideNode);
addedVariable = true;
}
}
}

return addedVariable;
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,8 @@ public override ImmutableArray<DiagnosticDescriptor> SupportedDiagnostics
TimerAnalyzer.V2Rule,
CancellationTokenAnalyzer.Rule,
BindingAnalyzer.Rule,
MethodInvocationAnalyzer.Rule);
MethodInvocationAnalyzer.Rule,
DependencyInjectionAnalyzer.Rule);
}
}

Expand Down Expand Up @@ -65,7 +66,8 @@ private void RegisterAnalyzers(CompilationAnalysisContext context)
| ThreadTaskAnalyzer.RegisterDiagnostic(context, semanticModel, methodDeclaration)
| TimerAnalyzer.RegisterDiagnostic(context, semanticModel, methodDeclaration)
| CancellationTokenAnalyzer.RegisterDiagnostic(context, methodDeclaration)
| BindingAnalyzer.RegisterDiagnostic(context, methodDeclaration))
| BindingAnalyzer.RegisterDiagnostic(context, semanticModel, methodDeclaration)
| DependencyInjectionAnalyzer.RegisterDiagnostic(context, methodDeclaration))
{
methodInvocationAnalyzer.RegisterDiagnostics(context, methodInformation);
}
Expand Down
Loading

0 comments on commit ab353f0

Please sign in to comment.