Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge release/3.0.1xx to master #3113

Merged
merged 44 commits into from
Apr 16, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
3b824b6
Add tests for passing FilesToSkip to GenerateDepsFile
dsplaisted Mar 26, 2019
64acbcc
Stop generating runtime signature for deps.json
dsplaisted Mar 26, 2019
45bcfd4
Set package ID metadata on conflict items
dsplaisted Apr 5, 2019
acc8b26
Implement new deps file generation logic
dsplaisted Apr 5, 2019
76d7db9
Write RuntimeIdentifier to RuntimeTargets items in ResolvePackageAssets
dsplaisted Apr 5, 2019
4a21bb0
Use ResolvedFile class to pass resolved NuGet files to new deps file …
dsplaisted Apr 5, 2019
79a5043
Write RelativePath for resolved compile reference items in ResolvePac…
dsplaisted Apr 5, 2019
755536d
More implementation of new deps file generation logic
dsplaisted Apr 2, 2019
f740835
Integrate new deps file generation logic into GenerateDepsFile task
dsplaisted Apr 2, 2019
fc6de74
Use PathInPackage instead of RelativePath for item metadata
dsplaisted Apr 2, 2019
bcd5188
Write deps files with both old and new methods, and ensure they are e…
dsplaisted Apr 3, 2019
4b71244
Fix dependency calculation in deps file generation
dsplaisted Apr 3, 2019
916eb83
Remove conflict files from items used by GenerateDepsFile
dsplaisted Apr 3, 2019
c7549ff
Include more metadata on output from ResolvePackageAssets even if it …
dsplaisted Apr 3, 2019
d4637bd
Fix dictionary that wasn't case-insensitive, delete extra deps.json f…
dsplaisted Apr 3, 2019
7bb64cc
Add property to control which version of deps generation is used
dsplaisted Apr 3, 2019
cca7bfc
Use RID-less assets file target compilation assets when writing deps …
dsplaisted Apr 4, 2019
4ef1d17
Fix writing runtime pack libraries to deps.json
dsplaisted Apr 4, 2019
db4dc8d
Don't create empty asset groups for deps.json if all files are trimmed
dsplaisted Apr 4, 2019
6a1d766
Fix path for resources assets in deps.json
dsplaisted Apr 4, 2019
f2ed7de
Filter conflict runtime pack assets from deps file
dsplaisted Apr 4, 2019
abd3f36
Filter publish conflict packages from new deps file generation
dsplaisted Apr 5, 2019
91d36a9
Fix deps library lookup to handle multitargeted projects depending on…
dsplaisted Apr 5, 2019
bd10b02
Update tests that need new deps file generation mode
dsplaisted Apr 5, 2019
eb9afbb
Read PackageOverrides from targeting packs
dsplaisted Apr 5, 2019
030137f
Don't rely on heuristic to find NuGet package ID for a file
dsplaisted Apr 5, 2019
fc31c03
Fix platform library calculation for ASP.NET Core apps
dsplaisted Apr 5, 2019
f353115
Fix resource skipping tests on non-Windows
dsplaisted Apr 5, 2019
c7c062e
Match previous logic for what library dependencies are included in th…
dsplaisted Apr 6, 2019
e0ec4d0
Don't use fluent assertions in product code to validate that deps fil…
dsplaisted Apr 6, 2019
b0ee865
Code review feedback
dsplaisted Apr 6, 2019
b20aada
Fix generating error about mismatching deps files
dsplaisted Apr 6, 2019
68e6297
Fix GenerateDepsFile task not returning failure when error is logged
dsplaisted Apr 8, 2019
7f3c8b7
Don't remove compile conflicts from runtime assets and vice versa for…
dsplaisted Apr 8, 2019
ca946c4
Use string comparison instead of parsing and comparing json
dsplaisted Apr 8, 2019
adb4014
Don't put runtime pack dependencies in the "compile" projects in deps…
dsplaisted Apr 8, 2019
4754e44
Update deps file mismatch error text
dsplaisted Apr 8, 2019
a56fb4f
Update stage 0
dsplaisted Apr 10, 2019
c8f8b9d
Add System.Data.DataSetExtensions.dll to expected .NET Core 3 self-co…
dsplaisted Apr 10, 2019
3ce6af5
Install 2.2.4 shared framework
dsplaisted Apr 10, 2019
9ead3df
Update perf test sources
dsplaisted Apr 11, 2019
814b789
Merge pull request #3086 from dsplaisted/rewrite-deps-generation
dsplaisted Apr 11, 2019
e21799e
Merge branch 'master' into merges/release/3.0.1xx-to-master
nguerrera Apr 11, 2019
643d19e
Update stage 0
dsplaisted Apr 16, 2019
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion eng/restore-toolset.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ function InitializeCustomSDKToolset {
InstallDotNetSharedFramework "1.0.5"
InstallDotNetSharedFramework "1.1.2"
InstallDotNetSharedFramework "2.1.0"
InstallDotNetSharedFramework "2.2.3"
InstallDotNetSharedFramework "2.2.4"

CreateBuildEnvScript
InstallNuget
Expand Down
2 changes: 1 addition & 1 deletion eng/restore-toolset.sh
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ function InitializeCustomSDKToolset {
InstallDotNetSharedFramework "1.0.5"
InstallDotNetSharedFramework "1.1.2"
InstallDotNetSharedFramework "2.1.0"
InstallDotNetSharedFramework "2.2.3"
InstallDotNetSharedFramework "2.2.4"
}

# Installs additional shared frameworks for testing purposes
Expand Down
2 changes: 1 addition & 1 deletion global.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"tools": {
"dotnet": "3.0.100-preview5-011162",
"dotnet": "3.0.100-preview5-011317",
"vs-opt": {
"version": "15.9"
}
Expand Down
26 changes: 24 additions & 2 deletions src/Tasks/Common/ConflictResolution/ConflictItem.cs
Original file line number Diff line number Diff line change
Expand Up @@ -158,11 +158,33 @@ public string PackageId
{
if (_packageId == null)
{
_packageId = OriginalItem?.GetMetadata(MetadataNames.NuGetPackageId) ?? String.Empty;
_packageId = OriginalItem?.GetMetadata(MetadataNames.NuGetPackageId);

if (string.IsNullOrEmpty(_packageId))
{
_packageId = OriginalItem?.GetMetadata(MetadataKeys.PackageName) ?? string.Empty;
}

if (_packageId.Length == 0)
{
_packageId = NuGetUtils.GetPackageIdFromSourcePath(SourcePath) ?? String.Empty;
// We want to move away from using the heuristic of walking up the folder tree until
// we find a .nuspec in order to determine the package ID of a file. However, we
// don't want to accidentally stop having a package ID for a file that the heuristic
// would have found the package ID for. So to catch those cases, we throw an
// exception if the heuristic finds a package ID but we don't have the package ID
// from any other source
string packageIdFromPath = NuGetUtils.GetPackageIdFromSourcePath(SourcePath);
if (!string.IsNullOrEmpty(packageIdFromPath))
{
string path = OriginalItem?.ItemSpec;
if (string.IsNullOrEmpty(path))
{
path = SourcePath;
}
throw new InvalidOperationException("NuGetPackageId metadata not set on " + path);
}

_packageId = string.Empty;
}
}

Expand Down
2 changes: 1 addition & 1 deletion src/Tasks/Common/ConflictResolution/FrameworkListReader.cs
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ private static IEnumerable<ConflictItem> LoadConflictItems(string frameworkListP
}

ret.Add(new ConflictItem(assemblyName + ".dll",
packageId: null,
packageId: "TargetingPack",
assemblyVersion: assemblyVersion,
fileVersion: null));
}
Expand Down
2 changes: 1 addition & 1 deletion src/Tasks/Common/ConflictResolution/PackageOverride.cs
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@ private static IEnumerable<Tuple<string, Version>> CreateOverriddenPackages(stri
if (!string.IsNullOrEmpty(overriddenPackagesString))
{
overriddenPackagesString = overriddenPackagesString.Trim();
string[] overriddenPackagesAndVersions = overriddenPackagesString.Split(new char[] { ';' }, StringSplitOptions.RemoveEmptyEntries);
string[] overriddenPackagesAndVersions = overriddenPackagesString.Split(new char[] { ';', '\r', '\n', ' ' }, StringSplitOptions.RemoveEmptyEntries);
foreach (string overriddenPackagesAndVersion in overriddenPackagesAndVersions)
{
string trimmedOverriddenPackagesAndVersion = overriddenPackagesAndVersion.Trim();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ private ITaskItem CreateConflictTaskItem(ConflictItem conflict)
if (conflict.PackageId != null)
{
item.SetMetadata(nameof(ConflictItemType), conflict.ItemType.ToString());
item.SetMetadata(MetadataKeys.NuGetPackageId, conflict.PackageId);
}

return item;
Expand Down
3 changes: 2 additions & 1 deletion src/Tasks/Common/MetadataKeys.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ internal static class MetadataKeys
public const string IsImplicitlyDefined = "IsImplicitlyDefined";
public const string IsTopLevelDependency = "IsTopLevelDependency";
public const string AllowExplicitVersion = "AllowExplicitVersion";
public const string RelativePath = "RelativePath";

// Target Metadata
public const string RuntimeIdentifier = "RuntimeIdentifier";
Expand Down Expand Up @@ -61,7 +62,7 @@ internal static class MetadataKeys
public const string NuGetPackageId = "NuGetPackageId";
public const string NuGetPackageVersion = "NuGetPackageVersion";
public const string NuGetSourceType = "NuGetSourceType";
public const string RelativePath = "RelativePath";
public const string PathInPackage = "PathInPackage";
public const string PackageDirectory = "PackageDirectory";

// References
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0/osx.10.11-x64",
"signature": "6c0ed2851936790712df303bf7e27fc94dfbd644"
"signature": ""
},
"compilationOptions": {},
"targets": {
Expand Down Expand Up @@ -83,4 +83,4 @@
"hashPath": "system.spatial.5.7.0.nupkg.sha512"
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "92c4a7720ef26003e081a02c73de57101baa7210"
"signature": ""
},
"compilationOptions": {},
"targets": {
Expand Down Expand Up @@ -81,4 +81,4 @@
"hashPath": "system.spatial.5.7.0.nupkg.sha512"
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "da39a3ee5e6b4b0d3255bfef95601890afd80709"
"signature": ""
},
"compilationOptions": {},
"targets": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "da39a3ee5e6b4b0d3255bfef95601890afd80709"
"signature": ""
},
"compilationOptions": {},
"targets": {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "a813617eb7d3bb5d69fa51d48f64bc6455d8ca5f"
"signature": ""
},
"compilationOptions": {
"defines": [
Expand Down Expand Up @@ -2494,4 +2494,4 @@
"hashPath": "system.xml.xpath.xdocument.4.0.1.nupkg.sha512"
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "a813617eb7d3bb5d69fa51d48f64bc6455d8ca5f"
"signature": ""
},
"compilationOptions": {},
"targets": {
Expand Down Expand Up @@ -63,4 +63,4 @@
"hashPath": "system.runtime.serialization.primitives.4.1.1.nupkg.sha512"
}
}
}
}
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"runtimeTarget": {
"name": ".NETCoreApp,Version=v1.0",
"signature": "a813617eb7d3bb5d69fa51d48f64bc6455d8ca5f"
"signature": ""
},
"compilationOptions": {
"defines": [
Expand Down Expand Up @@ -2513,4 +2513,4 @@
"hashPath": "system.xml.xpath.xdocument.4.0.1.nupkg.sha512"
}
}
}
}
62 changes: 34 additions & 28 deletions src/Tasks/Microsoft.NET.Build.Tasks/DependencyContextBuilder.cs
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ namespace Microsoft.NET.Build.Tasks
{
internal class DependencyContextBuilder
{
private const string _runtimePackPrefix = "runtimepack.";

private readonly VersionFolderPathResolver _versionFolderPathResolver;
private readonly SingleProjectInfo _mainProjectInfo;
private readonly ProjectContext _projectContext;
Expand All @@ -34,6 +36,7 @@ internal class DependencyContextBuilder
private bool _includeMainProjectInDepsFile = true;
private HashSet<string> _usedLibraryNames;
private Dictionary<ReferenceInfo, string> _referenceLibraryNames;
private Dictionary<string, LockFileTargetLibrary> _compilationTargetLibraries;

public DependencyContextBuilder(SingleProjectInfo mainProjectInfo, ProjectContext projectContext, bool includeRuntimeFileVersions)
{
Expand All @@ -50,6 +53,12 @@ public DependencyContextBuilder(SingleProjectInfo mainProjectInfo, ProjectContex
// it needs to read the file versions
_packageResolver = NuGetPackageResolver.CreateResolver(projectContext.LockFile);
}

if (_projectContext.CompilationLockFileTarget != _projectContext.LockFileTarget)
{
_compilationTargetLibraries = _projectContext.CompilationLockFileTarget.Libraries
.ToDictionary(l => l.Name, StringComparer.OrdinalIgnoreCase);
}
}

/// <summary>
Expand Down Expand Up @@ -169,7 +178,7 @@ public DependencyContext Build()

var libraryLookup = new LockFileLookup(_projectContext.LockFile);

var runtimeSignature = GenerateRuntimeSignature(runtimeExports);
var runtimeSignature = string.Empty;

IEnumerable<RuntimeLibrary> runtimeLibraries = Enumerable.Empty<RuntimeLibrary>();
if (_includeMainProjectInDepsFile)
Expand Down Expand Up @@ -224,30 +233,6 @@ public DependencyContext Build()
new RuntimeFallbacks[] { });
}

private static string GenerateRuntimeSignature(IEnumerable<LockFileTargetLibrary> runtimeExports)
{
var sha1 = SHA1.Create();
var builder = new StringBuilder();
var packages = runtimeExports
.Where(libraryExport => libraryExport.IsPackage());
var separator = "|";
foreach (var libraryExport in packages)
{
builder.Append(libraryExport.Name);
builder.Append(separator);
builder.Append(libraryExport.Version.ToString());
builder.Append(separator);
}
var hash = sha1.ComputeHash(Encoding.UTF8.GetBytes(builder.ToString()));

builder.Clear();
foreach (var hashByte in hash)
{
builder.AppendFormat("{0:x2}", hashByte);
}
return builder.ToString();
}

private List<Dependency> GetProjectDependencies(
ProjectContext projectContext,
Dictionary<string, Dependency> dependencyLookup,
Expand Down Expand Up @@ -324,7 +309,7 @@ private RuntimeLibrary GetProjectRuntimeLibrary(
List<Dependency> dependencies = GetProjectDependencies(projectContext, dependencyLookup, includeCompilationLibraries);
foreach (var runtimePackGroup in _runtimePackAssets.GroupBy(asset => asset.PackageName + "/" + asset.PackageVersion))
{
dependencies.Add(new Dependency("runtimepack." + runtimePackGroup.First().PackageName, runtimePackGroup.First().PackageVersion));
dependencies.Add(new Dependency(_runtimePackPrefix + runtimePackGroup.First().PackageName, runtimePackGroup.First().PackageVersion));
}

return CreateRuntimeLibrary(
Expand Down Expand Up @@ -376,7 +361,7 @@ private IEnumerable<RuntimeLibrary> GetRuntimePackLibraries(IEnumerable<RuntimeP
};

return new RuntimeLibrary("runtimepack",
"runtimepack." + runtimePackAssetGroup.First().PackageName,
_runtimePackPrefix + runtimePackAssetGroup.First().PackageName,
runtimePackAssetGroup.First().PackageVersion,
hash: string.Empty,
runtimeAssemblyGroups,
Expand Down Expand Up @@ -479,7 +464,28 @@ private Library GetLibrary(
}
else
{
IEnumerable<string> assemblies = GetCompileTimeAssemblies(export, referenceProjectInfo);
IEnumerable<string> assemblies = Enumerable.Empty<string>();

// In some situations, the assets file will include compilation assets under the RID-specific
// target, but not under the RID-less target. The RID-less target is what is used for project
// compilation, so make sure we get those assets when writing the compile references to the assets
// file.
// This can happen when the runtime graph adds dependencies which don't have compile assets excluded.
// This was encountered with the 4.3.0 System.Security.Claims, System.Security.Principal.Windows, and
// System.Threading.Overlapped packages.
LockFileTargetLibrary exportWithCompileAssets;
if (_compilationTargetLibraries != null)
{
_compilationTargetLibraries.TryGetValue(export.Name, out exportWithCompileAssets);
}
else
{
exportWithCompileAssets = export;
}
if (exportWithCompileAssets != null)
{
assemblies = GetCompileTimeAssemblies(exportWithCompileAssets, referenceProjectInfo);
}

return new CompilationLibrary(
type.ToLowerInvariant(),
Expand Down
Loading