Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix stack overflow on assembly resolution #3658

Merged
merged 2 commits into from
Sep 29, 2017

Conversation

KevinRansom
Copy link
Member

@KevinRansom KevinRansom commented Sep 28, 2017

Well .... this is embarrassing ... as I have said before our build is dependent on a very old build of the coreclr for testing.

We had a bug based on a badly implemented LoadContext (Sorry about that ... all my fault), that works on this very old coreclr but was fixed soon after ... only .... well we couldn't update at that time. On a fixed coreclr this old bad but working code becomes a stack overflow. Because I believed it only impacted fsi.exe, and there is a lot more I want to do for that tool I wasn't in any rush to fix it --- bad descision because it also impacts TypeProviders which I forgot about.

Anyway ... because we still need to use the old coreclr for a bit longer ... only for testing of course ... the fix here tests to see if the mscorlib is of a very specific version ... the one we use for testing. We do the old thing ... if instead it is any other version in existence we do the right thing.

Anyway please don't hate on me ... I am doing plenty of that myself. I wish that I was smart enough to come up with a good excuse that blamed others but this one was all on me ...

Fixes: #3658

@KevinRansom
Copy link
Member Author

@dsyme @xenocons , do either of you have a repro to test. This fixes the issue that I know of but that one doesn't use a TP.

Thanks

Kevin

@ghost
Copy link

ghost commented Sep 28, 2017

Hi Kevin, We all have bugs :)

I have difficulties to compile fsharp on netcore 2 with debian at the moment, however as a repo:

dotnet /usr/share/dotnet/sdk/2.0.0/FSharp/fsc.exe --noframework -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/mscorlib.dll -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/System.Runtime.dll -r:/usr/share/dotnet/sdk/NuGetFallbackFolder/netstandard.library/2.0.0/build/netstandard2.0/ref/netstandard.dll -r:/home/xenocons/.nuget/packages/sqlprovider/1.1.11/lib/FSharp.Data.SqlProvider.dll test.fs

test.fs can be anything, and the bug occurs with the inclusion of sqlprovider (you will need to modify the path in the above command). For anyone else reading this links to #3654


let isOldCorlib =
let attributes = typeof<RuntimeTypeHandle>.GetTypeInfo().Assembly.GetCustomAttributes<AssemblyFileVersionAttribute>() |> Seq.toArray
if attributes.Length = 0 then false else attributes.[0].Version = "4.6.24410.01"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let attributes = typeof.GetTypeInfo().Assembly.GetCustomAttributes()
match Seq.tryHead attributes with
| Some a -> a.Version = "4.6.24410.01"
| _ -> false

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also I assume it should be moved down next to usage

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and the the bool you can remove the temporary bool completely ....

@dsyme
Copy link
Contributor

dsyme commented Sep 28, 2017

Great to have this fixed. If it unblocks development of type providers for .NET Core I'll be very very happy

@KevinRansom
Copy link
Member Author

Okay this addresses the stack overflow report. However it moves the cheese down the line a bit:

There are two further issues from this:

  1. when building f# with the dotnet/cli f# somehow ... I don't know how picks the dotnet/cli from the path.
c:\temp\repro>c:\microsoft\visualfsharp\Tools\dotnet20\dotnet.exe build /v:normal

   ... a ton of logging elided

         C:\Program Files\dotnet\sdk\2.0.2-vspre-006949\FSharp\RunFsc.cmd -o:obj\Debug\netcoreapp2.0\repro.dll
      
  1. ProvidedType definition doesn't implement IReflect.
C:\microsoft\visualfsharp\tests\testbin\release\coreclr\FSC>C:\microsoft\visualfsharp\Tools\dotnet20\dotnet C:\microsoft\visualfsharp\Tools\dotnet20\sdk\2.0.0-preview2-006502\FSharp\fsc.exe @c:\temp\repro\repro.rsp
Microsoft (R) F# Compiler version 4.1
Copyright (c) Microsoft Corporation. All Rights Reserved.

error FS3033: The type provider 'FSharp.Data.Sql.SqlTypeProvider' reported an error: Unable to cast object of type 'ProviderImplementation.ProvidedTypes.ProvidedTypeDefinition' to type 'System.Reflection.IReflectableType'.

error FS3020: One or more errors seen during provided type setup

@KevinRansom
Copy link
Member Author

  1. is a dotnet sdk issue not ours :-)

@KevinRansom
Copy link
Member Author

@dotnet-bot test Windows_NT Release_ci_part2 Build please

@ghost
Copy link

ghost commented Sep 29, 2017

@KevinRansom could you confirm if the point 2. that you raised above would be remediated by implementing IReflectableType on https://github.com/fsprojects/FSharp.TypeProviders.StarterPack ?

Edit: Possibly related #1496

@KevinRansom
Copy link
Member Author

@xenocons

I have been looking at it and am kind of confused. It is possible to GetTypeinfo() from a Type despite Type not implementing IReflectableType. So I need to understand why the runtime is failing to do it.

On CoreClr type doesn't implement IReclectableType.

CoreClr type

image

Anyway ... working on it ...

@dsyme
Copy link
Contributor

dsyme commented Oct 6, 2017

I’m trying to work out this business with IReflectableType when implementing F# type providers and running them within the .NET Core hosted compiler

Basically, we need to implement IReflectedType because FSharp.Core uses GetTypeInfo (and possibly for other reasons TBD). But we can’t implement IReflectedType within the .NET Standard or .NET Framework programming models

What’s the right way to go here?

  1. Is it expected that it be possible to implement TypeInfo within the .NET Standard programming model? Is this an oversight or by design?

  2. Should we be avoiding the use of GetTypeInfo in FSharp.Core.dll? If so we will need to quickly prioritize pushing out a new FSharp.Core suitable for this.

*Observation: We can’t implement IReflectedType

The central purpose of the Type Provider SDK is to provide an implementation of System.Type:

type ProvidedTypeDefinition(…) =
    inherit Type()
    …

However it seems this means we must also implement IReflectableType (See below for why)

abstract GetTypeInfo : unit -> TypeInfo

But AFAICS there is no public or protected way to construct a TypeInfo in either the .NET Standard 2.0 or .NET Framework 4.6.1. programming model. Trying this:

type ProvidedTypeDefinition(…) =
    inherit TypeInfo()

gives

C:\GitHub\dsyme\FSharp.TypeProviders.StarterPack\src\ProvidedTypes.fs(1973,5): error FS0509: Method or object constructor 'TypeInfo' not found 
[C:\GitHub\dsyme\FSharp.TypeProviders.StarterPack\src\FSharp.TypeProviders.SDK.fsproj]

Observation: We need to implement IReflectedType because FSharp.Core uses it

It is possible we could get away with just not implementing IReflectedType, or providing a dummy implementation of it. However, it turns out FSharp.Core uses it, e.g.

Microsoft.FSharp.Core.ReflectionAdapters.Type.GetCustomAttributes(Type this, Type attrTy, Boolean inherits)

Thanks
Don

@KevinRansom
Copy link
Member Author

@dsyme we can't implement IReflectableType because TypeInfo does not have a public constructor. I think this may be a reference assemblies thing. But I am still worrying about p2p refs right now.

@KevinRansom
Copy link
Member Author

KevinRansom commented Oct 6, 2017

@dsyme, @brettfo, @cartermp, @Pilchie

I had another chat with Immo. This is clearly a bug in the framework and it is unclear how it can be fixed. Type extensibility by deriving from Type is incompatible with reflection using TypeInfo.

  • The most correct fix would be to make the TypeInfo() constructor protected or public, allowing us to implement a TypeInfo() which would then delegate to the generated type. However, there exists no dotnet runtime anywhere with that public, so it would be somewhat limited in terms of reach.

  • What we will do is this ... build the compiler for netcoreapp2.0 and ensuring that we don't use GetTypeInfo(). This will mean that either ...
    we enable build for both netcoreapp2.0 and netcoreapp1.1 or we stop shipping with dotnet/cli 1.1.*

Given that VS now has a dotnet core 2.0 install, I doubt the dotnet core 1.0 release matters much to us now.

Obviously all of the usual Kevin is embarrassed about this excuses spring to the fore here. I don't believe we use GetTypeInfo in FSharp.Core so I think we can stick with a NetStandard 1.6 FSharp.Core.dll.

Nope I was wrong we have to not use GetTypeInfo on generated types and there are a couple of places in FSharp.Core.dll where we use GetTypeInfo, especially in quotations so we may need a netstandard 2.0 fsharp.core.dll too.

KevinRansom added a commit that referenced this pull request Oct 11, 2017
* Generate source for .resx files on build. (#3607)

* add build task to generate *.fs from *.resx files

* generate source for embedded resources in tests

* generate source for embedded resources in FSharp.Editor

* generate source for embedded resources in FSharp.LanguageService

* generate source for embedded resources in FSharp.ProjectSystem.FSharp

* generate source for embedded resources in FSharp.VS.FSI

* don't generate non-string resources when <=netstandard1.6

* update baseline error message for tests

The error output should be the exception message, not the exception type.

* perform up-to-date check before generating *.fs from *.resx

* remove non-idiomatic fold for an array comprehension

* correct newline replacement

* output more friendly error message

* throw if boolean value isn't explicitly `true` or `false`

* only generate object resource code on non `netstandard1.*` and `netcoreapp1.*` platforms

* ensure FSharp.Core specifies a target framework for resource generaton

* rename attributes to be non-ambiguous and properly include them

* fix order of file items in FSharp.Core

* Fix build.cmd for certain always-shown errors (like "unable to find registry key"), improve finding of VS2017 tools, fix DEVGUIDE.md (#3635)

* Fix build.com displaying the following error on each run "ERROR: The system was unable to find the specified registry key or value."

* Remove warning about reg.exe errors introduced in #3614 (in commit b548bd7, but unrelated to that fix), it is no longer necessary.

* Fix #3638, VS2017 Preview installation was not found when VS2017RTM is not installed. Remove comment introduced in #3614 (through commit 966bd7f)

* Fixing JaroWinkler tests with InvariantCulture and fixing async tests by removing Debugger.Break() (#3627)

* Fixing JaroWinkler tests to use InvariantCulture for number-to-string

* Fixing the crashing of test runners because of a Debugger.Break() in a test

* update to System.Collections.Immutable 1.3.1 (#3641)

* update to System.Collections.Immutable 1.3.1

* fixes

* fix assembly reference

* [WIP] Adds optimized emit for int64 constants (#3620)

* Adds optimized emit for int64 constants.

* Adds comment linking to the changing PR.

Thanks for this PR.

Kevin

* fix assembly reference (#3646)

* Remove a few more build warnings (#3647)

* fix assembly reference

* remove more build warnings

* fix build

* move BuildFromSource projects to their own directory

* Adds tests for emitted IL for new Int64 constants. (#3650)

* Enable FS as prefix and ignore invalid values for warnings (#3631)

* enable fs as prefix and ignore invalid values for warnings + tests

* Allow #pragma to validate warnings

* do it right

* use ordinal compare

* In both places

* Add fs prefix to warnaserror

* Fixup tests

* Fix stack overflow on assembly resolution (#3658)

* Fix stack overflow on tp assembly resolution

* Feedback

* Add impl files to file check results (#3659)

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project

* add ImplementationFiles to FSharpCheckFileResults

* make FSharpImplementationFileContents ctor internal

* throw if ImplementationFiles is called having keepAssemblyContents flag set to false

* add a test

* spelling and cosmetics

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd (#3672)

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* Remove ambiguous an irrelevant instruction, improved help and instructions

* Fix a scenario where the return code wasn't nonzero for error conditions, fixes not creating backup dir when not backing up

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project (#3657)

* bump FCS version (#3676)

* bump version

* Update RELEASE_NOTES.md

* Parsing improvements: no reactor, add parsing options, error severity options (#3601)

* Parse without reactor, add parsing options, error severity options

* Revert parsing APIs (fallback to the new ones), fix VFT projects

* Cache parse results after type check

* Add impl files to file check results (#3659)

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project

* add ImplementationFiles to FSharpCheckFileResults

* make FSharpImplementationFileContents ctor internal

* throw if ImplementationFiles is called having keepAssemblyContents flag set to false

* add a test

* spelling and cosmetics

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd (#3672)

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* Remove ambiguous an irrelevant instruction, improved help and instructions

* Fix a scenario where the return code wasn't nonzero for error conditions, fixes not creating backup dir when not backing up

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project (#3657)

* bump FCS version (#3676)

* bump version

* Update RELEASE_NOTES.md

* updates to make tests pass

* restore old behaviour of CheckFileInProjectAllowingStaleCachedResults (builder had been created by ParseFileInProject)

* restore use of CheckFileInProjectAllowingStaleCachedResults

* deprecate test relying on whacky behaviour of deprecated GetCheckResultsBeforeFileInProjectEvenIfStale

* Use ParseFile and FSharpParsingOptions instead of ParseFileInProject

* prepare FCS release with this feature

* whitespace cleanup (#3682)

* whitespace and docs (#3684)

* Preserve XML docs for in-memory project references (#3683)

* fix xmldocs for in-memory project references

* add test

* fix tests

* whitespace and comments (#3686)

* fix assembly reference

* whitespace and comments

* whitespace and comments

* whitespace and comments

* cherry pick two PRs from FCS (#3687)

* fix assembly reference

* remove line endings from all *.nuspec files

* ProjectCracker returns *.fsi files in FSharpProjectOptions.SourceFiles array (in addition to *.fs files, in right order)

* ProjectCracker raises exception if ProjectCrackerTool returns non null ProjectCrackerOptions.Error (new field)

* fix build on linux

* fix a test

* slashes

* revert slashes

* Update FSharp.Compiler.Service.ProjectCracker.nuspec

* try to fix travis

* try to fix travis

* list dependencies

* no obsolete pdb in nuget

* list dependencies

* cherry pick of fsharp/fsharp change

* bump FCS version number (#3688)

* Update FSharp.Compiler.Service.MSBuild.v12.nuspec

* fix FCS nuget on windows

* fix-resource (#3690)

* Bump FSharp.Compiler.Tools to 4.1.27 and align mono build files (#3693)

* ri change from fsharp

* fix test

* bump FSC tools to 4.1.27

* remove fsharp.core from Mono GAC

* align mono directory

* fix typo

* install back versions with Mono

* fix typo

* update FCS doc generation (#3694)

* update DEVGUIDE to add addiitional steps before running build (#3725)

* Split templates out into a seperate vsix (#3720)

* merge error

* Merge issues
@KevinRansom KevinRansom deleted the issue3654 branch October 30, 2017 18:27
nosami pushed a commit to xamarin/visualfsharp that referenced this pull request Jan 26, 2022
* Generate source for .resx files on build. (dotnet#3607)

* add build task to generate *.fs from *.resx files

* generate source for embedded resources in tests

* generate source for embedded resources in FSharp.Editor

* generate source for embedded resources in FSharp.LanguageService

* generate source for embedded resources in FSharp.ProjectSystem.FSharp

* generate source for embedded resources in FSharp.VS.FSI

* don't generate non-string resources when <=netstandard1.6

* update baseline error message for tests

The error output should be the exception message, not the exception type.

* perform up-to-date check before generating *.fs from *.resx

* remove non-idiomatic fold for an array comprehension

* correct newline replacement

* output more friendly error message

* throw if boolean value isn't explicitly `true` or `false`

* only generate object resource code on non `netstandard1.*` and `netcoreapp1.*` platforms

* ensure FSharp.Core specifies a target framework for resource generaton

* rename attributes to be non-ambiguous and properly include them

* fix order of file items in FSharp.Core

* Fix build.cmd for certain always-shown errors (like "unable to find registry key"), improve finding of VS2017 tools, fix DEVGUIDE.md (dotnet#3635)

* Fix build.com displaying the following error on each run "ERROR: The system was unable to find the specified registry key or value."

* Remove warning about reg.exe errors introduced in dotnet#3614 (in commit b548bd7, but unrelated to that fix), it is no longer necessary.

* Fix dotnet#3638, VS2017 Preview installation was not found when VS2017RTM is not installed. Remove comment introduced in dotnet#3614 (through commit 966bd7f)

* Fixing JaroWinkler tests with InvariantCulture and fixing async tests by removing Debugger.Break() (dotnet#3627)

* Fixing JaroWinkler tests to use InvariantCulture for number-to-string

* Fixing the crashing of test runners because of a Debugger.Break() in a test

* update to System.Collections.Immutable 1.3.1 (dotnet#3641)

* update to System.Collections.Immutable 1.3.1

* fixes

* fix assembly reference

* [WIP] Adds optimized emit for int64 constants (dotnet#3620)

* Adds optimized emit for int64 constants.

* Adds comment linking to the changing PR.

Thanks for this PR.

Kevin

* fix assembly reference (dotnet#3646)

* Remove a few more build warnings (dotnet#3647)

* fix assembly reference

* remove more build warnings

* fix build

* move BuildFromSource projects to their own directory

* Adds tests for emitted IL for new Int64 constants. (dotnet#3650)

* Enable FS as prefix and ignore invalid values for warnings (dotnet#3631)

* enable fs as prefix and ignore invalid values for warnings + tests

* Allow #pragma to validate warnings

* do it right

* use ordinal compare

* In both places

* Add fs prefix to warnaserror

* Fixup tests

* Fix stack overflow on assembly resolution (dotnet#3658)

* Fix stack overflow on tp assembly resolution

* Feedback

* Add impl files to file check results (dotnet#3659)

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project

* add ImplementationFiles to FSharpCheckFileResults

* make FSharpImplementationFileContents ctor internal

* throw if ImplementationFiles is called having keepAssemblyContents flag set to false

* add a test

* spelling and cosmetics

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd (dotnet#3672)

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* Remove ambiguous an irrelevant instruction, improved help and instructions

* Fix a scenario where the return code wasn't nonzero for error conditions, fixes not creating backup dir when not backing up

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project (dotnet#3657)

* bump FCS version (dotnet#3676)

* bump version

* Update RELEASE_NOTES.md

* Parsing improvements: no reactor, add parsing options, error severity options (dotnet#3601)

* Parse without reactor, add parsing options, error severity options

* Revert parsing APIs (fallback to the new ones), fix VFT projects

* Cache parse results after type check

* Add impl files to file check results (dotnet#3659)

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project

* add ImplementationFiles to FSharpCheckFileResults

* make FSharpImplementationFileContents ctor internal

* throw if ImplementationFiles is called having keepAssemblyContents flag set to false

* add a test

* spelling and cosmetics

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd (dotnet#3672)

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* This adds backup, restore, coloration and many more checks to the update-vsintegration.cmd

* Remove ambiguous an irrelevant instruction, improved help and instructions

* Fix a scenario where the return code wasn't nonzero for error conditions, fixes not creating backup dir when not backing up

* add LanguageServiceProfiling project to internals visible to list of FSharp.Compiler.Private project (dotnet#3657)

* bump FCS version (dotnet#3676)

* bump version

* Update RELEASE_NOTES.md

* updates to make tests pass

* restore old behaviour of CheckFileInProjectAllowingStaleCachedResults (builder had been created by ParseFileInProject)

* restore use of CheckFileInProjectAllowingStaleCachedResults

* deprecate test relying on whacky behaviour of deprecated GetCheckResultsBeforeFileInProjectEvenIfStale

* Use ParseFile and FSharpParsingOptions instead of ParseFileInProject

* prepare FCS release with this feature

* whitespace cleanup (dotnet#3682)

* whitespace and docs (dotnet#3684)

* Preserve XML docs for in-memory project references (dotnet#3683)

* fix xmldocs for in-memory project references

* add test

* fix tests

* whitespace and comments (dotnet#3686)

* fix assembly reference

* whitespace and comments

* whitespace and comments

* whitespace and comments

* cherry pick two PRs from FCS (dotnet#3687)

* fix assembly reference

* remove line endings from all *.nuspec files

* ProjectCracker returns *.fsi files in FSharpProjectOptions.SourceFiles array (in addition to *.fs files, in right order)

* ProjectCracker raises exception if ProjectCrackerTool returns non null ProjectCrackerOptions.Error (new field)

* fix build on linux

* fix a test

* slashes

* revert slashes

* Update FSharp.Compiler.Service.ProjectCracker.nuspec

* try to fix travis

* try to fix travis

* list dependencies

* no obsolete pdb in nuget

* list dependencies

* cherry pick of fsharp/fsharp change

* bump FCS version number (dotnet#3688)

* Update FSharp.Compiler.Service.MSBuild.v12.nuspec

* fix FCS nuget on windows

* fix-resource (dotnet#3690)

* Bump FSharp.Compiler.Tools to 4.1.27 and align mono build files (dotnet#3693)

* ri change from fsharp

* fix test

* bump FSC tools to 4.1.27

* remove fsharp.core from Mono GAC

* align mono directory

* fix typo

* install back versions with Mono

* fix typo

* update FCS doc generation (dotnet#3694)

* update DEVGUIDE to add addiitional steps before running build (dotnet#3725)

* Split templates out into a seperate vsix (dotnet#3720)

* merge error

* Merge issues
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants