Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FileSystem access should go via IFileSystem (+refactoring) #11112

Merged
merged 17 commits into from
May 10, 2021

Conversation

vzarytovskii
Copy link
Member

@vzarytovskii vzarytovskii commented Feb 15, 2021

This is a work in progress change, will be in which all FS work should go via the IFileSystem.

I've decided to make changes in smaller batches, to minimize the area:

  • All FS access should go via IFileSystem.
  • ByteMemory reading should go thru IFileSystem instead of being in bytes.fs/ilread.fs
  • IFileSystem should be accessed via configs (TcConfig, il cenv, ILReaderOptions, ILWriterOptions, IMetadatReader, PEReader, etc).
  • Service should accept IFileSystem.

@runfoapp runfoapp bot mentioned this pull request Feb 23, 2021
@vzarytovskii vzarytovskii changed the title [WIP] Filesystem access should go via IFileSystem [WIP] Filesystem access should go via IFileSystem (+refactoring) Apr 27, 2021
@vzarytovskii
Copy link
Member Author

Oops, accidently got rid of some fsi files, going to fix it...

@vzarytovskii vzarytovskii marked this pull request as ready for review April 29, 2021 16:45
@@ -3990,27 +3990,153 @@ FSharp.Compiler.EditorServices.XmlDocable: Int32 line
FSharp.Compiler.EditorServices.XmlDocable: Microsoft.FSharp.Collections.FSharpList`1[System.String] get_paramNames()
FSharp.Compiler.EditorServices.XmlDocable: Microsoft.FSharp.Collections.FSharpList`1[System.String] paramNames
FSharp.Compiler.EditorServices.XmlDocable: System.String ToString()
FSharp.Compiler.IO.ByteArrayMemory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are these now public? They should be internal only.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since we expose ByteMemory type now via the IFileSystem interface, I was thinking to make ByteArrayMemory (and maybe other ones) public too, as default implementations, so FCS consumers can use it in their IFileSystem implementations. WDYT?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ByteMemory and friends should really be internal if possible.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vzarytovskii and I talked a bit, we will expose these types but will rename them with the FSharp prefix and will be given experimental attributes as we might make them internal again and/or change them as we continue to iterate on this.

…s - only use mmaped files to read big binary files (i.e. in the ilreader)
type IFileSystem =
abstract AssemblyLoader: IAssemblyLoader

abstract OpenFileForReadShim: filePath: string * ?useMemoryMappedFile: bool * ?shouldShadowCopy: bool -> ByteMemory
Copy link
Member Author

@vzarytovskii vzarytovskii Apr 30, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure whether useMemoryMappedFile needs to be in the signature or not since it is the implementation details.
But we somehow need to indicate if we want to use mmap or not (we only want to use it for big files, it does not make sense to use it for source files since it locks the file and we need to either reuse the map when accessing from another process or shadow copy all source files).

Copy link
Contributor

@TIHan TIHan May 4, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed this a bit today. I think it's fine that you can't open an already memory-mapped file if it's not shadow copied; we expect that.

We will iterate more on this when we need to, but otherwise it looks good.

@vzarytovskii vzarytovskii reopened this May 3, 2021
@vzarytovskii vzarytovskii reopened this May 3, 2021
Copy link
Contributor

@TIHan TIHan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really good work. I love that we are consolidating all of the file system related calls into one place. I know we will iterate more on this, but what we have here looks good.

One thing to note, in the future if we could not have as many formatting changes that would help the reviewing process.

@vzarytovskii
Copy link
Member Author

One thing to note, in the future if we could not have as many formatting changes that would help the reviewing process.

Yeah, it was already too late when I noticed that my editor did all those changes, sorry :(

@vzarytovskii vzarytovskii reopened this May 7, 2021
@vzarytovskii vzarytovskii changed the title [WIP] Filesystem access should go via IFileSystem (+refactoring) FileSystem access should go via IFileSystem (+refactoring) May 10, 2021
@vzarytovskii vzarytovskii merged commit 306e3c5 into dotnet:main May 10, 2021
@vzarytovskii vzarytovskii deleted the fs-shim branch May 10, 2021 17:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants