Improve the "scan-all" behavior of all project systems

[filipw]

As reported on chat today - the SearchOption.AllDirectories is a bit too much and can cause OmniSharp to load a long time.

[filipw]

Actually, this is not just scripting - MSBuild project system does the same thing if it doesn't have SLN

omnisharp-roslyn/src/OmniSharp.MSBuild/MSBuildProjectSystem.cs

Line 134 in beb61b8

return Directory.GetFiles(_environment.TargetDirectory, "*.csproj", SearchOption.AllDirectories);

project.json project system tries to work around it by having a depth limit of 5

omnisharp-roslyn/src/OmniSharp.DotNet/Projects/ProjectSearcher.cs

Line 12 in a4dced2

return Search(directory, maxDepth: 5);

At the very least, in all cases, we could exclude the common nasty folder names like node_modules, bower_components or .-prefixed folders etc.

[DustinCampbell]

I think there are a couple of things to do here.

1. Allow the user to be able to specify folders in an omnisharp.json to exclude from the project search.
2. Don't wait until projects are synchronously loaded before OmniSharp is finished loading. The project search should continue after OmniSharp has finished its initial configuration.

FWIW, I actually dislike the depth limit because it's hard to tell why a project didn't load if it happens to be lower in the directory tree.

[bjorkstromm]

As reported here, Cake Project System also suffers from this.

Time to think of some default exclude patterns and create a helper for enumerating files? Maybe we can start from default .gitignore` that Github uses. Additionally, make the exclude patterns configurable via OmniSharp configuration system.