Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Streamlining and minor cleanup of code in the PrettyNaming module #578

Closed
wants to merge 1 commit into from

Conversation

jack-pappas
Copy link
Contributor

I've made some small changes to the PrettyNaming module to streamline it for performance. Specifically:

  • Compilation and decompilation of operators is now memoized for custom (non-built-in) operators to improve performance.
  • The IsInfixOperator function was creating (potentially) multiple lists each time it was called. I've changed the lists to arrays since they're never modified, and I've lifted these arrays outside of the function body so they're only created once and re-used.
  • Some repetitive if-then-else expressions have been changed to use match instead so they can be compiled into a switch opcode when appropriate.
  • A few places in the code were unnecessarily using the .Substring(...) operation; they have been changed to use non-allocating comparisons instead (e.g., .StartsWith(...)).
  • The IsValidPrefixOperatorUse, IsValidPrefixOperatorDefinitionName, and IsPrefixOperator functions would, in certain specific cases, allocate a closure. It doesn't appear that this is actually necessary though -- a small modification to the code eliminates the variable capture that causes the allocation, allowing the lambda to be lifted out of the functions.

I've done some rudimentary benchmarking, and this change appears to have no significant impact (better or worse) when I compile these changes into a new fsc-proto and use that to rebuild the FSharp.Compiler.dll. However, these changes do appear to improve compile times by a small but consistent amount when I use the fsc-proto to compile the various flavors of FSharp.Core. Based on the data produced by --times, these improvements appear to come from reducing the number of Gen0 and Gen1 collections during the [Parse inputs] and [Typecheck] compilation phases.

These modifications should not cause any externally-visible differences in behavior (i.e., from the perspective of callers in other parts of the compiler).

@msftclas
Copy link

Hi @jack-pappas, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.microsoft.com.

TTYL, MSBOT;

@jack-pappas
Copy link
Contributor Author

Do I need to sign the CLA again? I thought I'd already filed this one.

@forki
Copy link
Contributor

forki commented Aug 10, 2015

@jack-pappas there was a change of the CLA process a couple of months ago. the new one is much faster/easier to sign and doesn't need employers approval. did you sign this one?

let private decompileCustomOpName =
// Memoize this operation. Custom operators are typically used more than once
// so this avoids repeating decompilation.
let decompiledOperators = Dictionary<_,_> (System.StringComparer.Ordinal)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please put this as a private module-level definition and mark it with // +++ GLOBAL STATE

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason not to use ConcurentDictionary? I'd prefer that unless there's a proven case against it.

@dsyme
Copy link
Contributor

dsyme commented Aug 10, 2015

I've taken a close look and it all seems like reasonable cleanup. But I'd appreciate someone else checking the code closely too.

@dsyme
Copy link
Contributor

dsyme commented Aug 10, 2015

I've left some comments too.

@msftclas
Copy link

@jack-pappas, Thanks for signing the contribution license agreement so quickly! Actual humans will now validate the agreement and then evaluate the PR.

Thanks, MSBOT;

@@ -293,7 +417,7 @@ module internal Microsoft.FSharp.Compiler.PrettyNaming
res <- res && n.[i] >= '0' && n.[i] <= '9';
res

type NameArityPair = NameArityPair of string*int
type NameArityPair = NameArityPair of name:string * arity:int
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Named DU fields is a 3.1 feature, but our internal build uses a pre-3.1 compiler to bootstrap. So for now we can't do this.

jack-pappas added a commit to jack-pappas/visualfsharp that referenced this pull request Aug 10, 2015
@jack-pappas
Copy link
Contributor Author

@dsyme I've made some of your requested changes and updated this PR (I'll squash them before the PR is merged).

I wasn't able to make CompileOp and DecompileOp private to the module; when I did that and tried to compile, I got a number of errors:

"C:\visualfsharp\src\fsharp-proto-build.proj" (default target) (1) ->
"C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj" (Build target) (3) ->
(CoreCompile target) ->
  C:\visualfsharp\src\fsharp\ast.fs(1713,23): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ast.fs(1714,19): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ast.fs(1715,46): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ast.fs(1726,37): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(3684,74): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(3689,93): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(3692,95): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(3696,93): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(4611,36): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(4614,118): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\pars.fsy(4617,36): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(837,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(838,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(839,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(840,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(842,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(844,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(845,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(846,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(847,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(848,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(849,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(850,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(851,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TcGlobals.fs(852,110): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TastOps.fs(3024,26): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\NameResolution.fs(200,38): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\NameResolution.fs(202,36): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ConstraintSolver.fs(1172,100): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ConstraintSolver.fs(1174,97): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ConstraintSolver.fs(1183,36): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ConstraintSolver.fs(1228,157): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\ConstraintSolver.fs(1230,154): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(971,20): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(1289,151): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(1290,137): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(1291,124): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(1393,52): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(5741,55): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(7092,27): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(7104,83): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(7126,83): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\TypeChecker.fs(7537,69): error FS0039: The value or constructor 'CompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\CompileOps.fs(720,47): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\CompileOps.fs(722,49): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\CompileOps.fs(724,24): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\CompileOps.fs(733,46): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]
  C:\visualfsharp\src\fsharp\CompileOps.fs(736,42): error FS0039: The value or constructor 'DecompileOpName' is not defined [C:\visualfsharp\src\fsharp\FSharp.Compiler-proto\FSharp.Compiler-proto.fsproj]

    0 Warning(s)
    48 Error(s)

I tried modifying the code to use ConcurrentDictionary<_,_> but ran into a bug in the compiler's type inferencer; I'll open an issue for this, but basically there's no way to call the ConcurrentDictionary<'TKey, 'T>.GetOrAdd('TKey, System.Func<'TKey, 'T>) overload. So, we can either stick with the dictionary + lock (for now, or permanently) or I can implement this in another way using ConcurrentDictionary which will have the downside of being slightly less efficient (when called concurrently for the same key); either way is fine with me, just let me know what you prefer.

@jack-pappas
Copy link
Contributor Author

@latkin I've removed the named fields and updated the code.

if s.StartsWith("get_", System.StringComparison.Ordinal) ||
s.StartsWith("set_", System.StringComparison.Ordinal)
then Some (s.Substring(4, s.Length - 4))
else None
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we simplify this further, so there is only a single check for "get_" and "set_" prefixes?

    /// Try to chop "get_" or "set_" from a string
    let TryChopPropertyName (s: string) =
        // extract the logical name from any mangled name produced by MakeMemberDataAndMangledNameForMemberVal
        if s.Length <= 4 then None else
        let s = chopStringTo s '.'
        if s.StartsWith("get_", System.StringComparison.Ordinal) ||
           s.StartsWith("set_", System.StringComparison.Ordinal)
            then Some (s.Substring(4, s.Length - 4))
            else None

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was thinking about making this change when doing the cleanup. It's a good change, I'll make it now and update the PR.

@latkin
Copy link
Contributor

latkin commented Aug 11, 2015

Overall LGTM, just some minor suggestions on improving clarity

@latkin
Copy link
Contributor

latkin commented Aug 11, 2015

@dsyme do you have a style preference for this repo regarding .NET method calls: with/without parens? e.g. resizeArray.Add x vs resizeArray.Add(x). This PR favors the former.

@dsyme
Copy link
Contributor

dsyme commented Aug 11, 2015

@latkin There is no strict preference, but more recent code tends to use the former, so it is ok to move code in that direction.

@jack-pappas
Copy link
Contributor Author

@latkin I cleaned up the loop in the decompileCustomOpName function per your suggestion.

One other thing I noticed was that there seemed to be more GC's reported in the --times output when I switched that one bit of code to use the for .. in .. do syntax to iterate over the string (as compared to indexing into the string). Not sure why that is, so I've changed the code back for now.

I've squashed the commits so this PR can be merged whenever you're ready.

@latkin
Copy link
Contributor

latkin commented Aug 12, 2015

@jack-pappas Great! Would be interesting to find out what's up with the string loop, but no worries in the meantime. I'll try to get this merged soon.

System.Text.StringBuilder (maxPossibleOpCharCount)

// Start decompiling just after the operator prefix.
match decompile sb opNamePrefixLen with
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thought was to have this simply return string directly - instead of returning Some(sb.ToString()) just return sb.ToString(), instead of None return opName

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@latkin Good point. I've made the change and updated the PR.

Compilation and decompilation of operators is now memoized for custom
(non-built-in) operators to improve performance.
Lifted creation of lists (now arrays) in the IsInfixOperator function
so they aren't re-created (potentially) on each function call.
Simplification to the TryChopPropertyName function per @latkin's suggestion.
Optimized the recursive function which performs the decompilation within 'decompileCustomOpName' so it's tail recursive and avoids allocating substrings and some Option<_> instances.

// extract the logical name from any mangled name produced by MakeMemberDataAndMangledNameForMemberVal
if s.Length <= 4 then None else
let s = chopStringTo s '.'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies, my earlier suggestion to use this simplification was bogus. This now fails/causes crash for properties with . in them (e.g. backtick identifiers). I'll fix before merging...

@latkin
Copy link
Contributor

latkin commented Aug 12, 2015

Looks like you are actively updating this. Let me know when it's done so I can sync locally and re-run tests.

@jack-pappas
Copy link
Contributor Author

@latkin I'm finished making changes if you're going to fix that one piece of code before merging. Or, I can fix it and update the PR -- just let me know.

@latkin
Copy link
Contributor

latkin commented Aug 12, 2015

Got it, re-running now. I'll take care of the TryChopPropertyName thing, sorry about that.

latkin pushed a commit that referenced this pull request Aug 13, 2015
closes #578

commit 7e1c8410a3df45df62830f1dd29df18ac89b69a6
Author: latkin <latkin@microsoft.com>
Date:   Wed Aug 12 14:58:45 2015 -0700

    Fix logic in TryChopPropertyName

commit cf72133
Author: Jack Pappas <jack-pappas@users.noreply.github.com>
Date:   Sun Aug 9 15:54:28 2015 -0400

    Streamlining and minor cleanup of code in the PrettyNaming module.
    Compilation and decompilation of operators is now memoized for custom
    (non-built-in) operators to improve performance.
    Lifted creation of lists (now arrays) in the IsInfixOperator function
    so they aren't re-created (potentially) on each function call.
    Simplification to the TryChopPropertyName function per @latkin's suggestion.
    Optimized the recursive function which performs the decompilation within 'decompileCustomOpName' so it's tail recursive and avoids allocating substrings and some Option<_> instances.
@latkin
Copy link
Contributor

latkin commented Aug 13, 2015

Applied to OOB branch

@latkin latkin closed this Aug 13, 2015
@latkin latkin added the fixed label Aug 13, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants