Performance macro usage tracking

[ChrisRackauckas]

Right now @inbounds and @muladd are used in the loops. @fastmath is not used because of JuliaLang/julia#22275, but it seems like because of JuliaLang/julia#18654 finished we can in the near future look at replacing @muladd usage by @fastmath (of course after the splat bugs are fixed).

The master issue for "long arithmetic line issues" is the following: JuliaLang/julia#22370 . It's the main blocker/constraint on design right now. This all could be fixed by JuliaLang/julia#22545?

[Armavica]

I have always been told to never enable -ffast-math in scientific programming, so I am a little worried about its introduction in this library. Not only this will probably change all the results in a compiler-dependent way, but it will make programs much more difficult to debug and we will also lose NaN propagation and error checking (like division by zero). Or is Julia's @fastmath safer than gcc's or llvm's?

[ChrisRackauckas]

Yes, it is less safe loses NaN propagation. There is a tradeoff. Maybe there is a good design that allows this to be a user's choice, like some kind of environment variable. We already have @tight_loop_macros setup so that way performance macros can be instantly applied throughout *DiffEq packages, so maybe we can use that infrastructure to make it user-overridable.

[ChrisRackauckas]

I was assured that Pkg3 would have a way to set "package options", so @tight_loop_macros will be used to allow users the choice of applying @fastmath and other things (@threads) globally to all inner loops.

[ChrisRackauckas]

Everything is being changed to broadcast, so this can be controlled by the user in the array type.