-
Notifications
You must be signed in to change notification settings - Fork 345
Close on the design of native-size numbers #1471
Comments
The biggest issue with this is you cannot undo C# mapping of IL e. g. For further consideration see https://github.com/DotNetCross/NativeInts for how types could be implemented in IL. |
I feel that this is skimmed over a bit too quickly; can we expand on what the specific incompatibilities are? Given that most/all existing interop mappings will just choose |
I see the value here. Though i question if it's necessary to have a specific language keyword for this. I think it would be fine if someone had to write IntN. And, if they really wanted to write 'nint', they could just say: "using nint = System.IntN". |
I'm not sure about the general usefulness of Other discussions on this topic suggested putting this into a separate ("Apple-specific") library, since it is only useful for their unusual usage of a float typedef in public function signatures. Is that not an acceptable solution? |
to emit math with The proposal is a bit too vague on that. Do we have an Basically, what is the expected IL for the following: nint x = 21;
x = x + x; |
I am interested in more details about compatibility concerns. Since the goal here is to allow efficient math in terms of native integers, just adding a bunch of compiler built-in checked/unchecked operators like Mapping to If the only concern is overload resolution between, say Basically, I see many advantages in mapping I do understand that for |
An alternative proposal for
(similarly The end result will be: // Metadata: IntPtr M1(IntPtr[] arr, IntPtr pos, int offset)
nint M1(nint[] arr, nint pos, int offset)
{
// can add nint and int
var i = pos + offset;
// can use nint as an array index
return arr[i];
}
// must cast 123 to nint explicitly for compat reasons.
M1(someArray, (nint)123, 45) The advantage here is that such |
And to answer my question above - there are indeed problems with mapping Turns out Sigh... |
@KrzysztofCwalina - the example of I assume that is how the underlying value is fetched when operating on the struct. |
But IntPtr.Zero uses static readonly field. Is there a reason for using get-only properties instead? |
Doesn't a static property like that if backed with a static readonly (which it should in this case) get basically collapsed to similar to a const by the hit anyway. If it's a property it allows for future behaviour change with no breaking of the api. |
Zero is constant; min and max value depend on native size which isn't known until runtime. (Though min value is a const for UIntN) |
@nietras, could you clarify the mapping issue? We map IntPtr to native int. We can map IntN to native int too. We don't map native int to BCL types except when here is metadata telling us what mapping is desired. Am I missing something? cc: @jkotas @mellinoe, there is a sentence alluding to the incompatibilites, but I will expand it. Thanks. @CyrusNajmabadi, without the alias, the types will forever feel like 2nd class citizens. But you are right that it's not absolutelly necessary, hence pri 2. @mellinoe, we considered doing what you suggested with GCFloat, for the reasons you listed. But it seems like the cost of doing proper language support is not that high (once we do this for ints), the type is very important to Xamarin, and it future proofs us in the case the type becomes more widely used. cc: @migueldeicaza @VSadov, the conversions to IntPtr/UIntPtr are real overloaded operators. See the asterix in the table. These will be used to unwrap. @VSadov, there are conversions UIntN->UINtPtr in the table. Admitably in an awkward place: see the second table. BTW, this is where the asterix operators are, which are APIs (overloaded operators) for the unwrapping scenarios. @zippec, as @benaadams said, some of the min/max values are not known till runtime. The others that can (e.g. UIntN.Zero) are properties for consistency with these that cannot. But we should consider making these consts. @jkotas? |
@mellinoe, I would very much rather see the support added to However, @jaredpar raised a very good point offline that the forced 'checked' behavior for some of the existing code (constructors, and some of the conversions, and operators) makes it very difficult since that means we would either have to take a breaking change to remove the forced 'checked' behavior or we would need to have inconsistent behavior when dealing with |
That being said, I think it would still be useful to finish fleshing out Regardless of whether the type was designed specifically for pointers (and I'm not sure I agree with this, since the remarks section and CLI spec both indicate otherwise), pointer arithmetic is a very valid and useful scenario which should be supported on the IntPtr type. |
Agreed IntPtr.Add (mypointer, offset) is ugly and hoop jumping. |
I agree completely with this, with some considerations below.
@KrzysztofCwalina Or perhaps I am missing something but given the following C# code: public struct nint
{
public IntPtr Value;
public nint(IntPtr value)
{
Value = value;
}
.... This is compiled into:
Notice how This works the other way too, of course, which can be seen in e.g. https://github.com/DotNetCross/NativeInts and of course the I believe we should simple accept that Anyway, the naming with So I do not understand the "metadata" part of the question, perhaps this is where I am missing something? 😕 |
@tannergooding - However the fact of adding operators to those types a while ago has essentially closed the door to implementing the math support as compiler intrinsics now - due to compatibility concerns. |
I understand that reasoning, but I'm skeptical that we'll ever see other libraries have such a pattern. It seems extremely unusual to me, and I don't think I've encountered another library doing something like it. I'm also not aware of other languages or standard libraries that have this kind of "native float" type, whereas native integers and integer pointers are universal and standard. I'd probably feel comfortable with more examples of this pattern being used. Do we know of any others than Related to |
How strongly do people feel about the naming of these? While I can see why things like
I'm not aware of the Why not just call them what they are? |
@tannergooding Couldn't we just add support for specifying |
The design explicitly calls for both |
@jaredpar, its a bit unprecedented (at least to the C# compiler, but not to compilers in general), but would it be crazy to have a compiler switch that says to ignore the existing operators defined by the framework and to emit pure add/add.ovf instructions instead? I believe that would let existing code continue to work and let new consumers, who explicitly want checked/unchecked behavior to opt-in. My thinking is that, overall, it may be worthwhile since we have 17 years worth of existing APIs/interop code (including in the framework itself) where we exclusively use IntPtr/UIntPtr. |
Imagine that command line option exists. How can I sanely review code which uses IntPtr now? It's impossible to understand the semantics unless I understand every single compilation in which it occurs. That is more of a problem now with the prevalence of shared projects where code is commonly used in multiple compilations. Astute readers may note: wait didn't @jaredpar just argue specifically against having a /checked command line option? Yes I did. That option makes it impossible to correctly interpret virtually any arithemetic operation in your source code. It's part of the reason it has so little usage.
Disagree strongly. A command line switch which changes the semantic behavior of the compiler is a bad feature. It makes it impossible to read code and understand what it does. |
I understand your viewpoint, but am not completely in agreement here. A large number of compilers have just such behavior for math (especially with regards to floating-point arithmetic -- /fp:fast for example). |
And it's a nightmare to manage. Your writing a precision sensitive finance library and a mistake is made in compilation, tracking that back is nigh on impossible. The nice thing about .net today is if I write a Monte Carlo and run it on an old çomputer in linux or on a Mac or whatever the numbers will match (with correct seeding). I also think this is why that native float is bad to have outside a specialized graphics lib. The numerical ramifications of flipping from a float to a double can be huge. |
Sure. And my feedback still applies. The presence of such switches make it harder to interpret what code is doing. It's entirely relying on the decision of an ambient authority. The decision which is often difficult to determine (saying this from experience trying to figure out how a series of makefiles translate into a final command line for the compiler). It also makes the code significantly less portable. Because the code and the ambient floating point option must be carried together.
I dislike |
@tannergooding code analyser as an entry barrier to writing simple high-level code?? No jokes, if you need to understand LOB space, best way is to spend 17 years doing it. Or ask somebody who did. Same goes for deserializing 64-bit values into 32-bit runtime, just ask someone competent for an explanation. |
@mihailik what's your problem? There is no reasonable excuse for acting this impolite, even if you personally fear this small addition to the C# language or just hate the world. I'm 100% sure most people in this discussion are competent programmers and the C# language design team has some very talented people thinking about these kinds of language changes. The feature is useful and the principal downside is additional complexity. So please leave your ad hominem arguments and unsubstantiated fear out of this discussion. |
@pentp not sure what problems you're talking about, or what hate/fear. Can you stick to the topic? Please. Marshalling of platform-dependent pointer-sized types is fundamentally unsafe. That is, extracting 64-bit value from stored location into runtime 32-bit location leads to unavoidable data corruption. There's no shame in missing this trivial fact, we all have different levels of competency and I for once am happy to spell such things out, as I did earlier. Same way I have spelled out risks of this feature to the wider community. @pentp please do not bring personal feelings into this, it's about material measurable risks to large stable and expensive codebases out there. As an active member of C# community for decades, and a long-term collaborator with Microsoft on CLR platform and other projects, I expect to see better moderation and more professional discussion driven by competent seasoned staff. When reasonable concerns are raised, they should be seriously taken into account and not brushed off. Treatment like 'if you fear for your code' or patronising contempt to legitimate feedback is very counterproductive to the ecosystem. I understand the proponents of this feature acting like this without Microsoft's mandate, but I expect the vendor and sponsor to put more effort in setting the rational, measured, evidence-based tone. Features are judged not on whether they are feared or desired, but on their merit. And showing how exactly a change makes a visible measurable improvement. Let's all remember that. |
Seconded.
And for those reasons, no one does this. Having |
Agreed on the let's avoid personal insults. I will tell you this, I have a lot of experience with the corporate .net world. Unsafe is almost never used, Intptr sounds dangerous to as does interop. When any of those are needed usually "specialists take care of it" but something that is a keyword in the language right along with int and unit... well I will just give that a shot. My problem here is you are proving a foot gun and making it seem very normal to carry it around d in your pocket. I would like to see some barrier to entry. Also I will say it again, just in case repetition works there is no good justification for native floats that I have seen other than a soon to be defunct Apple api. |
I do see a conceptual difference between IntPtr and native-sized integers (or floats). You would use the pointers to point to something in the memory and native-sized numeric types to do computations. You pass things to the underlying OS in IntPtrs but let the CPU do arithmetics in nints or nfloats. That also keeps IntPtr on the "unsafe" side and native-sized numerics on safe side. In this view, native floats would seem to be natural for doing numerical computations (e.g. graphics, geometry, physics etc.). It would even make sense to have nint/nfloat 64-bit on 64-bit CPU when the application itself is 32-bit. That said, the motivation above seem to be to improve the situation with pointer arithmetics, and in that case I am not entirely convinced introducing a new built-in type is worth saving the keystrokes. |
@jnm2 > And for those reasons, no one does this. The whole point of "LOB lobby" here is to keep these unsafe ideas out of the language proper:
|
@mihailik Those are not unsafe ideas. @tannergooding is making a reasonable point. It doesn't matter whether you use |
@jnm2 you've missed a tiny insignificant detail man ;-) You cannot deserialize 64-bit value on 32-bit platform. |
@mihailik Of course not, which is why you use whatever makes the most sense in memory to hold the size of values you are planning for. I'm really not sure what the problem is. If you need values 2^31 or greater, use |
@miloush, the purpose of extending |
Forgetting nint for a second. There is no such thing as a native sized float. Most fpu on cpus are fixed. As said before the fpu on the 8086 was 80bit internally ... having it native sized has one application I have ever heard of and that is the Apple API. I would love to hear of others |
@mihailik, you also cannot deserialize an array that has 20 elements into a container that can only hold 10 elements. You are completely correct that you shouldn't be serializing This doesn't stop badly designed LOB apps (or code in general) from being written to do as such and neither will it prevent users from doing as such just by throwing it behind some switch. The purpose of this proposal is to fill a very specific (and very much needed) scenario for Interop code. Having |
@jnm2 serializing and then deserializing a value of platform-specific type inherently leads to data corruption. Please reach me offline (on gmail) for further discussion, to avoid messing up the conversation. |
@Drawaes, correct. The only known use today (to my knowledge) is for better interop with Mac/iOS. It is also different from It does fall into a similar discussion in that it is a needed interop type. However, due to their being no underlying type it comes down to only being able to be implemented via a |
@tannergooding why don't you share that "very specific (and very much needed) scenario" and we avoid unnecessary hypothetising? |
@mihailik, I've listed it multiple times. Ease of interop with the underlying platform APIs (especially for Android, iOS, and Mac). As you've stated:
In the same vein, you cannot deserialize a 128-bit value on a 64-bit platform. Because of this condition, you cannot just create managed APIs (which wrap the platform APIs) that use anything other than Even under the argument that 64-bit is all we will ever need, that still leaves all of the 32-bit devices where code that wraps For better or worse, we've gotten by so far due to the shape of the Windows APIs we have had to wrap so-far. They are designed in such a way that most variable-sized types are actually either a handle or a 32-bit value (and never actually a The other platforms that now need support aren't designed the same. They often take an Outside of the platform APIs, there are other interop scenarios that fit the same bill (such as multimedia graphics applications, games, etc). The lowest level of code will be unsafe and directly wrap the underlying native code. It will then expose those APIs to higher level code in a safe manner. Even when done in a safe manner, the "safe" APIs still need to expose fields/properties as Just because I'm writing my own game using framework X, which itself wraps I should also be able to easily interop with other code (F#) that already supports all of the appropriate operators on Native sized numbers is something that no one can get away from because they are a part of the underlying platform (both hardware and software). We can try to abstract them away, but they are something everyone has to deal with and be aware of to some degree. The only reason you don't already have to work with |
@tannergooding when we discuss "very specific" scenario in context of compiler design, it's customary to present code samples. Is it something you can do? |
It is also obvious that we are going in circles and neither of us will ever convince the other of our POV. I believe most of the relevant arguments have been laid out (from both sides at this point) at this point and we probably won't get much further without derailing the issue entirely (although we might have done so already 😄) |
Without jumping into the other parts of the conversation, I just wanted to say that I hope this part doesn't get lost in the noise. I think |
What is the premise that nint is somehow for interop based on? Intptr currently serves this function. nint would be redundant for interop. What we need is nlong aka native long so interop can work on LP64 and LLP64 platforms. All the world is the former. Windows is the latter. This makes it the only item of everything proposed that is actually relevant to modern Mac/iOS. Yet, somehow, its also the only item being completely ignored. |
@tannergooding I see no relevance in your last statement. Language features need to come with justification that's more of a hard science than personal preference. That's why examples are useful. Thanks for your effort of picking those! The top example you've picked (thanks again!) illustrates the value of this feature in practice: using Alias2 = global::System.nfloat;
using Alias3 = nfloat;
namespace MonoTouch.Whatever {
enum NintEnum : nint { }
enum NuintEnum : nuint { }
class Foo {
nint x;
nuint y;
}
} To me it's crystal clear exactly how much value |
@OtherCrashOverride I agree with you here... Somehow, it seems that this Proposal focuses on We currently use p/invoke to call into a C interface for a C++ library (which cannot change it's type declarations as they need to stay binary compatible with existing clients). They expose masses of Now, the problem is, that while So as there's no single datatype we can use, so we currently define most structs and p/invoke signatures twice, using if() to guard the calls in higher level wrapper code. |
These full type names don't feel quite 'ECMA 335'-ish imho. |
Closing as |
@eerhardt Please link to the design docs, implementation etc. |
@dsyme here is the C# language spec for them https://github.com/dotnet/csharplang/blob/master/proposals/csharp-9.0/native-integers.md. It's effectively just exposing the underlying CLR native int tat's been around since 1.0 in the language much as we expose the other primitive types. Nothing changed about the runtime here. |
We worked on a design document for native-sized number types that we would like to add to the platform.
A draft of the design document has been just posted to https://github.com/dotnet/corefxlab/blob/master/docs/specs/nativesized.md
Feedback would be appreciated, especially on the “Design Considerations” section which describes open issues/questions/trade-offs.
The text was updated successfully, but these errors were encountered: