Skip to content
This repository has been archived by the owner on Aug 2, 2023. It is now read-only.

Close on the design of native-size numbers #1471

Closed
KrzysztofCwalina opened this issue Apr 17, 2017 · 224 comments
Closed

Close on the design of native-size numbers #1471

KrzysztofCwalina opened this issue Apr 17, 2017 · 224 comments
Assignees
Milestone

Comments

@KrzysztofCwalina
Copy link
Member

We worked on a design document for native-sized number types that we would like to add to the platform.

A draft of the design document has been just posted to https://github.com/dotnet/corefxlab/blob/master/docs/specs/nativesized.md

Feedback would be appreciated, especially on the “Design Considerations” section which describes open issues/questions/trade-offs.

@nietras
Copy link

nietras commented Apr 17, 2017

The biggest issue with this is you cannot undo C# mapping of IL e. g. native int to IntPtr, how will this be adressed?

For further consideration see https://github.com/DotNetCross/NativeInts for how types could be implemented in IL.

@mellinoe
Copy link
Contributor

We have considered adopting IntPtr and UIntPtr for these scenarios, but we decided against it because of compatibility reasons (it would require changing behavior of UIntPtr and IntPtr). We decided that separate types for integer and pointer scenarios is the best approach at this point.

I feel that this is skimmed over a bit too quickly; can we expand on what the specific incompatibilities are? Given that most/all existing interop mappings will just choose IntPtr or UIntPtr for these scenarios today, these new types at the very least will introduce a kind of "API schism" going forward.

@CyrusNajmabadi
Copy link
Member

I see the value here. Though i question if it's necessary to have a specific language keyword for this. I think it would be fine if someone had to write IntN. And, if they really wanted to write 'nint', they could just say: "using nint = System.IntN".

@mellinoe
Copy link
Contributor

I'm not sure about the general usefulness of FloatN. The usage of a typedef in a native signature (CGFloat is just a typedef for a 32- or 64-bit float) is a general problem not restricted to this Apple framework, and this proposal does not solve that; just one particular instance of it. Adding runtime support, language keywords, etc. for such an esoteric and unfriendly pattern feels like overkill to me.

Other discussions on this topic suggested putting this into a separate ("Apple-specific") library, since it is only useful for their unusual usage of a float typedef in public function signatures. Is that not an acceptable solution?

@VSadov
Copy link
Member

VSadov commented Apr 17, 2017

to emit math with IntN , compiler would need to unwrap to the underlying values, do the math, and then wrap the result back into IntN. Wrapping back is easy since there is a conversion operator, but how wnwrapping would happen?

The proposal is a bit too vague on that.
Even if it is intrinsically convertible, compiler will need to emit something.

Do we have an IntPtr value field, ToIntPtr property, or something else?

Basically, what is the expected IL for the following:

nint x = 21;
x = x + x;

@VSadov
Copy link
Member

VSadov commented Apr 17, 2017

I am interested in more details about compatibility concerns.

Since the goal here is to allow efficient math in terms of native integers, just adding a bunch of compiler built-in checked/unchecked operators like nint + nint -> nint , nint + int -> nint, ... would seem to do the job most efficiently (in terms of emitted IL).
Compiler then can emit that directly as checked/unchecked math on native ints/uints
This approach works nicely for pointer types today.

Mapping to IntPtr would naturally "just work" with interop and existing APIs.
For example dynamic COM binder knows about native ints. It would not understand IntN, without servicing.

If the only concern is overload resolution between, say int and nint, there could be other solutions to keep the existing behavior. For example we can keep the types not convertible implicitly except when dealing with literals.

Basically, I see many advantages in mapping nint directly to IntPtr when efficiency is concerned, so I wonder what disadvantages push us to use the wrapper approach?

I do understand that for nfloat we need a wrapper, but for nint we might not need to.

@VSadov
Copy link
Member

VSadov commented Apr 18, 2017

An alternative proposal for nint could be:

  • map nint directly to IntPtr
  • introduce built-in operators for IntPtr, including those that take int such as nint + int -> nint
    (emitted as plain add or add.ovf if in checked context)
  • allow nint when indexing arrays and in pointer element accesses.
  • introduce built-in explicit conversions from numeric integral types
    (emitted as conv.i , conv.u, conv.ovf.i or conv.ovf.i.un depending on context and operand's type)
  • introduce built-in explicit conversions to numeric integral types
    (emitted the same as conversions from long, i.e. conv.ovf.u8 when converting to ulong in checked)
  • do not introduce implicit conversions

(similarly nuint can be mapped to UIntPtr, but needs to use unsigned semantics for math and conversions)

The end result will be:

// Metadata:     IntPtr M1(IntPtr[] arr, IntPtr pos, int offset)
nint M1(nint[] arr, nint pos, int offset)
{
    // can add  nint and int
    var i = pos + offset;

    // can use  nint as an array index
    return arr[i];
}

// must cast 123 to nint explicitly for compat reasons.
M1(someArray, (nint)123, 45) 

The advantage here is that such nint works right away with everything that understands IntPtr.
Including scenarios where values are boxed, used dynamically, passed to COM interop as Variants and so on...

@VSadov
Copy link
Member

VSadov commented Apr 18, 2017

And to answer my question above - there are indeed problems with mapping nint directly to IntPtr in the way I've described.

Turns out IntPtr already defines some math operators and explicit conversions to/from ints and pointers. It probably felt like a good idea long time ago to add some math capabilities to this type, since C# would not provide any.
Now, when we would like to provide correct and efficient support via intrinsics, we will run into the risk of breaking some existing code.

Sigh...

@VSadov
Copy link
Member

VSadov commented Apr 18, 2017

@KrzysztofCwalina - the example of UIntN in the proposal is missing implicit conversion that goes the other way. UIntN->UIntPtr

I assume that is how the underlying value is fetched when operating on the struct.

@jveselka
Copy link

public struct IntN : IComparable<IntN>, IEquatable<IntN>, IFormattable {
    public static IntN MaxValue { get; } // note that Int32 uses const 
    public static IntN MinValue { get; } 
    …
}

But IntPtr.Zero uses static readonly field. Is there a reason for using get-only properties instead?

@Drawaes
Copy link
Contributor

Drawaes commented Apr 18, 2017

Doesn't a static property like that if backed with a static readonly (which it should in this case) get basically collapsed to similar to a const by the hit anyway. If it's a property it allows for future behaviour change with no breaking of the api.

@benaadams
Copy link
Member

Zero is constant; min and max value depend on native size which isn't known until runtime. (Though min value is a const for UIntN)

@KrzysztofCwalina
Copy link
Member Author

@nietras, could you clarify the mapping issue? We map IntPtr to native int. We can map IntN to native int too. We don't map native int to BCL types except when here is metadata telling us what mapping is desired. Am I missing something? cc: @jkotas

@mellinoe, there is a sentence alluding to the incompatibilites, but I will expand it. Thanks.

@CyrusNajmabadi, without the alias, the types will forever feel like 2nd class citizens. But you are right that it's not absolutelly necessary, hence pri 2.

@mellinoe, we considered doing what you suggested with GCFloat, for the reasons you listed. But it seems like the cost of doing proper language support is not that high (once we do this for ints), the type is very important to Xamarin, and it future proofs us in the case the type becomes more widely used. cc: @migueldeicaza

@VSadov, the conversions to IntPtr/UIntPtr are real overloaded operators. See the asterix in the table. These will be used to unwrap.

@VSadov, there are conversions UIntN->UINtPtr in the table. Admitably in an awkward place: see the second table. BTW, this is where the asterix operators are, which are APIs (overloaded operators) for the unwrapping scenarios.

@zippec, as @benaadams said, some of the min/max values are not known till runtime. The others that can (e.g. UIntN.Zero) are properties for consistency with these that cannot. But we should consider making these consts. @jkotas?

@tannergooding
Copy link
Member

@mellinoe, I would very much rather see the support added to System.IntPtr and System.UIntPtr as well, for all the various reasons.

However, @jaredpar raised a very good point offline that the forced 'checked' behavior for some of the existing code (constructors, and some of the conversions, and operators) makes it very difficult since that means we would either have to take a breaking change to remove the forced 'checked' behavior or we would need to have inconsistent behavior when dealing with IntPtr vs how we deal with everything else.

@tannergooding
Copy link
Member

That being said, I think it would still be useful to finish fleshing out IntPtr and UIntPtr with all the appropriate operators (and doing so in a consistent manner with how things already are for the available surface area).

Regardless of whether the type was designed specifically for pointers (and I'm not sure I agree with this, since the remarks section and CLI spec both indicate otherwise), pointer arithmetic is a very valid and useful scenario which should be supported on the IntPtr type.

@Drawaes
Copy link
Contributor

Drawaes commented Apr 18, 2017

Agreed IntPtr.Add (mypointer, offset) is ugly and hoop jumping.

@nietras
Copy link

nietras commented Apr 18, 2017

much rather see the support added to System.IntPtr and System.UIntPtr

I agree completely with this, with some considerations below.

We don't map native int to BCL types except when here is metadata telling us what mapping is desired. Am I missing something?

@KrzysztofCwalina Or perhaps I am missing something but given the following C# code:

    public struct nint
    {
        public IntPtr Value;

        public nint(IntPtr value)
        {
            Value = value;
        }
    ....

This is compiled into:

.class public sequential ansi sealed beforefieldinit DotNetCross.NativeInts.nint
       extends [System.Runtime]System.ValueType
{
  .field public native int Value

  .method public hidebysig specialname rtspecialname 
          instance void  .ctor(native int 'value') cil managed
  {
    // Code size       8 (0x8)
    .maxstack  8
    IL_0000:  ldarg.0
    IL_0001:  ldarg.1
    IL_0002:  stfld      native int DotNetCross.NativeInts.nint::Value
    IL_0007:  ret
  } // end of method nint::.ctor

Notice how IntPtr is not used or referenced. Only built-in native int is used. Thus, IntPtr is simply speaking a placeholder for native int. The same goes for UIntPtr.

This works the other way too, of course, which can be seen in e.g. https://github.com/DotNetCross/NativeInts and of course the Unsafe class https://github.com/dotnet/corefx/blob/master/src/System.Runtime.CompilerServices.Unsafe/src/System.Runtime.CompilerServices.Unsafe.il but also in any other code using these types. IntPtr and UIntPtr as such do not exist in the generated code, as far as I can tell. Which is great from an efficiency point, but less than ideal from a naming consideration.

I believe we should simple accept that IntPtr and UIntPtr are defacto keywords for native int and unsigned native int, perhaps I am missing something in this, as I am not completely aware of why the IntPtr and UIntPtr types exist and there have not been made actual language keywords for these types as is the case for e.g. int and System.Int32.

Anyway, the naming with Ptr is less than ideal of course, but how can this be changed? I can't see how without breaking stuff... which means if we want, and I believe we do, an efficient (e.g. IL code size) and zero coupled way (e.g. not introducing new types that need to be referenced which means new "runtime"/framework versions etc. and thus no support for older runtimes) of using the IL native sized types, we have to use IntPtr and UIntPtr and then make these full fidelity types will all operators and so forth.

So I do not understand the "metadata" part of the question, perhaps this is where I am missing something? 😕

@VSadov
Copy link
Member

VSadov commented Apr 18, 2017

@tannergooding - IntPtr/native int and UIntPtr/native uint are indeed just pointer-sized integer types. That is why there are signed/unsigned versions - to do signed/unsigned math.

However the fact of adding operators to those types a while ago has essentially closed the door to implementing the math support as compiler intrinsics now - due to compatibility concerns.

@mellinoe
Copy link
Contributor

@mellinoe, we considered doing what you suggested with GCFloat, for the reasons you listed. But it seems like the cost of doing proper language support is not that high (once we do this for ints), the type is very important to Xamarin, and it future proofs us in the case the type becomes more widely used. cc: @migueldeicaza

I understand that reasoning, but I'm skeptical that we'll ever see other libraries have such a pattern. It seems extremely unusual to me, and I don't think I've encountered another library doing something like it. I'm also not aware of other languages or standard libraries that have this kind of "native float" type, whereas native integers and integer pointers are universal and standard. I'd probably feel comfortable with more examples of this pattern being used. Do we know of any others than CGFloat?

Related to FloatN: How is it constructed? Are there proposed literals being added to the language? I don't see conversions for it listed here; I guess there are implicit conversions from float and double?

@MichalStrehovsky
Copy link
Member

How strongly do people feel about the naming of these? While I can see why things like Int32/UInt32 don't conform to the framework naming rules on multiple fronts (the reason I assume is that int is a concept well known to every programmer and it's nice to keep them short):

  • DO NOT use abbreviations or contractions as part of identifier names
  • DO choose easily readable identifier names
  • DO favor readability over brevity.

I'm not aware of the N suffix being a standard in .NET for anything. Especially UIntN just feels like an unreadable soup of lower case and capital letters.

Why not just call them what they are? NativeInt/NativeUInt/NativeFloat? That way people don't have to spend time thinking why the N is a suffix in the proposed type name, but a prefix in the proposed language keyword.

@nietras
Copy link

nietras commented Apr 18, 2017

@jaredpar raised a very good point offline that the forced 'checked' behavior for some of the existing code (constructors, and some of the conversions, and operators) makes it very difficult since that means we would either have to take a breaking change to remove the forced 'checked' behavior

@tannergooding Couldn't we just add support for specifying unchecked then? Like for int. The generated code is pure IL and doesn't reference the IntPtr reference impl as such, so supporting unchecked/checked should be possible, although perhaps not with the same defaults as int.

@jaredpar
Copy link
Member

Couldn't we just add support for specifying unchecked then? Like for int.

The design explicitly calls for both checked and unchecked support to match the existing integral types. Need to support this case and IntPtr can't do that without breaking compat in some way due to the existing operators.

@tannergooding
Copy link
Member

@jaredpar, its a bit unprecedented (at least to the C# compiler, but not to compilers in general), but would it be crazy to have a compiler switch that says to ignore the existing operators defined by the framework and to emit pure add/add.ovf instructions instead?

I believe that would let existing code continue to work and let new consumers, who explicitly want checked/unchecked behavior to opt-in.

My thinking is that, overall, it may be worthwhile since we have 17 years worth of existing APIs/interop code (including in the framework itself) where we exclusively use IntPtr/UIntPtr.

@jaredpar
Copy link
Member

switch that says to ignore the existing operators defined by the framework and to emit pure add/add.ovf instructions instead?

Imagine that command line option exists. How can I sanely review code which uses IntPtr now? It's impossible to understand the semantics unless I understand every single compilation in which it occurs. That is more of a problem now with the prevalence of shared projects where code is commonly used in multiple compilations.

Astute readers may note: wait didn't @jaredpar just argue specifically against having a /checked command line option? Yes I did. That option makes it impossible to correctly interpret virtually any arithemetic operation in your source code. It's part of the reason it has so little usage.

My thinking is that, overall, it may be worthwhile since we have 17 years worth of existing

Disagree strongly. A command line switch which changes the semantic behavior of the compiler is a bad feature. It makes it impossible to read code and understand what it does.

@tannergooding
Copy link
Member

I understand your viewpoint, but am not completely in agreement here. A large number of compilers have just such behavior for math (especially with regards to floating-point arithmetic -- /fp:fast for example). #pragma directives are another example where compilers frequently modify behavior of code to suit particular needs.

@Drawaes
Copy link
Contributor

Drawaes commented Apr 18, 2017

And it's a nightmare to manage. Your writing a precision sensitive finance library and a mistake is made in compilation, tracking that back is nigh on impossible. The nice thing about .net today is if I write a Monte Carlo and run it on an old çomputer in linux or on a Mac or whatever the numbers will match (with correct seeding). I also think this is why that native float is bad to have outside a specialized graphics lib. The numerical ramifications of flipping from a float to a double can be huge.

@jaredpar
Copy link
Member

A large number of compilers have just such behavior for math (especially with regards to floating-point arithmetic -- /fp:fast for example

Sure. And my feedback still applies. The presence of such switches make it harder to interpret what code is doing. It's entirely relying on the decision of an ambient authority. The decision which is often difficult to determine (saying this from experience trying to figure out how a series of makefiles translate into a final command line for the compiler).

It also makes the code significantly less portable. Because the code and the ambient floating point option must be carried together.

#pragma directives are another example where compilers frequently modify behavior of code to suit particular needs.

I dislike #pragma for similar reasons but it at least is local to the file. Hence it's easy to determine for a casual code reviewer in a web browser what the behavior of the code in question is. After a lot of scrolling around at least 😦

@mihailik
Copy link

mihailik commented Jun 1, 2017

@tannergooding code analyser as an entry barrier to writing simple high-level code??

No jokes, if you need to understand LOB space, best way is to spend 17 years doing it. Or ask somebody who did.

Same goes for deserializing 64-bit values into 32-bit runtime, just ask someone competent for an explanation.

@pentp
Copy link

pentp commented Jun 1, 2017

@mihailik what's your problem? There is no reasonable excuse for acting this impolite, even if you personally fear this small addition to the C# language or just hate the world.

I'm 100% sure most people in this discussion are competent programmers and the C# language design team has some very talented people thinking about these kinds of language changes. The feature is useful and the principal downside is additional complexity.

So please leave your ad hominem arguments and unsubstantiated fear out of this discussion.

@mihailik
Copy link

mihailik commented Jun 1, 2017

@pentp not sure what problems you're talking about, or what hate/fear. Can you stick to the topic? Please.

Marshalling of platform-dependent pointer-sized types is fundamentally unsafe. That is, extracting 64-bit value from stored location into runtime 32-bit location leads to unavoidable data corruption.

There's no shame in missing this trivial fact, we all have different levels of competency and I for once am happy to spell such things out, as I did earlier.

Same way I have spelled out risks of this feature to the wider community. @pentp please do not bring personal feelings into this, it's about material measurable risks to large stable and expensive codebases out there.

As an active member of C# community for decades, and a long-term collaborator with Microsoft on CLR platform and other projects, I expect to see better moderation and more professional discussion driven by competent seasoned staff.

When reasonable concerns are raised, they should be seriously taken into account and not brushed off. Treatment like 'if you fear for your code' or patronising contempt to legitimate feedback is very counterproductive to the ecosystem. I understand the proponents of this feature acting like this without Microsoft's mandate, but I expect the vendor and sponsor to put more effort in setting the rational, measured, evidence-based tone.

Features are judged not on whether they are feared or desired, but on their merit. And showing how exactly a change makes a visible measurable improvement. Let's all remember that.

@jnm2
Copy link
Contributor

jnm2 commented Jun 1, 2017

There is no reasonable excuse for acting this impolite

Seconded.

@mihailik

Marshalling of platform-dependent pointer-sized types is fundamentally unsafe. That is, extracting 64-bit value from stored location into runtime 32-bit location leads to unavoidable data corruption.

And for those reasons, no one does this. Having nint will not cause people to start doing this.

@Drawaes
Copy link
Contributor

Drawaes commented Jun 1, 2017

Agreed on the let's avoid personal insults.

I will tell you this, I have a lot of experience with the corporate .net world. Unsafe is almost never used, Intptr sounds dangerous to as does interop. When any of those are needed usually "specialists take care of it" but something that is a keyword in the language right along with int and unit... well I will just give that a shot.

My problem here is you are proving a foot gun and making it seem very normal to carry it around d in your pocket. I would like to see some barrier to entry.

Also I will say it again, just in case repetition works there is no good justification for native floats that I have seen other than a soon to be defunct Apple api.

@miloush
Copy link

miloush commented Jun 1, 2017

I do see a conceptual difference between IntPtr and native-sized integers (or floats). You would use the pointers to point to something in the memory and native-sized numeric types to do computations. You pass things to the underlying OS in IntPtrs but let the CPU do arithmetics in nints or nfloats. That also keeps IntPtr on the "unsafe" side and native-sized numerics on safe side. In this view, native floats would seem to be natural for doing numerical computations (e.g. graphics, geometry, physics etc.). It would even make sense to have nint/nfloat 64-bit on 64-bit CPU when the application itself is 32-bit.

That said, the motivation above seem to be to improve the situation with pointer arithmetics, and in that case I am not entirely convinced introducing a new built-in type is worth saving the keystrokes.

@mihailik
Copy link

mihailik commented Jun 1, 2017

@jnm2 > And for those reasons, no one does this.

The whole point of "LOB lobby" here is to keep these unsafe ideas out of the language proper:

Likewise, with IntPtr, it is really no different than serializing any other variable sized piece of data (arrays, strings, structs like BigInteger, etc...). You need to store the size of the data and then store the data itself.

@jnm2
Copy link
Contributor

jnm2 commented Jun 1, 2017

@mihailik Those are not unsafe ideas. @tannergooding is making a reasonable point. It doesn't matter whether you use int, long, byte, nint, BigInteger- use whatever makes the most sense in memory, and when you serialize, serialize in a platform-, endian- and word-size-independent fashion.

@mihailik
Copy link

mihailik commented Jun 1, 2017

@jnm2 you've missed a tiny insignificant detail man ;-)

You cannot deserialize 64-bit value on 32-bit platform.

@jnm2
Copy link
Contributor

jnm2 commented Jun 1, 2017

@mihailik Of course not, which is why you use whatever makes the most sense in memory to hold the size of values you are planning for. I'm really not sure what the problem is. If you need values 2^31 or greater, use long or double or decimal or BigInteger or whatever makes the most sense in memory.

@tannergooding
Copy link
Member

@miloush, the purpose of extending IntPtr is specifically for platform-sized numerical arithmetic (think size_t). If the user is actually working with pointers/handles/opaque types, then doing actual pointer arithmetic is reasonable (because that is what you are actually working with).

@Drawaes
Copy link
Contributor

Drawaes commented Jun 1, 2017

Forgetting nint for a second. There is no such thing as a native sized float. Most fpu on cpus are fixed. As said before the fpu on the 8086 was 80bit internally ... having it native sized has one application I have ever heard of and that is the Apple API. I would love to hear of others

@tannergooding
Copy link
Member

@mihailik, you also cannot deserialize an array that has 20 elements into a container that can only hold 10 elements.

You are completely correct that you shouldn't be serializing IntPtrs (except in some very niche, low-level, circumstances). You also shouldn't be serializing pointers, the result from object.GetHashCode(), and you should salt your passwords.

This doesn't stop badly designed LOB apps (or code in general) from being written to do as such and neither will it prevent users from doing as such just by throwing it behind some switch.

The purpose of this proposal is to fill a very specific (and very much needed) scenario for Interop code. Having nint in your code is no more unsafe than using ref, out, ref readonly, extern, etc. And regardless of whether any particular platform drops support for 32-bit, the APIs that need to be interoped with (as declared) are still using a variable sized integer, the size of which could change in the future as hardware progresses (not to mention all the legacy codebases that will still exist and need to run/be interoped with).

@mihailik
Copy link

mihailik commented Jun 1, 2017

@jnm2 serializing and then deserializing a value of platform-specific type inherently leads to data corruption. Please reach me offline (on gmail) for further discussion, to avoid messing up the conversation.

@tannergooding
Copy link
Member

@Drawaes, correct.

The only known use today (to my knowledge) is for better interop with Mac/iOS. It is also different from nint in that it has no current runtime support (no underlying runtime primitive, handling, etc).

It does fall into a similar discussion in that it is a needed interop type. However, due to their being no underlying type it comes down to only being able to be implemented via a struct (rather than compiler magic -- unless the runtime were to actually add support). Being implemented as a struct comes with all kinds of downsides (no constants, no checked/unchecked, etc). As such, the conversation about nfloat will likely become even more involved (although possibly not as controversial 😄)

@mihailik
Copy link

mihailik commented Jun 1, 2017

@tannergooding why don't you share that "very specific (and very much needed) scenario" and we avoid unnecessary hypothetising?

@tannergooding
Copy link
Member

@mihailik, I've listed it multiple times. Ease of interop with the underlying platform APIs (especially for Android, iOS, and Mac).

As you've stated:

You cannot deserialize 64-bit value on 32-bit platform.

In the same vein, you cannot deserialize a 128-bit value on a 64-bit platform. Because of this condition, you cannot just create managed APIs (which wrap the platform APIs) that use anything other than IntPtr as it causes a future-compat concern. That is, if we ever gain hardware where the underlying platform size is larger than 64-bit, then all managed code now has to be rewritten/recompiled, rather than just working. We have already seen the difficulties and fallout of this in the transition from 16->32-bit and again for 32->64-bit.

Even under the argument that 64-bit is all we will ever need, that still leaves all of the 32-bit devices where code that wraps IntPtr now has to do twice the work for every simple add/sub operation (and significantly more work for mul/div operations). A lot of these 32-bit devices are embedded or mobile devices where battery life matters.

For better or worse, we've gotten by so far due to the shape of the Windows APIs we have had to wrap so-far. They are designed in such a way that most variable-sized types are actually either a handle or a 32-bit value (and never actually a size_t). They were also written in type-unsafe languages where this kind of stuff is allowed/easy to do.

The other platforms that now need support aren't designed the same. They often take an NSInteger and have it actually represent a platform-sized numeric value. They are often written in type-safe (or at least less unsafe) languages.

Outside of the platform APIs, there are other interop scenarios that fit the same bill (such as multimedia graphics applications, games, etc). The lowest level of code will be unsafe and directly wrap the underlying native code. It will then expose those APIs to higher level code in a safe manner. Even when done in a safe manner, the "safe" APIs still need to expose fields/properties as IntPtr when the underlying type is something like size_t.

Just because I'm writing my own game using framework X, which itself wraps Vulkan, doesn't mean that I should have to use unsafe code to interact with their APIs that expose the size of an array using nint.

I should also be able to easily interop with other code (F#) that already supports all of the appropriate operators on IntPtr.

Native sized numbers is something that no one can get away from because they are a part of the underlying platform (both hardware and software). We can try to abstract them away, but they are something everyone has to deal with and be aware of to some degree.

The only reason you don't already have to work with nint on a regular basis is because the runtime decided that the size of an array is capped at Int32.MaxValue. This itself has caused various issues in places over the years that have had to be worked around (there were issues working with streams larger than 4Gb for a while).

@mihailik
Copy link

mihailik commented Jun 1, 2017

@tannergooding when we discuss "very specific" scenario in context of compiler design, it's customary to present code samples. Is it something you can do?

@tannergooding
Copy link
Member

@tannergooding
Copy link
Member

It is also obvious that we are going in circles and neither of us will ever convince the other of our POV. I believe most of the relevant arguments have been laid out (from both sides at this point) at this point and we probably won't get much further without derailing the issue entirely (although we might have done so already 😄)

@mellinoe
Copy link
Contributor

mellinoe commented Jun 1, 2017

Forgetting nint for a second. There is no such thing as a native sized float. Most fpu on cpus are fixed. As said before the fpu on the 8086 was 80bit internally ... having it native sized has one application I have ever heard of and that is the Apple API. I would love to hear of others

Without jumping into the other parts of the conversation, I just wanted to say that I hope this part doesn't get lost in the noise. I think NativeFloat is misguided, in name at the very least.

@OtherCrashOverride
Copy link

What is the premise that nint is somehow for interop based on? Intptr currently serves this function. nint would be redundant for interop. What we need is nlong aka native long so interop can work on LP64 and LLP64 platforms. All the world is the former. Windows is the latter. This makes it the only item of everything proposed that is actually relevant to modern Mac/iOS. Yet, somehow, its also the only item being completely ignored.

@mihailik
Copy link

mihailik commented Jun 2, 2017

@tannergooding I see no relevance in your last statement. Language features need to come with justification that's more of a hard science than personal preference. That's why examples are useful.

Thanks for your effort of picking those! The top example you've picked (thanks again!) illustrates the value of this feature in practice:

using Alias2 = global::System.nfloat;
	using Alias3 = nfloat;
	
	namespace MonoTouch.Whatever {
	
		enum NintEnum : nint { }
		enum NuintEnum : nuint { }
	
		class Foo {
			nint x;
			nuint y;
		}
	}

To me it's crystal clear exactly how much value nint introduces here. Hope it helps to settle the matter!

@markusschaber
Copy link

markusschaber commented Nov 18, 2018

@OtherCrashOverride I agree with you here...

Somehow, it seems that this Proposal focuses on int type for interop - however, on all relevant platforms, an int in C is 32 bit nowadays.

We currently use p/invoke to call into a C interface for a C++ library (which cannot change it's type declarations as they need to stay binary compatible with existing clients). They expose masses of long and unsigned long parameters, return values and struct members.

Now, the problem is, that while long is 32 bit on Windows x86 and x64, as well as Linux x86, it's 64 bit wide on linux x64. (We did not yet check ARM, or MacOS etc...)

So as there's no single datatype we can use, so we currently define most structs and p/invoke signatures twice, using if() to guard the calls in higher level wrapper code.

@ericwj
Copy link

ericwj commented Nov 8, 2019

These full type names don't feel quite 'ECMA 335'-ish imho.

@eerhardt
Copy link
Member

Closing as nint and nuint have been implemented in .NET 5.

@dsyme
Copy link
Contributor

dsyme commented Sep 28, 2020

@eerhardt Please link to the design docs, implementation etc.

@jaredpar
Copy link
Member

@dsyme here is the C# language spec for them https://github.com/dotnet/csharplang/blob/master/proposals/csharp-9.0/native-integers.md. It's effectively just exposing the underlying CLR native int tat's been around since 1.0 in the language much as we expose the other primitive types. Nothing changed about the runtime here.

@dsyme
Copy link
Contributor

dsyme commented Sep 28, 2020

I see, thank you. Yes, F# has had this since v0.5. Good to see C# catching up.

Cc @cartermp @TIHan

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests