[Proposal] Add Support For Pointers

[AzureSky12]

One of the things I'm sad VB does not have is support for pointers. I do plenty of high performance stuff in VB and it's pain to have to go and make a C# Class Library to plug into my projects just to be able handle memory-related stuff I often need to do.

Please can we have pointers in VB? That would be awesome!

[rolfbjarne]

This adds a lot of complexity for something that's IMHO not useful for most developers.

You can also work around it in VB by using the Marshal class - Marshal.ReadIntPtr, etc. (although the code admittedly turns out quite ugly). Also it's probably not as performant as C# unsafe, since I'm not sure how much the JIT is able to optimize it.

[Bill-McC]

I like the idea of pointers, but have less and less use for them these days , declining alongside the amount of pinvoke code I write ;)
So the question is the cost/benefit. I think I'd rather see work done to allow C# classes/partial classes inside a VB project; having the C# compiler a service the vb one calls on. So for the stuff you have to use C#, it'd be a much simpler experience. The big plus is you'd get to use all C# features that VB might be missing in the future.

[AzureSky12]

How do I add the Discussion label to my OP? I see it's meant to have a Discussion label. (I'm new to Github so not used to this.)

[AdamSpeight2008]

@AzureSky12 Only team member can add labels. See #8

[AdamSpeight2008]

Far as I know partial works across languages, I think you just have to make sure the namespaces match.

[AzureSky12]

@rolfbjarne Thanks for the Marshal.ReadIntPtr suggestion. I don't think I've tried that one before, I may have though, don't recall.

Hm, does it really make things on the back end that much more complex? I wouldn't have imaged so but then again I don't work on the back end so I wouldn't know, LOL.

[AzureSky12]

@Bill-McC Makes sense, Partial classes is a good idea.

[AzureSky12]

I still would love to see VB have support for pointers. I would like VB to be a first class language. Why settle for less than first class? (Ok, I see I may even have an unintended pun in there, LOL.)

[craigajohnson]

Partial classes as per @Bill-McC in the same project would be extremely excellent.

[rskar-git]

Pointers are powerful, on that I agree, but they are so troublesome in practice that one really should avoid making direct use of them. Even in C++, the usual paradigm is to make use of references, "smart" pointers, STL and Boost rather than ("raw") pointers whenever possible (see e.g. http://softwareengineering.stackexchange.com/questions/56935/why-are-pointers-not-recommended-when-coding-with-c, https://www.meetingcpp.com/index.php/br/items/cpp-future-and-the-pointer.html).

In the case of VB, I would prefer patterns/features in the language that the compiler can recognize and optimize on, such as it does for the With statement today. (BTW, has anyone checked out my proposal for Dim ByRef yet?)

For any regular situation that would benefit from pointer arithmetic, that should be something in the Framework. For the truly novel ones, there's no shame in reaching for "unsafe" C# or C/C++.

Finally, in this day and age of large CPU caches (see "CPU Caches and Why You Care" by Scott Meyers), branch predictions, hyper-threading etc., and the continuing bane of buffer overruns and memory leaks, we should be super trepidacious about pointers and pointer arithmetic. Usually, the more of that we can leave to the compiler, the better.

[TrendyTim]

@Bill-McC i was thinking the same thing.

Then when i started to reply i thought (sorry if this goes overboard) not only just [partial] classes, but even abandoning the whole c# / vb / f# project artificial separation, let anyone add any language file to any project and just have it work

Even better would be

Public Class TestClass
Private vbVar as String
#Language CS
  public static void Main()
   {
      vbVar = "From C#";
     MoreVBCode();
   }
#End Language
Sub MoreVBCode()
End Sub
End Class

Though i know there would be limitations (and perhaps only allowed in a .dotnet file instead of vb and cs files, although then you'd either need to specify the default language in some way), and i wouldn't expect to be able to switch languages in the middle of a method, with those limitations it could kinda be like an inline partial class so the compiler could handle it like that.

Although i have no idea how feasible that is. At the very least intermixing CS files and VB files in the same project would be a great start, and save me from having to have C# satellite assemblies .

Then we can truly be a unified platform, and languages can focus on their strengths without the other side being left out of nice features (not to say that lots of features shouldn't be cross implemented, because they should).

[Bill-McC]

@TrendyTim Although that'd be cool, it'd add an extra level of complexity, you'd have issues with exposing code file information to the code block such as namespaces, using vs imports. It could turn out that it is not that complex, but I'd set the target for partial classes, thus minimizing the metadata needed for each compiler. It'd probably make the code editing experience better as well.

[xieguigang]

Pointer in the C language can be simulated using System Marshal API in visualBasic.
For example, here is the memory pointer example in Visualbasic for processing the bitmap image in fast speed:

Image/Bitmap/BitmapScale.vb#L109

Dim byts As Marshal.Byte = rgbValues

 ' Set every third value to 255. A 24bpp bitmap will binarization.  
Do While Not rgbValues.NullEnd(3)
    ' Get the red channel
    iR = rgbValues(2)
    ' Get the green channel
    iG = rgbValues(1)
    ' Get the blue channel
    iB = rgbValues(0)
    ' If the gray value more than threshold and then set a white pixel.
    If (iR + iG + iB) / 3 > 100 Then
        ' White pixel
        byts(2) = 255
        byts(1) = 255
        byts(0) = 255
    Else
        ' Black pixel
        byts(2) = 0
        byts(1) = 0
        byts(0) = 0
    End If

    ' Move forward the memory pointer by step 4 bytes
    byts += style
Loop

[AnthonyDGreen]

Could you be more explicit about the high performance "stuff" and where pointers are giving you better performance? Is it just optimizing out array bounds checks?

[tannergooding]

I'm not sure that "high performance" and "pointers" are strictly dependent on each other unless you have a ton of other features (like fixed buffers of arbitrary types so you can do "AOSOA" (array of structures of arrays) and the like (even C# doesn't have this, yet).

All the other pointer performance optimizations are things that can mostly be done without pointers (passing things as readonly ref as compared to const *)

I think one of the few things that is useful is pointer arithmetic, which could be made available by exposing the IntPtr and UIntPtr operators declared by the CLI spec (see https://github.com/dotnet/csharplang/blob/master/proposals/intptr-operators.md for more details).

[KlausLoeffelmann]

I would definitely not like pointers for VB. They are not VB. The more I think about the new language strategy Mads and Anthony wrote about, the more I think how much I've been mistaken, when I thought years ago Visual Basic needed features like pointers to be on par with C#. But if I wanted that feature that VB didn't have, but C# had, and that other feature that C# had, but VB did not, and so on - why shouldn't I have used C# to begin with? Just because I like "EndIf" over "}"? When we noticed almost a decade ago that VB was losing more and more follower, we thought it was because it missed all the cooled features C# had. Really?

Meanwhile I think Visual Basic lost so much marked share because it emerged with an identity crisis from VB6. Its new features were somewhat too difficult to comprehend on the go for the typical VB6 developer on the one hand, and it was so incompatible to VB6 on the other hand, that the actual benefits over VB6 and C# (background compilation (C#), Late Binding (C#), Type Safety (VB6), .NET Framework (VB6)) got lost in the "complaining noise". C#, on the other hand, had a clear target audience. VB's target audience, however, seemed no longer be the VB6 Dev, and not yet the .NET Framework Devs. So, instead of focusing on keeping VB as a perfect RAD tool and concentrating on specializing on this (My Namespace, Code Snippets, Background Compilation, later easier yet more sophisticated LINQ support), people always took what VB had for granted, but wanted all of what C# strived for. The "Eier legende Wollmilchsau" (egg-laying-wool-milk-pig) as we say in German. Visual Basic did not lose so much of its base because it did not get the "cool" features C# got. Visual Basic did lose its base, because it did not get the cool Visual Basic-Spirit features for RAD it was supposed to get. Pointers, I think, are definitely not one of them.

My 2 Cents.

[Bill-McC]

Hi Klaus, I agree that pointers should not be a high priority for the language. When you say, the solution is just to use C#, I think there in is the real problem. Let’s say you have an application and you’ve found performance to be poor in one small part of it: to fine tune it you have to add a new project to your solution, mark the assembly friends because you don’t want to public expose your api, and shift part or all of your class/structures into C#. Wouldn’t it be much more RAD to just add a partial class file to your existing project and have that file appear to be part of your project but in C# ? As to the rise of C# over VB, I disagree as to the cause. It’s not because VB is or isn’t better than C#, it is to do with positioning and the market outside of Microsoft. Positioning inside Microsoft has and always be a problem. Microsoft has a history of selling a product/technology only then to dump it years later. We’ve seen it with Zune, Band, Windows Phone, WPF vs Silverlight, Lightswitch, InfoPath etc. So business leaders should rightly be asking why use VB if the safer bet long term is C# ? That’s why the statement of equal siblings was important. People misread that to mean language features, when it wasn’t: it was about support from within Microsoft so as each child could reach its full potential. For a while we saw it, and a lot of teams put a lot of effort to include VB in their samples and API even though they were primarily C# libraries. But it also often came off as second best, with poor quality machine generated style translations. People tired of this, so many used C# to ensure they had the latest and most accurate and tested code. And then there is the web and android, apple. All preferring languages based on C variants. In the internet I don’t think VBScript even exists anymore: again something Microsoft seems to have dropped. When developing for Android, you can do it in VB, but there’s hiccups along the way and historically it has been void. Sadly, today VB sits much like Windows Phone does, with Microsoft saying they support it, and they still churn stuff out there, but the market place questions that support and hence most developers have gone. Investors measure support by how much money and resources a company puts into a product, and weigh that against existing market penetration and future possible growth: those sums simply don’t stack up in favour of VB

[Nukepayload2]

In my opinion, allow to pass pointers as parameter is enough for VB.

// Generated from metadata.
public unsafe class JpgDecoder
{
    public byte* UnsafeBackBuffer {get; }
    public long BackBufferSize {get; }
    // Other members.
}

With New JpgDecoder
    Using strm As New UnmanagedMemoryStream(.UnsafeBackBuffer, .BackBufferSize)
        ' Process pixel data.
    End Using
End With

[Nukepayload2]

Have you seen new memory interop APIs in corefx such as ReadOnlySpan(Of T) ?
How can we read data from a block of unmanaged memory with ReadOnlySpan(Of T) in Visual Basic without using reflection?

' Declare 
Private Declare Sub GetBuffer lib "mynativelibrary" (buffer As IntPtr, length As Integer)
Private _head As Byte() = Encoding.UTF8.GetBytes("NCGR")
' Use
Dim head As New ReadOnlySpan(Of Byte)(_head) ' Works fine.
Dim span As New ReadOnlySpan(Of Byte)(buffer.ToPointer, length) ' Oops, buffer.ToPointer is not supported :( Unable to benefit from new memory interop APIs .
If length >= head.Length Then
    If span.StartsWith(head) Then
        ' ...
    End If
End If

[paul1956]

VB Needs to support Span(Of T) without direct access to pointers. The reason I much prefer VB over C# is its lack of Pointers and the issues they cause.. I haven't tried but the code below would be nice if it worked.

Dim span As New ReadOnlySpan(Of Byte)(AddressOf buffer, length)

[AnthonyDGreen]

This is why we added support for consuming ref returning APIs. See this blog post for an explanation. Pointers are not required. That was back when we were calling spans slices.

[Nukepayload2]

But we can't create Span(Of T) or ReadOnlySpan(Of T) with IntPtr and length. Neither Sub New(IntPtr, Integer) ' In Span or ReadOnlySpan nor Public Shared Function UnsafePtrToSpan(Of T As Structure)(ptr As IntPtr, length As Integer) As Span(Of T) ' In System.Runtime.InteropServices.Marshal exists.

[AnthonyDGreen]

Ok. We'd add those APIs loooooooooooooooooong before we'd add pointers to the language.

[AnthonyDGreen]

@terrajobst @jaredpar @VSadov

Hey guys, looks like we need to add these APIs for VB interop with spans. Was this just an oversight?

[xieguigang]

Read and write memory by using pointer can implemented by the Copy API from System.Runtime.InteropServices.Marshal

Here is how I implements a Generic Unmanaged memory pointer in VB:

• A generic pointer class: Marshal/IntPtr(Of%20T).vb#L114
• A Byte Pointer based on this generic memory pointer: Marshal/Pointers.vb#L71
• And last build a bitmap buffer object based on the memory pointer: BitmapBuffer.vb

[Nukepayload2]

@xieguigang My project need to unify memory read and write operations with Span(Of T) and I don't want to use Reflection.Emit or write SpanFactory in c#.

[paul1956]

@AnthonyDGreen I think you are saying that Span(Of T) returns a ref return and that now works with VB, great but to me that is an implementation detail. Your question to "Hey Guys" is the real issue not Pointer support. I would hope with Span(Of T), Unsafe will over time go away in C# and will never come to VB and over time more and more API's will take a Span(Of T) instead of or in addition to a IntPtr.

[terrajobst]

@AnthonyDGreen

I think the idea was that VB customers should be able to get spans from anything but a raw pointer and I believe we've that covered. We didn't add IntPtr constructors because we had no evidence that they would be useful.

[paul1956]

@terrajobst many Windows API's return an IntPtr and today in VB I mostly pass them to other API's and sometimes check against IntPr(0). If data processing is required I end up in C# and Unsafe code. With the advent of Span(Of T) Unsafe code shouldn't be necessary in either language nor should being required to deal with Pointers. I think to make this possible there needs to be the missing API's that @AnthonyDGreen asked about or possibly new prototypes for the WIndow's API's that return a Span instead of an IntPtr.