Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for enumerating non-default code pages added with RegisterProvider (System.Text.Encoding) #25804

Closed
mklement0 opened this issue Apr 7, 2018 · 11 comments
Labels
area-System.Text.Encoding question Answer questions and provide assistance, not an issue with source code or documentation.
Milestone

Comments

@mklement0
Copy link

The documentation for System.Text.Encoding.GetEncodings() tells us (emphasis added):

The list of supported encodings returned by the GetEncodings method does not include any
additional encodings made available by any EncodingProvider implementations that
were registered by calls to the RegisterProvider method.

Is there no way to enumerate these additional encodings?

Unfortunately, System.Text.CodePagesEncodingProvider.Instance lacks a method for enumerating them.

@iSazonov
Copy link
Contributor

iSazonov commented Apr 7, 2018

I would say that it is very surprising that after registering a new provider we can not get the full list of supported code pages.

@danmoseley
Copy link
Member

@tarekgh

@iSazonov
Copy link
Contributor

iSazonov commented Apr 8, 2018

First we discussed this in PowerShell Core repo. Some cmdlets have -Encoding parameter. If we want to add IntelliSense support for full set of code pages then we have the problem.

@tarekgh
Copy link
Member

tarekgh commented Apr 9, 2018

Although I can see the issue, generally people interested in specific codepage encoding they already knew this codepage number and can just use it. another scenario would be getting the codepage name or number from some contents (like Xml, html, network stream,...etc.) and create encoding object. in such cases enumeration is not needed anyway.

Could you tell exactly your scenario you want this? I understand power shell case but again this is not the scenario people use it. looking at the usage of System.Text.Encoding.GetEncodings, I am seeing it has very low usage.

If you really in need for that, you can file an API proposal for adding GetEncodings method in the class EncodingProvider. and have the implementation support it.

@mklement0
Copy link
Author

@tarekgh:

To elaborate on the scenario mentioned by @iSazonov:

PowerShell Core users may want to discover available encodings and/or be assisted in selecting a specific encoding with tab completion:

Get-Content file.txt -Encoding <tab>  # should show all available encodings

The above is currently limited to the .NET Core default encodings, even though PowerShell registers additional encodings on startup.

@tarekgh
Copy link
Member

tarekgh commented Apr 9, 2018

@mklement0 I understand this and I am not objecting it. my point is this is not a real app scenario or real user scenario. in another word, my experience is not many people need to enumerate the code pages.

Feel free to submit an API proposal as I mentioned in my previous reply.

@mklement0
Copy link
Author

mklement0 commented Apr 9, 2018

@tarekgh:

Thanks; I am glad you understand. It sounded like you wanted more details, based on this question of yours, in response to which I posted my elaboration.

Could you tell exactly your scenario you want this?

As for deeming this not a real-world scenario:

but again this is not the scenario people use it.
looking at the usage of System.Text.Encoding.GetEncodings,

Of course, it's impossible to see usage from the PowerShell side of things yet, given that the feature currently doesn't exist - for lack of API support (as well as the relative youth of PowerShell Core overall and, specifically, the -Encoding parameter's ability to accept Encoding instances directly).

I will submit an API proposal.

@iSazonov
Copy link
Contributor

iSazonov commented Apr 9, 2018

Also we have a request to implement new cmdlet for converting encoding like iconv. iconv has --list parameter so users can get a list of supported code pages. Today we can't do this effectively.
I suppose we could add a new overload (maybe with optional parameter) GetEncodings(bool IncludeExtendedCodePages = false).

@tarekgh
Copy link
Member

tarekgh commented Apr 9, 2018

given that the feature currently doesn't exist - for lack of API support

Does the desktop version of power shell support Get-Content file.txt -Encoding ? I mean the version of power shell which used to depend on the full framework and not net core.

if so, this would be a good thing to look at the usage of this functionality in power shell there.

@mklement0
Copy link
Author

mklement0 commented Apr 9, 2018

@tarekgh:

Unfortunately, Windows PowerShell (the Desktop edition of PowerShell built on the full .NET framework) does not support Encoding instances directly (only a limited set of enumeration values that map onto a handful of common encodings).

In PowerShell Core's cross-platform world, encoding conversions strike me as more common, and presumably that's why that ability was added.

@iSazonov's example of an iconv-like cmdlet - green-lighted, but not yet implemented - is indeed another good use case for the ability to enumerate all available encodings.

@mklement0
Copy link
Author

@tarekgh:

Please see my API proposal at dotnet/corefx#28944.

@msftgits msftgits transferred this issue from dotnet/corefx Jan 31, 2020
@msftgits msftgits added this to the 2.1.0 milestone Jan 31, 2020
@ghost ghost locked as resolved and limited conversation to collaborators Dec 17, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
area-System.Text.Encoding question Answer questions and provide assistance, not an issue with source code or documentation.
Projects
None yet
Development

No branches or pull requests

5 participants