Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support different Encodings #203

Closed
ghaith opened this issue Jul 22, 2021 · 0 comments · Fixed by #210
Closed

Support different Encodings #203

ghaith opened this issue Jul 22, 2021 · 0 comments · Fixed by #210
Assignees
Labels
bug Something isn't working

Comments

@ghaith
Copy link
Collaborator

ghaith commented Jul 22, 2021

Is your feature request related to a problem? Please describe.
We currently only parse UTF-8 files. We should allow different encoding using a flag.

Describe the solution you'd like
A --encoding parameter sets the encoding the files are using.
We could use encoding_rs to read the file.

Describe alternatives you've considered
We could try to use encoding detection and automatically detect what each file was, but I feel this might be overkill.

Additional context
Many Windows-based files would be using the standard Windows encoding in EU it is CP-1252
This ensures we can parse these files without converting them to UTF-8

@ghaith ghaith added the bug Something isn't working label Jul 23, 2021
@ghaith ghaith self-assigned this Jul 23, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant