Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UnicodeEncodeError in Non-UTF8 Terminals #179

Closed
wants to merge 7 commits into from

Conversation

przadka
Copy link
Collaborator

@przadka przadka commented Jun 19, 2023

This PR addresses an issue (Backblaze#888) where executing b2 --help in a non-UTF8 terminal would raise a UnicodeEncodeError due to improper handling of Unicode characters. The fix ensures that Unicode characters are correctly encoded irrespective of the terminal's locale. Unit tests have been added to prevent this issue from reoccurring in future releases.

@przadka
Copy link
Collaborator Author

przadka commented Jun 20, 2023

@ppolewicz Adding terminal output to show this fixes the problem:

ubuntu@ip-172-31-34-231:~$ locale
LANG=en_US
LANGUAGE=en_US:
LC_CTYPE=en_US
LC_NUMERIC=en_US
LC_TIME=en_US
LC_COLLATE=en_US
LC_MONETARY=en_US
LC_MESSAGES=en_US
LC_PAPER=en_US
LC_NAME=en_US
LC_ADDRESS=en_US
LC_TELEPHONE=en_US
LC_MEASUREMENT=en_US
LC_IDENTIFICATION=en_US
LC_ALL=
ubuntu@ip-172-31-34-231:~$ b2   
Traceback (most recent call last):
  File "/home/ubuntu/.local/bin/b2", line 8, in <module>
    sys.exit(main())
  File "/home/ubuntu/.local/lib/python3.10/site-packages/b2/console_tool.py", line 3530, in main
    exit_status = ct.run_command(sys.argv)
  File "/home/ubuntu/.local/lib/python3.10/site-packages/b2/console_tool.py", line 3370, in run_command
    args = parser.parse_args(argv[1:])
  File "/usr/lib/python3.10/argparse.py", line 1838, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 1871, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/lib/python3.10/argparse.py", line 2108, in _parse_known_args
    self.error(_('the following arguments are required: %s') %
  File "/home/ubuntu/.local/lib/python3.10/site-packages/b2/arg_parser.py", line 87, in error
    self.print_help()
  File "/usr/lib/python3.10/argparse.py", line 2568, in print_help
    self._print_message(self.format_help(), file)
  File "/usr/lib/python3.10/argparse.py", line 2574, in _print_message
    file.write(message)
UnicodeEncodeError: 'latin-1' codec can't encode character '\u2022' in position 1325: ordinal not in range(256)
ubuntu@ip-172-31-34-231:~/B2_Command_Line_Tool/dist$ ./b2 
b2 [-h]
   {authorize-account,authorize_account,cancel-all-unfinished-large-files,cancel_all_unfinished_large_files,cancel-large-file,cancel_large_file,clear-account,clear_account,copy-file-by-id,copy_file_by_id,create-bucket,create_bucket,create-key,create_key,delete-bucket,delete_bucket,delete-file-version,delete_file_version,delete-key,delete_key,download-file-by-id,download_file_by_id,download-file-by-name,download_file_by_name,get-account-info,get_account_info,get-bucket,get_bucket,get-file-info,get_file_info,get-download-auth,get_download_auth,get-download-url-with-auth,get_download_url_with_auth,hide-file,hide_file,list-buckets,list_buckets,list-keys,list_keys,list-parts,list_parts,list-unfinished-large-files,list_unfinished_large_files,ls,rm,make-url,make_url,make-friendly-url,make_friendly_url,sync,update-bucket,update_bucket,upload-file,upload_file,update-file-legal-hold,update_file_legal_hold,update-file-retention,update_file_retention,replication-setup,replication_setup,replication-delete,replication_delete,replication-pause,replication_pause,replication-unpause,replication_unpause,replication-status,replication_status,version,license,install-autocomplete,install_autocomplete}
   ...

  This program provides command-line access to the B2 service.

  There are two flows of authorization:

    * call b2 authorize-account and have the credentials cached in sqlite
    * set B2_APPLICATION_KEY_ID and B2_APPLICATION_KEY environment variables when running this program

  This program caches authentication-related and other data in a local SQLite database. The location of this database is determined in the following way:

  If profile arg is provided: * XDG_CONFIG_HOME/b2/db-<profile>.sqlite, if XDG_CONFIG_HOME env var is set * ~/.b2db-{profile}.sqlite

@ppolewicz ppolewicz closed this Jun 20, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants