UnicodeEncodeError: 'charmap' codec can't encode characters in position

[Sparviero-Sughero]

I'm trying to use whisper on Windows 11.
I wrote a simple py code:

def do_transcribe(file, language_opt, model_opt, output_dir, output_format, fp16_opt, verbose_opt, device):
	# prepare filename
	filename_no_ext = os.path.basename(os.path.realpath(file))
	filename_no_ext = os.path.splitext(filename_no_ext)[0]
	# load model
	model = whisper.load_model("large", device)
	# do trascription
	result = model.transcribe(file, language=language_opt, fp16=fp16_opt, verbose=verbose_opt)
	# save files
	os.makedirs(output_dir, exist_ok=True)
	output_writer = whisper.utils.get_writer("all", output_dir)
	output_writer(result, filename_no_ext)

Sometime give me this error:

[15:47:34] 4/790) Working on "file 2 Project.mp3"... Traceback (most recent call last):
  File "W:\myproj\srt_openai_for_windows.py", line 57, in <module>
    do_transcribe(file, language, model, output_dir, output_format, fp16, verbose, device)
  File "W:\myproj\srt_openai_for_windows.py", line 18, in do_transcribe
    result = model.transcribe(file, language=language_opt, fp16=fp16_opt, verbose=verbose_opt)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\user\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.12_qbz5n2kfra8p0\LocalCache\local-packages\Python312\site-packages\whisper\transcribe.py", line 466, in transcribe
    print(make_safe(line))
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.12_3.12.752.0_x64__qbz5n2kfra8p0\Lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode character '\u671f' in position 29: character maps to <undefined>

Just to test, I've modified utils.py to force "utf-8" encode.

system_encoding = sys.getdefaultencoding()
system_encoding = "utf-8"

Same results.
Any idea?

[suvraadeep]

the error is likely due to the attempt to write Unicode characters to a file or terminal using a default encoding that does not support those characters. To fix this, you can explicitly specify the encoding when writing the output. For example, when opening a file for writing, you can use the following:
with open('output_file.txt', 'w', encoding='utf-8') as f:
f.write(your_unicode_string)

[Sparviero-Sughero]

in utils.py (line 100)

        with open(output_path, "w", encoding="utf-8") as f:
            self.write_result(result, file=f, options=options, **kwargs)

I didn't see other open() command.

I added to my code

print(f"[D] Current encoding is: ", system_encoding)

and I'm using utf-8.

[Sparviero-Sughero]

if I run the python script, it gives no problems
It seems that the problem occurs when I launch the python script from a php script

[Sparviero-Sughero]

solved creating a batch file to run from cmd.exe
I don't like this solution, but it was the fastest

[jitendra-koodo]

can you please share your batch file?