Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Convert process can still encounter decoding errors #3035

Closed
MaxLohove opened this issue Dec 3, 2024 · 1 comment · Fixed by #3041
Closed

Convert process can still encounter decoding errors #3035

MaxLohove opened this issue Dec 3, 2024 · 1 comment · Fixed by #3041
Labels
bug Something isn't working

Comments

@MaxLohove
Copy link

Describe the bug

The issue at hand is similar to #540.
It concerns the conversion of jupyter notebook files in utf-8 format to marimo notebooks.

marimo convert my_notebook.ipynb > my_app.py

marimo convert my_notebook.ipynb -o my_app.py
Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "<my_path>.venv\Scripts\marimo.exe_main
.py", line 8, in
sys.exit(main())
~~~~^^
File "<my_path>.venv\Lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, **kwargs)
~~~~~~~~~^^^^^^^^^^^^^^^^^
File "<my_path>.venv\Lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "<my_path>.venv\Lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^
File "<my_path>.venv\Lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<my_path>.venv\Lib\site-packages\click\core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "<my_path>.venv\Lib\site-packages\marimo_cli\convert\commands.py", line 68, in convert
f.write(notebook)
~~~~~~~^^^^^^^^^^
File "C:\Users\mlohove\AppData\Roaming\uv\python\cpython-3.13.0-windows-x86_64-none\Lib\encodings\cp1252.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_table)[0]
~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
UnicodeEncodeError: 'charmap' codec can't encode characters in position 395-397: character maps to

I was able to trace the error to

if output:
        # Make dirs if needed
        maybe_make_dirs(output)
        with open(output, "w") as f:
            f.write(notebook)
        echo(f"Converted notebook saved to {output}")
    else:
        echo(notebook)

in marimo/_cli/convert/commands.py

Replacing
with open(output, "w") as f:
with
open(output, "w", encoding="utf-8") as f:

Allowed me to convert my notebook.

marimo convert my_notebook.ipynb -o my_app.py
Converted notebook saved to my_app.py

Environment

{
"marimo": "0.9.28",
"OS": "Windows",
"OS Version": "11",
"Processor": "Intel64 Family 6 Model 142 Stepping 10, GenuineIntel",
"Python Version": "3.13.0",
"Binaries": {
"Browser": "131.0.6778.86",
"Node": "v18.16.0"
},
"Dependencies": {
"click": "8.1.7",
"docutils": "0.21.2",
"itsdangerous": "2.2.0",
"jedi": "0.19.1",
"markdown": "3.7",
"narwhals": "1.15.1",
"packaging": "24.1",
"psutil": "6.0.0",
"pygments": "2.18.0",
"pymdown-extensions": "10.12",
"pyyaml": "6.0.2",
"ruff": "0.8.1",
"starlette": "0.41.3",
"tomlkit": "0.13.2",
"typing-extensions": "4.12.2",
"uvicorn": "0.32.1",
"websockets": "14.1"
},
"Optional Dependencies": {
"pandas": "2.2.3",
"pyarrow": "18.1.0"
}
}

Code to reproduce

No response

@MaxLohove MaxLohove added the bug Something isn't working label Dec 3, 2024
@mscolnick
Copy link
Contributor

mscolnick commented Dec 3, 2024

thanks for the issue and fix! i will put up a PR for your suggested fix.

is it odd the tests pass on windows (ill look into this)

mscolnick added a commit that referenced this issue Dec 3, 2024
Fixes #3035, 

but it is odd that the tests pass without this.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants