Unable to add file to DVC - Getting a 'win32timezone' error

[chris-steenkamp]

DVC version: 0.66.9
Platform: Windows 10 x64
Method of installation: exe

I am busy going through the "Getting Started" section of the website and on the Add Files section I am running the dvc add data/data.xml command to add the data.xml file to DVC.

I get the following error:
ERROR: unexpected error - No module named 'win32timezone'
and the file is not added.

[nightscape]

I just observed the same problem.
I had some old versions of dvc installed via scoop and tested with them.
The problem was not present in 0.66.3 and is present in 0.66.6.
I don't have 0.66.4 and 0.66.5 installed and afaik it's not easy to install old versions with scoop.

[chris-steenkamp]

As a workaround, creating a Python virtual environment and installing dvc via pip allows me add the file as expected. The library which provides the win32timezone module is pywin32.

[nightscape]

I managed to install 0.66.5 and the problem is not present there, so it seems to have been introduced between 0.66.5 and 0.66.6.
Here a code comparison between the two versions:
https://github.com/iterative/dvc/compare/ee70822bd434adb36c41e20c63939af69bf0bc64..522579be1ebde78992853a2dc25af3b06893c466

[efiop]

Hi @chris-steenkamp @nightscape ! Thanks for the report! Indeed, looks like it is a bug related to our binary packages (windows exe, mac pkg and linux deb/rpm). A workaround for that is to uninstall dvc and install it from pip, brew or conda. Investigating the issue right now.

[efiop]

@chris-steenkamp @nightscape Btw, guys, could you please share why you've chosen to use our exe installer instead of installing using pip or conda?

[efiop]

@chris-steenkamp @nightscape One more thing, could you run that command with -v added (e.g. dvc add data -v) and share the full log, please?

EDIT: nevermind, I am able to reproduce;

ERROR: unexpected error - No module named 'win32timezone'
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc\main.py", line 50, in main
  File "dvc\command\add.py", line 25, in run
  File "dvc\repo\__init__.py", line 35, in wrapper
  File "dvc\repo\scm_context.py", line 4, in run
  File "dvc\repo\add.py", line 48, in add
  File "dvc\repo\add.py", line 84, in _create_stages
  File "dvc\stage.py", line 484, in create
  File "dvc\stage.py", line 380, in _stage_fname
  File "dvc\utils\fs.py", line 73, in contains_symlink_up_to
  File "dvc\system.py", line 268, in is_symlink
  File "dvc\system.py", line 191, in _getdirinfo
ModuleNotFoundError: No module named 'win32timezone'
----------------------------------

[efiop]

Fix is #2763 . Also added a simple test to our dvc-test repo(testing existing packages) so we will get that covered in the future. Will release a new version ASAP.

[nightscape]

@efiop if that information is still useful: I installed DVC via scoop which uses the .exe file instead of pip or conda.
Thanks for fixing this so quickly!

[chris-steenkamp]

@chris-steenkamp @nightscape Btw, guys, could you please share why you've chosen to use our exe installer instead of installing using pip or conda?

Thanks for the quick turnaround! The reason I am using the .exe (rather than pip) is because the Installation Docs seem to recommend downloading the installer by default.

[efiop]

@nightscape Oh, I didn't even know about scoop! Looks like @r15ch13 added dvc https://github.com/ScoopInstaller/Main/blob/master/bucket/dvc.json and keeps it up to date! 🔥

@chris-steenkamp Got it! Thanks for the feedback!

@nightscape @chris-steenkamp A few more questions, if you don't mind 🙂Are you guys familiar with chocolatey or scoop? Would you prefer to us recommending one of them instead? Are you using standalone python or conda? What if we would recommend one of those?

We don't have anyone that is using windows for development in our team, so we are not sure what would be the preferred method of installation on windows. Coming from *nix, it feels right to use chocolatey or scoop(maybe there is even some other pkg manager that I'm not aware of) to install and manage packages in general, but we are not sure if our typical user is familiar with those 🙁 As to installing standalone python or conda, those seem like a language bias and we don't recommend those as dvc is language-agnostic and it would be weird(and might be not trivial) to make R developer install python specifically for dvc.

[SiriusBHU]

it seems that this bug is not fixed by now, when using exe installer 🙁...

[pmrowla]

@SiriusBHU can you please try re-running the command that gave you an error with the -v flag and then post the full error output here

[TheBigTicket13]

@SiriusBHU I encountered the same issue when using dvc push command. I have also used .exe installer.
I uninstalled v.2.9.3 and picked some older version from dvc releases, i.e. v.2.7.3.
Now dvc push command is performed successfully with no error.

It seeems that issue came up in versions 2.8+

[efiop]

@TheBigTicket13 @SiriusBHU Btw, where did you install it from (downloaded our exe, installed from chocolatey, scoop, conda, pip, etc?)? Could you show dvc doctor, please?

[TheBigTicket13]

@efiop DVC version which cause the problem was downloaded from this page I choose Windows option from download button. I don't have this version installed anymore, only the v2.7.3 which is working fine, so I think dvc doctor will not helpful

[Guixudong]

使用pip安装了2.8.3版本的dvc，运行dvc push报错，以下是错误日志：

2022-03-24 11:55:28,513 ERROR: unexpected error - No module named 'win32timezone'
------------------------------------------------------------
Traceback (most recent call last):
  File "dvc\output.py", line 855, in get_dir_cache
  File "dvc\objects\__init__.py", line 20, in check
  File "dvc\objects\db\base.py", line 175, in check
  File "dvc\objects\file.py", line 68, in check
  File "dvc\objects\file.py", line 73, in _check_hash
  File "dvc\objects\stage.py", line 72, in get_file_hash
  File "dvc\objects\stage.py", line 47, in _get_file_hash
  File "dvc\fs\local.py", line 164, in info
  File "fsspec\implementations\local.py", line 80, in info
FileNotFoundError: [WinError 3] ϵͳ▒Ҳ▒▒▒ָ▒▒▒▒·▒▒▒▒: 'E:/dvcrepo/.dvc/cache/7a/950a4ae9f11dc87b046d83b2b128f8.dir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "dvc\main.py", line 55, in main
  File "dvc\command\base.py", line 45, in do_run
  File "dvc\command\data_sync.py", line 57, in run
  File "dvc\repo\__init__.py", line 50, in wrapper
  File "dvc\repo\push.py", line 37, in push
  File "dvc\repo\__init__.py", line 420, in used_objs
  File "dvc\repo\index.py", line 186, in used_objs
  File "dvc\stage\__init__.py", line 660, in get_used_objs
  File "dvc\output.py", line 933, in get_used_objs
  File "dvc\output.py", line 877, in _collect_used_dir_cache
  File "dvc\output.py", line 859, in get_dir_cache
  File "dvc\data_cloud.py", line 114, in pull
  File "dvc\objects\transfer.py", line 153, in transfer
  File "dvc\objects\status.py", line 166, in compare_status
  File "dvc\objects\status.py", line 122, in status
  File "dvc\objects\status.py", line 48, in _indexed_dir_hashes
  File "dvc\objects\db\base.py", line 419, in list_hashes_exists
  File "concurrent\futures\_base.py", line 619, in result_iterator
  File "concurrent\futures\_base.py", line 444, in result
  File "concurrent\futures\_base.py", line 389, in __get_result
  File "concurrent\futures\thread.py", line 57, in run
  File "dvc\objects\db\base.py", line 410, in exists_with_progress
  File "dvc\fs\fsspec_wrapper.py", line 136, in exists
  File "funcy\objects.py", line 50, in __get__
  File "funcy\objects.py", line 28, in __get__
  File "dvc\fs\ssh.py", line 117, in fs
  File "fsspec\spec.py", line 76, in __call__
  File "sshfs\spec.py", line 77, in __init__
  File "fsspec\asyn.py", line 91, in wrapper
  File "fsspec\asyn.py", line 71, in sync
  File "fsspec\asyn.py", line 25, in _runner
  File "asyncio\tasks.py", line 494, in wait_for
  File "sshfs\utils.py", line 27, in wrapper
  File "sshfs\spec.py", line 92, in _connect
  File "contextlib.py", line 568, in enter_async_context
  File "asyncssh\misc.py", line 220, in __aenter__
  File "asyncssh\connection.py", line 6855, in connect
  File "asyncssh\connection.py", line 297, in _connect
  File "asyncio\base_events.py", line 1050, in create_connection
  File "asyncio\base_events.py", line 1068, in _create_connection_transport
  File "asyncssh\connection.py", line 6846, in conn_factory
  File "asyncssh\connection.py", line 2750, in __init__
  File "asyncssh\gss_win32.py", line 148, in __init__
  File "sspi.py", line 200, in __init__
ModuleNotFoundError: No module named 'win32timezone'

[dberenbaum]

@Guixudong How did you install DVC? Could you show dvc doctor? Does this error happen every time you run dvc push?

[dberenbaum]

@Guixudong I see you also have opened a new issue in #7505. Please ignore this thread and continue discussion there, thank you!