Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add skip if transformers not available for next item prediction notebook test #899

Merged
merged 3 commits into from
Nov 23, 2022
Merged

Add skip if transformers not available for next item prediction notebook test #899

merged 3 commits into from
Nov 23, 2022

Conversation

oliverholworthy
Copy link
Member

Goals ⚽

Enable correct notebook tests to run when merlin models installed in env without transformers dependency

Implementation Details 🚧

  • renames test for next item prediction notebook for clarify
  • adds skip for test if transformers is not installed

Testing Details 🔍

changes to next item prediction notebook test.

@oliverholworthy oliverholworthy self-assigned this Nov 22, 2022
@nvidia-merlin-bot
Copy link

Click to view CI Results
GitHub pull request #899 of commit 532f6e3e9933a75eed250a0e22002bc70fb3e8ee, no merge conflicts.
Running as SYSTEM
Setting status of 532f6e3e9933a75eed250a0e22002bc70fb3e8ee to PENDING with url http://merlin-infra1.nvidia.com:8080/job/merlin_models/1927/ and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/899/*:refs/remotes/origin/pr/899/* # timeout=10
 > git rev-parse 532f6e3e9933a75eed250a0e22002bc70fb3e8ee^{commit} # timeout=10
Checking out Revision 532f6e3e9933a75eed250a0e22002bc70fb3e8ee (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 532f6e3e9933a75eed250a0e22002bc70fb3e8ee # timeout=10
Commit message: "Rename test for next item prediction notebook"
 > git rev-list --no-walk 41f0fdf2f89461dbb8b41f55695f17f6687639bd # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins6120982526450279027.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
GLOB sdist-make: /var/jenkins_home/workspace/merlin_models/models/setup.py
py38-gpu recreate: /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
py38-gpu installdeps: -rrequirements/dev.txt, tensorflow<2.10
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu inst: /var/jenkins_home/workspace/merlin_models/models/.tox/.tmp/package/4/merlin-models-0.9.0+54.g532f6e3e.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,appdirs==1.4.4,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.14,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==20.8b1,bleach==5.0.1,bokeh==3.0.2,boto3==1.24.75,botocore==1.29.14,Brotli==1.0.9,build==0.9.0,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,check-manifest==0.48,clang==5.0,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flake8==5.0.4,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,interrogate==1.5.0,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,isort==5.10.1,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter==1.0.0,jupyter-cache==0.4.3,jupyter-console==6.4.4,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mccabe==0.7.0,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.9.0+54.g532f6e3e,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy==0.991,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathspec==0.10.2,pathtools==0.1.2,pbr==5.11.0,pep517==0.13.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycodestyle==2.9.1,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,pyflakes==2.5.0,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,qtconsole==5.4.0,QtPy==2.3.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.44,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typed-ast==1.5.4,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,xyzservices==2022.9.0,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='1968470124'
py38-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-d1cp9p1q
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-d1cp9p1q
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 025f10da5f96c9ae470ee31133801da05fabfbf9
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+19.g025f10d) (2022.5.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+19.g025f10d) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+19.g025f10d) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (3.19.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (1.10.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+19.g025f10d) (0.55.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (21.3)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (7.0.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.8.0+19.g025f10d) (2022.3.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (1.2.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.8.0+19.g025f10d) (4.64.1)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (1.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (0.12.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (2.4.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (2.0.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (1.0.4)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (6.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (1.7.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+19.g025f10d) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+19.g025f10d) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.8.0+19.g025f10d) (65.5.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.8.0+19.g025f10d) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+19.g025f10d) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+19.g025f10d) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+19.g025f10d) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.8.0+19.g025f10d) (1.52.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.8.0+19.g025f10d) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.8.0+19.g025f10d) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.8.0+19.g025f10d) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.8.0+19.g025f10d-py3-none-any.whl size=118886 sha256=7ecb6fdf152c4acf67aaa38ad818cb6630ed140cfd521d27bcd4eb3300c5cc93
  Stored in directory: /tmp/pip-ephem-wheel-cache-r6lvnsk3/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.8.0+19.g025f10d
py38-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/nvtabular.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/nvtabular.git
  Cloning https://github.com/NVIDIA-Merlin/nvtabular.git to /tmp/pip-req-build-1uv53p6o
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/nvtabular.git /tmp/pip-req-build-1uv53p6o
  Resolved https://github.com/NVIDIA-Merlin/nvtabular.git to commit e5b7351deb9e4885c4038aa0bbc9f146d8477a0e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from nvtabular==1.6.0+6.ge5b7351d) (0.8.0+19.g025f10d)
Collecting merlin-dataloader>=0.0.2
  Downloading merlin-dataloader-0.0.2.tar.gz (44 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 44.1/44.1 kB 1.3 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+6.ge5b7351d) (1.8.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.5.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.3.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.19.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.10.0)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.55.1)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (21.3)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (7.0.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.3.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.64.1)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (from scipy->nvtabular==1.6.0+6.ge5b7351d) (1.20.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.4.3)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.12.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.4.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.0.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.4)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (5.8.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.7.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (65.5.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (3.0.9)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2022.2.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.52.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+6.ge5b7351d) (6.0.1)
Building wheels for collected packages: nvtabular, merlin-dataloader
  Building wheel for nvtabular (pyproject.toml): started
  Building wheel for nvtabular (pyproject.toml): finished with status 'done'
  Created wheel for nvtabular: filename=nvtabular-1.6.0+6.ge5b7351d-cp38-cp38-linux_x86_64.whl size=257597 sha256=00b8d6d3a36e468d2a182e1c1fc98bd957f33a454608d43114610fb1112a1d77
  Stored in directory: /tmp/pip-ephem-wheel-cache-b5xzl0g6/wheels/8f/d9/f9/30f2cdc5bf8787fae6fdfe55afd6e1b493e619ec32c32ec40b
  Building wheel for merlin-dataloader (pyproject.toml): started
  Building wheel for merlin-dataloader (pyproject.toml): finished with status 'done'
  Created wheel for merlin-dataloader: filename=merlin_dataloader-0.0.2-py3-none-any.whl size=29203 sha256=69f7420bf5c647fc9a35e2a10f115fc44332cc8cd3c92d167f00eb98d82fc45a
  Stored in directory: /tmp/pip-ephem-wheel-cache-b5xzl0g6/wheels/76/ef/ed/cb880e3ef5192ec5940e26fd9442247b569fb0cf8602f97137
Successfully built nvtabular merlin-dataloader
Installing collected packages: merlin-dataloader, nvtabular
  Attempting uninstall: nvtabular
    Found existing installation: nvtabular 1.1.1
    Not uninstalling nvtabular at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'nvtabular'. No files were found to uninstall.
Successfully installed merlin-dataloader-0.0.2 nvtabular-1.6.0+6.ge5b7351d
py38-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/py38-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-3.0.2, cov-4.0.0
collected 884 items

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  0%]

tests/unit/datasets/test_ecommerce.py ..sss.s                            [  1%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  2%]

tests/unit/datasets/test_social.py .                                     [  2%]

tests/unit/datasets/test_synthetic.py ..........                         [  3%]

tests/unit/implicit/test_implicit.py .                                   [  3%]

tests/unit/lightfm/test_lightfm.py .                                     [  3%]

tests/unit/tf/test_core.py ......                                        [  4%]

tests/unit/tf/test_loader.py ................                            [  6%]

tests/unit/tf/test_public_api.py .                                       [  6%]

tests/unit/tf/blocks/test_cross.py ...........                           [  7%]

tests/unit/tf/blocks/test_dlrm.py ..........                             [  8%]

tests/unit/tf/blocks/test_interactions.py ...                            [  9%]

tests/unit/tf/blocks/test_mlp.py ....................................... [ 13%]

............................                                             [ 16%]

tests/unit/tf/blocks/test_optimizer.py s................................ [ 20%]

.....................                                                    [ 22%]

tests/unit/tf/blocks/retrieval/test_base.py .                            [ 22%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 23%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ............            [ 24%]

tests/unit/tf/blocks/sampling/test_cross_batch.py .                      [ 24%]

tests/unit/tf/blocks/sampling/test_in_batch.py .                         [ 24%]

tests/unit/tf/core/test_aggregation.py .........                         [ 25%]

tests/unit/tf/core/test_base.py ..                                       [ 26%]

tests/unit/tf/core/test_combinators.py s.....................            [ 28%]

tests/unit/tf/core/test_encoder.py ..                                    [ 28%]

tests/unit/tf/core/test_index.py ...                                     [ 29%]

tests/unit/tf/core/test_prediction.py ..                                 [ 29%]

tests/unit/tf/core/test_tabular.py ......                                [ 29%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 30%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 30%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 30%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 30%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 30%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 30%]

tests/unit/tf/examples/test_07_train_traditional_models.py .             [ 30%]

tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]

[ 30%]

tests/unit/tf/examples/test_usecase_data_parallel.py .                   [ 30%]

tests/unit/tf/examples/test_usecase_ecommerce_session_based.py .         [ 31%]

tests/unit/tf/examples/test_usecase_incremental_training_layer_freezing.py . [ 31%]

[ 31%]

tests/unit/tf/examples/test_usecase_pretrained_embeddings.py .           [ 31%]

tests/unit/tf/examples/test_usecase_retrieval_with_hpo.py .              [ 31%]

tests/unit/tf/examples/test_usecase_transformers_next_item_prediction.py . [ 31%]

[ 31%]

tests/unit/tf/horovod/test_horovod.py ...                                [ 31%]

tests/unit/tf/inputs/test_base.py .                                      [ 32%]

tests/unit/tf/inputs/test_continuous.py ........                         [ 32%]

tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]

........                                                                 [ 37%]

tests/unit/tf/inputs/test_tabular.py ..................                  [ 39%]

tests/unit/tf/layers/test_queue.py ..............                        [ 41%]

tests/unit/tf/losses/test_losses.py .......................              [ 43%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 44%]

tests/unit/tf/metrics/test_metrics_topk.py .........................     [ 47%]

tests/unit/tf/models/test_base.py s.........................             [ 50%]

tests/unit/tf/models/test_benchmark.py ..                                [ 50%]

tests/unit/tf/models/test_ranking.py ..................................  [ 54%]

tests/unit/tf/models/test_retrieval.py ................................. [ 57%]

...........................................                              [ 62%]

tests/unit/tf/outputs/test_base.py ......                                [ 63%]

tests/unit/tf/outputs/test_classification.py ......                      [ 64%]

tests/unit/tf/outputs/test_contrastive.py ..............                 [ 65%]

tests/unit/tf/outputs/test_regression.py ..                              [ 65%]

tests/unit/tf/outputs/test_sampling.py ....                              [ 66%]

tests/unit/tf/outputs/test_topk.py .                                     [ 66%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 66%]

tests/unit/tf/prediction_tasks/test_multi_task.py ................       [ 68%]

tests/unit/tf/prediction_tasks/test_next_item.py .....                   [ 69%]

tests/unit/tf/prediction_tasks/test_regression.py .....                  [ 69%]

tests/unit/tf/prediction_tasks/test_retrieval.py .                       [ 69%]

tests/unit/tf/prediction_tasks/test_sampling.py ......                   [ 70%]

tests/unit/tf/transformers/test_block.py ......................          [ 72%]

tests/unit/tf/transformers/test_transforms.py ..........                 [ 74%]

tests/unit/tf/transforms/test_bias.py ..                                 [ 74%]

tests/unit/tf/transforms/test_features.py s............................. [ 77%]

.......................s......                                           [ 81%]

tests/unit/tf/transforms/test_negative_sampling.py .........             [ 82%]

tests/unit/tf/transforms/test_noise.py .....                             [ 82%]

tests/unit/tf/transforms/test_sequence.py ....................           [ 84%]

tests/unit/tf/transforms/test_tensor.py ...                              [ 85%]

tests/unit/tf/utils/test_batch.py ....                                   [ 85%]

tests/unit/tf/utils/test_dataset.py ..                                   [ 85%]

tests/unit/tf/utils/test_tf_utils.py .....                               [ 86%]

tests/unit/torch/test_dataset.py .........                               [ 87%]

tests/unit/torch/test_public_api.py .                                    [ 87%]

tests/unit/torch/block/test_base.py ....                                 [ 88%]

tests/unit/torch/block/test_mlp.py .                                     [ 88%]

tests/unit/torch/features/test_continuous.py ..                          [ 88%]

tests/unit/torch/features/test_embedding.py ..............               [ 90%]

tests/unit/torch/features/test_tabular.py ....                           [ 90%]

tests/unit/torch/model/test_head.py ............                         [ 91%]

tests/unit/torch/model/test_model.py ..                                  [ 92%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 92%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 93%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 94%]

tests/unit/utils/test_schema_utils.py ................................   [ 97%]

tests/unit/xgb/test_xgboost.py ....................                      [100%]


=============================== warnings summary ===============================

../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "


../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_advertising.py: 1 warning

tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 10 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 6 warnings

tests/unit/tf/core/test_index.py: 8 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 38 warnings

tests/unit/tf/models/test_retrieval.py: 121 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning

tests/unit/tf/transformers/test_block.py: 16 warnings

tests/unit/tf/transforms/test_bias.py: 2 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_noise.py: 1 warning

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 9 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 3 warnings

tests/unit/xgb/test_xgboost.py: 18 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 9 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 7 warnings

tests/unit/tf/core/test_index.py: 3 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 36 warnings

tests/unit/tf/models/test_retrieval.py: 87 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/transformers/test_block.py: 10 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 7 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 2 warnings

tests/unit/xgb/test_xgboost.py: 17 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 1 warning

tests/unit/datasets/test_entertainment.py: 1 warning

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 2 warnings

tests/unit/tf/core/test_prediction.py: 1 warning

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 4 warnings

tests/unit/tf/inputs/test_embedding.py: 9 warnings

tests/unit/tf/inputs/test_tabular.py: 8 warnings

tests/unit/tf/models/test_ranking.py: 20 warnings

tests/unit/tf/models/test_retrieval.py: 10 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings

tests/unit/xgb/test_xgboost.py: 12 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:970: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning

tests/unit/tf/core/test_index.py: 4 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/models/test_retrieval.py: 59 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings

tests/unit/tf/utils/test_batch.py: 2 warnings

/tmp/autograph_generated_fileqxf83o8m.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:602: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

elif isinstance(self.feature_names, collections.Sequence):


tests/unit/tf/core/test_encoder.py: 1 warning

tests/unit/tf/core/test_index.py: 5 warnings

tests/unit/tf/models/test_retrieval.py: 30 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

tests/unit/tf/utils/test_dataset.py: 1 warning

/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.

warnings.warn(


tests/unit/tf/models/test_base.py::test_model_pre_post[True]

tests/unit/tf/models/test_base.py::test_model_pre_post[False]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]

tests/unit/tf/models/test_base.py::test_freeze_sequential_block

tests/unit/tf/models/test_base.py::test_freeze_unfreeze

tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.

super(SGD, self).init(name, **kwargs)


tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]

tests/unit/tf/transformers/test_block.py::test_retrieval_transformer[True]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))


tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]

tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

return py_builtins.overload_of(f)(*args)


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}


tests/unit/xgb/test_xgboost.py::test_without_dask_client

tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]

tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple

tests/unit/xgb/test_xgboost.py::TestEvals::test_default

tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid

tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data

/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].

warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")


tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first

self.make_current()


tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 34483 instead

warnings.warn(


tests/unit/xgb/test_xgboost.py: 14 warnings

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/xgb/test_xgboost.py: 11 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.

mask = pd.Series(mask)


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                                 Stmts   Miss  Cover


merlin/datasets/init.py                                              2      0   100%

merlin/datasets/advertising/init.py                                  2      0   100%

merlin/datasets/advertising/criteo/init.py                           0      0   100%

merlin/datasets/advertising/criteo/dataset.py                           79     49    38%

merlin/datasets/advertising/criteo/transformed/init.py               0      0   100%

merlin/datasets/ecommerce/init.py                                    4      0   100%

merlin/datasets/ecommerce/aliccp/init.py                             0      0   100%

merlin/datasets/ecommerce/aliccp/dataset.py                            141     87    38%

merlin/datasets/ecommerce/aliccp/raw/init.py                         0      0   100%

merlin/datasets/ecommerce/aliccp/transformed/init.py                 0      0   100%

merlin/datasets/ecommerce/booking/init.py                            0      0   100%

merlin/datasets/ecommerce/booking/dataset.py                           127    100    21%

merlin/datasets/ecommerce/booking/raw/init.py                        0      0   100%

merlin/datasets/ecommerce/booking/transformed/init.py                0      0   100%

merlin/datasets/ecommerce/dressipi/init.py                           0      0   100%

merlin/datasets/ecommerce/dressipi/dataset.py                           45     37    18%

merlin/datasets/ecommerce/dressipi/preprocessed/init.py              0      0   100%

merlin/datasets/ecommerce/large/init.py                              0      0   100%

merlin/datasets/ecommerce/small/init.py                              0      0   100%

merlin/datasets/ecommerce/transactions/init.py                       0      0   100%

merlin/datasets/entertainment/init.py                                2      0   100%

merlin/datasets/entertainment/movielens/1m-raw/init.py               0      0   100%

merlin/datasets/entertainment/movielens/1m-raw/ratings/init.py       0      0   100%

merlin/datasets/entertainment/movielens/1m/init.py                   0      0   100%

merlin/datasets/entertainment/movielens/25m/init.py                  0      0   100%

merlin/datasets/entertainment/movielens/100k/init.py                 0      0   100%

merlin/datasets/entertainment/movielens/init.py                      0      0   100%

merlin/datasets/entertainment/movielens/dataset.py                     152    122    20%

merlin/datasets/entertainment/music_streaming/init.py                0      0   100%

merlin/datasets/social/init.py                                       0      0   100%

merlin/datasets/synthetic.py                                           147      8    95%

merlin/datasets/testing/init.py                                      0      0   100%

merlin/datasets/testing/sequence_testing/init.py                     0      0   100%

merlin/models/init.py                                                2      0   100%

merlin/models/_version.py                                              354    205    42%

merlin/models/api.py                                                    14      5    64%

merlin/models/config/init.py                                         0      0   100%

merlin/models/config/schema.py                                          62      0   100%

merlin/models/implicit/init.py                                      27      4    85%

merlin/models/io.py                                                     15      0   100%

merlin/models/lightfm/init.py                                       23      0   100%

merlin/models/loader/init.py                                         0      0   100%

merlin/models/loader/backend.py                                        379     41    89%

merlin/models/loader/dataframe_iter.py                                  21     17    19%

merlin/models/loader/tf_utils.py                                        57     27    53%

merlin/models/loader/utils.py                                           40     15    62%

merlin/models/tf/init.py                                            70      0   100%

merlin/models/tf/blocks/init.py                                      0      0   100%

merlin/models/tf/blocks/cross.py                                        44      0   100%

merlin/models/tf/blocks/dlrm.py                                         49      2    96%

merlin/models/tf/blocks/experts.py                                      99     17    83%

merlin/models/tf/blocks/interaction.py                                 108     40    63%

merlin/models/tf/blocks/mlp.py                                         117      7    94%

merlin/models/tf/blocks/optimizer.py                                   173     12    93%

merlin/models/tf/blocks/retrieval/init.py                            0      0   100%

merlin/models/tf/blocks/retrieval/base.py                              175     62    65%

merlin/models/tf/blocks/retrieval/matrix_factorization.py               35      1    97%

merlin/models/tf/blocks/retrieval/two_tower.py                          30      0   100%

merlin/models/tf/blocks/sampling/init.py                             0      0   100%

merlin/models/tf/blocks/sampling/base.py                                29      2    93%

merlin/models/tf/blocks/sampling/cross_batch.py                         46      2    96%

merlin/models/tf/blocks/sampling/in_batch.py                            35      0   100%

merlin/models/tf/blocks/sampling/queue.py                              115     12    90%

merlin/models/tf/core/init.py                                        0      0   100%

merlin/models/tf/core/aggregation.py                                   241     45    81%

merlin/models/tf/core/base.py                                          242     50    79%

merlin/models/tf/core/combinators.py                                   426     53    88%

merlin/models/tf/core/encoder.py                                       175     28    84%

merlin/models/tf/core/index.py                                         106     16    85%

merlin/models/tf/core/prediction.py                                     50      1    98%

merlin/models/tf/core/tabular.py                                       280     29    90%

merlin/models/tf/distributed/init.py                                 0      0   100%

merlin/models/tf/distributed/backend.py                                  9      2    78%

merlin/models/tf/inputs/init.py                                      0      0   100%

merlin/models/tf/inputs/base.py                                         60     10    83%

merlin/models/tf/inputs/continuous.py                                   39      3    92%

merlin/models/tf/inputs/embedding.py                                   458     32    93%

merlin/models/tf/loader.py                                             268     94    65%

merlin/models/tf/losses/init.py                                      4      0   100%

merlin/models/tf/losses/base.py                                          9      0   100%

merlin/models/tf/losses/listwise.py                                     13      0   100%

merlin/models/tf/losses/pairwise.py                                    115      1    99%

merlin/models/tf/metrics/init.py                                     2      0   100%

merlin/models/tf/metrics/evaluation.py                                 105     48    54%

merlin/models/tf/metrics/topk.py                                       198     48    76%

merlin/models/tf/models/init.py                                      0      0   100%

merlin/models/tf/models/base.py                                        756    101    87%

merlin/models/tf/models/benchmark.py                                    16      0   100%

merlin/models/tf/models/ranking.py                                      67      3    96%

merlin/models/tf/models/retrieval.py                                    78      4    95%

merlin/models/tf/models/utils.py                                        10      1    90%

merlin/models/tf/outputs/init.py                                     0      0   100%

merlin/models/tf/outputs/base.py                                       123     17    86%

merlin/models/tf/outputs/classification.py                              91      1    99%

merlin/models/tf/outputs/contrastive.py                                147     10    93%

merlin/models/tf/outputs/regression.py                                   9      0   100%

merlin/models/tf/outputs/sampling/init.py                            0      0   100%

merlin/models/tf/outputs/sampling/base.py                               78     21    73%

merlin/models/tf/outputs/sampling/in_batch.py                           37      1    97%

merlin/models/tf/outputs/sampling/popularity.py                         27      1    96%

merlin/models/tf/outputs/topk.py                                        98      6    94%

merlin/models/tf/prediction_tasks/init.py                            0      0   100%

merlin/models/tf/prediction_tasks/base.py                              207     37    82%

merlin/models/tf/prediction_tasks/classification.py                     68     17    75%

merlin/models/tf/prediction_tasks/multi.py                               7      0   100%

merlin/models/tf/prediction_tasks/next_item.py                          59      6    90%

merlin/models/tf/prediction_tasks/regression.py                         35      2    94%

merlin/models/tf/prediction_tasks/retrieval.py                          73      3    96%

merlin/models/tf/transformers/init.py                                0      0   100%

merlin/models/tf/transformers/block.py                                 101      2    98%

merlin/models/tf/transformers/transforms.py                             63      0   100%

merlin/models/tf/transforms/init.py                                  0      0   100%

merlin/models/tf/transforms/bias.py                                    111      9    92%

merlin/models/tf/transforms/features.py                                435     36    92%

merlin/models/tf/transforms/negative_sampling.py                        77      3    96%

merlin/models/tf/transforms/noise.py                                    43      1    98%

merlin/models/tf/transforms/regularization.py                           17      1    94%

merlin/models/tf/transforms/sequence.py                                282     42    85%

merlin/models/tf/transforms/tensor.py                                  158     13    92%

merlin/models/tf/typing.py                                               7      0   100%

merlin/models/tf/utils/init.py                                       0      0   100%

merlin/models/tf/utils/batch_utils.py                                   85      4    95%

merlin/models/tf/utils/repr_utils.py                                    69      4    94%

merlin/models/tf/utils/search_utils.py                                  34     22    35%

merlin/models/tf/utils/testing_utils.py                                206     36    83%

merlin/models/tf/utils/tf_utils.py                                     209     42    80%

merlin/models/torch/init.py                                         12      0   100%

merlin/models/torch/block/init.py                                    0      0   100%

merlin/models/torch/block/base.py                                      167     32    81%

merlin/models/torch/block/mlp.py                                        38      5    87%

merlin/models/torch/dataset.py                                          68      5    93%

merlin/models/torch/features/init.py                                 0      0   100%

merlin/models/torch/features/base.py                                     4      0   100%

merlin/models/torch/features/continuous.py                              22      0   100%

merlin/models/torch/features/embedding.py                              165     12    93%

merlin/models/torch/features/tabular.py                                 65      8    88%

merlin/models/torch/losses.py                                           28     28     0%

merlin/models/torch/model/init.py                                    0      0   100%

merlin/models/torch/model/base.py                                      286     65    77%

merlin/models/torch/model/prediction_task.py                            24      0   100%

merlin/models/torch/tabular/init.py                                  0      0   100%

merlin/models/torch/tabular/aggregation.py                              75      0   100%

merlin/models/torch/tabular/base.py                                    247     39    84%

merlin/models/torch/tabular/transformations.py                          67      3    96%

merlin/models/torch/typing.py                                            6      0   100%

merlin/models/torch/utils/init.py                                    0      0   100%

merlin/models/torch/utils/data_utils.py                                117    117     0%

merlin/models/torch/utils/examples_utils.py                             39     39     0%

merlin/models/torch/utils/torch_utils.py                                80     22    72%

merlin/models/utils/init.py                                          0      0   100%

merlin/models/utils/constants.py                                         3      0   100%

merlin/models/utils/dataset.py                                          38      4    89%

merlin/models/utils/dependencies.py                                     26     19    27%

merlin/models/utils/doc_utils.py                                        10      0   100%

merlin/models/utils/example_utils.py                                    31      2    94%

merlin/models/utils/misc_utils.py                                      118     90    24%

merlin/models/utils/nvt_utils.py                                        27     24    11%

merlin/models/utils/registry.py                                        101     29    71%

merlin/models/utils/schema_utils.py                                     90     19    79%

merlin/models/xgb/init.py                                          124      4    97%


TOTAL                                                                11581   2346    80%


=========================== short test summary info ============================

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:64: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:80: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:94: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:115: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [5] ../../../.local/lib/python3.8/site-packages/tensorflow/python/framework/test_util.py:2746: Not a test.

========= 871 passed, 13 skipped, 1453 warnings in 1824.62s (0:30:24) ==========

___________________________________ summary ____________________________________

py38-gpu: commands succeeded

congratulations :)

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins16995859626760258600.sh

@nvidia-merlin-bot
Copy link

Click to view CI Results
GitHub pull request #899 of commit 6b38b9b37f1a8bfde76dc46818e7212bc2035204, no merge conflicts.
Running as SYSTEM
Setting status of 6b38b9b37f1a8bfde76dc46818e7212bc2035204 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/merlin_models/1941/ and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/899/*:refs/remotes/origin/pr/899/* # timeout=10
 > git rev-parse 6b38b9b37f1a8bfde76dc46818e7212bc2035204^{commit} # timeout=10
Checking out Revision 6b38b9b37f1a8bfde76dc46818e7212bc2035204 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6b38b9b37f1a8bfde76dc46818e7212bc2035204 # timeout=10
Commit message: "Merge branch 'main' into update-notebook-test-next-item-prediction"
 > git rev-list --no-walk 7cf0f5fbd0185d31701d8d46a994b0da45c8f66a # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins3607316891752599742.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
GLOB sdist-make: /var/jenkins_home/workspace/merlin_models/models/setup.py
py38-gpu recreate: /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
py38-gpu installdeps: -rrequirements/dev.txt, tensorflow<2.10
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu inst: /var/jenkins_home/workspace/merlin_models/models/.tox/.tmp/package/4/merlin-models-0.9.0+57.g6b38b9b3.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,appdirs==1.4.4,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.15,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==20.8b1,bleach==5.0.1,bokeh==3.0.2,boto3==1.24.75,botocore==1.29.15,Brotli==1.0.9,build==0.9.0,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,check-manifest==0.48,clang==5.0,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flake8==5.0.4,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,interrogate==1.5.0,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,isort==5.10.1,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter==1.0.0,jupyter-cache==0.4.3,jupyter-console==6.4.4,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mccabe==0.7.0,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.9.0+57.g6b38b9b3,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy==0.991,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathspec==0.10.2,pathtools==0.1.2,pbr==5.11.0,pep517==0.13.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycodestyle==2.9.1,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,pyflakes==2.5.0,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,qtconsole==5.4.0,QtPy==2.3.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.44,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typed-ast==1.5.4,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,xyzservices==2022.9.0,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='771681066'
py38-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-h4kptnih
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-h4kptnih
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 025f10da5f96c9ae470ee31133801da05fabfbf9
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+12.g025f10d) (0.55.1)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (1.10.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+12.g025f10d) (2022.3.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+12.g025f10d) (2022.3.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+12.g025f10d) (2022.5.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (7.0.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (1.2.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (3.19.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (4.64.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+12.g025f10d) (1.3.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+12.g025f10d) (21.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (0.4.3)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (0.12.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (5.4.1)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (1.7.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (6.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (8.1.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (1.0.4)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (2.0.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (5.8.0)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+12.g025f10d) (65.5.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+12.g025f10d) (0.38.1)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+12.g025f10d) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.9.0+12.g025f10d) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+12.g025f10d) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+12.g025f10d) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+12.g025f10d) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+12.g025f10d) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+12.g025f10d) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.9.0+12.g025f10d) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+12.g025f10d) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.9.0+12.g025f10d-py3-none-any.whl size=118882 sha256=ce34fb65a1c7babe18ed432c01e98d4ece10d53c40ca74972e57fdcdd69f054f
  Stored in directory: /tmp/pip-ephem-wheel-cache-h0ezo6xg/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.9.0+12.g025f10d
py38-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/nvtabular.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/nvtabular.git
  Cloning https://github.com/NVIDIA-Merlin/nvtabular.git to /tmp/pip-req-build-misg3xfj
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/nvtabular.git /tmp/pip-req-build-misg3xfj
  Resolved https://github.com/NVIDIA-Merlin/nvtabular.git to commit fc7f0b2620ae311db13ee8180cbf7226ef09d6de
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Collecting merlin-dataloader>=0.0.2
  Downloading merlin-dataloader-0.0.3.tar.gz (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.3/48.3 kB 1.5 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+8.gfc7f0b26) (1.8.1)
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from nvtabular==1.6.0+8.gfc7f0b26) (0.9.0+12.g025f10d)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (0.55.1)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.10.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2022.3.0)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2022.3.0)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2022.5.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (7.0.0)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.2.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (3.19.5)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (4.64.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.3.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (21.3)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (from scipy->nvtabular==1.6.0+8.gfc7f0b26) (1.20.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (0.4.3)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (0.12.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2.2.0)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (5.4.1)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.7.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (3.1.2)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (6.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (8.1.3)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.0.4)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2.0.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (5.8.0)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (65.5.1)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (0.38.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2022.2.1)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.52.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.2.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+8.gfc7f0b26) (6.0.1)
Building wheels for collected packages: nvtabular, merlin-dataloader
  Building wheel for nvtabular (pyproject.toml): started
  Building wheel for nvtabular (pyproject.toml): finished with status 'done'
  Created wheel for nvtabular: filename=nvtabular-1.6.0+8.gfc7f0b26-cp38-cp38-linux_x86_64.whl size=257586 sha256=f3d7ea03c77e4a281b67c8d962cb8bf14a45cd869ae19bfe911eeb99f6431de0
  Stored in directory: /tmp/pip-ephem-wheel-cache-3qtdbp_g/wheels/8f/d9/f9/30f2cdc5bf8787fae6fdfe55afd6e1b493e619ec32c32ec40b
  Building wheel for merlin-dataloader (pyproject.toml): started
  Building wheel for merlin-dataloader (pyproject.toml): finished with status 'done'
  Created wheel for merlin-dataloader: filename=merlin_dataloader-0.0.3-py3-none-any.whl size=37647 sha256=5928d934236909d45532072106cea5ebcf5f5661872cbed4d08f4396b47b3e31
  Stored in directory: /tmp/pip-ephem-wheel-cache-3qtdbp_g/wheels/1c/a3/4a/0feebb30e0c8cb7ba7046544390b43c7017a2195232f5305a1
Successfully built nvtabular merlin-dataloader
Installing collected packages: merlin-dataloader, nvtabular
  Attempting uninstall: nvtabular
    Found existing installation: nvtabular 1.1.1
    Not uninstalling nvtabular at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'nvtabular'. No files were found to uninstall.
Successfully installed merlin-dataloader-0.0.3 nvtabular-1.6.0+8.gfc7f0b26
py38-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/py38-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-3.0.2, cov-4.0.0
collected 886 items

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  0%]

tests/unit/datasets/test_ecommerce.py ..sss.s                            [  1%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  2%]

tests/unit/datasets/test_social.py .                                     [  2%]

tests/unit/datasets/test_synthetic.py ..........                         [  3%]

tests/unit/implicit/test_implicit.py .                                   [  3%]

tests/unit/lightfm/test_lightfm.py .                                     [  3%]

tests/unit/tf/test_core.py ......                                        [  4%]

tests/unit/tf/test_loader.py ................                            [  6%]

tests/unit/tf/test_public_api.py .                                       [  6%]

tests/unit/tf/blocks/test_cross.py ...........                           [  7%]

tests/unit/tf/blocks/test_dlrm.py ..........                             [  8%]

tests/unit/tf/blocks/test_interactions.py ...                            [  9%]

tests/unit/tf/blocks/test_mlp.py ....................................... [ 13%]

............................                                             [ 16%]

tests/unit/tf/blocks/test_optimizer.py s................................ [ 20%]

.....................                                                    [ 22%]

tests/unit/tf/blocks/retrieval/test_base.py .                            [ 22%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 23%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ............            [ 24%]

tests/unit/tf/blocks/sampling/test_cross_batch.py .                      [ 24%]

tests/unit/tf/blocks/sampling/test_in_batch.py .                         [ 24%]

tests/unit/tf/core/test_aggregation.py .........                         [ 25%]

tests/unit/tf/core/test_base.py ..                                       [ 25%]

tests/unit/tf/core/test_combinators.py s.....................            [ 28%]

tests/unit/tf/core/test_encoder.py ..                                    [ 28%]

tests/unit/tf/core/test_index.py ...                                     [ 29%]

tests/unit/tf/core/test_prediction.py ..                                 [ 29%]

tests/unit/tf/core/test_tabular.py ......                                [ 29%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 30%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 30%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 30%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 30%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 30%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 30%]

tests/unit/tf/examples/test_07_train_traditional_models.py .             [ 30%]

tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py F [ 30%]

[ 30%]

tests/unit/tf/examples/test_usecase_data_parallel.py .                   [ 30%]

tests/unit/tf/examples/test_usecase_ecommerce_session_based.py .         [ 31%]

tests/unit/tf/examples/test_usecase_incremental_training_layer_freezing.py F [ 31%]

[ 31%]

tests/unit/tf/examples/test_usecase_pretrained_embeddings.py .           [ 31%]

tests/unit/tf/examples/test_usecase_retrieval_with_hpo.py .              [ 31%]

tests/unit/tf/examples/test_usecase_transformers_next_item_prediction.py . [ 31%]

[ 31%]

tests/unit/tf/horovod/test_horovod.py ...                                [ 31%]

tests/unit/tf/inputs/test_base.py .                                      [ 31%]

tests/unit/tf/inputs/test_continuous.py ........                         [ 32%]

tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]

........                                                                 [ 37%]

tests/unit/tf/inputs/test_tabular.py ..................                  [ 39%]

tests/unit/tf/layers/test_queue.py ..............                        [ 41%]

tests/unit/tf/losses/test_losses.py .......................              [ 43%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 44%]

tests/unit/tf/metrics/test_metrics_topk.py .........................     [ 47%]

tests/unit/tf/models/test_base.py s.........................             [ 50%]

tests/unit/tf/models/test_benchmark.py ..                                [ 50%]

tests/unit/tf/models/test_ranking.py ..................................  [ 54%]

tests/unit/tf/models/test_retrieval.py ................................. [ 57%]

.............................................                            [ 62%]

tests/unit/tf/outputs/test_base.py ......                                [ 63%]

tests/unit/tf/outputs/test_classification.py ......                      [ 64%]

tests/unit/tf/outputs/test_contrastive.py ..............                 [ 65%]

tests/unit/tf/outputs/test_regression.py ..                              [ 66%]

tests/unit/tf/outputs/test_sampling.py ....                              [ 66%]

tests/unit/tf/outputs/test_topk.py .                                     [ 66%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 66%]

tests/unit/tf/prediction_tasks/test_multi_task.py ................       [ 68%]

tests/unit/tf/prediction_tasks/test_next_item.py .....                   [ 69%]

tests/unit/tf/prediction_tasks/test_regression.py .....                  [ 69%]

tests/unit/tf/prediction_tasks/test_retrieval.py .                       [ 69%]

tests/unit/tf/prediction_tasks/test_sampling.py ......                   [ 70%]

tests/unit/tf/transformers/test_block.py ......................          [ 73%]

tests/unit/tf/transformers/test_transforms.py ..........                 [ 74%]

tests/unit/tf/transforms/test_bias.py ..                                 [ 74%]

tests/unit/tf/transforms/test_features.py s............................. [ 77%]

.......................s......                                           [ 81%]

tests/unit/tf/transforms/test_negative_sampling.py .........             [ 82%]

tests/unit/tf/transforms/test_noise.py .....                             [ 82%]

tests/unit/tf/transforms/test_sequence.py ....................           [ 84%]

tests/unit/tf/transforms/test_tensor.py ...                              [ 85%]

tests/unit/tf/utils/test_batch.py ....                                   [ 85%]

tests/unit/tf/utils/test_dataset.py ..                                   [ 86%]

tests/unit/tf/utils/test_tf_utils.py .....                               [ 86%]

tests/unit/torch/test_dataset.py .........                               [ 87%]

tests/unit/torch/test_public_api.py .                                    [ 87%]

tests/unit/torch/block/test_base.py ....                                 [ 88%]

tests/unit/torch/block/test_mlp.py .                                     [ 88%]

tests/unit/torch/features/test_continuous.py ..                          [ 88%]

tests/unit/torch/features/test_embedding.py ..............               [ 90%]

tests/unit/torch/features/test_tabular.py ....                           [ 90%]

tests/unit/torch/model/test_head.py ............                         [ 91%]

tests/unit/torch/model/test_model.py ..                                  [ 92%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 93%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 93%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 94%]

tests/unit/utils/test_schema_utils.py ................................   [ 97%]

tests/unit/xgb/test_xgboost.py ....................                      [100%]


=================================== FAILURES ===================================

_________________ test_usecase_accelerate_training_by_lazyadam _________________


tb = <testbook.client.TestbookNotebookClient object at 0x7fa8c077c0d0>


@testbook(
    REPO_ROOT / p,
    timeout=180,
    execute=False,
)
@pytest.mark.skipif(not HAS_GPU, reason="No GPU available")
def test_usecase_accelerate_training_by_lazyadam(tb):
    tb.inject(
        """
        import os
        os.environ["NUM_ROWS"] = "1000"
        """
    )





  tb.execute()





tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py:25:




../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)




self = <testbook.client.TestbookNotebookClient object at 0x7fa8c077c0d0>

cell = {'cell_type': 'code', 'execution_count': 7, 'id': '0500ad25-29e0-40c8-85bc-6e3864107c6a', 'metadata': {'execution': {'...e_train_function_3861]']}], 'source': 'model1.compile(optimizer="adam")\nmodel1.fit(train, batch_size=1024, epochs=1)'}

cell_index = 12

exec_reply = {'buffers': [], 'content': {'ename': 'ResourceExhaustedError', 'engine_info': {'engine_id': -1, 'engine_uuid': 'ca3137...e, 'engine': 'ca313709-8df1-499e-86fb-d2858e42181b', 'started': '2022-11-23T15:16:55.909142Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           model1.compile(optimizer="adam")

E           model1.fit(train, batch_size=1024, epochs=1)

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mResourceExhaustedError�[0m                    Traceback (most recent call last)

E           Cell �[0;32mIn [7], line 2�[0m

E           �[1;32m      1�[0m model1�[38;5;241m.�[39mcompile(optimizer�[38;5;241m=�[39m�[38;5;124m"�[39m�[38;5;124madam�[39m�[38;5;124m"�[39m)

E           �[0;32m----> 2�[0m �[43mmodel1�[49m�[38;5;241;43m.�[39;49m�[43mfit�[49m�[43m(�[49m�[43mtrain�[49m�[43m,�[49m�[43m �[49m�[43mbatch_size�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m1024�[39;49m�[43m,�[49m�[43m �[49m�[43mepochs�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m1�[39;49m�[43m)�[49m

E

E           File �[0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:910�[0m, in �[0;36mBaseModel.fit�[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, train_metrics_steps, pre, **kwargs)�[0m

E           �[1;32m    907�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_reset_compile_cache()

E           �[1;32m    908�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39mtrain_pre �[38;5;241m=�[39m pre

E           �[0;32m--> 910�[0m out �[38;5;241m=�[39m �[38;5;28;43msuper�[39;49m�[43m(�[49m�[43m)�[49m�[38;5;241;43m.�[39;49m�[43mfit�[49m�[43m(�[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mfit_kwargs�[49m�[43m)�[49m

E           �[1;32m    912�[0m �[38;5;28;01mif�[39;00m pre:

E           �[1;32m    913�[0m     �[38;5;28;01mdel�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mtrain_pre

E

E           File �[0;32m~/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py:67�[0m, in �[0;36mfilter_traceback..error_handler�[0;34m(*args, **kwargs)�[0m

E           �[1;32m     65�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m �[38;5;28;01mas�[39;00m e:  �[38;5;66;03m# pylint: disable=broad-except�[39;00m

E           �[1;32m     66�[0m   filtered_tb �[38;5;241m=�[39m process_traceback_frames(e�[38;5;241m.�[39m__traceback_)

E           �[0;32m---> 67�[0m   �[38;5;28;01mraise�[39;00m e�[38;5;241m.�[39mwith_traceback(filtered_tb) �[38;5;28;01mfrom�[39;00m �[38;5;28mNone�[39m

E           �[1;32m     68�[0m �[38;5;28;01mfinally�[39;00m:

E           �[1;32m     69�[0m   �[38;5;28;01mdel�[39;00m filtered_tb

E

E           File �[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py:54�[0m, in �[0;36mquick_execute�[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)�[0m

E           �[1;32m     52�[0m �[38;5;28;01mtry�[39;00m:

E           �[1;32m     53�[0m   ctx�[38;5;241m.�[39mensure_initialized()

E           �[0;32m---> 54�[0m   tensors �[38;5;241m=�[39m pywrap_tfe�[38;5;241m.�[39mTFE_Py_Execute(ctx�[38;5;241m.�[39m_handle, device_name, op_name,

E           �[1;32m     55�[0m                                       inputs, attrs, num_outputs)

E           �[1;32m     56�[0m �[38;5;28;01mexcept�[39;00m core�[38;5;241m.�[39m_NotOkStatusException �[38;5;28;01mas�[39;00m e:

E           �[1;32m     57�[0m   �[38;5;28;01mif�[39;00m name �[38;5;129;01mis�[39;00m �[38;5;129;01mnot�[39;00m �[38;5;28;01mNone�[39;00m:

E

E           �[0;31mResourceExhaustedError�[0m: Graph execution error:

E

E           Detected at node 'Adam/Adam/update_17/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_21400/3741080137.py", line 2, in 

E                 model1.fit(train, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_17/mul_4'

E           Detected at node 'Adam/Adam/update_17/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_21400/3741080137.py", line 2, in 

E                 model1.fit(train, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_17/mul_4'

E           2 root error(s) found.

E             (0) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_17/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           	 [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E             (1) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_17/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           0 successful operations.

E           0 derived errors ignored. [Op:__inference_train_function_3861]

E           ResourceExhaustedError: Graph execution error:

E

E           Detected at node 'Adam/Adam/update_17/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_21400/3741080137.py", line 2, in 

E                 model1.fit(train, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_17/mul_4'

E           Detected at node 'Adam/Adam/update_17/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_21400/3741080137.py", line 2, in 

E                 model1.fit(train, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_17/mul_4'

E           2 root error(s) found.

E             (0) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_17/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           	 [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E             (1) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_17/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           0 successful operations.

E           0 derived errors ignored. [Op:__inference_train_function_3861]


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-11-23 15:16:48.962363: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-11-23 15:16:53.100080: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0

2022-11-23 15:16:53.100184: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 8139 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-11-23 15:16:53.100833: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1

2022-11-23 15:16:53.100893: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13875 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

2022-11-23 15:16:53.101439: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 2

2022-11-23 15:16:53.101490: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 13875 MB memory:  -> device: 2, name: Tesla P100-DGXS-16GB, pci bus id: 0000:0e:00.0, compute capability: 6.0

2022-11-23 15:16:53.102102: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 3

2022-11-23 15:16:53.102152: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 13875 MB memory:  -> device: 3, name: Tesla P100-DGXS-16GB, pci bus id: 0000:0f:00.0, compute capability: 6.0

2022-11-23 15:17:10.253256: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)

Reported by CUDA: Free memory/Total memory: 3735552/17069309952

2022-11-23 15:17:10.253314: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit:                      8534360064

InUse:                      4237262556

MaxInUse:                   4237262556

NumAllocs:                         249

MaxAllocSize:               1083564064

Reserved:                            0

PeakReserved:                        0

LargestFreeBlock:                    0


2022-11-23 15:17:10.253338: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;

2022-11-23 15:17:10.253347: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4

2022-11-23 15:17:10.253354: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 33

2022-11-23 15:17:10.253359: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8

2022-11-23 15:17:10.253365: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2

2022-11-23 15:17:10.253371: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 9

2022-11-23 15:17:10.253377: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 8

2022-11-23 15:17:10.253383: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5

2022-11-23 15:17:10.253389: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 6

2022-11-23 15:17:10.253395: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 4

2022-11-23 15:17:10.253401: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 4

2022-11-23 15:17:10.253407: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1

2022-11-23 15:17:10.253413: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4

2022-11-23 15:17:10.253439: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3

2022-11-23 15:17:10.253447: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3

2022-11-23 15:17:10.253453: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 204800, 1

2022-11-23 15:17:10.253459: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 4

2022-11-23 15:17:10.253465: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 3

2022-11-23 15:17:10.253471: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 4

2022-11-23 15:17:10.253477: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5

2022-11-23 15:17:10.253483: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4

2022-11-23 15:17:10.253489: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 3

2022-11-23 15:17:10.253495: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 3

2022-11-23 15:17:10.253501: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 5

2022-11-23 15:17:10.253507: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 3

2022-11-23 15:17:10.253513: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3

2022-11-23 15:17:10.253541: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory

2022-11-23 15:17:10.259147: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)

Reported by CUDA: Free memory/Total memory: 3735552/17069309952

2022-11-23 15:17:10.259173: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit:                      8534360064

InUse:                      4647339328

MaxInUse:                   4647339328

NumAllocs:                         267

MaxAllocSize:               1083564064

Reserved:                            0

PeakReserved:                        0

LargestFreeBlock:                    0


2022-11-23 15:17:10.259194: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;

2022-11-23 15:17:10.259203: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4

2022-11-23 15:17:10.259209: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 34

2022-11-23 15:17:10.259215: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8

2022-11-23 15:17:10.259221: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2

2022-11-23 15:17:10.259227: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 10

2022-11-23 15:17:10.259232: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10

2022-11-23 15:17:10.259238: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5

2022-11-23 15:17:10.259244: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 6

2022-11-23 15:17:10.259250: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5

2022-11-23 15:17:10.259256: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5

2022-11-23 15:17:10.259262: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1

2022-11-23 15:17:10.259268: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 5

2022-11-23 15:17:10.259274: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 61440, 3

2022-11-23 15:17:10.259280: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 65536, 3

2022-11-23 15:17:10.259286: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 204800, 1

2022-11-23 15:17:10.259305: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5

2022-11-23 15:17:10.259312: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 5

2022-11-23 15:17:10.259318: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5

2022-11-23 15:17:10.259324: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5

2022-11-23 15:17:10.259330: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 5

2022-11-23 15:17:10.259336: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 5

2022-11-23 15:17:10.259342: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 5

2022-11-23 15:17:10.259348: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 5

2022-11-23 15:17:10.259354: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 5

2022-11-23 15:17:10.259360: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3

2022-11-23 15:17:10.259369: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

_______________ test_usecase_incremental_training_layer_freezing _______________


tb = <testbook.client.TestbookNotebookClient object at 0x7fa97404f520>


@testbook(
    REPO_ROOT / p,
    timeout=180,
    execute=False,
)
def test_usecase_incremental_training_layer_freezing(tb):
    tb.inject(
        """
        import os
        os.environ["NUM_ROWS"] = "1000"
        """
    )





  tb.execute()





tests/unit/tf/examples/test_usecase_incremental_training_layer_freezing.py:22:




../../../.local/lib/python3.8/site-packages/testbook/client.py:147: in execute

super().execute_cell(cell, index)

../../../.local/lib/python3.8/site-packages/nbclient/util.py:84: in wrapped

return just_run(coro(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/nbclient/util.py:62: in just_run

return loop.run_until_complete(coro)

/usr/lib/python3.8/asyncio/base_events.py:616: in run_until_complete

return future.result()

../../../.local/lib/python3.8/site-packages/nbclient/client.py:965: in async_execute_cell

await self._check_raise_for_error(cell, cell_index, exec_reply)




self = <testbook.client.TestbookNotebookClient object at 0x7fa97404f520>

cell = {'cell_type': 'code', 'execution_count': 8, 'id': '791e06ec-c0cb-4c0f-9e41-7e5c8fa1dc4e', 'metadata': {'execution': {'...: 'model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.01))\nmodel.fit(day_1, batch_size=1024, epochs=1)'}

cell_index = 13

exec_reply = {'buffers': [], 'content': {'ename': 'ResourceExhaustedError', 'engine_info': {'engine_id': -1, 'engine_uuid': '27a48c...e, 'engine': '27a48cf0-129c-496d-b85f-de3ee8862823', 'started': '2022-11-23T15:18:47.324145Z', 'status': 'error'}, ...}


async def _check_raise_for_error(
    self, cell: NotebookNode, cell_index: int, exec_reply: t.Optional[t.Dict]
) -> None:

    if exec_reply is None:
        return None

    exec_reply_content = exec_reply['content']
    if exec_reply_content['status'] != 'error':
        return None

    cell_allows_errors = (not self.force_raise_errors) and (
        self.allow_errors
        or exec_reply_content.get('ename') in self.allow_error_names
        or "raises-exception" in cell.metadata.get("tags", [])
    )
    await run_hook(self.on_cell_error, cell=cell, cell_index=cell_index)
    if not cell_allows_errors:





      raise CellExecutionError.from_cell_and_msg(cell, exec_reply_content)





E           nbclient.exceptions.CellExecutionError: An error occurred while executing the following cell:

E           ------------------

E           model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.01))

E           model.fit(day_1, batch_size=1024, epochs=1)

E           ------------------

E

E           �[0;31m---------------------------------------------------------------------------�[0m

E           �[0;31mResourceExhaustedError�[0m                    Traceback (most recent call last)

E           Cell �[0;32mIn [8], line 2�[0m

E           �[1;32m      1�[0m model�[38;5;241m.�[39mcompile(optimizer�[38;5;241m=�[39mtf�[38;5;241m.�[39mkeras�[38;5;241m.�[39moptimizers�[38;5;241m.�[39mAdam(learning_rate�[38;5;241m=�[39m�[38;5;241m0.01�[39m))

E           �[0;32m----> 2�[0m �[43mmodel�[49m�[38;5;241;43m.�[39;49m�[43mfit�[49m�[43m(�[49m�[43mday_1�[49m�[43m,�[49m�[43m �[49m�[43mbatch_size�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m1024�[39;49m�[43m,�[49m�[43m �[49m�[43mepochs�[49m�[38;5;241;43m=�[39;49m�[38;5;241;43m1�[39;49m�[43m)�[49m

E

E           File �[0;32m~/workspace/merlin_models/models/merlin/models/tf/models/base.py:910�[0m, in �[0;36mBaseModel.fit�[0;34m(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_batch_size, validation_freq, max_queue_size, workers, use_multiprocessing, train_metrics_steps, pre, **kwargs)�[0m

E           �[1;32m    907�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39m_reset_compile_cache()

E           �[1;32m    908�[0m     �[38;5;28mself�[39m�[38;5;241m.�[39mtrain_pre �[38;5;241m=�[39m pre

E           �[0;32m--> 910�[0m out �[38;5;241m=�[39m �[38;5;28;43msuper�[39;49m�[43m(�[49m�[43m)�[49m�[38;5;241;43m.�[39;49m�[43mfit�[49m�[43m(�[49m�[38;5;241;43m�[39;49m�[38;5;241;43m�[39;49m�[43mfit_kwargs�[49m�[43m)�[49m

E           �[1;32m    912�[0m �[38;5;28;01mif�[39;00m pre:

E           �[1;32m    913�[0m     �[38;5;28;01mdel�[39;00m �[38;5;28mself�[39m�[38;5;241m.�[39mtrain_pre

E

E           File �[0;32m~/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py:67�[0m, in �[0;36mfilter_traceback..error_handler�[0;34m(*args, **kwargs)�[0m

E           �[1;32m     65�[0m �[38;5;28;01mexcept�[39;00m �[38;5;167;01mException�[39;00m �[38;5;28;01mas�[39;00m e:  �[38;5;66;03m# pylint: disable=broad-except�[39;00m

E           �[1;32m     66�[0m   filtered_tb �[38;5;241m=�[39m process_traceback_frames(e�[38;5;241m.�[39m__traceback_)

E           �[0;32m---> 67�[0m   �[38;5;28;01mraise�[39;00m e�[38;5;241m.�[39mwith_traceback(filtered_tb) �[38;5;28;01mfrom�[39;00m �[38;5;28mNone�[39m

E           �[1;32m     68�[0m �[38;5;28;01mfinally�[39;00m:

E           �[1;32m     69�[0m   �[38;5;28;01mdel�[39;00m filtered_tb

E

E           File �[0;32m~/.local/lib/python3.8/site-packages/tensorflow/python/eager/execute.py:54�[0m, in �[0;36mquick_execute�[0;34m(op_name, num_outputs, inputs, attrs, ctx, name)�[0m

E           �[1;32m     52�[0m �[38;5;28;01mtry�[39;00m:

E           �[1;32m     53�[0m   ctx�[38;5;241m.�[39mensure_initialized()

E           �[0;32m---> 54�[0m   tensors �[38;5;241m=�[39m pywrap_tfe�[38;5;241m.�[39mTFE_Py_Execute(ctx�[38;5;241m.�[39m_handle, device_name, op_name,

E           �[1;32m     55�[0m                                       inputs, attrs, num_outputs)

E           �[1;32m     56�[0m �[38;5;28;01mexcept�[39;00m core�[38;5;241m.�[39m_NotOkStatusException �[38;5;28;01mas�[39;00m e:

E           �[1;32m     57�[0m   �[38;5;28;01mif�[39;00m name �[38;5;129;01mis�[39;00m �[38;5;129;01mnot�[39;00m �[38;5;28;01mNone�[39;00m:

E

E           �[0;31mResourceExhaustedError�[0m: Graph execution error:

E

E           Detected at node 'Adam/Adam/update_19/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_22194/2071866865.py", line 2, in 

E                 model.fit(day_1, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_19/mul_4'

E           Detected at node 'Adam/Adam/update_19/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_22194/2071866865.py", line 2, in 

E                 model.fit(day_1, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_19/mul_4'

E           2 root error(s) found.

E             (0) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_19/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           	 [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E             (1) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_19/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           0 successful operations.

E           0 derived errors ignored. [Op:__inference_train_function_4007]

E           ResourceExhaustedError: Graph execution error:

E

E           Detected at node 'Adam/Adam/update_19/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_22194/2071866865.py", line 2, in 

E                 model.fit(day_1, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_19/mul_4'

E           Detected at node 'Adam/Adam/update_19/mul_4' defined at (most recent call last):

E               File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main

E                 return _run_code(code, main_globals, None,

E               File "/usr/lib/python3.8/runpy.py", line 87, in _run_code

E                 exec(code, run_globals)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel_launcher.py", line 17, in 

E                 app.launch_new_instance()

E               File "/usr/local/lib/python3.8/dist-packages/traitlets/config/application.py", line 978, in launch_instance

E                 app.start()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelapp.py", line 712, in start

E                 self.io_loop.start()

E               File "/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/platform/asyncio.py", line 215, in start

E                 self.asyncio_loop.run_forever()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever

E                 self._run_once()

E               File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once

E                 handle._run()

E               File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run

E                 self._context.run(self._callback, *self._args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 510, in dispatch_queue

E                 await self.process_one()

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 499, in process_one

E                 await dispatch(*args)

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 406, in dispatch_shell

E                 await result

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/kernelbase.py", line 730, in execute_request

E                 reply_content = await reply_content

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/ipkernel.py", line 383, in do_execute

E                 res = shell.run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/ipykernel/zmqshell.py", line 528, in run_cell

E                 return super().run_cell(*args, **kwargs)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2885, in run_cell

E                 result = self._run_cell(

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 2940, in _run_cell

E                 return runner(coro)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/async_helpers.py", line 129, in pseudo_sync_runner

E                 coro.send(None)

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3139, in run_cell_async

E                 has_raised = await self.run_ast_nodes(code_ast.body, cell_name,

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3318, in run_ast_nodes

E                 if await self.run_code(code, result, async=asy):

E               File "/usr/local/lib/python3.8/dist-packages/IPython/core/interactiveshell.py", line 3378, in run_code

E                 exec(code_obj, self.user_global_ns, self.user_ns)

E               File "/tmp/ipykernel_22194/2071866865.py", line 2, in 

E                 model.fit(day_1, batch_size=1024, epochs=1)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 910, in fit

E                 out = super().fit(**fit_kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 64, in error_handler

E                 return fn(*args, **kwargs)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1409, in fit

E                 tmp_logs = self.train_function(iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1051, in train_function

E                 return step_function(self, iterator)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1040, in step_function

E                 outputs = model.distribute_strategy.run(run_step, args=(data,))

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/engine/training.py", line 1030, in run_step

E                 outputs = model.train_step(data)

E               File "/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/models/base.py", line 724, in train_step

E                 self.optimizer.minimize(loss, self.trainable_variables, tape=tape)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 539, in minimize

E                 return self.apply_gradients(grads_and_vars, name=name)

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 678, in apply_gradients

E                 return tf.internal.distribute.interim.maybe_merge_call(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 723, in _distributed_apply

E                 update_op = distribution.extended.update(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 701, in apply_grad_to_update_var

E                 return self._resource_apply_sparse_duplicate_indices(

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/optimizer_v2.py", line 1326, in _resource_apply_sparse_duplicate_indices

E                 return self._resource_apply_sparse(summed_grad, handle, unique_indices,

E               File "/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/adam.py", line 214, in _resource_apply_sparse

E                 v_t = tf.compat.v1.assign(v, v * coefficients['beta_2_t'],

E           Node: 'Adam/Adam/update_19/mul_4'

E           2 root error(s) found.

E             (0) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_19/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           	 [[StatefulPartitionedCall/cond/pivot_t/_131/_53]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E             (1) RESOURCE_EXHAUSTED:  failed to allocate memory

E           	 [[{{node Adam/Adam/update_19/mul_4}}]]

E           Hint: If you want to see a list of allocated tensors when OOM happens, add report_tensor_allocations_upon_oom to RunOptions for current allocation info. This isn't available when running in Eager mode.

E

E           0 successful operations.

E           0 derived errors ignored. [Op:__inference_train_function_4007]


../../../.local/lib/python3.8/site-packages/nbclient/client.py:862: CellExecutionError

----------------------------- Captured stderr call -----------------------------

2022-11-23 15:18:40.324297: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

2022-11-23 15:18:44.494201: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 0

2022-11-23 15:18:44.494305: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:0 with 8139 MB memory:  -> device: 0, name: Tesla P100-DGXS-16GB, pci bus id: 0000:07:00.0, compute capability: 6.0

2022-11-23 15:18:44.495064: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 1

2022-11-23 15:18:44.495124: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:1 with 13875 MB memory:  -> device: 1, name: Tesla P100-DGXS-16GB, pci bus id: 0000:08:00.0, compute capability: 6.0

2022-11-23 15:18:44.495732: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 2

2022-11-23 15:18:44.495783: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:2 with 13875 MB memory:  -> device: 2, name: Tesla P100-DGXS-16GB, pci bus id: 0000:0e:00.0, compute capability: 6.0

2022-11-23 15:18:44.496392: I tensorflow/core/common_runtime/gpu/gpu_process_state.cc:222] Using CUDA malloc Async allocator for GPU: 3

2022-11-23 15:18:44.496444: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1532] Created device /job:localhost/replica:0/task:0/device:GPU:3 with 13875 MB memory:  -> device: 3, name: Tesla P100-DGXS-16GB, pci bus id: 0000:0f:00.0, compute capability: 6.0

2022-11-23 15:19:01.779291: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)

Reported by CUDA: Free memory/Total memory: 3735552/17069309952

2022-11-23 15:19:01.779355: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit:                      8534360064

InUse:                      4264588520

MaxInUse:                   4264588520

NumAllocs:                         280

MaxAllocSize:               1083564064

Reserved:                            0

PeakReserved:                        0

LargestFreeBlock:                    0


2022-11-23 15:19:01.779378: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;

2022-11-23 15:19:01.779389: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4

2022-11-23 15:19:01.779398: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 36

2022-11-23 15:19:01.779406: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8

2022-11-23 15:19:01.779415: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2

2022-11-23 15:19:01.779423: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 8

2022-11-23 15:19:01.779431: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10

2022-11-23 15:19:01.779440: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 3

2022-11-23 15:19:01.779448: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 6

2022-11-23 15:19:01.779456: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5

2022-11-23 15:19:01.779464: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5

2022-11-23 15:19:01.779472: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 512, 6

2022-11-23 15:19:01.779481: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1

2022-11-23 15:19:01.779516: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4

2022-11-23 15:19:01.779527: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 32768, 6

2022-11-23 15:19:01.779535: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 84480, 1

2022-11-23 15:19:01.779543: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 122880, 3

2022-11-23 15:19:01.779551: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 131072, 3

2022-11-23 15:19:01.779560: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5

2022-11-23 15:19:01.779568: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 3

2022-11-23 15:19:01.779576: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5

2022-11-23 15:19:01.779584: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5

2022-11-23 15:19:01.779592: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 3

2022-11-23 15:19:01.779600: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 5

2022-11-23 15:19:01.779609: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 3

2022-11-23 15:19:01.779617: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 5

2022-11-23 15:19:01.779625: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 3

2022-11-23 15:19:01.779633: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3

2022-11-23 15:19:01.779661: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory

2022-11-23 15:19:01.780938: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:288] gpu_async_0 cuMemAllocAsync failed to allocate 1083564064 bytes: CUDA error: out of memory (CUDA_ERROR_OUT_OF_MEMORY)

Reported by CUDA: Free memory/Total memory: 3735552/17069309952

2022-11-23 15:19:01.780963: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:293] Stats: Limit:                      8534360064

InUse:                      4279990600

MaxInUse:                   4279990600

NumAllocs:                         285

MaxAllocSize:               1083564064

Reserved:                            0

PeakReserved:                        0

LargestFreeBlock:                    0


2022-11-23 15:19:01.780980: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:56] Histogram of current allocation: (allocation_size_in_bytes, nb_allocation_of_that_sizes), ...;

2022-11-23 15:19:01.780988: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1, 4

2022-11-23 15:19:01.780995: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4, 36

2022-11-23 15:19:01.781002: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 8, 8

2022-11-23 15:19:01.781009: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 40, 2

2022-11-23 15:19:01.781016: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 128, 10

2022-11-23 15:19:01.781023: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 160, 10

2022-11-23 15:19:01.781029: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 192, 5

2022-11-23 15:19:01.781036: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 256, 6

2022-11-23 15:19:01.781043: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 288, 5

2022-11-23 15:19:01.781050: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 480, 5

2022-11-23 15:19:01.781057: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 512, 6

2022-11-23 15:19:01.781063: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1028, 1

2022-11-23 15:19:01.781070: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 3168, 4

2022-11-23 15:19:01.781091: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 32768, 6

2022-11-23 15:19:01.781099: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 84480, 1

2022-11-23 15:19:01.781106: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 122880, 3

2022-11-23 15:19:01.781113: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 131072, 3

2022-11-23 15:19:01.781120: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 584352, 5

2022-11-23 15:19:01.781126: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 823872, 3

2022-11-23 15:19:01.781133: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 4324736, 5

2022-11-23 15:19:01.781140: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 7426048, 5

2022-11-23 15:19:01.781146: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 15401440, 4

2022-11-23 15:19:01.781153: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 18678720, 5

2022-11-23 15:19:01.781160: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 39970560, 3

2022-11-23 15:19:01.781167: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 56589504, 5

2022-11-23 15:19:01.781173: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 135407776, 3

2022-11-23 15:19:01.781180: E tensorflow/core/common_runtime/gpu/gpu_cudamallocasync_allocator.cc:59] 1083564064, 3

2022-11-23 15:19:01.781190: W tensorflow/core/framework/op_kernel.cc:1733] RESOURCE_EXHAUSTED: failed to allocate memory

Error in atexit._run_exitfuncs:

Traceback (most recent call last):

File "/usr/lib/python3.8/logging/init.py", line 2127, in shutdown

h.close()

File "/usr/local/lib/python3.8/dist-packages/absl/logging/init.py", line 934, in close

self.stream.close()

File "/usr/local/lib/python3.8/dist-packages/ipykernel/iostream.py", line 438, in close

self.watch_fd_thread.join()

AttributeError: 'OutStream' object has no attribute 'watch_fd_thread'

=============================== warnings summary ===============================

../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "


../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_advertising.py: 1 warning

tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 10 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 6 warnings

tests/unit/tf/core/test_index.py: 8 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 38 warnings

tests/unit/tf/models/test_retrieval.py: 123 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning

tests/unit/tf/transformers/test_block.py: 16 warnings

tests/unit/tf/transforms/test_bias.py: 2 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_noise.py: 1 warning

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 9 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 3 warnings

tests/unit/xgb/test_xgboost.py: 18 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 9 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 7 warnings

tests/unit/tf/core/test_index.py: 3 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 36 warnings

tests/unit/tf/models/test_retrieval.py: 89 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/transformers/test_block.py: 10 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 7 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 2 warnings

tests/unit/xgb/test_xgboost.py: 17 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 1 warning

tests/unit/datasets/test_entertainment.py: 1 warning

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 2 warnings

tests/unit/tf/core/test_prediction.py: 1 warning

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 4 warnings

tests/unit/tf/inputs/test_embedding.py: 9 warnings

tests/unit/tf/inputs/test_tabular.py: 8 warnings

tests/unit/tf/models/test_ranking.py: 20 warnings

tests/unit/tf/models/test_retrieval.py: 10 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings

tests/unit/xgb/test_xgboost.py: 12 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:970: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning

tests/unit/tf/core/test_index.py: 4 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/models/test_retrieval.py: 63 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings

tests/unit/tf/utils/test_batch.py: 2 warnings

/tmp/autograph_generated_fileocq71bot.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:602: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

elif isinstance(self.feature_names, collections.Sequence):


tests/unit/tf/core/test_encoder.py: 1 warning

tests/unit/tf/core/test_index.py: 5 warnings

tests/unit/tf/models/test_retrieval.py: 30 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

tests/unit/tf/utils/test_dataset.py: 1 warning

/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.

warnings.warn(


tests/unit/tf/models/test_base.py::test_model_pre_post[True]

tests/unit/tf/models/test_base.py::test_model_pre_post[False]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]

tests/unit/tf/models/test_base.py::test_freeze_sequential_block

tests/unit/tf/models/test_base.py::test_freeze_unfreeze

tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.

super(SGD, self).init(name, **kwargs)


tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]

tests/unit/tf/transformers/test_block.py::test_retrieval_transformer[True]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))


tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]

tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

return py_builtins.overload_of(f)(*args)


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}


tests/unit/xgb/test_xgboost.py::test_without_dask_client

tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]

tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple

tests/unit/xgb/test_xgboost.py::TestEvals::test_default

tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid

tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data

/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].

warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")


tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first

self.make_current()


tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective

/var/jenkins_home/.local/lib/python3.8/site-packages/distributed/node.py:180: UserWarning: Port 8787 is already in use.

Perhaps you already have a cluster running?

Hosting the HTTP server on port 38833 instead

warnings.warn(


tests/unit/xgb/test_xgboost.py: 14 warnings

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/xgb/test_xgboost.py: 11 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.

mask = pd.Series(mask)


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                                 Stmts   Miss  Cover


merlin/datasets/init.py                                              2      0   100%

merlin/datasets/advertising/init.py                                  2      0   100%

merlin/datasets/advertising/criteo/init.py                           0      0   100%

merlin/datasets/advertising/criteo/dataset.py                           79     49    38%

merlin/datasets/advertising/criteo/transformed/init.py               0      0   100%

merlin/datasets/ecommerce/init.py                                    4      0   100%

merlin/datasets/ecommerce/aliccp/init.py                             0      0   100%

merlin/datasets/ecommerce/aliccp/dataset.py                            141     87    38%

merlin/datasets/ecommerce/aliccp/raw/init.py                         0      0   100%

merlin/datasets/ecommerce/aliccp/transformed/init.py                 0      0   100%

merlin/datasets/ecommerce/booking/init.py                            0      0   100%

merlin/datasets/ecommerce/booking/dataset.py                           127    100    21%

merlin/datasets/ecommerce/booking/raw/init.py                        0      0   100%

merlin/datasets/ecommerce/booking/transformed/init.py                0      0   100%

merlin/datasets/ecommerce/dressipi/init.py                           0      0   100%

merlin/datasets/ecommerce/dressipi/dataset.py                           45     37    18%

merlin/datasets/ecommerce/dressipi/preprocessed/init.py              0      0   100%

merlin/datasets/ecommerce/large/init.py                              0      0   100%

merlin/datasets/ecommerce/small/init.py                              0      0   100%

merlin/datasets/ecommerce/transactions/init.py                       0      0   100%

merlin/datasets/entertainment/init.py                                2      0   100%

merlin/datasets/entertainment/movielens/1m-raw/init.py               0      0   100%

merlin/datasets/entertainment/movielens/1m-raw/ratings/init.py       0      0   100%

merlin/datasets/entertainment/movielens/1m/init.py                   0      0   100%

merlin/datasets/entertainment/movielens/25m/init.py                  0      0   100%

merlin/datasets/entertainment/movielens/100k/init.py                 0      0   100%

merlin/datasets/entertainment/movielens/init.py                      0      0   100%

merlin/datasets/entertainment/movielens/dataset.py                     152    122    20%

merlin/datasets/entertainment/music_streaming/init.py                0      0   100%

merlin/datasets/social/init.py                                       0      0   100%

merlin/datasets/synthetic.py                                           147      8    95%

merlin/datasets/testing/init.py                                      0      0   100%

merlin/datasets/testing/sequence_testing/init.py                     0      0   100%

merlin/models/init.py                                                2      0   100%

merlin/models/_version.py                                              354    205    42%

merlin/models/api.py                                                    14      5    64%

merlin/models/config/init.py                                         0      0   100%

merlin/models/config/schema.py                                          62      0   100%

merlin/models/implicit/init.py                                      27      4    85%

merlin/models/io.py                                                     15      0   100%

merlin/models/lightfm/init.py                                       23      0   100%

merlin/models/loader/init.py                                         0      0   100%

merlin/models/loader/backend.py                                        379     41    89%

merlin/models/loader/dataframe_iter.py                                  21     17    19%

merlin/models/loader/tf_utils.py                                        57     27    53%

merlin/models/loader/utils.py                                           40     15    62%

merlin/models/tf/init.py                                            70      0   100%

merlin/models/tf/blocks/init.py                                      0      0   100%

merlin/models/tf/blocks/cross.py                                        44      0   100%

merlin/models/tf/blocks/dlrm.py                                         49      2    96%

merlin/models/tf/blocks/experts.py                                      99     17    83%

merlin/models/tf/blocks/interaction.py                                 108     40    63%

merlin/models/tf/blocks/mlp.py                                         117      7    94%

merlin/models/tf/blocks/optimizer.py                                   173     12    93%

merlin/models/tf/blocks/retrieval/init.py                            0      0   100%

merlin/models/tf/blocks/retrieval/base.py                              175     62    65%

merlin/models/tf/blocks/retrieval/matrix_factorization.py               35      1    97%

merlin/models/tf/blocks/retrieval/two_tower.py                          30      0   100%

merlin/models/tf/blocks/sampling/init.py                             0      0   100%

merlin/models/tf/blocks/sampling/base.py                                29      2    93%

merlin/models/tf/blocks/sampling/cross_batch.py                         46      2    96%

merlin/models/tf/blocks/sampling/in_batch.py                            35      0   100%

merlin/models/tf/blocks/sampling/queue.py                              115     12    90%

merlin/models/tf/core/init.py                                        0      0   100%

merlin/models/tf/core/aggregation.py                                   241     45    81%

merlin/models/tf/core/base.py                                          242     50    79%

merlin/models/tf/core/combinators.py                                   426     54    87%

merlin/models/tf/core/encoder.py                                       175     28    84%

merlin/models/tf/core/index.py                                         106     16    85%

merlin/models/tf/core/prediction.py                                     50      1    98%

merlin/models/tf/core/tabular.py                                       280     32    89%

merlin/models/tf/distributed/init.py                                 0      0   100%

merlin/models/tf/distributed/backend.py                                  9      2    78%

merlin/models/tf/inputs/init.py                                      0      0   100%

merlin/models/tf/inputs/base.py                                         64     10    84%

merlin/models/tf/inputs/continuous.py                                   39      3    92%

merlin/models/tf/inputs/embedding.py                                   458     32    93%

merlin/models/tf/loader.py                                             268     94    65%

merlin/models/tf/losses/init.py                                      4      0   100%

merlin/models/tf/losses/base.py                                          9      0   100%

merlin/models/tf/losses/listwise.py                                     13      0   100%

merlin/models/tf/losses/pairwise.py                                    115      1    99%

merlin/models/tf/metrics/init.py                                     2      0   100%

merlin/models/tf/metrics/evaluation.py                                 105     48    54%

merlin/models/tf/metrics/topk.py                                       198     48    76%

merlin/models/tf/models/init.py                                      0      0   100%

merlin/models/tf/models/base.py                                        758    103    86%

merlin/models/tf/models/benchmark.py                                    16      0   100%

merlin/models/tf/models/ranking.py                                      67      3    96%

merlin/models/tf/models/retrieval.py                                    78      4    95%

merlin/models/tf/models/utils.py                                        10      1    90%

merlin/models/tf/outputs/init.py                                     0      0   100%

merlin/models/tf/outputs/base.py                                       123     17    86%

merlin/models/tf/outputs/classification.py                              91      1    99%

merlin/models/tf/outputs/contrastive.py                                147     10    93%

merlin/models/tf/outputs/regression.py                                   9      0   100%

merlin/models/tf/outputs/sampling/init.py                            0      0   100%

merlin/models/tf/outputs/sampling/base.py                               78     21    73%

merlin/models/tf/outputs/sampling/in_batch.py                           37      1    97%

merlin/models/tf/outputs/sampling/popularity.py                         27      1    96%

merlin/models/tf/outputs/topk.py                                        98      6    94%

merlin/models/tf/prediction_tasks/init.py                            0      0   100%

merlin/models/tf/prediction_tasks/base.py                              207     37    82%

merlin/models/tf/prediction_tasks/classification.py                     68     17    75%

merlin/models/tf/prediction_tasks/multi.py                               7      0   100%

merlin/models/tf/prediction_tasks/next_item.py                          59      6    90%

merlin/models/tf/prediction_tasks/regression.py                         35      2    94%

merlin/models/tf/prediction_tasks/retrieval.py                          73      3    96%

merlin/models/tf/transformers/init.py                                0      0   100%

merlin/models/tf/transformers/block.py                                 101      2    98%

merlin/models/tf/transformers/transforms.py                             63      0   100%

merlin/models/tf/transforms/init.py                                  0      0   100%

merlin/models/tf/transforms/bias.py                                    111      9    92%

merlin/models/tf/transforms/features.py                                435     36    92%

merlin/models/tf/transforms/negative_sampling.py                        77      3    96%

merlin/models/tf/transforms/noise.py                                    43      1    98%

merlin/models/tf/transforms/regularization.py                           17      1    94%

merlin/models/tf/transforms/sequence.py                                282     42    85%

merlin/models/tf/transforms/tensor.py                                  158     13    92%

merlin/models/tf/typing.py                                               7      0   100%

merlin/models/tf/utils/init.py                                       0      0   100%

merlin/models/tf/utils/batch_utils.py                                   85      4    95%

merlin/models/tf/utils/repr_utils.py                                    69      7    90%

merlin/models/tf/utils/search_utils.py                                  34     22    35%

merlin/models/tf/utils/testing_utils.py                                206     36    83%

merlin/models/tf/utils/tf_utils.py                                     209     42    80%

merlin/models/torch/init.py                                         12      0   100%

merlin/models/torch/block/init.py                                    0      0   100%

merlin/models/torch/block/base.py                                      167     32    81%

merlin/models/torch/block/mlp.py                                        38      5    87%

merlin/models/torch/dataset.py                                          68      5    93%

merlin/models/torch/features/init.py                                 0      0   100%

merlin/models/torch/features/base.py                                     4      0   100%

merlin/models/torch/features/continuous.py                              22      0   100%

merlin/models/torch/features/embedding.py                              165     12    93%

merlin/models/torch/features/tabular.py                                 65      8    88%

merlin/models/torch/losses.py                                           28     28     0%

merlin/models/torch/model/init.py                                    0      0   100%

merlin/models/torch/model/base.py                                      286     65    77%

merlin/models/torch/model/prediction_task.py                            24      0   100%

merlin/models/torch/tabular/init.py                                  0      0   100%

merlin/models/torch/tabular/aggregation.py                              75      0   100%

merlin/models/torch/tabular/base.py                                    247     39    84%

merlin/models/torch/tabular/transformations.py                          67      3    96%

merlin/models/torch/typing.py                                            6      0   100%

merlin/models/torch/utils/init.py                                    0      0   100%

merlin/models/torch/utils/data_utils.py                                117    117     0%

merlin/models/torch/utils/examples_utils.py                             39     39     0%

merlin/models/torch/utils/torch_utils.py                                80     22    72%

merlin/models/utils/init.py                                          0      0   100%

merlin/models/utils/constants.py                                         3      0   100%

merlin/models/utils/dataset.py                                          38      4    89%

merlin/models/utils/dependencies.py                                     26     19    27%

merlin/models/utils/doc_utils.py                                        10      0   100%

merlin/models/utils/example_utils.py                                    31      2    94%

merlin/models/utils/misc_utils.py                                      118     90    24%

merlin/models/utils/nvt_utils.py                                        27     24    11%

merlin/models/utils/registry.py                                        101     29    71%

merlin/models/utils/schema_utils.py                                     90     19    79%

merlin/models/xgb/init.py                                          124      4    97%


TOTAL                                                                11587   2355    80%


=========================== short test summary info ============================

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:64: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:80: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:94: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:115: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [5] ../../../.local/lib/python3.8/site-packages/tensorflow/python/framework/test_util.py:2746: Not a test.

==== 2 failed, 871 passed, 13 skipped, 1461 warnings in 1790.83s (0:29:50) =====

ERROR: InvocationError for command /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/bin/python -m pytest --cov-report term --cov merlin -rxs tests/unit (exited with code 1)

___________________________________ summary ____________________________________

ERROR:   py38-gpu: commands failed

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins17290709220691557620.sh

@rnyak
Copy link
Contributor

rnyak commented Nov 23, 2022

rerun tests

@rnyak rnyak added this to the Merlin 22.12 milestone Nov 23, 2022
@nvidia-merlin-bot
Copy link

Click to view CI Results
GitHub pull request #899 of commit 6b38b9b37f1a8bfde76dc46818e7212bc2035204, no merge conflicts.
Running as SYSTEM
Setting status of 6b38b9b37f1a8bfde76dc46818e7212bc2035204 to PENDING with url http://merlin-infra1.nvidia.com:8080/job/merlin_models/1945/ and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_models
using credential nvidia-merlin-bot
 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/models/ # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/models/
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/models/ +refs/pull/899/*:refs/remotes/origin/pr/899/* # timeout=10
 > git rev-parse 6b38b9b37f1a8bfde76dc46818e7212bc2035204^{commit} # timeout=10
Checking out Revision 6b38b9b37f1a8bfde76dc46818e7212bc2035204 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 6b38b9b37f1a8bfde76dc46818e7212bc2035204 # timeout=10
Commit message: "Merge branch 'main' into update-notebook-test-next-item-prediction"
 > git rev-list --no-walk 6602b84c88b2d669ae411533f271fd399a31385e # timeout=10
[merlin_models] $ /bin/bash /tmp/jenkins1554438950151999798.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: testbook in /usr/local/lib/python3.8/dist-packages (0.4.2)
Requirement already satisfied: nbclient>=0.4.0 in /usr/local/lib/python3.8/dist-packages (from testbook) (0.6.8)
Requirement already satisfied: nbformat>=5.0.4 in /usr/local/lib/python3.8/dist-packages (from testbook) (5.5.0)
Requirement already satisfied: jupyter-client>=6.1.5 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (7.3.5)
Requirement already satisfied: traitlets>=5.2.2 in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (5.4.0)
Requirement already satisfied: nest-asyncio in /usr/local/lib/python3.8/dist-packages (from nbclient>=0.4.0->testbook) (1.5.5)
Requirement already satisfied: jupyter_core in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.11.1)
Requirement already satisfied: jsonschema>=2.6 in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (4.16.0)
Requirement already satisfied: fastjsonschema in /usr/local/lib/python3.8/dist-packages (from nbformat>=5.0.4->testbook) (2.16.1)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (0.18.1)
Requirement already satisfied: attrs>=17.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (22.1.0)
Requirement already satisfied: pkgutil-resolve-name>=1.3.10 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (1.3.10)
Requirement already satisfied: importlib-resources>=1.4.0 in /usr/local/lib/python3.8/dist-packages (from jsonschema>=2.6->nbformat>=5.0.4->testbook) (5.9.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (2.8.2)
Requirement already satisfied: pyzmq>=23.0 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (24.0.0)
Requirement already satisfied: entrypoints in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (0.4)
Requirement already satisfied: tornado>=6.2 in /usr/local/lib/python3.8/dist-packages (from jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (6.2)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from importlib-resources>=1.4.0->jsonschema>=2.6->nbformat>=5.0.4->testbook) (3.8.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.8.2->jupyter-client>=6.1.5->nbclient>=0.4.0->testbook) (1.15.0)
GLOB sdist-make: /var/jenkins_home/workspace/merlin_models/models/setup.py
py38-gpu recreate: /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
py38-gpu installdeps: -rrequirements/dev.txt, tensorflow<2.10
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu inst: /var/jenkins_home/workspace/merlin_models/models/.tox/.tmp/package/4/merlin-models-0.9.0+57.g6b38b9b3.zip
WARNING: Discarding $PYTHONPATH from environment, to override specify PYTHONPATH in 'passenv' in your configuration.
py38-gpu installed: absl-py==1.2.0,aiohttp==3.8.1,aiosignal==1.2.0,alabaster==0.7.12,alembic==1.8.1,anyio==3.6.1,appdirs==1.4.4,argon2-cffi==21.3.0,argon2-cffi-bindings==21.2.0,astroid==2.5.6,asttokens==2.0.8,astunparse==1.6.3,asv==0.5.1,asvdb==0.4.2,async-timeout==4.0.2,attrs==22.1.0,autopage==0.5.1,awscli==1.27.15,Babel==2.10.3,backcall==0.2.0,beautifulsoup4==4.11.1,betterproto==1.2.5,black==20.8b1,bleach==5.0.1,bokeh==3.0.2,boto3==1.24.75,botocore==1.29.15,Brotli==1.0.9,build==0.9.0,cachetools==5.2.0,certifi==2019.11.28,cffi==1.15.1,chardet==3.0.4,charset-normalizer==2.1.1,check-manifest==0.48,clang==5.0,click==8.1.3,cliff==4.1.0,cloudpickle==2.2.0,cmaes==0.9.0,cmake==3.24.1.1,cmd2==2.4.2,colorama==0.4.4,colorlog==6.7.0,contourpy==1.0.5,coverage==6.5.0,cuda-python==11.7.1,cupy-cuda117==10.6.0,cycler==0.11.0,Cython==0.29.32,dask==2022.1.1,dbus-python==1.2.16,debugpy==1.6.3,decorator==5.1.1,defusedxml==0.7.1,dill==0.3.5.1,distlib==0.3.6,distributed==2022.5.1,distro==1.7.0,dm-tree==0.1.6,docker-pycreds==0.4.0,docutils==0.16,emoji==1.7.0,entrypoints==0.4,execnet==1.9.0,executing==1.0.0,faiss==1.7.2,faiss-gpu==1.7.2,fastai==2.7.9,fastapi==0.85.0,fastavro==1.6.1,fastcore==1.5.27,fastdownload==0.0.7,fastjsonschema==2.16.1,fastprogress==1.0.3,fastrlock==0.8,feast==0.19.4,fiddle==0.2.2,filelock==3.8.0,flake8==5.0.4,flatbuffers==1.12,fonttools==4.37.3,frozenlist==1.3.1,fsspec==2022.5.0,gast==0.4.0,gevent==21.12.0,geventhttpclient==2.0.2,gitdb==4.0.9,GitPython==3.1.27,google==3.0.0,google-api-core==2.10.1,google-auth==2.11.1,google-auth-oauthlib==0.4.6,google-pasta==0.2.0,googleapis-common-protos==1.52.0,graphviz==0.20.1,greenlet==1.1.3,grpcio==1.41.0,grpcio-channelz==1.49.0,grpcio-reflection==1.48.1,grpclib==0.4.3,h11==0.13.0,h2==4.1.0,h5py==3.7.0,HeapDict==1.0.1,horovod==0.26.1,hpack==4.0.0,httptools==0.5.0,hugectr2onnx==0.0.0,huggingface-hub==0.9.1,hyperframe==6.0.1,idna==2.8,imagesize==1.4.1,implicit==0.6.1,importlib-metadata==4.12.0,importlib-resources==5.9.0,iniconfig==1.1.1,interrogate==1.5.0,ipykernel==6.15.3,ipython==8.5.0,ipython-genutils==0.2.0,ipywidgets==7.7.0,isort==5.10.1,jedi==0.18.1,Jinja2==3.1.2,jmespath==1.0.1,joblib==1.2.0,json5==0.9.10,jsonschema==4.16.0,jupyter==1.0.0,jupyter-cache==0.4.3,jupyter-console==6.4.4,jupyter-core==4.11.1,jupyter-server==1.18.1,jupyter-server-mathjax==0.2.5,jupyter-sphinx==0.3.2,jupyter_client==7.3.5,jupyterlab==3.4.7,jupyterlab-pygments==0.2.2,jupyterlab-widgets==1.1.0,jupyterlab_server==2.15.1,keras==2.9.0,Keras-Preprocessing==1.1.2,kiwisolver==1.4.4,lazy-object-proxy==1.8.0,libclang==14.0.6,libcst==0.4.7,lightfm==1.16,lightgbm==3.3.2,linkify-it-py==1.0.3,llvmlite==0.39.1,locket==1.0.0,lxml==4.9.1,Mako==1.2.4,Markdown==3.4.1,markdown-it-py==1.1.0,MarkupSafe==2.1.1,matplotlib==3.6.0,matplotlib-inline==0.1.6,mccabe==0.7.0,mdit-py-plugins==0.2.8,merlin-core==0.6.0+1.g5926fcf,merlin-models==0.9.0+57.g6b38b9b3,merlin-systems==0.5.0+4.g15074ad,mistune==2.0.4,mmh3==3.0.0,mpi4py==3.1.3,msgpack==1.0.4,multidict==6.0.2,mypy==0.991,mypy-extensions==0.4.3,myst-nb==0.13.2,myst-parser==0.15.2,natsort==8.1.0,nbclassic==0.4.3,nbclient==0.6.8,nbconvert==7.0.0,nbdime==3.1.1,nbformat==5.5.0,nest-asyncio==1.5.5,ninja==1.10.2.3,notebook==6.4.12,notebook-shim==0.1.0,numba==0.56.2,numpy==1.22.4,nvidia-pyindex==1.0.9,# Editable install with no version control (nvtabular==1.4.0+8.g95e12d347),-e /usr/local/lib/python3.8/dist-packages,nvtx==0.2.5,oauthlib==3.2.1,oldest-supported-numpy==2022.8.16,onnx==1.12.0,onnxruntime==1.11.1,opt-einsum==3.3.0,optuna==3.0.3,packaging==21.3,pandas==1.3.5,pandavro==1.5.2,pandocfilters==1.5.0,parso==0.8.3,partd==1.3.0,pathspec==0.10.2,pathtools==0.1.2,pbr==5.11.0,pep517==0.13.0,pexpect==4.8.0,pickleshare==0.7.5,Pillow==9.2.0,pkgutil_resolve_name==1.3.10,platformdirs==2.5.2,plotly==5.11.0,pluggy==1.0.0,prettytable==3.5.0,prometheus-client==0.14.1,promise==2.3,prompt-toolkit==3.0.31,proto-plus==1.19.6,protobuf==3.19.5,psutil==5.9.2,ptyprocess==0.7.0,pure-eval==0.2.2,py==1.11.0,pyarrow==7.0.0,pyasn1==0.4.8,pyasn1-modules==0.2.8,pybind11==2.10.0,pycodestyle==2.9.1,pycparser==2.21,pydantic==1.10.2,pydot==1.4.2,pyflakes==2.5.0,Pygments==2.13.0,PyGObject==3.36.0,pynvml==11.4.1,pyparsing==3.0.9,pyperclip==1.8.2,pyrsistent==0.18.1,pytest==7.1.3,pytest-cov==4.0.0,pytest-xdist==3.0.2,python-apt==2.0.0+ubuntu0.20.4.8,python-dateutil==2.8.2,python-dotenv==0.21.0,python-rapidjson==1.8,pytz==2022.2.1,PyYAML==5.4.1,pyzmq==24.0.0,qtconsole==5.4.0,QtPy==2.3.0,regex==2022.9.13,requests==2.22.0,requests-oauthlib==1.3.1,requests-unixsocket==0.2.0,rsa==4.7.2,s3fs==2022.2.0,s3transfer==0.6.0,sacremoses==0.0.53,scikit-build==0.15.0,scikit-learn==1.1.2,scipy==1.8.1,seedir==0.3.0,Send2Trash==1.8.0,sentry-sdk==1.9.8,setproctitle==1.3.2,setuptools-scm==7.0.5,shortuuid==1.0.9,six==1.15.0,sklearn==0.0,smmap==5.0.0,sniffio==1.3.0,snowballstemmer==2.2.0,sortedcontainers==2.4.0,soupsieve==2.3.2.post1,Sphinx==5.3.0,sphinx-multiversion==0.2.4,sphinx-togglebutton==0.3.1,sphinx_external_toc==0.3.0,sphinxcontrib-applehelp==1.0.2,sphinxcontrib-copydirs @ git+https://github.com/mikemckiernan/sphinxcontrib-copydirs.git@bd8c5d79b3f91cf5f1bb0d6995aeca3fe84b670e,sphinxcontrib-devhelp==1.0.2,sphinxcontrib-htmlhelp==2.0.0,sphinxcontrib-jsmath==1.0.1,sphinxcontrib-qthelp==1.0.3,sphinxcontrib-serializinghtml==1.1.5,SQLAlchemy==1.4.44,stack-data==0.5.0,starlette==0.20.4,stevedore==4.1.1,stringcase==1.2.0,supervisor==4.1.0,tabulate==0.8.10,tblib==1.7.0,tdqm==0.0.1,tenacity==8.0.1,tensorboard==2.9.1,tensorboard-data-server==0.6.1,tensorboard-plugin-wit==1.8.1,tensorflow==2.9.2,tensorflow-estimator==2.9.0,tensorflow-gpu==2.9.2,tensorflow-io-gcs-filesystem==0.27.0,tensorflow-metadata==1.10.0,termcolor==2.0.1,terminado==0.15.0,testbook==0.4.2,threadpoolctl==3.1.0,tinycss2==1.1.1,tokenizers==0.10.3,toml==0.10.2,tomli==2.0.1,toolz==0.12.0,torch==1.12.1+cu113,torchmetrics==0.3.2,tornado==6.2,tox==3.26.0,tqdm==4.64.1,traitlets==5.4.0,transformers==4.12.0,transformers4rec==0.1.12+2.gbcc939255,treelite==2.3.0,treelite-runtime==2.3.0,tritonclient==2.25.0,typed-ast==1.5.4,typing-inspect==0.8.0,typing_extensions==4.3.0,uc-micro-py==1.0.1,urllib3==1.26.12,uvicorn==0.18.3,uvloop==0.17.0,versioneer==0.20,virtualenv==20.16.5,wandb==0.13.3,watchfiles==0.17.0,wcwidth==0.2.5,webencodings==0.5.1,websocket-client==1.4.1,websockets==10.3,Werkzeug==2.2.2,widgetsnbextension==3.6.0,wrapt==1.12.1,xgboost==1.6.2,xyzservices==2022.9.0,yarl==1.8.1,zict==2.2.0,zipp==3.8.1,zope.event==4.5.0,zope.interface==5.4.0
py38-gpu run-test-pre: PYTHONHASHSEED='668111163'
py38-gpu run-test: commands[0] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/core.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-req-build-mst_7_6o
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-req-build-mst_7_6o
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 78f1f0b0952fd14b76913b0dd258565c06694abe
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (3.19.5)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+13.g78f1f0b) (0.55.1)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (1.10.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+13.g78f1f0b) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+13.g78f1f0b) (1.3.5)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+13.g78f1f0b) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core==0.9.0+13.g78f1f0b) (2022.5.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (7.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (21.3)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core==0.9.0+13.g78f1f0b) (1.2.5)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (0.4.3)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (0.12.0)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (6.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (1.7.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (2.4.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (1.0.4)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+13.g78f1f0b) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+13.g78f1f0b) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core==0.9.0+13.g78f1f0b) (65.5.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core==0.9.0+13.g78f1f0b) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+13.g78f1f0b) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+13.g78f1f0b) (2022.2.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+13.g78f1f0b) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core==0.9.0+13.g78f1f0b) (1.52.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core==0.9.0+13.g78f1f0b) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core==0.9.0+13.g78f1f0b) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core==0.9.0+13.g78f1f0b) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.9.0+13.g78f1f0b-py3-none-any.whl size=118889 sha256=8f2c874026410daf52ff2eeae983f45b4195c3b6b96c2eb5ebe3dd548c975f78
  Stored in directory: /tmp/pip-ephem-wheel-cache-hz0003tq/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
  Attempting uninstall: merlin-core
    Found existing installation: merlin-core 0.3.0+12.g78ecddd
    Not uninstalling merlin-core at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'merlin-core'. No files were found to uninstall.
Successfully installed merlin-core-0.9.0+13.g78f1f0b
py38-gpu run-test: commands[1] | python -m pip install --upgrade git+https://github.com/NVIDIA-Merlin/nvtabular.git
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting git+https://github.com/NVIDIA-Merlin/nvtabular.git
  Cloning https://github.com/NVIDIA-Merlin/nvtabular.git to /tmp/pip-req-build-w59i421o
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/nvtabular.git /tmp/pip-req-build-w59i421o
  Resolved https://github.com/NVIDIA-Merlin/nvtabular.git to commit ff186c6fac1bca957d4edd76e8e81c77ca6a649e
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: merlin-core>=0.2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from nvtabular==1.6.0+10.gff186c6f) (0.9.0+13.g78f1f0b)
Collecting merlin-dataloader>=0.0.2
  Downloading merlin-dataloader-0.0.3.tar.gz (48 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 48.3/48.3 kB 1.6 MB/s eta 0:00:00
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: scipy in /usr/local/lib/python3.8/dist-packages (from nvtabular==1.6.0+10.gff186c6f) (1.8.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (3.19.5)
Requirement already satisfied: numba>=0.54 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (0.55.1)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.10.0)
Requirement already satisfied: distributed>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2022.3.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.3.5)
Requirement already satisfied: dask>=2022.3.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2022.3.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (4.64.1)
Requirement already satisfied: fsspec==2022.5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2022.5.0)
Requirement already satisfied: pyarrow>=5.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (7.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (21.3)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.2.5)
Requirement already satisfied: numpy<1.25.0,>=1.17.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (from scipy->nvtabular==1.6.0+10.gff186c6f) (1.20.3)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (0.4.3)
Requirement already satisfied: pyyaml>=5.3.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (5.4.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (0.12.0)
Requirement already satisfied: tornado>=6.0.3 in ./.tox/py38-gpu/lib/python3.8/site-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (6.2)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (8.1.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.7.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (3.1.2)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2.4.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.0.4)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (0.38.1)
Requirement already satisfied: setuptools in ./.tox/py38-gpu/lib/python3.8/site-packages (from numba>=0.54->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (65.5.1)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2022.2.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.2.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.52.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (0.2.1)
Requirement already satisfied: six>=1.5 in /var/jenkins_home/.local/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas<1.4.0dev0,>=1.2.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.15.0)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in ./.tox/py38-gpu/lib/python3.8/site-packages (from jinja2->distributed>=2022.3.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core>=0.2.0->nvtabular==1.6.0+10.gff186c6f) (6.0.1)
Building wheels for collected packages: nvtabular, merlin-dataloader
  Building wheel for nvtabular (pyproject.toml): started
  Building wheel for nvtabular (pyproject.toml): finished with status 'done'
  Created wheel for nvtabular: filename=nvtabular-1.6.0+10.gff186c6f-cp38-cp38-linux_x86_64.whl size=257603 sha256=13b35051c1d9018665fa2323d9b43542428899452da87dd1ef91ab07b6c08c4e
  Stored in directory: /tmp/pip-ephem-wheel-cache-cg30e74m/wheels/8f/d9/f9/30f2cdc5bf8787fae6fdfe55afd6e1b493e619ec32c32ec40b
  Building wheel for merlin-dataloader (pyproject.toml): started
  Building wheel for merlin-dataloader (pyproject.toml): finished with status 'done'
  Created wheel for merlin-dataloader: filename=merlin_dataloader-0.0.3-py3-none-any.whl size=37647 sha256=1e4eb4bb2180a4768b5ab8d7e6d4a81bf74f1bea920b09bb6e4fdd045cf6542f
  Stored in directory: /tmp/pip-ephem-wheel-cache-cg30e74m/wheels/1c/a3/4a/0feebb30e0c8cb7ba7046544390b43c7017a2195232f5305a1
Successfully built nvtabular merlin-dataloader
Installing collected packages: merlin-dataloader, nvtabular
  Attempting uninstall: nvtabular
    Found existing installation: nvtabular 1.1.1
    Not uninstalling nvtabular at /var/jenkins_home/.local/lib/python3.8/site-packages, outside environment /var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu
    Can't uninstall 'nvtabular'. No files were found to uninstall.
Successfully installed merlin-dataloader-0.0.3 nvtabular-1.6.0+10.gff186c6f
py38-gpu run-test: commands[2] | python -m pytest --cov-report term --cov merlin -rxs tests/unit
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.3, pluggy-1.0.0
cachedir: .tox/py38-gpu/.pytest_cache
rootdir: /var/jenkins_home/workspace/merlin_models/models, configfile: pyproject.toml
plugins: anyio-3.5.0, xdist-3.0.2, cov-4.0.0
collected 886 items

tests/unit/config/test_schema.py ....                                    [  0%]

tests/unit/datasets/test_advertising.py .s                               [  0%]

tests/unit/datasets/test_ecommerce.py ..sss.s                            [  1%]

tests/unit/datasets/test_entertainment.py ....sss.                       [  2%]

tests/unit/datasets/test_social.py .                                     [  2%]

tests/unit/datasets/test_synthetic.py ..........                         [  3%]

tests/unit/implicit/test_implicit.py .                                   [  3%]

tests/unit/lightfm/test_lightfm.py .                                     [  3%]

tests/unit/tf/test_core.py ......                                        [  4%]

tests/unit/tf/test_loader.py ................                            [  6%]

tests/unit/tf/test_public_api.py .                                       [  6%]

tests/unit/tf/blocks/test_cross.py ...........                           [  7%]

tests/unit/tf/blocks/test_dlrm.py ..........                             [  8%]

tests/unit/tf/blocks/test_interactions.py ...                            [  9%]

tests/unit/tf/blocks/test_mlp.py ....................................... [ 13%]

............................                                             [ 16%]

tests/unit/tf/blocks/test_optimizer.py s................................ [ 20%]

.....................                                                    [ 22%]

tests/unit/tf/blocks/retrieval/test_base.py .                            [ 22%]

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py ..           [ 23%]

tests/unit/tf/blocks/retrieval/test_two_tower.py ............            [ 24%]

tests/unit/tf/blocks/sampling/test_cross_batch.py .                      [ 24%]

tests/unit/tf/blocks/sampling/test_in_batch.py .                         [ 24%]

tests/unit/tf/core/test_aggregation.py .........                         [ 25%]

tests/unit/tf/core/test_base.py ..                                       [ 25%]

tests/unit/tf/core/test_combinators.py s.....................            [ 28%]

tests/unit/tf/core/test_encoder.py ..                                    [ 28%]

tests/unit/tf/core/test_index.py ...                                     [ 29%]

tests/unit/tf/core/test_prediction.py ..                                 [ 29%]

tests/unit/tf/core/test_tabular.py ......                                [ 29%]

tests/unit/tf/examples/test_01_getting_started.py .                      [ 30%]

tests/unit/tf/examples/test_02_dataschema.py .                           [ 30%]

tests/unit/tf/examples/test_03_exploring_different_models.py .           [ 30%]

tests/unit/tf/examples/test_04_export_ranking_models.py .                [ 30%]

tests/unit/tf/examples/test_05_export_retrieval_model.py .               [ 30%]

tests/unit/tf/examples/test_06_advanced_own_architecture.py .            [ 30%]

tests/unit/tf/examples/test_07_train_traditional_models.py .             [ 30%]

tests/unit/tf/examples/test_usecase_accelerate_training_by_lazyadam.py . [ 30%]

[ 30%]

tests/unit/tf/examples/test_usecase_data_parallel.py .                   [ 30%]

tests/unit/tf/examples/test_usecase_ecommerce_session_based.py .         [ 31%]

tests/unit/tf/examples/test_usecase_incremental_training_layer_freezing.py . [ 31%]

[ 31%]

tests/unit/tf/examples/test_usecase_pretrained_embeddings.py .           [ 31%]

tests/unit/tf/examples/test_usecase_retrieval_with_hpo.py .              [ 31%]

tests/unit/tf/examples/test_usecase_transformers_next_item_prediction.py . [ 31%]

[ 31%]

tests/unit/tf/horovod/test_horovod.py ...                                [ 31%]

tests/unit/tf/inputs/test_base.py .                                      [ 31%]

tests/unit/tf/inputs/test_continuous.py ........                         [ 32%]

tests/unit/tf/inputs/test_embedding.py ................................. [ 36%]

........                                                                 [ 37%]

tests/unit/tf/inputs/test_tabular.py ..................                  [ 39%]

tests/unit/tf/layers/test_queue.py ..............                        [ 41%]

tests/unit/tf/losses/test_losses.py .......................              [ 43%]

tests/unit/tf/metrics/test_metrics_popularity.py .....                   [ 44%]

tests/unit/tf/metrics/test_metrics_topk.py .........................     [ 47%]

tests/unit/tf/models/test_base.py s.........................             [ 50%]

tests/unit/tf/models/test_benchmark.py ..                                [ 50%]

tests/unit/tf/models/test_ranking.py ..................................  [ 54%]

tests/unit/tf/models/test_retrieval.py ................................. [ 57%]

.............................................                            [ 62%]

tests/unit/tf/outputs/test_base.py ......                                [ 63%]

tests/unit/tf/outputs/test_classification.py ......                      [ 64%]

tests/unit/tf/outputs/test_contrastive.py ..............                 [ 65%]

tests/unit/tf/outputs/test_regression.py ..                              [ 66%]

tests/unit/tf/outputs/test_sampling.py ....                              [ 66%]

tests/unit/tf/outputs/test_topk.py .                                     [ 66%]

tests/unit/tf/prediction_tasks/test_classification.py ..                 [ 66%]

tests/unit/tf/prediction_tasks/test_multi_task.py ................       [ 68%]

tests/unit/tf/prediction_tasks/test_next_item.py .....                   [ 69%]

tests/unit/tf/prediction_tasks/test_regression.py .....                  [ 69%]

tests/unit/tf/prediction_tasks/test_retrieval.py .                       [ 69%]

tests/unit/tf/prediction_tasks/test_sampling.py ......                   [ 70%]

tests/unit/tf/transformers/test_block.py ......................          [ 73%]

tests/unit/tf/transformers/test_transforms.py ..........                 [ 74%]

tests/unit/tf/transforms/test_bias.py ..                                 [ 74%]

tests/unit/tf/transforms/test_features.py s............................. [ 77%]

.......................s......                                           [ 81%]

tests/unit/tf/transforms/test_negative_sampling.py .........             [ 82%]

tests/unit/tf/transforms/test_noise.py .....                             [ 82%]

tests/unit/tf/transforms/test_sequence.py ....................           [ 84%]

tests/unit/tf/transforms/test_tensor.py ...                              [ 85%]

tests/unit/tf/utils/test_batch.py ....                                   [ 85%]

tests/unit/tf/utils/test_dataset.py ..                                   [ 86%]

tests/unit/tf/utils/test_tf_utils.py .....                               [ 86%]

tests/unit/torch/test_dataset.py .........                               [ 87%]

tests/unit/torch/test_public_api.py .                                    [ 87%]

tests/unit/torch/block/test_base.py ....                                 [ 88%]

tests/unit/torch/block/test_mlp.py .                                     [ 88%]

tests/unit/torch/features/test_continuous.py ..                          [ 88%]

tests/unit/torch/features/test_embedding.py ..............               [ 90%]

tests/unit/torch/features/test_tabular.py ....                           [ 90%]

tests/unit/torch/model/test_head.py ............                         [ 91%]

tests/unit/torch/model/test_model.py ..                                  [ 92%]

tests/unit/torch/tabular/test_aggregation.py ........                    [ 93%]

tests/unit/torch/tabular/test_tabular.py ...                             [ 93%]

tests/unit/torch/tabular/test_transformations.py .......                 [ 94%]

tests/unit/utils/test_schema_utils.py ................................   [ 97%]

tests/unit/xgb/test_xgboost.py ....................                      [100%]


=============================== warnings summary ===============================

../../../../../usr/lib/python3/dist-packages/requests/init.py:89

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.12) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "


../../../.local/lib/python3.8/site-packages/flatbuffers/compat.py:19

/var/jenkins_home/.local/lib/python3.8/site-packages/flatbuffers/compat.py:19: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

import imp


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:36: DeprecationWarning: NEAREST is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.NEAREST or Dither.NONE instead.

'nearest': pil_image.NEAREST,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:37: DeprecationWarning: BILINEAR is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BILINEAR instead.

'bilinear': pil_image.BILINEAR,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:38: DeprecationWarning: BICUBIC is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BICUBIC instead.

'bicubic': pil_image.BICUBIC,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:39: DeprecationWarning: HAMMING is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.HAMMING instead.

'hamming': pil_image.HAMMING,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:40: DeprecationWarning: BOX is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.BOX instead.

'box': pil_image.BOX,


../../../.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/utils/image_utils.py:41: DeprecationWarning: LANCZOS is deprecated and will be removed in Pillow 10 (2023-07-01). Use Resampling.LANCZOS instead.

'lanczos': pil_image.LANCZOS,


tests/unit/datasets/test_advertising.py: 1 warning

tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 10 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 6 warnings

tests/unit/tf/core/test_index.py: 8 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 38 warnings

tests/unit/tf/models/test_retrieval.py: 123 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/prediction_tasks/test_retrieval.py: 1 warning

tests/unit/tf/transformers/test_block.py: 16 warnings

tests/unit/tf/transforms/test_bias.py: 2 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_noise.py: 1 warning

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 9 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 3 warnings

tests/unit/xgb/test_xgboost.py: 18 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.ITEM_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.ITEM: 'item'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 3 warnings

tests/unit/datasets/test_entertainment.py: 4 warnings

tests/unit/datasets/test_social.py: 1 warning

tests/unit/datasets/test_synthetic.py: 9 warnings

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_core.py: 6 warnings

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/test_cross.py: 5 warnings

tests/unit/tf/blocks/test_dlrm.py: 9 warnings

tests/unit/tf/blocks/test_interactions.py: 2 warnings

tests/unit/tf/blocks/test_mlp.py: 60 warnings

tests/unit/tf/blocks/test_optimizer.py: 30 warnings

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 11 warnings

tests/unit/tf/core/test_aggregation.py: 6 warnings

tests/unit/tf/core/test_base.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 7 warnings

tests/unit/tf/core/test_index.py: 3 warnings

tests/unit/tf/core/test_prediction.py: 2 warnings

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_base.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 7 warnings

tests/unit/tf/inputs/test_embedding.py: 20 warnings

tests/unit/tf/inputs/test_tabular.py: 18 warnings

tests/unit/tf/models/test_base.py: 28 warnings

tests/unit/tf/models/test_benchmark.py: 2 warnings

tests/unit/tf/models/test_ranking.py: 36 warnings

tests/unit/tf/models/test_retrieval.py: 89 warnings

tests/unit/tf/outputs/test_base.py: 6 warnings

tests/unit/tf/outputs/test_classification.py: 6 warnings

tests/unit/tf/outputs/test_contrastive.py: 19 warnings

tests/unit/tf/outputs/test_regression.py: 2 warnings

tests/unit/tf/prediction_tasks/test_classification.py: 2 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 5 warnings

tests/unit/tf/transformers/test_block.py: 10 warnings

tests/unit/tf/transforms/test_features.py: 10 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 10 warnings

tests/unit/tf/transforms/test_sequence.py: 15 warnings

tests/unit/tf/utils/test_batch.py: 7 warnings

tests/unit/tf/utils/test_dataset.py: 2 warnings

tests/unit/torch/block/test_base.py: 4 warnings

tests/unit/torch/block/test_mlp.py: 1 warning

tests/unit/torch/features/test_continuous.py: 1 warning

tests/unit/torch/features/test_embedding.py: 4 warnings

tests/unit/torch/features/test_tabular.py: 4 warnings

tests/unit/torch/model/test_head.py: 12 warnings

tests/unit/torch/model/test_model.py: 2 warnings

tests/unit/torch/tabular/test_aggregation.py: 6 warnings

tests/unit/torch/tabular/test_transformations.py: 2 warnings

tests/unit/xgb/test_xgboost.py: 17 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.USER_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.USER: 'user'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/datasets/test_ecommerce.py: 1 warning

tests/unit/datasets/test_entertainment.py: 1 warning

tests/unit/implicit/test_implicit.py: 1 warning

tests/unit/lightfm/test_lightfm.py: 1 warning

tests/unit/tf/test_loader.py: 1 warning

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py: 2 warnings

tests/unit/tf/blocks/retrieval/test_two_tower.py: 2 warnings

tests/unit/tf/core/test_combinators.py: 11 warnings

tests/unit/tf/core/test_encoder.py: 2 warnings

tests/unit/tf/core/test_prediction.py: 1 warning

tests/unit/tf/horovod/test_horovod.py: 1 warning

tests/unit/tf/inputs/test_continuous.py: 4 warnings

tests/unit/tf/inputs/test_embedding.py: 9 warnings

tests/unit/tf/inputs/test_tabular.py: 8 warnings

tests/unit/tf/models/test_ranking.py: 20 warnings

tests/unit/tf/models/test_retrieval.py: 10 warnings

tests/unit/tf/prediction_tasks/test_multi_task.py: 16 warnings

tests/unit/tf/prediction_tasks/test_regression.py: 3 warnings

tests/unit/tf/transforms/test_negative_sampling.py: 9 warnings

tests/unit/xgb/test_xgboost.py: 12 warnings

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/merlin/schema/tags.py:148: UserWarning: Compound tags like Tags.SESSION_ID have been deprecated and will be removed in a future version. Please use the atomic versions of these tags, like [<Tags.SESSION: 'session'>, <Tags.ID: 'id'>].

warnings.warn(


tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_matrix_factorization.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/blocks/retrieval/test_two_tower.py::test_matrix_factorization_embedding_export

tests/unit/tf/inputs/test_embedding.py::test_embedding_features_exporting_and_loading_pretrained_initializer

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/inputs/embedding.py:970: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

embeddings_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(embeddings)))


tests/unit/tf/blocks/retrieval/test_two_tower.py: 1 warning

tests/unit/tf/core/test_index.py: 4 warnings

tests/unit/tf/horovod/test_horovod.py: 3 warnings

tests/unit/tf/models/test_retrieval.py: 63 warnings

tests/unit/tf/prediction_tasks/test_next_item.py: 3 warnings

tests/unit/tf/utils/test_batch.py: 2 warnings

/tmp/autograph_generated_file434_udmi.py:8: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead

ag.converted_call(ag__.ld(warnings).warn, ("The 'warn' method is deprecated, use 'warning' instead", ag__.ld(DeprecationWarning), 2), None, fscope)


tests/unit/tf/core/test_combinators.py::test_parallel_block_select_by_tags

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/core/tabular.py:602: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working

elif isinstance(self.feature_names, collections.Sequence):


tests/unit/tf/core/test_encoder.py: 1 warning

tests/unit/tf/core/test_index.py: 5 warnings

tests/unit/tf/models/test_retrieval.py: 30 warnings

tests/unit/tf/utils/test_batch.py: 4 warnings

tests/unit/tf/utils/test_dataset.py: 1 warning

/var/jenkins_home/workspace/merlin_models/models/merlin/models/utils/dataset.py:75: DeprecationWarning: unique_rows_by_features is deprecated and will be removed in a future version. Please use unique_by_tag instead.

warnings.warn(


tests/unit/tf/models/test_base.py::test_model_pre_post[True]

tests/unit/tf/models/test_base.py::test_model_pre_post[False]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.1]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.3]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.5]

tests/unit/tf/transforms/test_noise.py::test_stochastic_swap_noise[0.7]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/util/dispatch.py:1082: UserWarning: tf.keras.backend.random_binomial is deprecated, and will be removed in a future version.Please use tf.keras.backend.random_bernoulli instead.

return dispatch_target(*args, **kwargs)


tests/unit/tf/models/test_base.py::test_freeze_parallel_block[True]

tests/unit/tf/models/test_base.py::test_freeze_sequential_block

tests/unit/tf/models/test_base.py::test_freeze_unfreeze

tests/unit/tf/models/test_base.py::test_unfreeze_all_blocks

/var/jenkins_home/.local/lib/python3.8/site-packages/keras/optimizers/optimizer_v2/gradient_descent.py:108: UserWarning: The lr argument is deprecated, use learning_rate instead.

super(SGD, self).init(name, **kwargs)


tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_base.py::test_retrieval_model_query

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_v2_export_embeddings

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[True]

tests/unit/tf/models/test_retrieval.py::test_youtube_dnn_topk_evaluation[False]

tests/unit/tf/transformers/test_block.py::test_retrieval_transformer[True]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/utils/tf_utils.py:298: DeprecationWarning: This function is deprecated in favor of cupy.from_dlpack

tensor_cupy = cupy.fromDlpack(to_dlpack(tf.convert_to_tensor(tensor)))


tests/unit/tf/models/test_ranking.py::test_deepfm_model_only_categ_feats[False]

tests/unit/tf/models/test_ranking.py::test_deepfm_model_categ_and_continuous_feats[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_3/parallel_block_2/sequential_block_3/sequential_block_2/private__dense_1/dense_1/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_categorical_one_hot[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_model_hashed_cross[False]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_2/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[True]

tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/workspace/merlin_models/models/merlin/models/tf/transforms/features.py:569: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_embedding_custom_inputblock[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/autograph/impl/api.py:371: UserWarning: Please make sure input features to be categorical, detect user_age has no categorical tag

return py_builtins.overload_of(f)(*args)


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_onehot_multihot_feature_interaction[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_5/sequential_block_9/sequential_block_8/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/models/test_ranking.py::test_wide_deep_model_wide_feature_interaction_multi_optimizer[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape_1:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Reshape:0", shape=(None, 1), dtype=float32), dense_shape=Tensor("gradient_tape/model/parallel_block_4/sequential_block_6/sequential_block_5/private__dense_3/dense_3/embedding_lookup_sparse/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_as_classfication_model[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_causal_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/prepare_transformer_inputs_1/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/bert_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask_1/GatherV2:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/boolean_mask/GatherV2:0", shape=(None, 48), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/prepare_transformer_inputs_5/RaggedToTensor/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_3:0", shape=(None,), dtype=int64), values=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Reshape_2:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/Cast:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_1:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/tf/transformers/test_block.py::test_transformer_with_masked_language_modeling_check_eval_masked[False]

/var/jenkins_home/.local/lib/python3.8/site-packages/tensorflow/python/framework/indexed_slices.py:444: UserWarning: Converting sparse IndexedSlices(IndexedSlices(indices=Tensor("gradient_tape/model/gpt2_block/replace_masked_embeddings/RaggedWhere/RaggedTile_2/Reshape_3:0", shape=(None,), dtype=int32), values=Tensor("gradient_tape/model/concat_features/RaggedConcat/Slice_3:0", shape=(None, None), dtype=float32), dense_shape=Tensor("gradient_tape/model/concat_features/RaggedConcat/Shape_1:0", shape=(2,), dtype=int32))) to a dense Tensor of unknown shape. This may consume a large amount of memory.

warnings.warn(


tests/unit/torch/block/test_mlp.py::test_mlp_block

/var/jenkins_home/workspace/merlin_models/models/tests/unit/torch/_conftest.py:151: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow. Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor. (Triggered internally at  ../torch/csrc/utils/tensor_new.cpp:201.)

return {key: torch.tensor(value) for key, value in data.items()}


tests/unit/xgb/test_xgboost.py::test_without_dask_client

tests/unit/xgb/test_xgboost.py::TestXGBoost::test_music_regression

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs0-DaskDeviceQuantileDMatrix]

tests/unit/xgb/test_xgboost.py::test_gpu_hist_dmatrix[fit_kwargs1-DaskDMatrix]

tests/unit/xgb/test_xgboost.py::TestEvals::test_multiple

tests/unit/xgb/test_xgboost.py::TestEvals::test_default

tests/unit/xgb/test_xgboost.py::TestEvals::test_train_and_valid

tests/unit/xgb/test_xgboost.py::TestEvals::test_invalid_data

/var/jenkins_home/workspace/merlin_models/models/merlin/models/xgb/init.py:344: UserWarning: Ignoring list columns as inputs to XGBoost model: ['item_genres', 'user_genres'].

warnings.warn(f"Ignoring list columns as inputs to XGBoost model: {list_column_names}.")


tests/unit/xgb/test_xgboost.py::TestXGBoost::test_unsupported_objective

/var/jenkins_home/workspace/merlin_models/models/.tox/py38-gpu/lib/python3.8/site-packages/tornado/ioloop.py:350: DeprecationWarning: make_current is deprecated; start the event loop first

self.make_current()


tests/unit/xgb/test_xgboost.py: 14 warnings

/usr/local/lib/python3.8/dist-packages/xgboost/dask.py:884: RuntimeWarning: coroutine 'Client._wait_for_workers' was never awaited

client.wait_for_workers(n_workers)

Enable tracemalloc to get traceback where the object was allocated.

See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.


tests/unit/xgb/test_xgboost.py: 11 warnings

/usr/local/lib/python3.8/dist-packages/cudf/core/dataframe.py:1183: DeprecationWarning: The default dtype for empty Series will be 'object' instead of 'float64' in a future version. Specify a dtype explicitly to silence this warning.

mask = pd.Series(mask)


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                                 Stmts   Miss  Cover


merlin/datasets/init.py                                              2      0   100%

merlin/datasets/advertising/init.py                                  2      0   100%

merlin/datasets/advertising/criteo/init.py                           0      0   100%

merlin/datasets/advertising/criteo/dataset.py                           79     49    38%

merlin/datasets/advertising/criteo/transformed/init.py               0      0   100%

merlin/datasets/ecommerce/init.py                                    4      0   100%

merlin/datasets/ecommerce/aliccp/init.py                             0      0   100%

merlin/datasets/ecommerce/aliccp/dataset.py                            141     87    38%

merlin/datasets/ecommerce/aliccp/raw/init.py                         0      0   100%

merlin/datasets/ecommerce/aliccp/transformed/init.py                 0      0   100%

merlin/datasets/ecommerce/booking/init.py                            0      0   100%

merlin/datasets/ecommerce/booking/dataset.py                           127    100    21%

merlin/datasets/ecommerce/booking/raw/init.py                        0      0   100%

merlin/datasets/ecommerce/booking/transformed/init.py                0      0   100%

merlin/datasets/ecommerce/dressipi/init.py                           0      0   100%

merlin/datasets/ecommerce/dressipi/dataset.py                           45     37    18%

merlin/datasets/ecommerce/dressipi/preprocessed/init.py              0      0   100%

merlin/datasets/ecommerce/large/init.py                              0      0   100%

merlin/datasets/ecommerce/small/init.py                              0      0   100%

merlin/datasets/ecommerce/transactions/init.py                       0      0   100%

merlin/datasets/entertainment/init.py                                2      0   100%

merlin/datasets/entertainment/movielens/1m-raw/init.py               0      0   100%

merlin/datasets/entertainment/movielens/1m-raw/ratings/init.py       0      0   100%

merlin/datasets/entertainment/movielens/1m/init.py                   0      0   100%

merlin/datasets/entertainment/movielens/25m/init.py                  0      0   100%

merlin/datasets/entertainment/movielens/100k/init.py                 0      0   100%

merlin/datasets/entertainment/movielens/init.py                      0      0   100%

merlin/datasets/entertainment/movielens/dataset.py                     152    122    20%

merlin/datasets/entertainment/music_streaming/init.py                0      0   100%

merlin/datasets/social/init.py                                       0      0   100%

merlin/datasets/synthetic.py                                           147      8    95%

merlin/datasets/testing/init.py                                      0      0   100%

merlin/datasets/testing/sequence_testing/init.py                     0      0   100%

merlin/models/init.py                                                2      0   100%

merlin/models/_version.py                                              354    205    42%

merlin/models/api.py                                                    14      5    64%

merlin/models/config/init.py                                         0      0   100%

merlin/models/config/schema.py                                          62      0   100%

merlin/models/implicit/init.py                                      27      4    85%

merlin/models/io.py                                                     15      0   100%

merlin/models/lightfm/init.py                                       23      0   100%

merlin/models/loader/init.py                                         0      0   100%

merlin/models/loader/backend.py                                        379     41    89%

merlin/models/loader/dataframe_iter.py                                  21     17    19%

merlin/models/loader/tf_utils.py                                        57     27    53%

merlin/models/loader/utils.py                                           40     15    62%

merlin/models/tf/init.py                                            70      0   100%

merlin/models/tf/blocks/init.py                                      0      0   100%

merlin/models/tf/blocks/cross.py                                        44      0   100%

merlin/models/tf/blocks/dlrm.py                                         49      2    96%

merlin/models/tf/blocks/experts.py                                      99     17    83%

merlin/models/tf/blocks/interaction.py                                 108     40    63%

merlin/models/tf/blocks/mlp.py                                         117      7    94%

merlin/models/tf/blocks/optimizer.py                                   173     12    93%

merlin/models/tf/blocks/retrieval/init.py                            0      0   100%

merlin/models/tf/blocks/retrieval/base.py                              175     62    65%

merlin/models/tf/blocks/retrieval/matrix_factorization.py               35      1    97%

merlin/models/tf/blocks/retrieval/two_tower.py                          30      0   100%

merlin/models/tf/blocks/sampling/init.py                             0      0   100%

merlin/models/tf/blocks/sampling/base.py                                29      2    93%

merlin/models/tf/blocks/sampling/cross_batch.py                         46      2    96%

merlin/models/tf/blocks/sampling/in_batch.py                            35      0   100%

merlin/models/tf/blocks/sampling/queue.py                              115     12    90%

merlin/models/tf/core/init.py                                        0      0   100%

merlin/models/tf/core/aggregation.py                                   241     45    81%

merlin/models/tf/core/base.py                                          242     50    79%

merlin/models/tf/core/combinators.py                                   426     53    88%

merlin/models/tf/core/encoder.py                                       175     28    84%

merlin/models/tf/core/index.py                                         106     16    85%

merlin/models/tf/core/prediction.py                                     50      1    98%

merlin/models/tf/core/tabular.py                                       280     29    90%

merlin/models/tf/distributed/init.py                                 0      0   100%

merlin/models/tf/distributed/backend.py                                  9      2    78%

merlin/models/tf/inputs/init.py                                      0      0   100%

merlin/models/tf/inputs/base.py                                         64     10    84%

merlin/models/tf/inputs/continuous.py                                   39      3    92%

merlin/models/tf/inputs/embedding.py                                   458     32    93%

merlin/models/tf/loader.py                                             268     94    65%

merlin/models/tf/losses/init.py                                      4      0   100%

merlin/models/tf/losses/base.py                                          9      0   100%

merlin/models/tf/losses/listwise.py                                     13      0   100%

merlin/models/tf/losses/pairwise.py                                    115      1    99%

merlin/models/tf/metrics/init.py                                     2      0   100%

merlin/models/tf/metrics/evaluation.py                                 105     48    54%

merlin/models/tf/metrics/topk.py                                       198     48    76%

merlin/models/tf/models/init.py                                      0      0   100%

merlin/models/tf/models/base.py                                        758    101    87%

merlin/models/tf/models/benchmark.py                                    16      0   100%

merlin/models/tf/models/ranking.py                                      67      3    96%

merlin/models/tf/models/retrieval.py                                    78      4    95%

merlin/models/tf/models/utils.py                                        10      1    90%

merlin/models/tf/outputs/init.py                                     0      0   100%

merlin/models/tf/outputs/base.py                                       123     17    86%

merlin/models/tf/outputs/classification.py                              91      1    99%

merlin/models/tf/outputs/contrastive.py                                147     10    93%

merlin/models/tf/outputs/regression.py                                   9      0   100%

merlin/models/tf/outputs/sampling/init.py                            0      0   100%

merlin/models/tf/outputs/sampling/base.py                               78     21    73%

merlin/models/tf/outputs/sampling/in_batch.py                           37      1    97%

merlin/models/tf/outputs/sampling/popularity.py                         27      1    96%

merlin/models/tf/outputs/topk.py                                        98      6    94%

merlin/models/tf/prediction_tasks/init.py                            0      0   100%

merlin/models/tf/prediction_tasks/base.py                              207     37    82%

merlin/models/tf/prediction_tasks/classification.py                     68     17    75%

merlin/models/tf/prediction_tasks/multi.py                               7      0   100%

merlin/models/tf/prediction_tasks/next_item.py                          59      6    90%

merlin/models/tf/prediction_tasks/regression.py                         35      2    94%

merlin/models/tf/prediction_tasks/retrieval.py                          73      3    96%

merlin/models/tf/transformers/init.py                                0      0   100%

merlin/models/tf/transformers/block.py                                 101      2    98%

merlin/models/tf/transformers/transforms.py                             63      0   100%

merlin/models/tf/transforms/init.py                                  0      0   100%

merlin/models/tf/transforms/bias.py                                    111      9    92%

merlin/models/tf/transforms/features.py                                435     36    92%

merlin/models/tf/transforms/negative_sampling.py                        77      3    96%

merlin/models/tf/transforms/noise.py                                    43      1    98%

merlin/models/tf/transforms/regularization.py                           17      1    94%

merlin/models/tf/transforms/sequence.py                                282     42    85%

merlin/models/tf/transforms/tensor.py                                  158     13    92%

merlin/models/tf/typing.py                                               7      0   100%

merlin/models/tf/utils/init.py                                       0      0   100%

merlin/models/tf/utils/batch_utils.py                                   85      4    95%

merlin/models/tf/utils/repr_utils.py                                    69      4    94%

merlin/models/tf/utils/search_utils.py                                  34     22    35%

merlin/models/tf/utils/testing_utils.py                                206     36    83%

merlin/models/tf/utils/tf_utils.py                                     209     42    80%

merlin/models/torch/init.py                                         12      0   100%

merlin/models/torch/block/init.py                                    0      0   100%

merlin/models/torch/block/base.py                                      167     32    81%

merlin/models/torch/block/mlp.py                                        38      5    87%

merlin/models/torch/dataset.py                                          68      5    93%

merlin/models/torch/features/init.py                                 0      0   100%

merlin/models/torch/features/base.py                                     4      0   100%

merlin/models/torch/features/continuous.py                              22      0   100%

merlin/models/torch/features/embedding.py                              165     12    93%

merlin/models/torch/features/tabular.py                                 65      8    88%

merlin/models/torch/losses.py                                           28     28     0%

merlin/models/torch/model/init.py                                    0      0   100%

merlin/models/torch/model/base.py                                      286     65    77%

merlin/models/torch/model/prediction_task.py                            24      0   100%

merlin/models/torch/tabular/init.py                                  0      0   100%

merlin/models/torch/tabular/aggregation.py                              75      0   100%

merlin/models/torch/tabular/base.py                                    247     39    84%

merlin/models/torch/tabular/transformations.py                          67      3    96%

merlin/models/torch/typing.py                                            6      0   100%

merlin/models/torch/utils/init.py                                    0      0   100%

merlin/models/torch/utils/data_utils.py                                117    117     0%

merlin/models/torch/utils/examples_utils.py                             39     39     0%

merlin/models/torch/utils/torch_utils.py                                80     22    72%

merlin/models/utils/init.py                                          0      0   100%

merlin/models/utils/constants.py                                         3      0   100%

merlin/models/utils/dataset.py                                          38      4    89%

merlin/models/utils/dependencies.py                                     26     19    27%

merlin/models/utils/doc_utils.py                                        10      0   100%

merlin/models/utils/example_utils.py                                    31      2    94%

merlin/models/utils/misc_utils.py                                      118     90    24%

merlin/models/utils/nvt_utils.py                                        27     24    11%

merlin/models/utils/registry.py                                        101     29    71%

merlin/models/utils/schema_utils.py                                     90     19    79%

merlin/models/xgb/init.py                                          124      4    97%


TOTAL                                                                11587   2346    80%


=========================== short test summary info ============================

SKIPPED [1] tests/unit/datasets/test_advertising.py:20: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:64: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:80: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:94: ALI-CCP data is not available, pass it through env variable $DATA_PATH_ALICCP

SKIPPED [1] tests/unit/datasets/test_ecommerce.py:115: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [3] tests/unit/datasets/test_entertainment.py:44: No data-dir available, pass it through env variable $INPUT_DATA_DIR

SKIPPED [5] ../../../.local/lib/python3.8/site-packages/tensorflow/python/framework/test_util.py:2746: Not a test.

========= 873 passed, 13 skipped, 1460 warnings in 1807.54s (0:30:07) ==========

___________________________________ summary ____________________________________

py38-gpu: commands succeeded

congratulations :)

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/models/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[merlin_models] $ /bin/bash /tmp/jenkins12965861824981804588.sh

@rnyak rnyak merged commit 01cb7ef into NVIDIA-Merlin:main Nov 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@rnyak rnyak rnyak approved these changes

Assignees

@oliverholworthy oliverholworthy

Labels
ci
Projects
None yet
Milestone
Merlin 22.12
Development

Successfully merging this pull request may close these issues.
3 participants
@oliverholworthy @nvidia-merlin-bot @rnyak