Gradio demo available

[fffiloni]

Hello 👋

I’ve crafted a Gradio App available here: https://huggingface.co/spaces/fffiloni/Meigen-MultiTalk

Works with L40S, multi-GPU ready.
Shared UI on HF is limited to 5 seconds audio max and 12 sample_steps. Results with 12 steps are satisfying enough for the demo purpose.

Performances on L40S are quite similar than on A100, maybe loosing around 10 seconds at each step.

You can duplicate the space to your own account to unlock these limitations ;)

Enjoy !

PS: see the requirements.txt file to understand how to handle some of your struggles installing dependencies 🤗

[yzhang2016]

So great.

[nguyenkienleho]

hi can i install this with RTX 5090 thank you for sharing

[Pblaz]

Hi, the app seems to be broken

[fffiloni]

@Pblaz : This is due to the gradio queue being full :)

You might want to duplicate to skip it

[laurbrick]

When I duplicate the Demo on HuggingFace I get a stripped down interface that is not fully functioning. Is there a way to get the exact Demo version shown?

[fffiloni]

Thank you for the heads up, the link under the duplicate button was wrong, it is solved now :)

here the working link just in case: https://huggingface.co/spaces/fffiloni/Meigen-MultiTalk?duplicate=true

[laurbrick]

So I have 2 issues now - after duplicating the space again and running the test image, I got this error:

File "/home/user/app/generate_multitalk.py", line 442, in generate
audio_embedding = get_embedding(human_speech, wav2vec_feature_extractor, audio_encoder)
File "/home/user/app/generate_multitalk.py", line 275, in get_embedding
embeddings = audio_encoder(audio_feature, seq_len=int(video_length), output_hidden_states=True)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "/home/user/app/src/audio_analysis/wav2vec2.py", line 23, in forward
self.config.output_attentions = True
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/transformers/configuration_utils.py", line 204, in setattr
super().setattr(key, value)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/transformers/configuration_utils.py", line 341, in output_attentions
raise ValueError(
ValueError: The output_attentions attribute is not supported when using the attn_implementation set to sdpa. Please set it to 'eager' instead.
[INFO] Removed temporary file: /tmp/tmpj26akpna.json
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/blocks.py", line 2220, in process_api
result = await self.call_function(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/blocks.py", line 1731, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/utils.py", line 904, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/app.py", line 219, in infer
raise RuntimeError("Inference failed. Check inference.log for details.")
RuntimeError: Inference failed. Check inference.log for details.

Possibly - a Wav2Vec2 + Transformers config clash.

ValueError: The output_attentions attribute is not supported when using the attn_implementation set to sdpa. Please set it to 'eager' instead.

Your audio_encoder (a Wav2Vec2 model from Hugging Face) is using the new SDPA (scaled dot product attention) backend, which does not support output_attentions=True. But your code is explicitly enabling it here:

self.config.output_attentions = True

I am not a coder and rely on LLM's to help me with open source installs and this is what it said. Also I am new to Huggingface and it doesn't look like I can access the .py to make any edits to fix this. Is this true?

My second comment is -I am attempting to get this to run on RUNPOD - using a RTX 6000 ADA. I am having runtime failures still (tensor issues) but in my gradio interface I have many other advanced controls. See image attached. Any chance you can get those on Huggingface?

Thanks,
Laurence

[sanebg]

Any chance to run this on local 5090?!

[giuseppeCoccia]

Hi @fffiloni , I have tried running your Gradio app but every time it fails with the error "Inference failed. Check inference.log for details."

I have also duplicated the space to my own account, and in this case I have encountered 2 issues:

1. If I use Chrome, then the app doesn't show anything, i.e. there is just a black page. I have also tried clearing all data and cookies but the problem persists.
2. If I use Firefox then the usual UI appears, but when I try to run inference I get the same issue that I get when running your own space, that is this error message "Inference failed. Check inference.log for details." The full log is exactly the same as @laurbrick

[2025-07-10 15:26:28,183] INFO: offload_model is not specified, set to True.
[2025-07-10 15:26:28,183] INFO: Generation job args: Namespace(task='multitalk-14B', size='multitalk-480', frame_num=81, ckpt_dir='weights/Wan2.1-I2V-14B-480P', wav2vec_dir='weights/chinese-wav2vec2-base', offload_model=True, ulysses_size=1, ring_size=1, t5_fsdp=False, t5_cpu=False, dit_fsdp=False, save_file='meigen_multitalk_result_12_steps_20250710152621714884', audio_save_dir='save_audio', base_seed=42, input_json='/tmp/tmpnrmazw6k.json', motion_frame=25, mode='streaming', sample_steps=12, sample_shift=7, sample_text_guide_scale=5.0, sample_audio_guide_scale=4.0, num_persistent_param_in_dit=11000000000, use_teacache=True, teacache_thresh=0.2, use_apg=False, apg_momentum=-0.75, apg_norm_threshold=55)
[2025-07-10 15:26:28,183] INFO: Generation model config: {'name': 'Config: Wan MultiTalk AI2V 14B', 't5_model': 'umt5_xxl', 't5_dtype': torch.bfloat16, 'text_len': 512, 'param_dtype': torch.bfloat16, 'num_train_timesteps': 1000, 'sample_fps': 16, 'sample_neg_prompt': 'bright tones, overexposed, static, blurred details, subtitles, style, works, paintings, images, static, overall gray, worst quality, low quality, JPEG compression residue, ugly, incomplete, extra fingers, poorly drawn hands, poorly drawn faces, deformed, disfigured, misshapen limbs, fused fingers, still picture, messy background, three legs, many people in the background, walking backwards', 't5_checkpoint': 'models_t5_umt5-xxl-enc-bf16.pth', 't5_tokenizer': 'google/umt5-xxl', 'clip_model': 'clip_xlm_roberta_vit_h_14', 'clip_dtype': torch.float16, 'clip_checkpoint': 'models_clip_open-clip-xlm-roberta-large-vit-huge-14.pth', 'clip_tokenizer': 'xlm-roberta-large', 'vae_checkpoint': 'Wan2.1_VAE.pth', 'vae_stride': (4, 8, 8), 'patch_size': (1, 2, 2), 'dim': 5120, 'ffn_dim': 13824, 'freq_dim': 256, 'num_heads': 40, 'num_layers': 40, 'window_size': (-1, -1), 'qk_norm': True, 'cross_attn_norm': True, 'eps': 1e-06}
Traceback (most recent call last):
File "/home/user/app/generate_multitalk.py", line 508, in
generate(args)
File "/home/user/app/generate_multitalk.py", line 429, in generate
audio_embedding_1 = get_embedding(new_human_speech1, wav2vec_feature_extractor, audio_encoder)
File "/home/user/app/generate_multitalk.py", line 275, in get_embedding
embeddings = audio_encoder(audio_feature, seq_len=int(video_length), output_hidden_states=True)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
File "/home/user/app/src/audio_analysis/wav2vec2.py", line 23, in forward
self.config.output_attentions = True
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/transformers/configuration_utils.py", line 204, in setattr
super().setattr(key, value)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/transformers/configuration_utils.py", line 341, in output_attentions
raise ValueError(
ValueError: The output_attentions attribute is not supported when using the attn_implementation set to sdpa. Please set it to 'eager' instead.
[INFO] Removed temporary file: /tmp/tmpnrmazw6k.json
Traceback (most recent call last):
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/queueing.py", line 625, in process_events
response = await route_utils.call_process_api(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/route_utils.py", line 322, in call_process_api
output = await app.get_blocks().process_api(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/blocks.py", line 2220, in process_api
result = await self.call_function(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/blocks.py", line 1731, in call_function
prediction = await anyio.to_thread.run_sync( # type: ignore
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/to_thread.py", line 56, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 2470, in run_sync_in_worker_thread
return await future
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 967, in run
result = context.run(func, *args)
File "/home/user/.pyenv/versions/3.10.18/lib/python3.10/site-packages/gradio/utils.py", line 904, in wrapper
response = f(*args, **kwargs)
File "/home/user/app/app.py", line 219, in infer
raise RuntimeError("Inference failed. Check inference.log for details.")
RuntimeError: Inference failed. Check inference.log for details.

Any idea on what can be going wrong and how to fix it?
Thanks in advance for your help

[doggiedaddy69]

Any chance to run this on local 5090?!

Following

[samyaksgupta]

hey i am getting the same error while running , showing the same inference error,

[alexyeoh]

even after duplicating the space, it's not working anymore. I am always hit with the same Inference Error too.

[fffiloni]

Hello everyone 👋

sorry for the late response, i was off for a couple of days.

The problem was due to an incompatibility with newest transformers version, so i fixed the demo by specifying the right module versions.

I’ve tested it, and everything should be back running properly now 🤗

[samyaksgupta]

hey @fffiloni its still not working , spaces says queue too long and that is understandable, but when i am dockerizing it, it still shows inference error check inference.log while generating with the examples present, also i cannot see "use via api" section under the app when its being ported to my localhost, maybe cause its set to false in app.py.

Edit: hey i made it work, the dockerfile isn't updated (1 month old update) properly so i had to manually change some pip packages, even after all of that it didn't work cause the gpu was running a lot of things and it needs a lot of computing power and clean gpu to run, anyways
Thanks though great work!