Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RPCS3/PlayStation Home's videos not displaying properly, distorted but audio plays. #11751

Open
PSHFigment opened this issue Apr 10, 2022 · 18 comments · Fixed by #13413
Open

RPCS3/PlayStation Home's videos not displaying properly, distorted but audio plays. #11751

PSHFigment opened this issue Apr 10, 2022 · 18 comments · Fixed by #13413
Labels
Firmware: HLE Video

Comments

@PSHFigment
Copy link

Quick summary

RPCS3 doesn't support or incorrectly supports MP4 format of video codec's used in PlayStation Home.

Details

RPCS3 doesn't correctly display Home's video format. visuals distorted but audio plays fine.
(PS3, display of video in Bowling Alley.)
6371df6edb4894de94281c6d31a59ee7
(RPCS3, display of video in Bowling Alley.)
7d76fa561550611fd71272af33790958
(Original Video from PlayStation Home content snapshot..)
https://user-images.githubusercontent.com/74638626/162596647-f56fdc1c-d730-406c-9d45-5637caa88e83.MP4

Logs

https://cdn.discordapp.com/attachments/838553670831308820/962539682044997672/RPCS3.log

Specs

Processor Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz 3.60 GHz
Installed RAM 32.0 GB
GPU NVIDIA GeForce RTX 2080 Ti
System type 64-bit operating system, x64-based processor
Edition Windows 10 Home
Version 21H1
OS build 19043.1586
&
Ryzen 5800H
GeForce 3050ti
16GB Ram
Windows 11
@NagatoDEV
Copy link

NagatoDEV commented Apr 10, 2022 •
edited
Loading

Information regarding PlayStation Home Supported Video Codec Formats:

Greetings RPCS3 developers, I hope this message finds you well. I'm one of the developers for the Destination Home project and I wanted to add some additional information/details that can possibly give insight on this issue.

To add to this post PlayStation Home supports the following media/video codecs.

  • MPEG-4 Part 2 (MPEG-4 Video) Simple Profile - up to Level 3
  • MPEG-4 Part 2 (MPEG-4 Video) Advanced Simple Profile - up to Level 5
  • MPEG-4 Part 10 (AVC/H.264) - up to Level 3.1

Furthermore, it also supports the following video container formats:

  • MPEG-4 Part 14 (MP4)
  • MPEG-2 Part 1 Transport Stream (TS)

I know that Home as well can play video bit rates up to the maximum allowed for each supported MPEG-4 profile level e.g., up to 8 Mbps for MPEG-4 & 14 Mbps for AVC/H.264. From research, I know there are some general guidelines that the Home client will accept for video content. Note this, is just what I've seen from various media content for scenes on Home.

Format Width Height FPS
MPEG-4 Part 2, Simple Profile 176 144 15 FPS
MPEG-4 Part 2, Simple Profile 352 288 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 352 288 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 640 480 aprox. 29 FPS
MPEG-4 Part 2, Advanced Simple Profile 720 576 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 320 240 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 640 480 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 720 480 aprox. 29 FPS
MPEG-4 Part 10 (AVC/H.264) 1280 720 aprox. 29 FPS

PlayStation Home Supported Audio Codecs:

  • MP3: Up to 320kbps, 48kHz, Stereo
  • AAC: Up to 320kbps, 48kHz, Stereo |
    • AAC audio is considered as a video by the client due to the format container being the same as MPEG-4.
  • HE-AAC v1 (AAC+): Up to 320kbps, 48kHz, Stereo
  • HE-AAC v2 (eAAC+): Up to 320kbps, 48kHz, Stereo

Our current bug/issue

It seems on our end, that the audio from content is being played just fine. However, as a showcase from the image below you can clearly see the screen is distorted.

NPIA00005_screenshot_2022_04_10_00_33_22

In contrast on original hardware (PS3) you can see that in the same location the video content is playing as intended, for the end-user.

PlayStationAHome_Developer_HEN-QA_Launcher

The original video content displayed on PlayStation Home

NowPlaying_AnimalHouse_ChainSwing.mp4

This video (NowPlaying_AnimalHouse_ChainSwing.mp4) has the following properties:

General

  • Complete name: C:\Users\Nagato\Desktop\NowPlaying_AnimalHouse_ChainSwing.mp4
  • Format: MPEG-4
  • Format profile: Base Media / Version 2
  • Codec ID: mp42 (mp41/isom)
  • File size: 7.27 MiB
  • Duration: 3 min 27 s
  • Overall bit rate: 295 kb/s

Video

  • ID: 1
  • Format: AVC
  • Format/Info: Advanced Video Codec
  • Format profile: Baseline@L3.1
  • Format settings: 1 Ref Frames
  • Format settings, CABAC: No
  • Format settings, Reference frames: 1 frame
  • Format settings, GOP: M=1, N=30
  • Codec ID: avc1
  • Codec ID/Info: Advanced Video Coding
  • Duration: 3 min 27 s
  • Source duration: 3 min 27 s
  • Bit rate: 96.1 kb/s
  • Width: 854 pixels
  • Height: 480 pixels
  • Display aspect ratio: 16:9
  • Frame rate mode: Constant
  • Frame rate: 29.970 (29970/1000) FPS
  • Color space: YUV
  • Chroma subsampling: 4:2:0
  • Bit depth: 8 bits
  • Scan type: Progressive
  • Bits/(Pixel*Frame): 0.008
  • Stream size: 2.37 MiB (33%)
  • Source stream size: 2.37 MiB (33%)
  • mdhd_Duration: 207140
  • Codec configuration box: avcC

Audio

ID: 2

  • Format: AAC LC
  • Format/Info: Advanced Audio Codec Low Complexity
  • Codec ID: mp4a-40-2
  • Duration: 3 min 27 s
  • Bit rate mode: Constant
  • Bit rate: 192 kb/s
  • Channel(s): 2 channels
  • Channel layout: L R
  • Sampling rate: 48.0 kHz
  • Frame rate: 46.875 FPS (1024 SPF)
  • Compression mode: Lossy
  • Stream size: 4.83 MiB (66%)
  • mdhd_Duration: 207061

RPCS3 Logs

To provide some documentation, here were the original logs from my current gameplay on RPCS3.

My Specs

  • OS NameMicrosoft Windows 11 Home
  • Version -10.0.22000 Build 22000
  • Other OS Description - Not Available
  • OS Manufacturer - Microsoft Corporation
  • System Name - LAPTOP-BR7PP4O2
  • System Manufacturer - LENOVO
  • System Model - 82JW
  • System Type x64-based PC
  • System SKU - LENOVO_MT_82JW_BU_idea_FM_Legion 5 15ACH6
  • Processor - AMD Ryzen 7 5800H with Radeon Graphics, 3201 Mhz, 8 Core(s), 16 Logical Processor(s)
  • BIOS Version/Date - LENOVO HHCN20WW, 10/1/2021
  • SMBIOS Version -3.3
  • Embedded Controller Version - 1.20
  • BIOS Mode - UEFI
  • BaseBoard Manufacturer -LENOVO
  • BaseBoard Product -LNVNB161216
  • BaseBoard Version - SDK0T76461 WIN
  • Platform Role - Mobile
  • Secure Boot State On
  • PCR7 Configuration - Elevation Required to View
  • Windows Directory - C:\Windows
  • System Directory - C:\Windows\system32
  • Boot Device - \Device\HarddiskVolume1
  • Locale - United States
  • Hardware Abstraction Layer Version = "10.0.22000.527"
  • User Name - LAPTOP-BR7PP4O2\Nagato
  • Time Zone - Eastern Daylight Time
  • Installed Physical Memory (RAM) - 16.0 GB
  • Total Physical Memory - 13.9 GB
  • Available Physical Memory - 4.63 GB
  • Total Virtual Memory - 23.9 GB
  • Available Virtual Memory - 6.41 GB
  • Page File Space - 10.0 GB

@Megamouse
Copy link
Contributor

Thanks for the detailed info. But please hide or remove the huge log and upload a file attachment instead

@NagatoDEV
Copy link

Thanks for the detailed info. But please hide or remove the huge log and upload a file attachment instead

Hey Megamouse, I've redacted the huge logfile from the original post & thank you in advance for looking into this issue. Down below here are the logs in a .txt file RPCS3_PlayStation_Home_Video_Bug (3).txt

Further investigation regarding PlayStation Home's video content:

NPIA00005_screenshot_2022_04_11_19_24_20

After further analysis, it seems that some video content on Home runs on RPCS3 as intended in certain spaces. For instance, in the Konami Penthouse public scene, the video is parsing information for the client. In turn, the end-user will be able to see the video along with its original audio. For documentation purposes here is a quick video of Konami_clothing_PSHOME_Profile3.mp4 running along with RPCS3 logs for your review.

2022-04-11.19-20-20.mp4

Not so sure why some video content is working without any issues in certain areas versus others, but hopefully this brief synopsis can help you out.

Konami_clothing_PSHOME_Profile3.mp4 Properties:

Here is the original Konami_clothing_PSHOME_Profile3.mp4 file that you can review as well.

Konami_clothing_PSHOME_Profile3.mp4

General:

  • Complete name: C:\Users\Calde\Downloads\Konami_clothing_PSHOME_Profile3.mp4
  • Format: QuickTime
  • Format/Info: Original Apple specifications
  • File size: 6.89 MiB
  • Duration: 40 s 407 ms
  • Overall bit rate mode: Variable
  • Overall bit rate: 1 431 kb/s
  • Encoded date: UTC 2011-06-17 22:44:55
  • Tagged date: UTC 2011-06-17 22:44:55
  • Writing library: Apple QuickTime
  • FileExtension_Invalid: braw mov qt

Video:

  • ID: 2
  • Format: MPEG-4 Visual
  • Format profile: Simple@L4a
  • Format settings, BVOP: No
  • Format settings, QPel: No
  • Format settings, GMC: No warppoints
  • Format settings, Matrix: Default (H.263)
  • Codec ID: mp4v-20
  • Duration: 40 s 407 ms
  • Bit rate mode: Variable
  • Bit rate: 1 300 kb/s
  • Nominal bit rate: 1 400 kb/s
  • Maximum bit rate: 2 476 kb/s
  • Width: 640 pixels
  • Height: 352 pixels
  • Display aspect ratio: 16:9
  • Frame rate mode: Constant
  • Frame rate: 29.970 (29970/1000) FPS
  • Color space: YUV
  • Chroma subsampling: 4:2:0
  • Bit depth: 8 bits
  • Scan type: Progressive
  • Compression mode: Lossy
  • Bits/(Pixel*Frame): 0.193
  • Stream size: 6.26 MiB (91%)
  • Language: English
  • Encoded date : UTC 2011-06-17 22:44:55
  • Tagged date: UTC 2011-06-17 22:44:55

Audio:

  • ID: 1
  • Format: AAC LC
  • Format/Info: Advanced Audio Codec Low Complexity
  • Codec ID : mp4a-40-2
  • Duration: 40 s 333 ms
  • Bit rate mode: Variable
  • Bit rate: 128 kb/s
  • Maximum bit rate: 136 kb/s
  • Channel(s: 2 channels
  • Channel layout: L R
  • Sampling rate: 44.1 kHz
  • Frame rate: 43.066 FPS (1024 SPF)
  • Compression mode: Lossy
  • Stream size: 630 KiB (9%)
  • Language: English
  • Encoded date: UTC 2011-06-17 22:44:55
  • Tagged date: UTC 2011-06-17 22:44:55

Other #1

  • ID: 3
  • Type: Object description
  • Format: System Core
  • Codec ID: mp4s-02
  • Duration: 40 s 407 ms
  • Language English
  • Encoded date: UTC 2011-06-17 22:44:55
  • Tagged date: UTC 2011-06-17 22:44:55
  • mdhd_Duration: 1000

Other #2:

  • ID: 4
  • Type: Scene description
  • Format: System Core
  • Codec ID: mp4s-02
  • Duration: 40 s 407 ms
  • Language: English
  • Encoded date: UTC 2011-06-17 22:44:55
  • Tagged date: UTC 2011-06-17 22:44:55
  • mdhd_Duration: 1000

My Specs:

  • OS Name - Microsoft Windows 11 Home
  • Version -10.0.22000 Build 22000
  • Other OS Description - Not Available
  • OS Manufacturer - Microsoft Corporation
  • System Name - LAPTOP-BR7PP4O2
  • System Manufacturer - LENOVO
  • System Model - 82JW
  • System Type x64-based PC
  • System SKU - LENOVO_MT_82JW_BU_idea_FM_Legion 5 15ACH6
  • Processor - AMD Ryzen 7 5800H with Radeon Graphics, 3201 Mhz, 8 Core(s), 16 Logical Processor(s)
  • BIOS Version/Date - LENOVO HHCN20WW, 10/1/2021
  • SMBIOS Version -3.3
  • Embedded Controller Version - 1.20
  • BIOS Mode - UEFI
  • BaseBoard Manufacturer -LENOVO
  • BaseBoard Product -LNVNB161216
  • BaseBoard Version - SDK0T76461 WIN
  • Platform Role - Mobile
  • Secure Boot State On
  • PCR7 Configuration - Elevation Required to View
  • Windows Directory - C:\Windows
  • System Directory - C:\Windows\system32
  • Boot Device - \Device\HarddiskVolume1
  • Locale - United States
  • Hardware Abstraction Layer Version = "10.0.22000.527"
  • User Name - LAPTOP-BR7PP4O2\Nagato
  • Time Zone - Eastern Daylight Time
  • Installed Physical Memory (RAM) - 16.0 GB
  • Total Physical Memory - 13.9 GB
  • Available Physical Memory - 4.63 GB
  • Total Virtual Memory - 23.9 GB
  • Available Virtual Memory - 6.41 GB
  • Page File Space - 10.0 GB

@Megamouse
Copy link
Contributor

please try with #11754

@PSHFigment
Copy link
Author

PSHFigment commented Apr 13, 2022 •
edited
Loading

Screen is completely black now instead of glitching with this client (audio works but no video.), this gets spammed into the log as well.

E SYS: 'cellVdecDecodeAu' failed with 0x80610101 : CELL_VDEC_ERROR_ARG, vdec=1, auInfo=1, upper=0, startAddr=0x42daf380 [1]

2022-04-12_23-37-27.1.mp4

@Megamouse

This comment was marked as outdated.

Sign in to view
@Megamouse
Copy link
Contributor

Please try again with the latest version

@NagatoDEV
Copy link

Hey Megamouse, sorry for the delayed response I have been busy with my academic career, and just saw this comment. However, I just tested out the RPCS3 client version rpcs3-v0.0.21-13475-6a671554_win64 & ran PlayStation Home, and unfortunately the client still only displays audio & the actual video footage seems to be distorted/fuzzy.

2022-04-22.16-35-26.mp4

Logs

As always, here are logs from this session.

@Megamouse
Copy link
Contributor

how can i reproduce this on my own?

@kd-11
Copy link
Contributor

kd-11 commented Apr 22, 2022

The newest error looks like output pitch is broken. Hmm, maybe it is worth trying enabling WCB and CPU blit to debug this.
But yea, we need some kind of steps to reproduce this one ourselves.

@Megamouse
Copy link
Contributor

It might still be a straightforward vdec pitch issue, similar to what i fixed for Uncharted 2.

@kd-11
Copy link
Contributor

kd-11 commented Apr 22, 2022

Yea. It seems like the data is there, just incorrect scanning pitch.

@NagatoDEV
Copy link

Hey Megamouse, here is the offline client you can look at it to replicate this issue, Home has an MLA issue in this client. However, with Home's navigator, you can travel to Central Plaza. Just for documentation purposes, here is a quick guide I posted on YouTube on how to fully travel to Central Plaza for debugging.

Download

https://github.com/NagatoDEV/PlayStation-Home-Master-Archive/releases/tag/v1.00

@illusion0001
Copy link
Contributor

What happens if you LLE libvdec?

@NagatoDEV
Copy link

Hey, illusion0001 that seemed to actually work by activating the LLE libvdec.sprx option in RPCS3 settings to activate with the following client rpcs3-v0.0.21-13475-6a671554_win64

Images/Videos

2022-04-22.20-49-41.mp4

NPIA00005_screenshot_2022_04_22_21_04_43

NPIA00005_screenshot_2022_04_22_21_10_25

@illusion0001
Copy link
Contributor

Thanks for confirming, only HLE issue now

@Megamouse
Copy link
Contributor

Megamouse commented Apr 23, 2022 •
edited
Loading

It appears that the rendering and the outgoing data are correct.
So the issue is caused earlier.
This means that either the data offsets or pitch is wrongly set by us, or the frame is not decoded correctly or the frame data is wrong in the first place.

I think it's just us doing some derpy stuff with av. I managed to get a slightly different picture by using a different method of determining the incoming data offsets, but it seems I only made it worse xD

@Megamouse Megamouse mentioned this issue Feb 18, 2023
Merged
@Megamouse Megamouse reopened this Mar 5, 2023
@Megamouse
Copy link
Contributor

The fix wasn't correct and broke other games.

@FlexBy420 FlexBy420 mentioned this issue Aug 19, 2023
Open
25 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Firmware: HLE Video
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cellVdec: fix pitch of RGBA/ARGB images Megamouse/rpcs3
5 participants
@kd-11 @Megamouse @illusion0001 @NagatoDEV @PSHFigment