Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Panic in tile_restoration_state on encoding YUV422 sample #1608

Closed
shssoichiro opened this issue Aug 29, 2019 · 9 comments
Closed

Panic in tile_restoration_state on encoding YUV422 sample #1608

shssoichiro opened this issue Aug 29, 2019 · 9 comments
Labels
bug

Comments

@shssoichiro
Copy link
Collaborator

I have the following test file: yuv422p8_input.tar.gz which is a 3-frame clip in YUV422. I have tested the same clip in YUV420 and YUV444 successfully. This YUV422 version crashes rav1e with the following backtrace:

➜ RUST_BACKTRACE=1 rav1e yuv422p8_input.y4m -o yuv422p8_output.ivf --quantizer 160
640x360 @ 25/1 fps

Run with RUST_BACKTRACE=full to include source snippets.
Backtrace (most recent call first):
  File "src/tiling/tile_restoration_state.rs", line 208, in rav1e::encoder::encode_tile
  File "/home/soichiro/repos/rav1e/src/encoder.rs", line 2759, in rav1e::encoder::encode_tile_group::{{closure}}
  File "rust:src/libcore/ops/function.rs", line 249, in core::ops::function::impls::<impl core::ops::function::FnMut<A> for &F>::call_mut
  File "rust:src/libcore/ops/function.rs", line 279, in core::ops::function::impls::<impl core::ops::function::FnOnce<A> for &mut F>::call_once
  File "rust:src/libcore/option.rs", line 416, in core::option::Option<T>::map
  File "rust:src/libcore/iter/adapters/mod.rs", line 570, in <core::iter::adapters::Map<I,F> as core::iter::traits::iterator::Iterator>::next
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 178, in rayon::iter::plumbing::Folder::consume_iter
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/map.rs", line 248, in <rayon::iter::map::MapFolder<C,F> as rayon::iter::plumbing::Folder<T>>::consume_iter                                                                                                                                                                                    
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 110, in rayon::iter::plumbing::Producer::fold_with
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 438, in rayon::iter::plumbing::bridge_producer_consumer::helper
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 397, in rayon::iter::plumbing::bridge_producer_consumer
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 373, in <rayon::iter::plumbing::bridge::Callback<C> as rayon::iter::plumbing::ProducerCallback<I>>::callback                                                                                                                                                           
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/vec.rs", line 69, in <rayon::vec::IntoIter<T> as rayon::iter::IndexedParallelIterator>::with_producer
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/plumbing/mod.rs", line 357, in rayon::iter::plumbing::bridge
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/vec.rs", line 34, in <rayon::vec::IntoIter<T> as rayon::iter::ParallelIterator>::drive_unindexed
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/map.rs", line 49, in <rayon::iter::map::Map<I,F> as rayon::iter::ParallelIterator>::drive_unindexed                                                                                                                                                                                           
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/unzip.rs", line 271, in <rayon::iter::unzip::UnzipB<I,OP,CA> as rayon::iter::ParallelIterator>::drive_unindexed                                                                                                                                                                               
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/collect/mod.rs", line 43, in rayon::iter::collect::special_extend
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/collect/mod.rs", line 137, in rayon::iter::collect::<impl rayon::iter::ParallelExtend<T> for alloc::vec::Vec<T>>::par_extend                                                                                                                                                                  
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/unzip.rs", line 221, in <rayon::iter::unzip::UnzipA<I,OP,FromB> as rayon::iter::ParallelIterator>::drive_unindexed                                                                                                                                                                            
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/collect/mod.rs", line 43, in rayon::iter::collect::special_extend
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/collect/mod.rs", line 137, in rayon::iter::collect::<impl rayon::iter::ParallelExtend<T> for alloc::vec::Vec<T>>::par_extend                                                                                                                                                                  
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/unzip.rs", line 54, in rayon::iter::unzip::execute_into
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/unzip.rs", line 38, in rayon::iter::unzip::execute
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/unzip.rs", line 69, in rayon::iter::unzip::unzip
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-1.1.0/src/iter/mod.rs", line 1835, in rayon::iter::ParallelIterator::unzip
  File "/home/soichiro/repos/rav1e/src/encoder.rs", line 2753, in rav1e::encoder::encode_tile_group
  File "/home/soichiro/repos/rav1e/src/encoder.rs", line 3341, in rav1e::encoder::encode_frame
  File "/home/soichiro/repos/rav1e/src/api/mod.rs", line 2046, in rav1e::api::ContextInner<T>::receive_packet
  File "/home/soichiro/repos/rav1e/src/api/mod.rs", line 1072, in rav1e::api::Context<T>::receive_packet::{{closure}}
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/thread_pool/mod.rs", line 132, in rayon_core::thread_pool::ThreadPool::install::{{closure}}
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 495, in rayon_core::registry::Registry::in_worker_cold::{{closure}}
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/job.rs", line 115, in <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::{{closure}}                                                                                                                                                                                        
  File "rust:src/libstd/panic.rs", line 315, in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  File "rust:src/libstd/panicking.rs", line 296, in std::panicking::try::do_call
  File "rust:/src/libpanic_unwind/lib.rs", line 82, in std::panicking::try
  File "rust:src/libstd/panic.rs", line 394, in std::panic::catch_unwind
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/unwind.rs", line 17, in rayon_core::unwind::halt_unwinding
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/job.rs", line 115, in <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/job.rs", line 59, in rayon_core::job::JobRef::execute
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 734, in rayon_core::registry::WorkerThread::execute
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 718, in rayon_core::registry::WorkerThread::wait_until_cold
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 692, in rayon_core::registry::WorkerThread::wait_until
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 812, in rayon_core::registry::main_loop
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 58, in rayon_core::registry::ThreadBuilder::run
  File "/home/soichiro/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.5.0/src/registry.rs", line 103, in <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}}                                                                                                                                                                      
  File "rust:src/libstd/sys_common/backtrace.rs", line 77, in std::sys_common::backtrace::__rust_begin_short_backtrace
  File "rust:src/libstd/thread/mod.rs", line 470, in std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}
  File "rust:src/libstd/panic.rs", line 315, in <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once
  File "rust:src/libstd/panicking.rs", line 296, in std::panicking::try::do_call
  File "rust:/src/libpanic_unwind/lib.rs", line 82, in std::panicking::try
  File "rust:src/libstd/panic.rs", line 394, in std::panic::catch_unwind
  File "rust:src/libstd/thread/mod.rs", line 469, in std::thread::Builder::spawn_unchecked::{{closure}}
  File "rust:src/libcore/ops/function.rs", line 231, in core::ops::function::FnOnce::call_once{{vtable.shim}}
  File "rust:src/liballoc/boxed.rs", line 746, in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
  File "rust:src/liballoc/boxed.rs", line 746, in <alloc::boxed::Box<F> as core::ops::function::FnOnce<A>>::call_once
  File "src/libstd/sys_common/thread.rs", line 13, in std::sys_common::thread::start_thread
  File "src/libstd/sys/unix/thread.rs", line 79, in std::sys::unix::thread::Thread::new::thread_start
  File "<unknown>", line 0, in start_thread
  File "<unknown>", line 0, in __clone
  File "<unknown>", line 0, in <unknown>

The application panicked (crashed).
  internal error: entered unreachable code
in src/tiling/tile_restoration_state.rs, line 237
thread: <unnamed>
@lu-zero
Copy link
Collaborator

lu-zero commented Aug 29, 2019

Is there a known-working commit?

@shssoichiro
Copy link
Collaborator Author

Yes, doing bisect now.

@shssoichiro
Copy link
Collaborator Author

First bad commit is 5271249

@YaLTeR
Copy link
Collaborator

YaLTeR commented Aug 29, 2019 

└─ target/debug/rav1e ~/Downloads/yuv422p8_input.y4m -o yuv422p8_output.ivf --quantizer 160
640x360 @ 25/1 fps
Backtrace (most recent call first):
  File "/home/yalter/Source/rust/rav1e-2/src/tiling/tile_state.rs", line 0, in rav1e::tiling::tile_restoration_state::TileRestorationStateMut::new
Backtrace (most recent call first):
  File "/home/yalter/Source/rust/rav1e-2/src/tiling/tile_state.rs", line 0, in rav1e::tiling::tile_restoration_state::TileRestorationStateMut::new
thread panicked while processing panic. aborting.
fish: “target/debug/rav1e ~/Downloads/…” terminated by signal SIGILL (Illegal instruction)

nice

@lu-zero
Copy link
Collaborator

lu-zero commented Aug 29, 2019

Disable the pretty backtrace, looks like there is a bug there :/

@YaLTeR
Copy link
Collaborator

YaLTeR commented Aug 29, 2019

The backtrace seems to work on the release build at least.

@rzumer
Copy link
Collaborator

rzumer commented Aug 29, 2019

I have the same crash trying to encode a frame sized 6820x4320 (4:2:0).

I tested the revision prior to the one that broke for this issue, and it seemed to work. My computer cannot handle it and froze trying to encode it, but since it did not crash I think it's likely that the same commit broke both.

xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
f1b5c79 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
a2d451d 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
5a6b596 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
2bdc019 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
9a84b92 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit to xiphmont/rav1e that referenced this issue Sep 5, 2019
@xiphmont
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
aa12165 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
xiphmont added a commit that referenced this issue Sep 5, 2019
@xiphmont
Fix for #1601, #1611 and #1629 and possible #1608
ef482d4 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
@xiphmont
Copy link
Contributor

xiphmont commented Sep 5, 2019

retest with master please? #1633 very possibly fixed this. If it did not, I'd like to know.

@shssoichiro
Copy link
Collaborator Author

@xiphmont Yes, this is fixed now. Thanks!

@YaLTeR The reason you get illegal instruction is because there's a bug in rustc with opt-level >= 2 and debug assertions enabled. The compiler optimizes away the code that should handle the panic, and you get an illegal instruction. It's hit me a number of times while debugging my code. We could go back to opt-level = 1, but that's pretty slow, unfortunately.

lu-zero pushed a commit to rust-av/rav1e that referenced this issue Nov 27, 2019
@xiphmont @lu-zero
Fix for xiph#1601, xiph#1611 and xiph#1629 and possible xiph#1608
d820e69 
This patch set attempts to both improve cooperation between LRF and
tiling setup, but also eliminate several illegal configurations.

The patch strategy is to consistently eliminate evaluation of
'stretch' superblocks in RDO of LRUs for the loop restoration filter.
This primarily affects the case where a tile consists of
primarily/exclusively SBs that belong to an LRU stretched from some
other tile.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@lu-zero @YaLTeR @shssoichiro @rzumer @xiphmont