IndexError: index 0 is out of bounds for axis 0 with size 0

[YingYa]

Hi, I am running into a weird indexing error for some samples.

I run the following command with us.gcr.io/broad-dsde-methods/cellbender:20230403:
cellbender remove-background --input RawMatrix.h5ad --output RawMatrix.remove_background.h5 --cuda

And get this error message:
cellbender:remove-background: CellBender 0.3.0
cellbender:remove-background: (Workflow hash 3722b516c3)
cellbender:remove-background: 2023-04-07 17:35:39
cellbender:remove-background: Running remove-background
cellbender:remove-background: Loading data from /path/to/RawMatrix.h5ad
cellbender:remove-background: AnnData format
cellbender:remove-background: Using .X for count data.
cellbender:remove-background: No feature type information specified in the input file. Including all features in the analysis.
cellbender:remove-background: Trimming features for inference.
cellbender:remove-background: 34263 features have nonzero counts.
Traceback (most recent call last):
File "/opt/conda/bin/cellbender", line 33, in
sys.exit(load_entry_point('cellbender', 'console_scripts', 'cellbender')())
File "/software/cellbender/cellbender/base_cli.py", line 112, in main
cli_dict[args.tool].run(args)
File "/software/cellbender/cellbender/remove_background/cli.py", line 181, in run
main(args)
File "/software/cellbender/cellbender/remove_background/cli.py", line 226, in main
run_remove_background(args)
File "/software/cellbender/cellbender/remove_background/run.py", line 79, in run_remove_background
SingleCellRNACountsDataset(input_file=args.input_file,
File "/software/cellbender/cellbender/remove_background/data/dataset.py", line 111, in init
self.priors = get_priors(umi_counts=counts, low_count_threshold=low_count_threshold)
File "/software/cellbender/cellbender/remove_background/data/priors.py", line 271, in get_priors
priors = get_cell_count_empty_count(
File "/software/cellbender/cellbender/remove_background/data/priors.py", line 186, in get_cell_count_empty_count
new_empty_count_prior, empty_count_upper_limit = _peak_density_given_cutoff(
File "/software/cellbender/cellbender/remove_background/data/priors.py", line 96, in _peak_density_given_cutoff
noncell_counts = np.sort(noncell_counts)[::-1][inds]
IndexError: index 0 is out of bounds for axis 0 with size 0

The input file, RawMatrix.h5ad, include all the empty droplets.

[sjfleming]

Hi @YingYa , interesting. Well, I thought this would be fixed in the latest version, so thank you for bringing it to my attention.

Is there any chance you can share that h5ad file with me? I'd love to use it to debug and see what the problem is. It looks like my heuristics for determining expected cells are failing on this sample.

Can you post a picture of the UMI curve here?
If you specify --expected-cells and --total-droplets-included, will this error still occur?

[YingYa]

Hi @sjfleming, thanks for your respond. Looking forward to the new version.

Yes, the h5ad file is a little big and has been uploaded to drive.google.com, could I have you email and then share the file with you?

The error still occur with --expected-cells and --total-droplets-included were specified.

The UMI curve was posted here:

[sjfleming]

Hi @YingYa , yes my email is sfleming@broadinstitute.org

Thank you! Hopefully I can fix the bug quickly and update the code.

[sjfleming]

Hi @YingYa , I am taking a look at this data now, thank you for sending. I was able to reproduce the same error, so this will be very helpful to debug.

I did have a question about the experiment though. Now that I zoom in on the UMI curve, I see that there appear to be 100k cells in this experiment.

Is that what you expected based on the experimental conditions? Is this a very very overloaded experiment targeting 100k cells?

[YingYa]

Hi @sjfleming, I am analyzing some single nuclear sequencing experiments, the expected cell for our experiment is about 10k cells, the targeting 100k cells is very very overloaded.

[sjfleming]

Hi @YingYa , alright, I have made some updates and I think I can now run your sample (at least I can run a test). See if this works for you:
us.gcr.io/broad-dsde-methods/cellbender:20230425

(This docker image is still at the development stage, and does not correspond to an official release.)

Here's a preliminary result of a short run on your dataset, which I ran using

cellbender remove-background \
    --cuda \
    --input test.RawMatrix.h5ad \
    --output issue190.h5 \
    --training-fraction 0.1 \
    --projected-ambient-count-threshold 1 \
    --epochs 50 \
    --expected-cells 100000 \
    --total-droplets-included 130000

issue190.pdf

[YingYa]

Hi @sjfleming , thanks. I will test that and give the feedback.

[sjfleming]

Closed by #238

[albert239825]

Hi, I'm still getting this error with cellbender version 0.3.0. The dataset I am using is available from here: https://cellxgene.cziscience.com/collections/a0c84e3f-a5ca-4481-b3a5-ccfda0a81ecc. I am using the MRCA: scRNA-seq of the mouse retina - all cells dataset. Thanks for the help!