Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unexpectedly small, yet non-zero sigma2 value #1191

Open
DrJesseHansen opened this issue Oct 9, 2024 · 1 comment
Open

unexpectedly small, yet non-zero sigma2 value #1191

DrJesseHansen opened this issue Oct 9, 2024 · 1 comment

Comments

@DrJesseHansen
Copy link

3D refinement with subvolumes gives the error cited. Subvolumes extracted from WarpTools 2.0.0/dev28. reconstruct_particle gives a very nice reconstruction.

mpirun --np 5 --oversubscribe relion_refine_mpi --o Refine3D/job001_3D/run --auto_refine --split_random_halves --i allparticles_bin8_3D.star --ref InitialModel/recon_3D.mrc --trust_ref_size --ini_high 30 --dont_combine_weights_via_disc --pool 3 --pad 2 --ctf --particle_diameter 400 --flatten_solvent --zero_mask --solvent_mask masks/fullmask_final.mrc --solvent_correct_fsc --oversampling 1 --healpix_order 4 --auto_local_healpix_order 4 --offset_range 2 --offset_step 2 --sym C1 --low_resol_join_halves 40 --norm --scale --j 4 --gpu "" --pipeline_control Refine3D/job108/

error:
`=== RELION MPI setup ===

  • Number of MPI processes = 5
  • Number of threads per MPI process = 4
  • Total number of threads therefore = 20
  • Leader (0) runs on host = gpu265
  • Follower 1 runs on host = gpu265
  • Follower 2 runs on host = gpu265
  • Follower 3 runs on host = gpu265
  • Follower 4 runs on host = gpu265
    ==========================
    uniqueHost gpu265 has 4 ranks.
    GPU-ids not specified for this rank, threads will automatically be mapped to available devices.
    Thread 0 on follower 1 mapped to device 0
    Thread 1 on follower 1 mapped to device 0
    Thread 2 on follower 1 mapped to device 0
    Thread 3 on follower 1 mapped to device 0
    GPU-ids not specified for this rank, threads will automatically be mapped to available devices.
    Thread 0 on follower 2 mapped to device 1
    Thread 1 on follower 2 mapped to device 1
    Thread 2 on follower 2 mapped to device 1
    Thread 3 on follower 2 mapped to device 1
    GPU-ids not specified for this rank, threads will automatically be mapped to available devices.
    Thread 0 on follower 3 mapped to device 2
    Thread 1 on follower 3 mapped to device 2
    Thread 2 on follower 3 mapped to device 2
    Thread 3 on follower 3 mapped to device 2
    GPU-ids not specified for this rank, threads will automatically be mapped to available devices.
    Thread 0 on follower 4 mapped to device 3
    Thread 1 on follower 4 mapped to device 3
    Thread 2 on follower 4 mapped to device 3
    Thread 3 on follower 4 mapped to device 3
    Running CPU instructions in double precision.
    WARNING: allparticles_bin8_3D.star seems to be from a previous version of Relion. Attempting conversion...
    You should make sure metadata in the optics group table after conversion is correct.
    Estimating initial noise spectra from at most 10 particles
    0/ 0 sec ............................................................(,_,">
    Auto-refine: Iteration= 1
    Auto-refine: Resolution= 29.4827 (no gain for 0 iter)
    Auto-refine: Changes in angles= 999 degrees; and in offsets= 999 Angstroms (no gain for 0 iter)
    Estimating accuracies in the orientational assignment ...
    0/ 0 sec ............................................................
    (,,">
    Auto-refine: Estimated accuracy angles= 11.35 degrees; offsets= 9.72928 Angstroms
    Auto-refine: WARNING: Iter = 1 The angular accuracy is worse than 10 degrees, so basically you cannot align your particles (yet)!
    Auto-refine: WARNING: You probably need not worry if the accuracy improves during the next few iterations.
    Auto-refine: WARNING: However, if the problem persists it may lead to spurious FSC curves, so be wary of inflated resolution estimates...
    Auto-refine: WARNING: Sometimes it is better to tune resolution yourself by adjusting T in a 3D-classification with a single class.
    CurrentResolution= 29.4827 Angstroms, which requires orientationSampling of at least 8.37209 degrees for a particle of diameter 400 Angstroms
    Oversampling= 0 NrHiddenVariableSamplingPoints= 1085
    OrientationalSampling= 3.75 NrOrientations= 155
    TranslationalSampling= 22.112 NrTranslations= 7
    =============================
    Oversampling= 1 NrHiddenVariableSamplingPoints= 69440
    OrientationalSampling= 1.875 NrOrientations= 1240
    TranslationalSampling= 11.056 NrTranslations= 56
    =============================
    Expectation iteration 1
    2.68/2.68 min ............................................................~~(,
    ,">
    Averaging half-reconstructions up to 40 Angstrom resolution to prevent diverging orientations ...
    Note that only for higher resolutions the FSC-values are according to the gold-standard!
    Calculating solvent-corrected gold-standard FSC ...
    WARNING: FSC curve between unmasked maps never drops below 0.8. Using unmasked FSC as FSC_true...
    WARNING: This message should go away during the later stages of refinement!
    Maximization...
    000/??? sec ~~(,_,"> [oo] DIRECT_A1D_ELEM(sigma2, i)= nan
    in: /nfs/scistore16/itgrp/schloegl/src/relion/src/backprojector.cpp, line 1088
    ERROR:
    BackProjector::reconstruct: ERROR: unexpectedly small, yet non-zero sigma2 value, this should not happen...
    === Backtrace ===
    relion_refine_mpi(_ZN11RelionErrorC1ERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEES7_l+0x6a) [0x559c09221b2a]
    relion_refine_mpi(+0x75709) [0x559c0918e709]
    relion_refine_mpi(_ZN14MlOptimiserMpi12maximizationEv+0x11b8) [0x559c0925e2e8]
    relion_refine_mpi(_ZN14MlOptimiserMpi7iterateEv+0x3a1) [0x559c0925ff71]
    relion_refine_mpi(main+0x52) [0x559c0920cf42]
    /usr/lib/x86_64-linux-gnu/libc.so.6(+0x2724a) [0x14b0e4e4624a]
    /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x85) [0x14b0e4e46305]
    relion_refine_mpi(_start+0x21) [0x559c09210951]
    ==================
    ERROR:
    BackProjector::reconstruct: ERROR: unexpectedly small, yet non-zero sigma2 value, this should not happen...

RELION version: 5.0-beta-3-commit-12cf15
exiting with an error ...
ERROR: cannot touch file: Refine3D/job108/RELION_JOB_EXIT_FAILURE

MPI_ABORT was invoked on rank 2 in communicator MPI_COMM_WORLD
with errorcode 13.

NOTE: invoking MPI_ABORT causes Open MPI to kill all MPI processes.
You may or may not see output from other processes, depending on
exactly when Open MPI kills them.

`

meanwhile, 3D classification runs well.

mpirun --np 5 --oversubscribe relion_refine_mpi --o Class3D/job002_bin8_3D/run --i allparticles_bin8_3D.star --ref InitialModel/recon_3D.mrc --firstiter_cc --trust_ref_size --ini_high 60 --dont_combine_weights_via_disc --pool 3 --pad 2 --ctf --iter 25 --tau2_fudge 1 --particle_diameter 400 --fast_subsets --K 1 --flatten_solvent --zero_mask --strict_highres_exp 30 --oversampling 1 --healpix_order 4 --sigma_ang 0.6 --offset_range 5 --offset_step 2 --sym C1 --norm --scale --j 5 --gpu ""

@bwmr
Copy link

bwmr commented Nov 14, 2024

Same thing for me, both with Warp 1.X and WarpTools - same particles work fine in Relion 3.1.4

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants