Import successfull but resulting star file contains only single movie for all tiit angles

[jamstujam]

Hello,

We recently installed Relion 5.0-beta-3.0 on our AlmaLinux 9.4 system and I am trying to use a tilt series through the workflow. However, I've run into an issue with the import. According to relion, the import is successful:

[20:49:46] Started import_tilt_series job. serialem.py:76
Creating output directory at Import/job020/tilt_series serialem.py:80
Found 1 mdoc files and 33 image files. serialem.py:86
Wrote tilt-series data STAR file serialem.py:127
Import/job020/tilt_series.star
writing per tilt-series STAR files... serialem.py:130
Working... ???????????????????????????????????????? 100% 0:00:00
Wrote STAR files for 1 tilt-series. serialem.py:150

However, when I look at the resulting star file for the tilt series, it lists the first tilt image (as listed in the mdoc) for all the tilt angles (column 1; see below). So, it seems to find the image files and mdoc file, but the output is not correct. I have double checked the files (symbolic link to original data using bith cp -s and ln -s), and the mdoc lists the correct movie for each tilt angle, so I'm at a bit of a loss on what to change.

I did try motion correction, however, it only corrects the one tilt movie noted in the star file.

Any advice is appreciated.

Best,
Jamie

Created by the starfile Python package (version 0.4.12) at 20:49:46 on 17/07/2024

data_cells2

loop_
_rlnMicrographMovieName #1
_rlnTomoTiltMovieFrameCount #2
_rlnTomoNominalStageTiltAngle #3
_rlnTomoNominalTiltAxisAngle #4
_rlnMicrographPreExposure #5
_rlnTomoNominalDefocus #6
frames2/12_Jul10_18.31.48.tif 11 0.000165 -88.600000 0.000000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 2.999830 -88.600000 3.300000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 6.000470 -88.600000 6.600000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 9.000130 -88.600000 9.900000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 11.999300 -88.600000 13.200000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 15.000400 -88.600000 16.500000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -2.999500 -88.600000 19.800000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -5.999650 -88.600000 23.100000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -8.999800 -88.600000 26.400000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -12.000000 -88.600000 29.700000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -15.000100 -88.600000 33.000000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 18.000600 -88.600000 36.300000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 21.000200 -88.600000 39.600000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 23.999900 -88.600000 42.900000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 26.999600 -88.600000 46.200000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 29.999200 -88.600000 49.500000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -17.999800 -88.600000 52.800000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -20.999400 -88.600000 56.100000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -23.999600 -88.600000 59.400000 -5.000000
frames2/12_Jul10_18.31.48.tif 11 -27.000700 -88.600000 62.700000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -29.999400 -88.600000 66.000000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 32.998900 -88.600000 69.300000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 35.998100 -88.600000 72.600000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 38.999200 -88.600000 75.900000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 41.997400 -88.600000 79.200000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 44.996100 -88.600000 82.500000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -32.999500 -88.600000 85.800000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -35.999700 -88.600000 89.100000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -38.999800 -88.600000 92.400000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -42.000000 -88.600000 95.700000 -5.000000
frames2/12_Jul10_18.31.48.tif 12 -45.000200 -88.600000 99.000000 -5.000000
frames2/12_Jul10_18.31.48.tif 13 47.997200 -88.600000 102.300000 -5.000000
frames2/12_Jul10_18.31.48.tif 13 -47.999800 -88.600000 105.600000 -5.000000

[EuanPyle]

hi can you paste the mdoc please?

[jamstujam]

Yes, of course. Apologies I was on vacation. I've attached the mdoc for the tilt series that I tried to import.

importmdoc.txt

[EuanPyle]

Not certain, but the problem may be to do with the dots in the file names. Maybe try replacing the dots in the file name (and the corresponding mdoc) with underscores and see what happens. Should be simple enough to write a script to do that but email me tomorrow if you need some help with that

[tsbrant]

Hello,
Was a resolution ever found for this issue? I'm encountering the same problem with one of my datasets.
Thanks!

[sroet]

@tsbrant My group ran into this issue last november and for us updating the filenames to only contain a single . solved it.

We used the following scripts to update the mdoc and rename the files (as always with code from the internet: please make sure you understand what these scripts do before running them!)

To update the mdoc(s) (make sure you back up your mdoc files before running this script):

import os
import re
# Directory containing the .mdoc files
directory = "./"
# Regular expression to match lines that start with 'SubFrameName'
pattern = re.compile(r'^(SubFramePath\s*=\s*)(.+)')
# Function to replace all but the last period with underscores
def replace_periods(line):
    # Split the line into 'SubFrameName' and the actual name
    prefix, name = pattern.match(line).groups()
    
    # Replace all periods but the last one with underscores
    parts = name.rsplit('.', 1)
    modified_name = parts[0].replace('.', '_') + '.' + parts[1]+'\n'
    
    return prefix + modified_name
# Iterate over all files in the directory
for filename in os.listdir(directory):
    if filename.endswith(".mdoc"):
        file_path = os.path.join(directory, filename)
        
        # Read the content of the file
        with open(file_path, 'r') as file:
            lines = file.readlines()
        
        # Process each line
        with open(file_path, 'w') as file:
            for line in lines:
                if line.startswith("SubFramePath"):
                    line = replace_periods(line)
                file.write(line)
print("Processing complete!")

to update the filenames:

import os
import argparse
def replace_periods_in_filenames(directory, dryrun=False):
    for filename in os.listdir(directory):
        # Split the filename into name and extension
        name, ext = os.path.splitext(filename)
        
        if ext != '.tif':
            continue
        # Replace all periods in the name with underscores
        new_name = name.replace('.', '_') + ext
        
        # Create full file paths
        old_file = os.path.join(directory, filename)
        new_file = os.path.join(directory, new_name)
        
        if old_file == new_file:
            continue  # Skip if the new name is the same as the old name
        
        if os.path.exists(new_file):
            print(f'[WARNING] Skipping: {new_name} already exists.')
        else:
            if dryrun:
                print(f'[DRY RUN] Would rename: {filename} -> {new_name}')
            else:
                os.rename(old_file, new_file)
                print(f'Renamed: {filename} -> {new_name}')
if __name__ == "__main__":
    # Set up command line arguments
    parser = argparse.ArgumentParser(description='Replace all but the last period in filenames with underscores.')
    parser.add_argument('directory', type=str, help='The directory containing files to rename.')
    parser.add_argument('--dryrun', action='store_true', help='If set, perform a dry run without renaming files.')
    args = parser.parse_args()
    # Call the function with the provided directory and dryrun option
    replace_periods_in_filenames(args.directory, dryrun=args.dryrun)

[tsbrant]

That seems to have worked - Thank you so much!