Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixing args and merge #2

Merged
merged 22 commits into from
Jul 6, 2022
Merged

Fixing args and merge #2

merged 22 commits into from
Jul 6, 2022

Conversation

AlecThomson
Copy link

Hey @mb010,

Big fix incoming!

Brings in updates from CASA6 updates to merge merge conflicts in the main repo.

Also reworks config and argument problems introduced into processMeerKAT.py. Previously, the default parameters were overdetermined between the args and the config file. The fallback default is now taken from the config file if not specified on the commandline.

Jordatious and others added 21 commits December 10, 2021 15:48
Various changes:
- Match taylor terms of outliers to main image
- Run science_image.py with MPI
- Make bash scripts executable for all, so other users that have permission can use them - e.g. to check progress on pipeline run
- Update default container to CASA 6.4
- Fix outlier naming bug that increased run-time
Slightly improved workaround for Singularity >= 3.6 (sensitive to different $PYTHONPATH within container), as suggest here:

sylabs/singularity#490 (comment)
…l outliers

- Revert to renaming the CASA 6 logs after script is run
- Update science imaging config defaults: 6k, 512 planes, 5k iterations, S/N=10 threshold
- Include any outliers above threshold within edge 1% of main image
Add manual model from Russ Taylor for polarisation calibrator J1130-1449, taken from observations under the MeerKAT calibrator project.
- When running PyBDSF in selfcal_part2, write out the gaussian catalogue instead of the source catalogue
- After this, use the position of the brightest gaussian for outliers, instead of the position (centroid) of the closest source
meanfreq was using Ghz when it should have been using MHz.
- Update to CASA 6.4.4 (and OpenMPI 4.0.3)
- Bugfixes for calc_refant.py since we moved to CASA 6 and field names instead of IDs
- Enable show_ant_stats.py to work out of the box
If badants isn't an empty list, exclude those antennas during the split step at the end of the pipeline.
Use the `targetfields` parameter from the config to identify phase centre of target for selecting outliers from within 2 degrees.
When outlier imaging is triggered at the beginning of the pipeline, it assumes a naming convention for the outliers, based on the name of the first target field. This is now retrieved from the config instead of being based on the MS intents (for non-compliant MSs).
science_image.py now produces an additional PB corrected image (using
katbeam) at the end of imaging. The cutoff for the PB can be specified
by the pbthreshold parameter in the imaging section of the config file.

The default container has been updated to use the new CASA 6.5 release,
the container also contains katbeam as a dependency.
Outlier imaging:
  - Improvement to run-time for calculating outliers, by using local copy of RACS catalogue instead of new tap query, and reading first sub-MS instead of large MMS (if present)
  - Config parameter for search radius for outliers from RACS
  - If 0.0 or '' used, calculate search radius by reading input MS
  - If whole catalogued flux of an outlier is < 1 mJy, or no source is catalogued, discard it

General:
  - Ensure MHz unit when checking validity of SPW selection
  - Select SPWs with '*' instead of '0' (e.g. when having already processed with the pipeline but wanting a different SPW selection)
  - Cosmetic changes and minor bug fixes
@mb010
Copy link
Owner

mb010 commented Jul 6, 2022

Looks good to me!
I'll merge and test it on my system (judging from the code changes it shouldn't break anything as far as I know).

Good work @AlecThomson !

@mb010 mb010 merged commit d5d900e into mb010:HPC_parameter Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants