Issue 68

[bennahugo]

@Athanaseus this builds against the current casacore, but we are missing those executables I mentioned on the KERN ticket. I'm committing these as a WIP pending passing the test. Hopefully we can simplify the docker to test the kern 7 debs once we found the issue with the kern casarest package

[bennahugo]

@Athanaseus #fd3af5d passes tests locally - the CI server should confirm. The only issue that I can currently see is that /usr/bin/python needs to be symlinked (as it is in a virtualenv) to python3. Everything installs and works inside a venv, but not quite on system level without additional trickery. Unfortunately I don't know where the issue stems from at the moment -- I have tried to eliminate 'python' from the Meqtrees universe for the past while and updated the hashbangs but still can't track this one down.

@o-smirnov perhaps you can provide some further thoughts here as this is originally your codebase:
This is what I get without the additional symlink

Using all interferometers
Making dirty image:  lwimager ms=/usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS data=DATA operation=image stokes=I mode=channel weight=natural wprojplanes=0 npix=512 cellsize=1arcsec spwid=0 field=0 padding=1.2 cachesize=512 prefervelocity=False robust=0 image=tmp.img
lwimager: Version 1.4.0
2022-01-26 20:41:47     INFO    Imager::open()  Opening MeasurementSet /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS
2022-01-26 20:41:47     INFO    imager::data selection  Selecting data
2022-01-26 20:41:47     INFO    imager::data selection  Selected 6552 out of 7560 visibilities.
2022-01-26 20:41:47     INFO    Imager::selectDataChannel()     Selecting channel 0 for spw 0
2022-01-26 20:41:47     INFO    imager::defineimage()   Defining image properties:nx=512 ny=512 cellx='1arcsec' celly='1arcsec' stokes=I' mode=channel nchan=1 start=0 step=1 spwids=[0] fieldid=0 facets=1 frame=1 distance='0m'
2022-01-26 20:41:47     INFO    imager::defineimage()    phaseCenter='field-0 ' mStart='Radialvelocity: 0' qStep='1 km/s'' mFreqStart='Frequency: 0
2022-01-26 20:41:47     INFO    imager::weight()        Weighting MS: Imaging weights will be changed
2022-01-26 20:41:47     INFO    imager::weight()        Natural weighting
2022-01-26 20:41:47     INFO    imager::setoptions()    Setting processing options
2022-01-26 20:41:47     INFO    imager::makeimage()     Calculating image (without full skyequation)
2022-01-26 20:41:47     INFO    imager::makeimage()     Making dirty image from observed data 
2022-01-26 20:41:47     INFO    imager::makeimage()     Image is : tmp.img
2022-01-26 20:41:48     INFO    imager::createFTMachine()       Performing interferometric gridding...
lwimager normally ended

real    0m0.489s
user    0m0.201s
sys     0m0.051s
Success, all *.img files will be removed after conversion to FITS
image2fits: Version 20090915GvD
The input image is: tmp.img*2
2022-01-26 20:41:48     INFO    ImageFitsConverter::ImageToFITS         Copying 'Expression: tmp.img*2' to file All pixels fit in memory (262144 pixels).
image2fits normally ended
E
======================================================================
ERROR: batch_test.testMeqtreesBatchJob
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/src/pyxis-1.7.4.3/Pyxis/recipes/meqtrees-batch-test/batch_test.py", line 161, in testMeqtreesBatchJob
    if not os.path.exists(os.path.join(PACKAGE_TEST_DIR, "WSRT.MS.MODEL_DATA.channel.1ch.fits")): raise RuntimeError("Output FITS file does not exist")
RuntimeError: Output FITS file does not exist

and this is with the symlink

The input image is: tmp.img*2
2022-01-26 20:47:21     INFO    ImageFitsConverter::ImageToFITS         Copying 'Expression: tmp.img*2' to file All pixels fit in memory (262144 pixels).
image2fits normally ended
lwimager: Version 1.4.0
2022-01-26 20:47:23     INFO    Imager::open()  Opening MeasurementSet /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS
2022-01-26 20:47:23     INFO    imager::data selection  Selecting data
2022-01-26 20:47:23     INFO    imager::data selection  Selected 6552 out of 7560 visibilities.
2022-01-26 20:47:23     INFO    Imager::selectDataChannel()     Selecting 64 channels, starting at 0, stepped by 1, for spw 0
2022-01-26 20:47:23     INFO    imager::defineimage()   Defining image properties:nx=1024 ny=1024 cellx='14.0625arcsec' celly='14.0625arcsec' stokes=IQUV' mode=channel nchan=1 start=0 step=64 spwids=[0] fieldid=0 facets=1 frame=1 distance='0m'
2022-01-26 20:47:23     INFO    imager::defineimage()    phaseCenter='field-0 ' mStart='Radialvelocity: 0' qStep='1 km/s'' mFreqStart='Frequency: 0
2022-01-26 20:47:23     INFO    imager::weight()        Weighting MS: Imaging weights will be changed
2022-01-26 20:47:23     INFO    imager::weight()        Radial weighting
2022-01-26 20:47:23     INFO    imager::filter()        Imaging weights will be tapered
2022-01-26 20:47:23     INFO    VisImagingWeight::setFilter()   Filtering for Gaussian of shape: 12 by 12 (arcsec) at p.a. 0 (degrees)
2022-01-26 20:47:23     INFO    imager::setoptions()    Setting processing options
2022-01-26 20:47:23     INFO    imager::makeimage()     Calculating image (without full skyequation)
2022-01-26 20:47:23     INFO    imager::makeimage()     Making dirty image from model data 
2022-01-26 20:47:23     INFO    imager::makeimage()     Image is : /usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.img
2022-01-26 20:47:23     INFO    imager::createFTMachine()       Performing interferometric gridding...
2022-01-26 20:47:24     INFO    ImageFitsConverter::ImageToFITS         Copying '/usr/local/lib/python3.8/dist-packages/Pyxis/recipes/meqtrees-batch-test/WSRT.MS.MODEL_DATA.channel.1ch.img' to file Copying plane by plane (1048576 pixels).
lwimager normally ended

So the second lwimager call does not execute... I'm a bit stumped to tracking this one down

[bennahugo]

Nvm... I found it at long last. This should now work. @Athanaseus all you need to do now is remove meqbrowser.py from the list of targets that get installed until we can more cleanly separate all the Qt required code from this repo

Ping me once you have located the issue with casarest -- then the dockerfile can be simplified. I will make a release of this for you once it has gone through the CI server