Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Subbeamnod routine cannot access plnum #153

Closed
jfrothingham opened this issue Dec 5, 2023 · 12 comments
Closed

Subbeamnod routine cannot access plnum #153

jfrothingham opened this issue Dec 5, 2023 · 12 comments
Assignees
Labels
beta Beta testing issue bug Something isn't working

Comments

@jfrothingham
Copy link

Unable to retrieve a subbeamnod scan because of apparent problems accessing 'plnum'

I've been following the SubBeamNod example (https://dysh.readthedocs.io/en/latest/examples/subbeamnod.html) with a different dataset.

I am using dysh-0.2.0b with Python3.11 on a computer that I think is running RedHat 8. (Thales data reduction machine)
Data is located at /home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas in the Green Bank filesystem.

Here is a screenshot of the error I'm seeing. I get this error regardless of whether I specify kwargs or not, and regardless of which scan I use.

image

However, sdfits.summary() shows that this scan does have values for PLNUM and FDNUM.

image

@tchamberlin tchamberlin added the bug Something isn't working label Dec 5, 2023
@jfrothingham
Copy link
Author

Yes, I think you can test against it. It's a dataset we were considering using as a training example

@tchamberlin
Copy link
Member

tchamberlin commented Dec 5, 2023

I can reproduce this:

sdfits = GBTFITSLoad("/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas")
sbn = sdfits.subbeamnod(scan=18, fdnum=0, plnum=1)

Output:

File <snip>dysh/fits/gbtfitsload.py:815, in GBTFITSLoad.subbeamnod(self, scan, bintable, **kwargs)
    813 df = df[df["IFNUM"].isin([ifnum])]
    814 df = df[df["FDNUM"].isin([fdnum])]
--> 815 df = df[df["PLNUM"].isin([plnum])]
    816 df_on = df[df["CAL"] == "T"]
    817 df_off = df[df["CAL"] == "F"]

UnboundLocalError: cannot access local variable 'plnum' where it is not associated with a value

Indeed, plnum is not defined, since rx is not "Rcvr26_40". But upon fixing that, I get

File <snip>/dysh/spectra/scan.py:228, in TPScan.__init__(self, gbtfits, scan, sigstate, calstate, scanrows, calrows, bintable, calibrate)
    225 # print("BINTABLE = ", bintable)
    226 # @TODO deal with data that crosses bintables
    227 if bintable is None:
--> 228     self._bintable_index = self._sdfits._find_bintable_and_row(self._scanrows[0])[0]
    229 else:
    230     self._bintable_index = bintable

IndexError: index 0 is out of bounds for axis 0 with size 0

@tchamberlin
Copy link
Member

tchamberlin commented Dec 5, 2023

After fixing the plnum bug (see 153-subbeamnod-routine-cannot-access-plnum)

It appears that this causes an error:

sdfits = GBTFITSLoad("/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas")
sbn = sdfits.subbeamnod(scan=18, fdnum=0, plnum=1)

But this does not:

sdfits = GBTFITSLoad("/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas/AGBT19B_096_08.raw.vegas.A.fits")
sbn = sdfits.subbeamnod(scan=18, fdnum=0, plnum=1)

The difference being that the former (the path in the original issue) is the path to the SDFITS folder, while the latter is the specific path of a specific bank (as per the example).

@mpound Should both of these paths work? i.e. is giving the SDFITS folder path a supported operation?

@jfrothingham
Copy link
Author

jfrothingham commented Dec 5, 2023

I should have attached screenshots of my full code, sorry. I ran dysh from within the folder "/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas" and did specify the path of the bank:

filename='AGBT19B_096_08.raw.vegas.A.fits'
sdfits = GBTFITSLoad(filename)
sbn = sdfits.subbeamnod(scan=18, fdnum=0)

This code produced the error I reported here.

@tchamberlin
Copy link
Member

Ah okay, then I think I fixed the bug. You can install the updated dysh code via:

pip install git+https://github.com/GreenBankObservatory/dysh.git@153-subbeamnod-routine-cannot-access-plnum

@tchamberlin tchamberlin self-assigned this Dec 5, 2023
@jfrothingham
Copy link
Author

I'm still encountering the issue, but I think that's due to installation problems on my end rather than anything to do with this bug.

@astrofle
Copy link
Collaborator

astrofle commented Dec 5, 2023

After fixing the plnum bug (see 153-subbeamnod-routine-cannot-access-plnum)

It appears that this causes an error:

sdfits = GBTFITSLoad("/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas")
sbn = sdfits.subbeamnod(scan=18, fdnum=0, plnum=1)

But this does not:

sdfits = GBTFITSLoad("/home/scratch/jfrothin/testdata/AGBT19B_096_08.raw.vegas/AGBT19B_096_08.raw.vegas.A.fits")
sbn = sdfits.subbeamnod(scan=18, fdnum=0, plnum=1)

The difference being that the former (the path in the original issue) is the path to the SDFITS folder, while the latter is the specific path of a specific bank (as per the example).

@mpound Should both of these paths work? i.e. is giving the SDFITS folder path a supported operation?

@tchamberlin yes. dysh should be able to load multiple fits files in a directory. This feature was introduced as part of 0.2.0. (Thanks for looking into this issue.)

@astrofle
Copy link
Collaborator

astrofle commented Dec 6, 2023

@jfrothingham I had to uninstall dysh and then reinstall using the command Thomas provided for this to work. Here's what I did (from the Python environment with dysh):

pip uninstall dysh
pip install git+https://github.com/GreenBankObservatory/dysh.git@153-subbeamnod-routine-cannot-access-plnum

Hope this helps.

@tchamberlin
Copy link
Member

Ah right! You can do it in one command via the -I/--ignore-installed flag:

pip install -I git+https://github.com/GreenBankObservatory/dysh.git@153-subbeamnod-routine-cannot-access-plnum

@jfrothingham
Copy link
Author

With the uninstall/reinstall, I can now successfully process subbeamnod data with dysh. Thanks all!

@mpound
Copy link
Collaborator

mpound commented Dec 6, 2023

Late to this party, but I do think there is still a potential bug in subbeamnod having to do with the receiver assumption Rcvr26_40, which I got from the GBTIDL implementation. I will work with Pedro to improve my understanding of which Rxs have or historically had polarization capability.

@astrofle
Copy link
Collaborator

astrofle commented Dec 6, 2023

@mpound currently, only Argus (the 3 mm 16 feed receiver being used for the GBT EDGE survey) has single polarization capabilities. For observations with every other spectral line receiver the data should have at least two polarizations. The if else clause in dysh.fits.GBTFITSLoad.subbeamnod is there because the Ka receiver (Rcvr26_40) has had its feeds mislabelled for some time now. So, if you want to calibrate the data properly for that particular receiver you have to swap the polarization and feed values. For the rest of the receivers the method should use the same polarization for the selected feeds. I hope that clears a bit the confusion.

Apologies for not explaining that earlier.

PS. There is an open item to fix the mislabelling of the Ka receiver feeds. That is also why there is a commented part of the else clause referring to the date of the observations. For observations taken after the Ka beam mislabelling issue is fixed the polarizations should not be swapped.

mpound added a commit that referenced this issue Jan 8, 2024
mpound added a commit that referenced this issue Jan 8, 2024
mpound added a commit that referenced this issue Jan 17, 2024
mpound added a commit that referenced this issue Jan 18, 2024
astrofle pushed a commit that referenced this issue Feb 27, 2024
@mpound mpound added the beta Beta testing issue label Jul 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
beta Beta testing issue bug Something isn't working
Projects
None yet
Development

When branches are created from issues, their pull requests are automatically linked.

4 participants