issue #1448 - Enable batch processing for P(r)

NOTE: This issue is not yet fixed.

Feature:

Selected data sent from the load data can now be sent to Pr window and a new tab will appear for each data set. Although the goal is batch calculate all data and neatly display in a table the current “non batch” tab method can be used for quick references of calculations for each file. In the nect commit I intend to implement a batch tab. Notes for those are in the bottom.

Dev note

-  1) Block automatic calculation after data is sent to Pr prospective whilst batch mode is deselected. (Allow users to hit calculate) Or block graphs from showing automatically as it clusters the interface.

-  2) Fix batch Mode
o  When batch mode is selected create only one tab (similar to fitting)
o  New interface for batch / same interface could be used with the addition of “Fit” / ” Graph” button conjunction with the 1st TODO.

3) Clean up Code. The current files ive modified/created are messy. Also comment code

…w and efficient. also a way to display/export all the data in a table is still required

Currently, Tabs works. Batch still has a little ironing out to do. mainly its interaction with setCurrentData(). Additionally, The data from Batch still needs to be sent to a Table and eventual reported/exported.

files calculated using Pr can now be batch Processed and saved to a table. However, a lot of work is still left to fix optimise and automate specific parts of this feature.

things still left to fix
- [ ] calculate all files before showing the table
- [ ] perform calculations without stop showing the plots as it slows down and clutters the interface
- [ ] optimise, clean and comment the code

Files imported into Pr interface can now automatically be processed and outputs displayed in a table. Code still requires to be optimised and cleaned. A better way to update the table is also required.

things still left to fix
- [x] calculate all files before showing the table
- [x] perform calculations without stop showing the plots as it slows down and clutters the interface
- [ ] optimise, clean and comment the code

However, im still getting errors with displayChange and setCurrentData even though the calculations and outputs are successful.

 Batch Processing still not working properly. Cleaned a bit of the code(That I was working on). 2D data shows a plot of the data however calculations still cannot be made. in the process of coding the bit gets a slice of the 2D data.

when 2D data is loaded into Pr interface a raw data plot should pop up. Using the Pr interface, parameters for the conversions can be inputted and when calculated should update the graphs and calculate the Pr values for those specific slices.

… be plotted on a single plot.

However, currently only one of the slices is calculated using Pr, this will be changed (Hopefully) in the next commit so that all sliced can be processed together similarly to 1D Batch and its outputs displayed in a table.

…lices.

the table populates the slices created and the button when pressed need to show plots of the slice.

calculate all button need to also be fixed. the data cannot be read as its not in the right object.

…dating the gui and getting/setting parameters.

 Furthermore, the data output ont the batch table will be from self._calculator. this means It's flexible for further expansions and is just as reliable as the gui. Although a few Issues still remain during calculation, one being Q range being set to 0 or bellow which causes the data set to not be calculated and the calculations of the previous data to be set for the 'current' data set (Hence you might see duplicate rows with different file names). In my previous commit in the branch 1D data could not be uploaded for individual calculation that bug has now been fixed. Code still needs to be cleaned up and commented.

this is a bit of a hacky way of getting it to work and is just to get an idea of how it might work. things to improve upon are :-

- [ ] Getting the table in GridPanel.py to work flexibly for both 1D batch aswell as 2D slices.

- [ ] cleaning up slices()

- [ ] making showBatchOutput() more portable

- [ ] creating batch like datalist support for 2D slices. (so that it can be batch processed just like 1D batch files)

- [ ] SORTING OUT THE ERRORS

- [ ] (Might need help) Fix the Wierd Parameter issue. (Currently set/default parameters change to estimates which makes it hard to use reliabely. Background, Qmin and Qmax is also not working as expected, it sets itself to a value bellow 0 causing it to not calculate for that slice.)

need to test, clean up code, and make sure that this doesn't affect 1D batch processing

…ionally, Q min and Q Max can be seen in the Batch results panel.

However, further testing and Code cleaning is required. Newly created pop up windows such as the batch results, 1D slice plot, and 2D image plots need to be connected with the interface and made to fit in withing the UI.

…ionally, Q min and Q Max can be seen in the Batch results panel.

However, further testing and Code cleaning is required. Newly created pop up windows such as the batch results, 1D slice plot, and 2D image plots need to be connected with the interface and made to fit in withing the UI.

… Pr fit and I(q) plots.

This Version should somewhat work besides the found bugs.
Testing and code cleaning still required.

- [ ]  Batch Processing might not yet be reliable. A row or 2 might duplicate if a parameter is not right eg: Qmax or NoOfTerms.

- [ ] Dropdown menu Goes back to previous Item if user decides to change.

- [ ] All New Windows Are outside the SasView Interface.

…u. Additionally the dropdown menu used to not set to new selected data which has been fixed. however, in this version 2D data slicing is bracken this is due to some changes with how raw batch data is handles.

… the dataset. when switched back to the dateset (using the dropdown menu) the parameters selected prevously will be set.

…ead to a dir so each data file could be assigned to a thread instance. estimationThread and estimationThreadNT might also need to be changed as such. however more testing is required.

slicing: slices data are turned into model using the createModelItemWithPlot function.
this is makes it much easier to move around as it can work just like 1D data.

1D batch: Parameters set before batch processing is remembered and used when processed using batch mode.

…imations".

Parameters whilst, Batch Processing is also remembered.
Estimated data doesn't mess with actual data unless suggested button is pressed.
However, Duplication issue still remains.

…ne using to Calculate all button is still a bit buggy.

Still a bit more tweaking is required for batch processing slices.

…D rawImage and shows slice using phi. Fixed buttons. Minor cleanups. fixed 2D slices from not being batched processed.

for Devs: Plots in Pr can be disabled using self._allowPlots

Plot Behaviour:
if Batch or Data2D Slices -> hide Plots
if Button(Calculate) or Single Data1D -> show Plots

… the lager functions into smaller functions. And Changes the Warnings for GridPanel.py.

# Conflicts:
#	src/sas/qtgui/Perspectives/Inversion/InversionPerspective.py

…during the time this branch was not updated with it.

… seem get to run with the following code

```
        GuiUtils.updateModelItemWithPlot(item, new_plot, new_plot.id)

        self.base.manager.communicator.plotUpdateSignal.emit([new_plot])
        self.base.manager.communicator.forcePlotDisplaySignal.emit([item, new_plot])
```

changes made using feedback from:

1) #2181 (comment)
removed as it was something that I was testing out but didn't end up using.

2) #2181 (comment)
Moved closeEvent() from InversionWidget.py to InversionPerspective.py.

3) #2181 (comment)
changed `from sas.sascalc.dataloader.manipulations import SectorQ` to `from sasdata.data_util.manipulations import SectorQ`

I've had to discard the tabbing order stuff that Piotr did, because it wasn't compatible with the new layout

issue #1448 - Enable batch processing for P(r)

NOTE: This issue is not yet fixed.

Feature:

Selected data sent from the load data can now be sent to Pr window and a new tab will appear for each data set. Although the goal is batch calculate all data and neatly display in a table the current “non batch” tab method can be used for quick references of calculations for each file. In the nect commit I intend to implement a batch tab. Notes for those are in the bottom.

Dev note

-  1) Block automatic calculation after data is sent to Pr prospective whilst batch mode is deselected. (Allow users to hit calculate) Or block graphs from showing automatically as it clusters the interface.

-  2) Fix batch Mode
o  When batch mode is selected create only one tab (similar to fitting)
o  New interface for batch / same interface could be used with the addition of “Fit” / ” Graph” button conjunction with the 1st TODO.

3) Clean up Code. The current files ive modified/created are messy. Also comment code

…w and efficient. also a way to display/export all the data in a table is still required

Currently, Tabs works. Batch still has a little ironing out to do. mainly its interaction with setCurrentData(). Additionally, The data from Batch still needs to be sent to a Table and eventual reported/exported.

files calculated using Pr can now be batch Processed and saved to a table. However, a lot of work is still left to fix optimise and automate specific parts of this feature.

things still left to fix
- [ ] calculate all files before showing the table
- [ ] perform calculations without stop showing the plots as it slows down and clutters the interface
- [ ] optimise, clean and comment the code

Files imported into Pr interface can now automatically be processed and outputs displayed in a table. Code still requires to be optimised and cleaned. A better way to update the table is also required.

things still left to fix
- [x] calculate all files before showing the table
- [x] perform calculations without stop showing the plots as it slows down and clutters the interface
- [ ] optimise, clean and comment the code

However, im still getting errors with displayChange and setCurrentData even though the calculations and outputs are successful.

 Batch Processing still not working properly. Cleaned a bit of the code(That I was working on). 2D data shows a plot of the data however calculations still cannot be made. in the process of coding the bit gets a slice of the 2D data.

when 2D data is loaded into Pr interface a raw data plot should pop up. Using the Pr interface, parameters for the conversions can be inputted and when calculated should update the graphs and calculate the Pr values for those specific slices.

… be plotted on a single plot.

However, currently only one of the slices is calculated using Pr, this will be changed (Hopefully) in the next commit so that all sliced can be processed together similarly to 1D Batch and its outputs displayed in a table.

…lices.

the table populates the slices created and the button when pressed need to show plots of the slice.

calculate all button need to also be fixed. the data cannot be read as its not in the right object.

…dating the gui and getting/setting parameters.

 Furthermore, the data output ont the batch table will be from self._calculator. this means It's flexible for further expansions and is just as reliable as the gui. Although a few Issues still remain during calculation, one being Q range being set to 0 or bellow which causes the data set to not be calculated and the calculations of the previous data to be set for the 'current' data set (Hence you might see duplicate rows with different file names). In my previous commit in the branch 1D data could not be uploaded for individual calculation that bug has now been fixed. Code still needs to be cleaned up and commented.

this is a bit of a hacky way of getting it to work and is just to get an idea of how it might work. things to improve upon are :-

- [ ] Getting the table in GridPanel.py to work flexibly for both 1D batch aswell as 2D slices.

- [ ] cleaning up slices()

- [ ] making showBatchOutput() more portable

- [ ] creating batch like datalist support for 2D slices. (so that it can be batch processed just like 1D batch files)

- [ ] SORTING OUT THE ERRORS

- [ ] (Might need help) Fix the Wierd Parameter issue. (Currently set/default parameters change to estimates which makes it hard to use reliabely. Background, Qmin and Qmax is also not working as expected, it sets itself to a value bellow 0 causing it to not calculate for that slice.)

need to test, clean up code, and make sure that this doesn't affect 1D batch processing

…ionally, Q min and Q Max can be seen in the Batch results panel.

However, further testing and Code cleaning is required. Newly created pop up windows such as the batch results, 1D slice plot, and 2D image plots need to be connected with the interface and made to fit in withing the UI.

…ionally, Q min and Q Max can be seen in the Batch results panel.

However, further testing and Code cleaning is required. Newly created pop up windows such as the batch results, 1D slice plot, and 2D image plots need to be connected with the interface and made to fit in withing the UI.

… Pr fit and I(q) plots.

This Version should somewhat work besides the found bugs.
Testing and code cleaning still required.

- [ ]  Batch Processing might not yet be reliable. A row or 2 might duplicate if a parameter is not right eg: Qmax or NoOfTerms.

- [ ] Dropdown menu Goes back to previous Item if user decides to change.

- [ ] All New Windows Are outside the SasView Interface.

…u. Additionally the dropdown menu used to not set to new selected data which has been fixed. however, in this version 2D data slicing is bracken this is due to some changes with how raw batch data is handles.

… the dataset. when switched back to the dateset (using the dropdown menu) the parameters selected prevously will be set.

…ead to a dir so each data file could be assigned to a thread instance. estimationThread and estimationThreadNT might also need to be changed as such. however more testing is required.

slicing: slices data are turned into model using the createModelItemWithPlot function.
this is makes it much easier to move around as it can work just like 1D data.

1D batch: Parameters set before batch processing is remembered and used when processed using batch mode.

…imations".

Parameters whilst, Batch Processing is also remembered.
Estimated data doesn't mess with actual data unless suggested button is pressed.
However, Duplication issue still remains.

…ne using to Calculate all button is still a bit buggy.

Still a bit more tweaking is required for batch processing slices.

…D rawImage and shows slice using phi. Fixed buttons. Minor cleanups. fixed 2D slices from not being batched processed.

for Devs: Plots in Pr can be disabled using self._allowPlots

Plot Behaviour:
if Batch or Data2D Slices -> hide Plots
if Button(Calculate) or Single Data1D -> show Plots

… the lager functions into smaller functions. And Changes the Warnings for GridPanel.py.

…during the time this branch was not updated with it.

… seem get to run with the following code

```
        GuiUtils.updateModelItemWithPlot(item, new_plot, new_plot.id)

        self.base.manager.communicator.plotUpdateSignal.emit([new_plot])
        self.base.manager.communicator.forcePlotDisplaySignal.emit([item, new_plot])
```

changes made using feedback from:

1) #2181 (comment)
removed as it was something that I was testing out but didn't end up using.

2) #2181 (comment)
Moved closeEvent() from InversionWidget.py to InversionPerspective.py.

3) #2181 (comment)
changed `from sas.sascalc.dataloader.manipulations import SectorQ` to `from sasdata.data_util.manipulations import SectorQ`