Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tabs, Batch Processing and 2D data slicing and processing for pr #2181

Open
wants to merge 150 commits into
base: release_6.0.0
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
150 commits
Select commit Hold shift + click to select a range
03b924d
Adds Tab support for Pr. (Batch Mode not yet functional)
ru4en Jul 1, 2022
2f0e64b
Added Batch Processing for Pr, However the interface is currently slo…
ru4en Jul 5, 2022
294c3cb
Added Tabs and Batch Processing for Pr.
ru4en Jul 6, 2022
c9f8df5
Added Batch Processing for Pr - work in progress
ru4en Jul 7, 2022
816eddd
Automatically Batch Process all selected files for Pr - work in progress
ru4en Jul 8, 2022
eac7699
Tab and Batch mode for Pr is now functional.
ru4en Jul 13, 2022
264bf8b
Fixed Batch Tabs for Pr inversion.
ru4en Jul 26, 2022
daad94a
2D data to 1D data for Pr is now functional.
ru4en Jul 29, 2022
3c1f051
Sliced 2D data can be imported to Pr inverter and multiple slices can…
ru4en Aug 2, 2022
024e208
buggy 2D to 1D plots fixed. The 1D plot shows the correct number of s…
ru4en Aug 2, 2022
18e660a
2D data has a new tab (Ui) with an outputs table.
ru4en Aug 3, 2022
916c69b
batch results are consistent now however there are a few bugs with up…
ru4en Aug 9, 2022
70981ad
Batch results will display Number of terms, reg const and Max Distance.
ru4en Aug 11, 2022
d67dd20
got 2D data slices to Show up on batch table.
ru4en Aug 11, 2022
f9dabdf
Working on Batch Processing for 2D slices.
ru4en Aug 13, 2022
a81bb3a
Batch Pr processing of 1D data and 2D slices is now functional. Addit…
ru4en Aug 15, 2022
44fbd7f
Batch Pr processing of 1D data and 2D slices is now functional. Addit…
ru4en Aug 15, 2022
689bedd
Fixed new tab (first tab) from not being able to set data to. Enables…
ru4en Aug 15, 2022
eb99cd2
Parameters for individual data sets can be set using the dropdown men…
ru4en Aug 24, 2022
9ce2941
set parameters for items in the dropdown menu of batch will saved for…
ru4en Aug 25, 2022
de06f70
Result Duplication Issue has slightly been patched by setting calcThr…
ru4en Aug 27, 2022
3593c91
made a few changed to slicing and 1D batch.
ru4en Aug 28, 2022
722e27b
got batch processing to spit out actual calculations rather then "est…
ru4en Aug 30, 2022
a878d19
Calculations on batch interface is more reliable now. Calculations do…
ru4en Sep 1, 2022
1436167
Slicer works a bit smother now.
ru4en Sep 1, 2022
480fb7d
Fixed Errors for 2D data and added a few catchers.
ru4en Sep 1, 2022
c8a3502
fixed exporting from table issue, buttons in slicer now updates the 2…
ru4en Sep 4, 2022
09b3b10
Plots Behaviour is designed to be more dynamic.
ru4en Sep 5, 2022
876c36e
Cleaned and Added Comments to InversionWidget.py, broke down a few of…
ru4en Sep 6, 2022
5a608ea
Cleaned up and worked on some finishing touches ready for PR.
ru4en Sep 7, 2022
cf37525
Merge branch 'main' into 2D-Data-Processing-for-Pr
ru4en Sep 8, 2022
47c301c
minor compatibility changes to allow changes made in the main branch …
ru4en Sep 8, 2022
6b360f1
Few housekeeping changes. tried to fix 2D plotting issue, but I can't…
ru4en Sep 15, 2022
3a3887e
Merge branch 'main' into 2D-Data-Processing-for-Pr
lucas-wilkins Feb 2, 2023
05cd452
Main errors corrected
lucas-wilkins Feb 2, 2023
0a0c399
Adds Tab support for Pr. (Batch Mode not yet functional)
ru4en Jul 1, 2022
09e0f88
Added Batch Processing for Pr, However the interface is currently slo…
ru4en Jul 5, 2022
0fb9791
Added Tabs and Batch Processing for Pr.
ru4en Jul 6, 2022
a6e308e
Added Batch Processing for Pr - work in progress
ru4en Jul 7, 2022
3f77a99
Automatically Batch Process all selected files for Pr - work in progress
ru4en Jul 8, 2022
0ee4dc2
Tab and Batch mode for Pr is now functional.
ru4en Jul 13, 2022
c9daff5
Fixed Batch Tabs for Pr inversion.
ru4en Jul 26, 2022
47fbd39
2D data to 1D data for Pr is now functional.
ru4en Jul 29, 2022
58a3ba8
Sliced 2D data can be imported to Pr inverter and multiple slices can…
ru4en Aug 2, 2022
a054495
buggy 2D to 1D plots fixed. The 1D plot shows the correct number of s…
ru4en Aug 2, 2022
68620b6
2D data has a new tab (Ui) with an outputs table.
ru4en Aug 3, 2022
f9f7602
batch results are consistent now however there are a few bugs with up…
ru4en Aug 9, 2022
4a1ac0d
Batch results will display Number of terms, reg const and Max Distance.
ru4en Aug 11, 2022
1f46d14
got 2D data slices to Show up on batch table.
ru4en Aug 11, 2022
dac109f
Working on Batch Processing for 2D slices.
ru4en Aug 13, 2022
575db58
Batch Pr processing of 1D data and 2D slices is now functional. Addit…
ru4en Aug 15, 2022
254f1c1
Batch Pr processing of 1D data and 2D slices is now functional. Addit…
ru4en Aug 15, 2022
2eaf971
Fixed new tab (first tab) from not being able to set data to. Enables…
ru4en Aug 15, 2022
4438331
Parameters for individual data sets can be set using the dropdown men…
ru4en Aug 24, 2022
e8ab2e9
set parameters for items in the dropdown menu of batch will saved for…
ru4en Aug 25, 2022
45f9116
Result Duplication Issue has slightly been patched by setting calcThr…
ru4en Aug 27, 2022
6a4d996
made a few changed to slicing and 1D batch.
ru4en Aug 28, 2022
ab2fc80
got batch processing to spit out actual calculations rather then "est…
ru4en Aug 30, 2022
c8091e0
Calculations on batch interface is more reliable now. Calculations do…
ru4en Sep 1, 2022
95107e2
Slicer works a bit smother now.
ru4en Sep 1, 2022
3505d8f
Fixed Errors for 2D data and added a few catchers.
ru4en Sep 1, 2022
282c455
fixed exporting from table issue, buttons in slicer now updates the 2…
ru4en Sep 4, 2022
893c487
Plots Behaviour is designed to be more dynamic.
ru4en Sep 5, 2022
6b89f1f
Cleaned and Added Comments to InversionWidget.py, broke down a few of…
ru4en Sep 6, 2022
a9dae54
Cleaned up and worked on some finishing touches ready for PR.
ru4en Sep 7, 2022
a661d9c
minor compatibility changes to allow changes made in the main branch …
ru4en Sep 8, 2022
c3f7929
Few housekeeping changes. tried to fix 2D plotting issue, but I can't…
ru4en Sep 15, 2022
08e01eb
Main errors corrected
lucas-wilkins Feb 2, 2023
b9a02d0
Move InversionWidget to PySide6
dehoni Jun 6, 2023
325f346
Resolve merge conflict with main
dehoni Jun 6, 2023
68ea756
introduce DataRole in InversionWidget
dehoni Jun 6, 2023
410c696
Merge branch 'main' into 2D-Data-Processing-for-Pr
dehoni Jun 14, 2023
5ec9a89
add tab_id to P(r) perspective
dehoni Jun 30, 2023
ae1bb33
add check if Pr has data, delete double removeData method
dehoni Jul 2, 2023
ebbfe14
fix saving when Pr absent
dehoni Jul 3, 2023
2634f75
fix Save Analysis for currentPage
dehoni Jul 3, 2023
a5ec34e
Merge branch 'main' into 2D-Data-Processing-for-Pr
dehoni Jul 4, 2023
5e804f2
Merge branch 'main' into 2D-Data-Processing-for-Pr
dehoni Jul 6, 2023
a8f98d0
refactor capturing 2Dslicer for Inversion
dehoni Jul 6, 2023
57f0c99
toogle 2D Data tab
dehoni Jul 8, 2023
be66aa3
implement swaping of data to fill empty tab
dehoni Jul 14, 2023
9ffc80f
Merge branch 'main' into 2D-Data-Processing-for-Pr
dehoni Aug 26, 2023
64e8cb4
Bring naming of Inversion Tabs in line with Fitting Perspective
dehoni Aug 28, 2023
ce0f725
populate first tab with data
dehoni Sep 1, 2023
9d52bba
fixing BatchTab naming
dehoni Sep 1, 2023
b2fd716
update q range when loading batch elements
dehoni Sep 4, 2023
290e51d
update batchresults to final result when calculate all
dehoni Sep 4, 2023
e4ab4b7
catch negative regularisation parameter and negative data
dehoni Sep 5, 2023
bac0a4f
Merge branch 'main' into 2D-Data-Processing-for-Pr
dehoni Sep 16, 2023
26e0877
add id to new plot
dehoni Sep 17, 2023
f061e3e
show BatchTable after completion of all threads
dehoni Sep 19, 2023
ae704d0
catch Corfunc bug when deleting data
dehoni Sep 19, 2023
cf4ac19
catch assertion error when deleting data
dehoni Sep 20, 2023
8005823
fix loading 2D data for new slicer and flagging batch+2D data loading
dehoni Sep 20, 2023
a6b3c9a
set default values for 2D plots
dehoni Sep 21, 2023
2762b11
rename starting angle for 2D slicing
dehoni Oct 8, 2023
a0a8c9a
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Oct 12, 2023
b986f61
Merge branch '2D-Data-Processing-for-Pr' of https://github.com/SasVie…
dehoni Oct 12, 2023
afb0be5
latest changes
dehoni Oct 12, 2023
f9cb5d6
Fixing 2D slicing, creating 1D data fowarded to normal batchmode
dehoni Oct 12, 2023
c45f273
simplify 2D slicer option, plotting not needed as 1D data exposed in …
dehoni Oct 22, 2023
a0157c8
emmbed 2D plot in Sasview
dehoni Oct 22, 2023
68979ac
setup communicators for 2D slicers, and plot sectors
dehoni Oct 25, 2023
62b3355
set sector to initial values
dehoni Oct 28, 2023
0a51118
allow successive slicing of 2D data set with different settings
dehoni Oct 28, 2023
b642759
live update slicer on change of parameter
dehoni Oct 28, 2023
c0201ee
allow plots for Pr batch
dehoni Oct 28, 2023
4fdadbf
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni Oct 28, 2023
5d5f27c
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
Nov 21, 2023
6bd8ebb
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
Dec 3, 2023
3351e31
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Mar 15, 2024
1a6d3b6
modify Output Panel Parameter list
dehoni Mar 25, 2024
8d8967c
Merge branch '2D-Data-Processing-for-Pr' of https://github.com/SasVie…
dehoni Mar 25, 2024
ba6825c
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni Mar 25, 2024
3a33735
catch modified behaviour of numpy.lstsq
dehoni Mar 25, 2024
3130d7e
Merge branch '2D-Data-Processing-for-Pr' of https://github.com/SasVie…
dehoni Mar 25, 2024
932d665
instigate regenerate documentation for inversion widget
dehoni Mar 25, 2024
c51b712
check if set of parameters is for batch P(r)
dehoni Apr 2, 2024
b45d07c
dynamically update calculated parameters for batch data list
dehoni Apr 2, 2024
a15292e
provide minimum value for dmax
dehoni Apr 2, 2024
beb2cc1
dynamic change the suggested regularisation parameters for batch inve…
dehoni Apr 4, 2024
dd6354e
check stop calculation, remove data, closing P(r) tabs
dehoni Apr 4, 2024
7daeabd
use inline string formating for plotting
dehoni Apr 4, 2024
29ccd73
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni Apr 6, 2024
1359efa
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni Apr 28, 2024
62e78d5
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni May 7, 2024
bfa9886
Modified slider stuff
lucas-wilkins May 9, 2024
29fa731
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
butlerpd May 13, 2024
384c8e9
remove 2D processing for P(r)
dehoni Jun 16, 2024
45a9818
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
dehoni Jul 14, 2024
35d2957
check and moan when loading 2D data
dehoni Jul 14, 2024
8ef427b
Invertor, Corfunc not working on 2D data, initialise suggested values…
dehoni Jul 15, 2024
f4d19ce
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Jul 18, 2024
4042788
Fix error string
lucas-wilkins Jul 19, 2024
ef7f234
key from Filename to Data - not sure why it has been changed?
lucas-wilkins Jul 19, 2024
a0c1fa4
Removed chisqr stuff, it didn't do anything anyway
lucas-wilkins Jul 19, 2024
4b8749b
Unused variable
lucas-wilkins Jul 19, 2024
ad85c7d
Alpha check rewrite
lucas-wilkins Jul 19, 2024
acdaaef
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Jul 19, 2024
de60d82
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Jul 21, 2024
4294f80
Merge branch 'release_6.0.0' into 2D-Data-Processing-for-Pr
lucas-wilkins Jul 27, 2024
9987982
setting (and keeping) q limits in P(r) options
dehoni Aug 4, 2024
053cac6
Merge branch '2D-Data-Processing-for-Pr' of https://github.com/SasVie…
dehoni Aug 4, 2024
bbbf464
making QRangeSlidersTest run
dehoni Aug 5, 2024
146b218
fix optional q-range selection via text fields in Inversion
dehoni Aug 19, 2024
16496f4
fixing incursion
dehoni Aug 19, 2024
66f038c
catch data with dy=0 crashing Inversion
dehoni Aug 20, 2024
8565ba6
no empty string QMIN and QMAX input defaults
dehoni Aug 20, 2024
9b2272d
reinstate batch table
dehoni Aug 26, 2024
befa7a9
plot Pr and data from Batchresult table
dehoni Aug 27, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion build_tools/release_notes/6.0.0_notes.txt
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@

## Anticipated for beta version
- PDB-based model saved to custom model (for S(q) calculations)?
- Batch Processing and 2D data slicing and processing for P(r)
- Batch Processing for P(r)
- Local documentation generator
- Log explorer
- Send To button with replacement options
Expand Down
18 changes: 16 additions & 2 deletions src/sas/qtgui/MainWindow/DataExplorer.py
Original file line number Diff line number Diff line change
Expand Up @@ -616,8 +616,22 @@ def updatePerspectiveWithProperties(self, key, value):
if 'pr_params' in value:
self.cbFitting.setCurrentIndex(self.cbFitting.findText('Inversion'))
params = value['pr_params']
self.sendItemToPerspective(items[0])
self._perspective().updateFromParameters(params)
# Make the perspective read the rest of the read data
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this conditional contains the same content as the branch above (for fitting) and the common code should be merged

if not isinstance(params, list):
params = [params]
for page in params:
# Check if this set of parameters is for a batch page
# if so, skip the update
if page['is_batch'][0] == 'True':
continue
tab_index = None
if 'tab_index' in page:
tab_index = page['tab_index'][0]
tab_index = int(tab_index)
# Send current model item to the perspective
self.sendItemToPerspective(items[0], tab_index=tab_index)
# Assign parameters to the most recent (current) page.
self._perspective().updateFromParameters(page)
if 'invar_params' in value:
self.cbFitting.setCurrentIndex(self.cbFitting.findText('Invariant'))
self.sendItemToPerspective(items[0])
Expand Down
6 changes: 3 additions & 3 deletions src/sas/qtgui/MainWindow/UnitTesting/GuiManagerTest.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,9 +32,9 @@ def manager(self, qapp):
'''Create/Destroy the GUI Manager'''
class MainWindow(MainSasViewWindow):
# Main window of the application
def __init__(self, reactor, parent=None):
screen_resolution = QtCore.QRect(0,0,640,480)
super(MainWindow, self).__init__(screen_resolution, parent)

def __init__(self, parent=None):
super(MainWindow, self).__init__( parent)

# define workspace for dialogs.
self.workspace = QMdiArea(self)
Expand Down
3 changes: 3 additions & 0 deletions src/sas/qtgui/Perspectives/Corfunc/CorfuncPerspective.py
Original file line number Diff line number Diff line change
Expand Up @@ -468,6 +468,9 @@ def setData(self, data_item: List[QStandardItem], is_batch=False):
model_item = data_item[0]
data = GuiUtils.dataFromItem(model_item)
self.data = data
if not isinstance(self.data, Data1D):
msg = "Invariant cannot be computed with 2D data."
raise ValueError(msg)
self._model_item = model_item

self.model.itemChanged.disconnect(self.model_changed)
Expand Down
15 changes: 8 additions & 7 deletions src/sas/qtgui/Perspectives/Corfunc/CorfuncSlider.py
Original file line number Diff line number Diff line change
Expand Up @@ -217,13 +217,14 @@ def extrapolation_parameters(self):
return ExtrapolationParameters(self._min, self._point_1, self._point_2, self._point_3, self._max)

@extrapolation_parameters.setter
def extrapolation_parameters(self, params: ExtrapolationParameters):
if self.find_parameter_problems(params) is None:
self._min = params.data_q_min
self._point_1 = params.point_1
self._point_2 = params.point_2
self._point_3 = params.point_3
self._max = params.data_q_max
def extrapolation_parameters(self, params: Optional[ExtrapolationParameters]):
if params is not None:
if self.find_parameter_problems(params) is None:
self._min = params.data_q_min
self._point_1 = params.point_1
self._point_2 = params.point_2
self._point_3 = params.point_3
self._max = params.data_q_max

self.update()

Expand Down
7 changes: 5 additions & 2 deletions src/sas/qtgui/Perspectives/Inversion/DMaxExplorerWidget.py
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,11 @@ def modelChanged(self, item):
plotable_xs.append(x)
except Exception as ex:
# This inversion failed, skip this D_max value
msg = "ExploreDialog: inversion failed "
msg += "for D_max=%s\n%s" % (str(x), ex)

msg = (f"ExploreDialog: inversion failed for D_max={x}\n"
f"{ex}\n"
f"Please adapt Inversion parameters")

logger.error(msg)

#Return the invertor to its original state
Expand Down
33 changes: 25 additions & 8 deletions src/sas/qtgui/Perspectives/Inversion/InversionLogic.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
import logging
import numpy as np

from sas.qtgui.Plotting.PlotterData import Data1D
from sas.qtgui.Plotting.PlotterData import Data1D, Data2D

PR_FIT_LABEL = r"$P_{fit}(r)$"
PR_LOADED_LABEL = r"$P_{loaded}(r)$"
Expand All @@ -27,6 +27,8 @@ def __init__(self, data=None):
self.data_is_loaded = False
if data is not None:
self.data_is_loaded = True
self.qmin = 0.0
self.qmax = np.inf

@property
def data(self):
Expand All @@ -42,7 +44,7 @@ def isLoadedData(self):
""" accessor """
return self.data_is_loaded

def new1DPlot(self, out, pr, q=None):
def new1DPlot(self, tab_id=1, out=None, pr=None, q=None):
lucas-wilkins marked this conversation as resolved.
Show resolved Hide resolved
"""
Create a new 1D data instance based on fitting results
"""
Expand Down Expand Up @@ -72,7 +74,9 @@ def new1DPlot(self, out, pr, q=None):
logger.info("Could not compute I(q) for q =", list((x[index])))

new_plot = Data1D(x, y)
new_plot.name = IQ_FIT_LABEL + f"[{self._data.name}]"
new_plot.is_data = False
new_plot.dy = np.zeros(len(y))
new_plot.name = "%s [%s]" % (IQ_FIT_LABEL, self._data.name)
new_plot.xaxis("\\rm{Q}", 'A^{-1}')
new_plot.yaxis("\\rm{Intensity} ", "cm^{-1}")
title = "I(q)"
Expand All @@ -81,7 +85,7 @@ def new1DPlot(self, out, pr, q=None):
# If we have a group ID, use it
if 'plot_group_id' in pr.info:
new_plot.group_id = pr.info["plot_group_id"]
new_plot.id = IQ_FIT_LABEL
new_plot.id = str(tab_id) +IQ_FIT_LABEL


# If we have used slit smearing, plot the smeared I(q) too
Expand Down Expand Up @@ -111,6 +115,8 @@ def new1DPlot(self, out, pr, q=None):

return new_plot



def newPRPlot(self, out, pr, cov=None):
"""
"""
Expand All @@ -124,7 +130,7 @@ def newPRPlot(self, out, pr, cov=None):
(y, dy) = pr.pr_err(out, cov, x)
new_plot = Data1D(x, y, dy=dy)

new_plot.name = PR_FIT_LABEL + f"[{self._data.name}]"
new_plot.name = "%s [%s]" % (PR_FIT_LABEL, self._data.name)
new_plot.xaxis("\\rm{r}", 'A')
new_plot.yaxis("\\rm{P(r)} ", "cm^{-3}")
new_plot.title = "P(r) fit"
Expand All @@ -135,12 +141,18 @@ def newPRPlot(self, out, pr, cov=None):

return new_plot

def add_errors(self, sigma=0.05):
def add_errors(self, sigma=1.0):
"""
Adds errors to data set is they are not available.
Uses $\Delta y = \sigma | y |$.
"""
self._data.dy = sigma * np.fabs(self._data.y)
if self.data.dy.size == 0.0:
self.data.dy = np.sqrt(np.fabs(self.data.y))*sigma

if self.data.dy is not None:
self.data.dy = np.where(self.data.dy < 0.0, np.sqrt(np.fabs(self.data.y))*sigma, self.data.dy)
self.data.dy = np.where(np.fabs(self.data.dy) < 1e-16, 1e16, self.data.dy)
return self.data.dy

def computeDataRange(self):
"""
Expand All @@ -156,8 +168,13 @@ def computeRangeFromData(self, data):
qmin, qmax = None, None
if isinstance(data, Data1D):
try:
qmin = min(data.x)
qmax = max(data.x)
#set q values where Intensity is zero,
#to qmax and exclude from minimum accepted q
#to avoid dodgy points around beam stop
usable_qrange=np.where(data.y <= 0, qmax, data.x)
qmin = min(usable_qrange)

except (ValueError, TypeError):
msg = "Unable to find min/max/length of \n data named %s" % \
self.data.filename
Expand Down
Loading
Loading