-
Notifications
You must be signed in to change notification settings - Fork 73
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add main functions for the computation of Kr maps #866
base: master
Are you sure you want to change the base?
Conversation
Comments & Corrections:
Tests:
|
This is a large commit, better do small commits with specific changes. Otherwise is difficult to follow the flow of changes. I understant that:
Do I miss other changes? There is a comment of many lines related with time-series, I understant this is the time-evolution part, that will be re-adapted in a new PR. @carhc Are you preparing to commit more tests of the functions used in the code? |
cb58021
to
f1c6ab6
Compare
063b20c
to
c7606ed
Compare
c7606ed
to
aedc73e
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is being done with peras
geom_comb = itertools.product(b_center[1], b_center[0]) | ||
r_values = np.array([np.sqrt(x**2+y**2)for x, y in itertools.product(b_center[1], b_center[0])]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We usually take x as the outer look and y as the inner one. Here geom_comb
will loop over x first, I suggest to change it, if it doesn't have any implications.
Also, you have two instances of this cartesian product. I suggest making a list and use it in the next line and below in the assignment to the columns
kr_map['bin'] = bin_index | ||
kr_map['counts'] = counts | ||
kr_map['R'] = r_values | ||
kr_map[['Y', 'X']] = pd.DataFrame(geom_comb) | ||
kr_map['in_active'] = kr_map['R'] <= r_max | ||
kr_map['has_min_counts'] = kr_map['counts'] >= n_min | ||
kr_map['fit_success'] = False | ||
kr_map['valid'] = False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since you are going to enumerate most columns anyway, why not create the dataframe using the syntax:
pd.DataFrame(dict(col0 = values0, ...)
?
def get_XY_bins(n_bins : np.array, | ||
XYrange : Tuple[float, float]): | ||
''' | ||
Returns the bins that will be used to make the map. | ||
|
||
Parameters | ||
--------- | ||
b_nins: np.array | ||
array of len = 2 containing the number of bins in X and Y | ||
XYrange: Tuple[float, float] | ||
Limits (mm) of X and Y for the map computation | ||
|
||
Returns | ||
--------- | ||
bins: Tuple[np.array, np.array] | ||
Bins in each direction (X,Y) (square map). | ||
''' | ||
bins_x = np.linspace(*XYrange, n_bins[0]+1) | ||
bins_y = np.linspace(*XYrange, n_bins[1]+1) | ||
return bins_x, bins_y | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Unless this function is used in many many places, I don't think it's worth it.
|
||
function = fit_output.fn | ||
|
||
res = y - function(x) | ||
std = res.std() | ||
|
||
return res, std |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
make this more compact: it can fit in one line.
p-value for the Shapiro-Wilk normality test. | ||
''' | ||
|
||
pval = stats.shapiro(residuals)[1] if (len(residuals) > 10) else 0. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is 10 a magic number?
k = map_bin.bin | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
superfluous variable, please remove
map_bin['res_std'] = std | ||
map_bin['chi2'] = chi2 | ||
map_bin['pval'] = pval | ||
map_bin['fit_success'] = True if ier in [1, 2, 3, 4] else False |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
map_bin['fit_success'] = True if ier in [1, 2, 3, 4] else False | |
map_bin['fit_success'] = ier in range(1, 5) |
map_bin['chi2'] = chi2 | ||
map_bin['pval'] = pval | ||
map_bin['fit_success'] = True if ier in [1, 2, 3, 4] else False | ||
map_bin['valid'] = map_bin['fit_success'] & map_bin['has_min_counts'] & map_bin['in_active'] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for booleans it's more idiomatic to use and
instead of &
def find_outliers(maps : pd.DataFrame, | ||
x2range : Tuple[float, float] = (0, 2)): | ||
''' | ||
For a given maps and deserved range, it returns a mask where values are | ||
within the interval. | ||
|
||
Parameters | ||
--------- | ||
maps: pd.DataFrame | ||
Map to check the outliers | ||
x2range : Tuple[float, float] | ||
Range for chi2 | ||
|
||
Returns | ||
--------- | ||
mask: pd.Series | ||
Mask. | ||
''' | ||
mask = in_range(maps.chi2, *x2range) | ||
return mask | ||
|
||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This function is also superfluous
outliers = maps.in_active | ||
outliers &= np.logical_not(find_outliers(new_map, x2range)) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
outliers = maps.in_active | |
outliers &= np.logical_not(find_outliers(new_map, x2range)) | |
outliers = maps.in_active & ~in_range(new_map.chi2, *x2range)) |
This PR addresses issue #863 (ICAROS: Calibration map creation) and lies on top of PR #865 (Data preparation functions).
It contains the main and auxiliary functions needed for the core of Icaro city. Many different functions have been defined in order to prepare, fit and produce the Kr calibration maps.
WIP: