Correct possible bug #29
Conversation
(1) Line 88: if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0): --> if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0): (2) over_sampling.py, Line 283-285, 377-379: synth_matrix[i * x_synth + j, (d - 1)] = data.iloc[i, (d - 1)] + data.iloc[ knn_matrix[i, neigh], (d - 1)] / 2 --> synth_matrix[i * x_synth + j, (d - 1)] = ( data.iloc[i, (d - 1)] + data.iloc[ knn_matrix[i, neigh], (d - 1)] ) / 2 (3) over_sampling.py, Line 267, 363: for z in feat_list_num: --> for z in feat_list_num[0:(d-1)]: (Because when z=d-1, the entry synth_matrix[x_synth * n + count, z] is not defined yet.)
| (d - 1)] = data.iloc[i, (d - 1)] + data.iloc[ | ||
| knn_matrix[i, neigh], (d - 1)] / 2 | ||
| (d - 1)] = (data.iloc[i, (d - 1)] + data.iloc[ | ||
| knn_matrix[i, neigh], (d - 1)]) / 2 |
| @@ -264,7 +264,7 @@ def over_sampling( | |||
|
|
|||
| ## generate synthetic y response variable by | |||
| ## inverse distance weighted | |||
| for z in feat_list_num: | |||
| for z in feat_list_num[0:(d - 1)]: | |||
There was a problem hiding this comment.
Why is this preferred (genuinely curious)?
There was a problem hiding this comment.
I made this change because synth_matrix[:,z] was only defined for 0:(d-1).
| @@ -85,7 +85,7 @@ def over_sampling( | |||
| num_dtypes = ["int64", "float64"] | |||
|
|
|||
| for j in range(d): | |||
| if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0): | |||
| if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0): | |||
There was a problem hiding this comment.
What is this doing that any() is not?
There was a problem hiding this comment.
all() assures all elements are positive.
any() only assures that some elements are positive (there may also be negative elements).
I made this change because I thought the former is expected here.
| @@ -360,7 +360,7 @@ def over_sampling( | |||
|
|
|||
| ## generate synthetic y response variable by | |||
| ## inverse distance weighted | |||
| for z in feat_list_num: | |||
| for z in feat_list_num[0:(d - 1)]: | |||
There was a problem hiding this comment.
Why is this preferred (genuinely curious)?
There was a problem hiding this comment.
I made this change because synth_matrix[:,z] was only defined for 0:(d-1).
| @@ -374,9 +374,9 @@ def over_sampling( | |||
| x_synth * n + count, feat_list_nom]) | |||
|
|
|||
| if a == b: | |||
| synth_matrix[x_synth * n + count, (d - 1)] = data.iloc[ | |||
| synth_matrix[x_synth * n + count, (d - 1)] = (data.iloc[ | |||
|
@seazuma I hope that you might be able to address some of the questions I had in your PR. Thank you. |
|
@nickkunz I apologize for having failed to notice your comments before. Thank you for letting me know again. I have replied to them. |
I appreciate your library smogn. I found some suspicious points in over_sampling.py and tried to fix them.
(1) Line 88:
if data.iloc[:, j].dtype in num_dtypes and any(data.iloc[:, j] > 0):
-->
if data.iloc[:, j].dtype in num_dtypes and all(data.iloc[:, j] > 0):
(2) over_sampling.py, Line 283-285, 377-379:
synth_matrix[i * x_synth + j,
(d - 1)] = data.iloc[i, (d - 1)] + data.iloc[
knn_matrix[i, neigh], (d - 1)] / 2
-->
synth_matrix[i * x_synth + j,
(d - 1)] = ( data.iloc[i, (d - 1)] + data.iloc[
knn_matrix[i, neigh], (d - 1)] ) / 2
(3) over_sampling.py, Line 267, 363:
for z in feat_list_num:
-->
for z in feat_list_num[0:(d-1)]:
(Because when z=d-1, the entry synth_matrix[x_synth * n + count, z] is not defined yet.)