string column metrics bug

[reza1615]

I am sorry there was a bug in my codes for string column description and it adds NaN to the results. so to solve it before computing any number please remove the NaN data from the calculations. this command

import numpy as np
df['a'] = df['a'].replace(np.nan, '', regex=True)

see http://alphatechadmin.pythonanywhere.com/dtale/popup/describe/1?selectedCol=City most of the numbers have 74 (NaN or missing number)

[aschonfeld]

@reza1615 I see 74 for the Missing: value but that seems correct. I can validate that when applying the custom filter City != City. Are you saying that these 74 nans should be updated to '' before we start building your column metrics? That seems a little off since these records are nan and not actually an empty string, but if thats how you think it should be treated thats fine with me.

[reza1615]

please check
start with space
end with space
have punctuation
theses cases the number 74 should be 0

[reza1615]

[aschonfeld]

Ahh, got it. Thank you. Only thing to make note of is that this update will make Min # Chars update to zero. Is this expected?

[reza1615]

yes it is right. we can remove 0 from the min result and get next min

[aschonfeld]

fixed in v1.28.0