Fill missing values

[reza1615]

It is good to have fill missing values. usallay I use these commands

• df.loc[:,'col']=df.loc[:,'col'].replace(r'^\s+$', np.nan, regex=True) # Replace 1 space or multiple space
• .replace(r'[Uu]nknown', np.nan, regex=True) # replace unknown or Unknown
• .replace(r'^ *[-_]+ *$', np.nan, regex=True) # replace '-' or ' - ' or ' ---- ' or '_'
• .fillna('')
• .fillna(0)
• .fillna(value)
• .fillna(df.mean()/min/max/fisrt/last/mid)
  Also
• SimpleImputer
• KNNImputer
• IterativeImputer
  https://www.linkedin.com/posts/justmarkham_sklearntips-machinelearning-python-activity-6650374643419090944-GHyR
  https://www.linkedin.com/posts/justmarkham_sklearntips-machinelearning-python-activity-6648932419095322624-hH5F

[reza1615]

Maybe it is better to have a replacement window with these sub-pages

• filling missing value is a sub-page
• replace value1 > value 2
• Replace outliers by max() or min() (see Outliers in the description window #137)

[reza1615]

I developed this notebook.

For UI I suggest to have some thing like this pipline
to replace
1-specific word (like "unknown") > null
2-specific char (like "-") > null
3-empty > nul
.
.
.
.
10- null > value or empty or remain null

Updated the code. Now it works properly.

[reza1615]

Now it works properly + Added

• SimpleImputer
• KNNImputer
• IterativeImputer
  fill missings3.zip

[aschonfeld]

@reza1615 This is great! Thanks for supplying the code. I'll work on integrating this and hopefully this can be used for #86 as well

[aschonfeld]

@reza1615 here's the first cut of this new functionality. It should also tackle #86 as well.

Let me know if you want me to add the functionality to create a new column as well as inplace updates?

[reza1615]

That is great thanks

[aschonfeld]

added v1.8.14