Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cross Validation Added #407

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

Cross Validation Added #407

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
cdc0e5a
new scoring metrics added
Apr 21, 2021
e737117
new changes
Apr 23, 2021
64e6c22
made the requested changes
Apr 23, 2021
babc4bf
removed unnecessary files and print statements + fixed _predict()
Apr 24, 2021
b1683b8
removed empty lines and spaces + fixed if..else indentation
Apr 26, 2021
ddd5d73
new changes
Apr 28, 2021
4b672cd
added Cross Validation results
May 13, 2021
1063207
cross validation
May 13, 2021
09b49f1
requested changes made
May 18, 2021
5f24173
single input for CV
May 20, 2021
eaa58dc
final suggested changes done
May 27, 2021
e175cfe
Merge branch 'master' of https://github.com/gramener/gramex
Jun 3, 2021
5f631c1
resolved conflicts
Jun 3, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,3 +155,4 @@ docs/_build

# License copied to conda build_dir
pkg/conda/LICENSE
env/*
18 changes: 15 additions & 3 deletions gramex/handlers/mlhandler.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,7 @@
from tornado.gen import coroutine
from tornado.web import HTTPError
from sklearn.metrics import get_scorer
from sklearn.model_selection import cross_val_predict, cross_val_score

op = os.path
MLCLASS_MODULES = [
Expand All @@ -40,7 +41,8 @@
'pipeline': True,
'nums': [],
'cats': [],
'target_col': None
'target_col': None,
'cv': True,
}
ACTIONS = ['predict', 'score', 'append', 'train', 'retrain']
DEFAULT_TEMPLATE = op.join(op.dirname(__file__), '..', 'apps', 'mlhandler', 'template.html')
Expand Down Expand Up @@ -112,14 +114,23 @@ def setup(cls, data=None, model={}, config_dir='', **kwargs):
data = cls._filtercols(data)
data = cls._filterrows(data)
cls.model = cls._assemble_pipeline(data, mclass=mclass, params=params)

# train the model
target = data[target_col]
train = data[[c for c in data if c != target_col]]
# cross validation
cls.cross_validation(train,target)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not required here.

gramex.service.threadpool.submit(
_fit, cls.model, train, target, cls.model_path, cls.name)
cls.config_store.flush()


@classmethod
def cross_validation(cls,train,target):
cv = cls.get_opt('cv',True)
if cv:
CVscore = cross_val_score(cls.model.steps[-1][1], X=train, y=target, cv=cv)
CVavg = sum(CVscore)/len(CVscore)
print('Cross Validation Score : ',CVavg)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CV should take place within the train method only.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more. 

if cv:
    cvscore = cross_val_score(mod, X=train, y=target, cv=cv)
else:
   # Do the usual .fit


@classmethod
def load_data(cls, default=pd.DataFrame()):
try:
Expand Down Expand Up @@ -351,6 +362,7 @@ def _train(self, data=None):
target = data[target_col]
train = data[[c for c in data if c != target_col]]
self.model = self._assemble_pipeline(data, force=True)
self.cross_validation(train,target)
_fit(self.model, train, target, self.model_path)
return {'score': self.model.score(train, target)}

Expand Down