Skip to content

[SPARK-14772][ML,PySpark]Python ML Params.copy treats uid, paramMaps … #12888

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hujy wants to merge 3 commits into from
Closed

[SPARK-14772][ML,PySpark]Python ML Params.copy treats uid, paramMaps … #12888

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2a05414
[SPARK-14772][ML,PySpark]Python ML Params.copy treats uid, paramMaps …
hujy May 4, 2016
f389ad2
copy default param map
hujy May 6, 2016
dc352cd
copy default param map
hujy May 6, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 12 additions & 8 deletions python/pyspark/ml/param/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ def getOrDefault(self, param):
return self._defaultParamMap[param]

@since("1.4.0")
def extractParamMap(self, extra=None):
def extractParamMap(self, extra=None, default=False):
Copy link
Contributor

@holdenk holdenk May 4, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please document this new param

"""
Extracts the embedded default param values and user-supplied
values, and then merges them with extra values from input into
Expand All @@ -366,17 +366,19 @@ def extractParamMap(self, extra=None):
user-supplied values < extra.

:param extra: extra param values
:param default: if just copy the default param map
:return: merged param map
"""
if extra is None:
if extra is None and not default:
extra = dict()
paramMap = self._defaultParamMap.copy()
paramMap.update(self._paramMap)
paramMap.update(extra)
if not default:
paramMap.update(self._paramMap)
paramMap.update(extra)
return paramMap

@since("1.4.0")
def copy(self, extra=None):
def copy(self, extra=None, default=False):
"""
Creates a copy of this instance with the same uid and some
extra params. The default implementation creates a
Expand All @@ -386,13 +388,14 @@ def copy(self, extra=None):
is not sufficient.

:param extra: Extra parameters to copy to the new instance
:param default: if just copy the default param map
:return: Copy of this instance
"""
if extra is None:
extra = dict()
that = copy.copy(self)
that._paramMap = {}
return self._copyValues(that, extra)
return self._copyValues(that, extra, default)

def _shouldOwn(self, param):
"""
Expand Down Expand Up @@ -463,18 +466,19 @@ def _setDefault(self, **kwargs):
self._defaultParamMap[p] = value
return self

def _copyValues(self, to, extra=None):
def _copyValues(self, to, extra=None, default=False):
"""
Copies param values from this instance to another instance for
params shared by them.

:param to: the target instance
:param extra: extra params to be copied
:param default: if just copy the default param map
:return: the target instance with param values copied
"""
if extra is None:
extra = dict()
paramMap = self.extractParamMap(extra)
paramMap = self.extractParamMap(extra, default)
for p in self.params:
if p in paramMap and to.hasParam(p.name):
to._set(**{p.name: paramMap[p]})
Expand Down