Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add initial support for ibis.random function #2060

Merged
merged 9 commits into from
Mar 14, 2020
1 change: 1 addition & 0 deletions docs/source/release.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Release Notes

* :feature:`2048` Introduce a top level vectorized UDF module (experimental). Implement element-wise UDF for pandas and PySpark backend.
* :release:`1.2.1 <pending>`
* :feature:`2060` Add initial support for ibis.random function
* :bug:`2061` CI: Fix CI builds related to new pandas 1.0 compatibility
* :feature:`1976` Add DenseRank, RowNumber, MinRank, Count, PercentRank/CumeDist window operations to OmniSciDB
* :bug:`2055` Fix "cudf" import on OmniSciDB backend
Expand Down
6 changes: 6 additions & 0 deletions ibis/expr/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,6 +187,7 @@
'param',
'pi',
'prevent_rewrite',
'random',
'range_window',
'row_number',
'rows_with_max_lookback',
Expand Down Expand Up @@ -1344,6 +1345,11 @@ def clip(arg, lower=None, upper=None):
return op.to_expr()


def random():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's create a ibis.expr.random; the goal is to produce a random scalar yes? reference the documentation https://docs.python.org/3/library/random.html, which i assume is the goal here.

op = ops.Random()
return op.to_expr()


def quantile(arg, quantile, interpolation='linear'):
"""
Return value at the given quantile, a la numpy.percentile.
Expand Down
5 changes: 5 additions & 0 deletions ibis/expr/operations.py
Original file line number Diff line number Diff line change
Expand Up @@ -2450,6 +2450,11 @@ def output_type(self):
return dt.timestamp.scalar_type()


class Random(Constant):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this could be RandomScalar. you could eventually expand this to a column op as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jrbourbeau do you still have any interesting on moving this forward? let me know if you need any help! thanks for having been working on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies for the delayed response. I've made updates in response to @jreback's initial feedback, please let me know if there's anything else you'd like me to include in this PR

def output_type(self):
return dt.float64.scalar_type()


class E(Constant):
def output_type(self):
return functools.partial(ir.FloatingScalar, dtype=dt.float64)
Expand Down
5 changes: 5 additions & 0 deletions ibis/sql/mysql/compiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -191,6 +191,10 @@ def _literal(t, expr):
return sa.literal(value)


def _random(t, expr):
return sa.func.random()


_operation_registry.update(
{
ops.Literal: _literal,
Expand All @@ -204,6 +208,7 @@ def _literal(t, expr):
ops.Log2: unary(sa.func.log2),
ops.Log10: unary(sa.func.log10),
ops.Round: _round,
ops.Random: _random,
# dates and times
ops.Date: unary(sa.func.date),
ops.DateAdd: infix_op('+'),
Expand Down
5 changes: 5 additions & 0 deletions ibis/sql/postgres/compiler.py
Original file line number Diff line number Diff line change
Expand Up @@ -607,6 +607,10 @@ def _literal(t, expr):
return sa.literal(value)


def _random(t, expr):
return sa.func.random()


def _day_of_week_index(t, expr):
(sa_arg,) = map(t.translate, expr.op().args)
return sa.cast(
Expand Down Expand Up @@ -682,6 +686,7 @@ def _day_of_week_name(t, expr):
ops.Max: _reduction('max'),
ops.Variance: _variance_reduction('var'),
ops.StandardDev: _variance_reduction('stddev'),
ops.Random: _random,
# now is in the timezone of the server, but we want UTC
ops.TimestampNow: lambda *args: sa.func.timezone('UTC', sa.func.now()),
ops.CumulativeAll: unary(sa.func.bool_and),
Expand Down
14 changes: 14 additions & 0 deletions ibis/tests/all/test_numeric.py
Original file line number Diff line number Diff line change
Expand Up @@ -405,3 +405,17 @@ def test_sa_default_numeric_precision_and_scale(

assert_equal(schema, expected)
con.drop_table(table_name, force=True)


@pytest.mark.only_on_backends([PostgreSQL, MySQL])
def test_random(con):
expr = ibis.random()
result = con.execute(expr)
assert isinstance(result, float)
assert 0 <= result < 1

# Ensure a different random number of produced on subsequent calls
result2 = con.execute(expr)
assert isinstance(result2, float)
assert 0 <= result2 < 1
assert result != result2