Fixed error with dialog expression when virtual column involved

[yrudman]

ISSUE:
When virtual column involved in expression for automate method than error raised during attempt to order service. Error operator 'token' is not supported... raised when MiqExpression trying to convert expression to sql.
The same expression works fine in other places, like report.

Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1558926

Automation vs Report:
Automation do special pre-processing of expression (before converting to sql) and substitute value in expression with supplied tokens (if any). This pre-processing includes adding temporary entry like :token=>1 and supposed to be removed after automation specific pre-processing finished.

There is place on UI side were those temporary tokens removed before saving record: https://github.com/ManageIQ/manageiq-ui-
classic/blob/f16e3f542c0463773ca19291ddd8884e8863aa9c/app/controllers/application_controller/filter.rb#L343 , but there is no clean-up on automation side during method invocation

FIX:
Remove temporary tokens from expression during token substitution.
It would allow to skip attempt to transfer expression to_sql here:

manageiq/lib/miq_expression.rb

Line 289 in 9e95f2b

sql = to_arel(@pexp, tz).to_sql if @pexp.present?

EXPRESSION:

BEFORE:

AFTER:

@miq-bot add-label bug

\cc @gtanzillo @gmcculloug

[yrudman]

@miq-bot assign @gtanzillo

[mkanoor]

This is the same as #16514

[yrudman]

@miq-bot add-label gaprindashvili/yes

[Fryguy]

@kbrock Please review

[gtanzillo]

This looks good. But it may be good to test with a more complex expression. Something like -

{"and"=>[{"="=>{"field"=>"Vm-active", "value"=>"true"}, :token => 1}, {"="=>{"field"=>"Vm-archived", "value"=>"true"}, :token => 1}], :token => 1}

Just a suggestion

[yrudman]

@gtanzillo ok, make sense, will do with this expression:
{"and"=>[{"="=>{"field"=>"Vm-active", "value"=>"true"}, :token => 1}, {"="=>{"field"=>"Vm-archived", "value"=>"true"}, :token => 1}]}. (there is no tokens added on the level of and or or)

[miq-bot]

Checked commits yrudman/manageiq@40ba300~...a199611 with ruby 2.3.3, rubocop 0.52.1, haml-lint 0.20.0, and yamllint 1.10.0
2 files checked, 0 offenses detected
Everything looks fine. 🍰

[kbrock]

deleting the :token looks good.
Not exactly sure where/why it was added.

I'm good with this.
But would it make more sense for the ui that added the :token key to remove it?

[simaishi]

Gaprindashvili backport details:

$ git log -1
commit 4a2af379cc61523e99bc32b0b0499e3199bdc697
Author: Gregg Tanzillo <gtanzill@redhat.com>
Date:   Thu Apr 5 14:26:27 2018 -0400

    Merge pull request #17215 from yrudman/remove-temporary-tokens-from-expression-after-substitution
    
    Fixed error with dialog expression when virtual column involved
    (cherry picked from commit 54589f650f2c4026a43dcf9b4230d77cd7b5d504)
    
    Fixes https://bugzilla.redhat.com/show_bug.cgi?id=1565139