Specifications: tags and tagged execution hooks do not work when a data table is present at spec-level (gauge 0.8.5)

[afeique]

Description

Tests were written in Python.

Creating tagged execution hooks does not work for specs (untested with scenarios).

The table of values referenced in this issue:

    | search_terms                              |
    | MKS Instruments                           | 
    | RF Power Generator                        |
    | PECVD plasma enhanced vapour deposition   |

Having a table of values at the specification level causes tags to be lost/ignored within the execution context. This can be verified by using the following execution hooks:

# ---------------
# Execution Hooks
# ---------------

@before_spec()
def before_spec(exec_cxt):
    print("\nbefore_scenario_hook: websearch\n")
    print(exec_cxt)
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec()
def after_spec(exec_cxt):
    print("\nafter_scenario_hook: websearch\n")
    print(exec_cxt)
    context.driver.quit()

This produces output like:

$ gauge --tags "websearch" specs
Python: 3.6.1
# Web Search

before_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: , file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: , is_failing: False, tags:  }, Step: { text: , is_failing: False } }
  ## User visits google.com      P P P P

     |search_terms   |
     |---------------|
     |MKS Instruments|
  ## User searches using google.com      P P P P

     |search_terms      |
     |------------------|
     |RF Power Generator|
  ## User searches using google.com      P P P P

     |search_terms                           |
     |---------------------------------------|
     |PECVD plasma enhanced vapour deposition|
  ## User searches using google.com      P P P P

after_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: , file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: User searches using google.com, is_failing: False, tags: google search }, Step: { text: Element "//div[@id='resultStats']" appears within "5" seconds, is_failing: False } }

Successfully generated html-report to => C:\test\gauge\reports\html-report
Specifications: 1 executed      1 passed        0 failed        0 skipped
Scenarios:      2 executed      2 passed        0 failed        0 skipped

Total time taken: 14.827s

In particular, notice that ExecutionInfo['Specification']['tags'] is EMPTY.

Removing the table of values and any scenarios that use <search_terms> results in ExecutionInfo['Specification']['tags'] being populated correctly. Instructions for doing this are provided in the Workarounds section under Steps to Reproduce.

Output when spec-level tables and scenarios that use them are removed:

$ gauge --tags "websearch" specs
Python: 3.6.1
# Web Search

before_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: websearch, file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: , is_failing: False, tags:  }, Step: { text: , is_failing: False } }
  ## User visits google.com      P P P P

after_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: websearch, file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: User visits google.com, is_failing: False, tags: visit google }, Step: { text: Look for "//input[@value='Google Search']", is_failing: False } }

Successfully generated html-report to => C:\test\gauge\reports\html-report
Specifications: 1 executed      1 passed        0 failed        0 skipped
Scenarios:      1 executed      1 passed        0 failed        0 skipped

Total time taken: 8.986s

In particular, notice that ExecutionInfo['Specification']['tags'] is NOT EMPTY.

This behavior may be a bug with Gauge itself.

Steps to reproduce

0. Install selenium. Install Chrome webdriver to PATH.

1. step_impl/websearch.py

Note that the @before_spec and @after_spec hooks are tagged with websearch

from getgauge.python import *
# step, after_suite, Messages, DataStoreFactory

from selenium import webdriver
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0
from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

# ---------------
# Execution Hooks
# ---------------

@before_spec("<websearch>")
def before_spec():
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec("<websearch>")
def after_spec():
    context.driver.quit()

class Context(object):
    pass

context = Context()

# --------------------------
# Gauge step implementations
# --------------------------

@step('Visit <url>')
def test_visit_url(url):
    if not (url.startswith("http://") or url.startswith("https://")):
        context.driver.get("http://"+ url)
    else:
        context.driver.get(url)
    assert url in context.driver.current_url

@step('Look for <xpath>')
def test_find_element(xpath):
    try:
        e = context.driver.find_element_by_xpath(xpath)
    except:
        Messages.write_message('Could not find element with xpath "%s"' % xpath)
    assert isinstance(e, WebElement) is True
    return e

@step('Element <xpath> appears within <wait_time> seconds')
def test_wait_and_find_element(xpath, wait_time):
    try:
        e = WebDriverWait(context.driver, float(wait_time)).until(EC.presence_of_element_located((By.XPATH, xpath)))
    except:
        Messages.write_message('Could not find element with xpath "%s"' % xpath)
    assert isinstance(e, WebElement) is True
    return e


@step('Enter <search_terms> into <xpath>')
def test_enter_data_into_element(search_terms, xpath):
    e = test_find_element(xpath)
    e.send_keys(search_terms)
    assert e.get_attribute('value') == search_terms

@step('Click <xpath>')
def test_click_element(xpath):
    e = test_find_element(xpath)
    e.click()

2. specs/websearch.spec

# Web Search

Tags: websearch

    | search_terms                              |
    | MKS Instruments                           | 
    | RF Power Generator                        |
    | PECVD plasma enhanced vapour deposition   |

This is a demo specification for puppeting a Google Chrome instance which is used to search the web.

To execute this specification, run

    gauge specs

## User visits google.com

Tags: visit google

In this scenario, the user visits google.com and sees: the google logo (img), a search box (input), and a search button (input).

* Visit "google.com"
* Look for "//img[@alt='Google']"
* Look for "//input[@name='q' and @value='']"
* Look for "//input[@value='Google Search']"

## User searches using google.com

Tags: google search

In this scenario, the user navigates to google.com and performs a few searches.

* Visit "google.com"
* Enter <search_terms> into "//input[@name='q']"
* Click "//button[@value='Search' and @aria-label='Google Search']"
* Element "//div[@id='resultStats']" appears within "5" seconds

3. Run gauge

gauge --tags "websearch" specs

Expected behaviour:
The before_spec and after_spec hooks are executed, correctly instantiating the Selenium webdriver at the start of the spec and then closing the browser at the end of the spec.

Actual behaviour:
None of the tagged-execution hooks are run.

Workarounds:

1. Remove <search_terms> table at specification level. Remove any scenarios that use <search_terms>.

Updated specs/websearch.spec

# Web Search

Tags: websearch

This is a demo specification for puppeting a Google Chrome instance which is used to search the web.

To execute this specification, run

    gauge specs

## User visits google.com

Tags: visit google

In this scenario, the user visits google.com and sees: the google logo (img), a search box (input), and a search button (input).

* Visit "google.com"
* Look for "//img[@alt='Google']"
* Look for "//input[@name='q' and @value='']"
* Look for "//input[@value='Google Search']"

This is undesirable because it requires removing tests, or defining them in an inconvenient manner (e.g. hardcoded values instead of dynamic values from table).

2. Use untagged execution hooks:

# ---------------
# Execution Hooks
# ---------------

@before_spec()
def before_spec():
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec()
def after_spec():
    context.driver.quit()

Unfortunately, now the before_spec and after_spec hooks will be run for ALL specs (undesirable for suites containing multiple disparate specs).

Version Information

• Gauge version: 0.8.5
• Gauge Python plugin version: 0.3.6
• Python version: 3.6.1 (conda env)
• getgauge package version, run pip3 show getgauge to get it: 0.2.0
• OS information: Windows 7 x64

[Apoorva-GA]

Fix should be available in nightly >= 12/07/17