Tagged execution hooks not working (gauge-python 0.3.6, getgauge 0.2.0, gauge 0.8.5)

[afeique]

Description

Creating tagged execution hooks does not work for specs (untested with scenarios).

The table of values referenced in this issue:

    | search_terms                              |
    | MKS Instruments                           | 
    | RF Power Generator                        |
    | PECVD plasma enhanced vapour deposition   |

Having a table of values at the specification level causes tags to be lost/ignored within the execution context. This can be verified by using the following execution hooks:

# ---------------
# Execution Hooks
# ---------------

@before_spec()
def before_spec(exec_cxt):
    print("\nbefore_scenario_hook: websearch\n")
    print(exec_cxt)
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec()
def after_spec(exec_cxt):
    print("\nafter_scenario_hook: websearch\n")
    print(exec_cxt)
    context.driver.quit()

This produces output like:

$ gauge --tags "websearch" specs
Python: 3.6.1
# Web Search

before_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: , file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: , is_failing: False, tags:  }, Step: { text: , is_failing: False } }
  ## User visits google.com      P P P P

     |search_terms   |
     |---------------|
     |MKS Instruments|
  ## User searches using google.com      P P P P

     |search_terms      |
     |------------------|
     |RF Power Generator|
  ## User searches using google.com      P P P P

     |search_terms                           |
     |---------------------------------------|
     |PECVD plasma enhanced vapour deposition|
  ## User searches using google.com      P P P P

after_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: , file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: User searches using google.com, is_failing: False, tags: google search }, Step: { text: Element "//div[@id='resultStats']" appears within "5" seconds, is_failing: False } }

Successfully generated html-report to => C:\test\gauge\reports\html-report
Specifications: 1 executed      1 passed        0 failed        0 skipped
Scenarios:      2 executed      2 passed        0 failed        0 skipped

Total time taken: 14.827s

In particular, notice that ExecutionInfo['Specification']['tags'] is EMPTY.

Removing the table of values and any scenarios that use <search_terms> results in ExecutionInfo['Specification']['tags'] being populated correctly. Instructions for doing this are provided in the Workarounds section under Steps to Reproduce.

Output when spec-level tables and scenarios that use them are removed:

$ gauge --tags "websearch" specs
Python: 3.6.1
# Web Search

before_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: websearch, file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: , is_failing: False, tags:  }, Step: { text: , is_failing: False } }
  ## User visits google.com      P P P P

after_scenario_hook: websearch

ExecutionInfo: { Specification: { name: Web Search, is_failing: False, tags: websearch, file_name: C:\test\gauge\specs\websearch.spec }, Scenario: { name: User visits google.com, is_failing: False, tags: visit google }, Step: { text: Look for "//input[@value='Google Search']", is_failing: False } }

Successfully generated html-report to => C:\test\gauge\reports\html-report
Specifications: 1 executed      1 passed        0 failed        0 skipped
Scenarios:      1 executed      1 passed        0 failed        0 skipped

Total time taken: 8.986s

In particular, notice that ExecutionInfo['Specification']['tags'] is NOT EMPTY.

This behavior may be a bug with Gauge itself.

Steps to reproduce

0. Install selenium. Install Chrome webdriver to PATH.

1. step_impl/websearch.py

Note that the @before_spec and @after_spec hooks are tagged with websearch

from getgauge.python import *
# step, after_suite, Messages, DataStoreFactory

from selenium import webdriver
from selenium.webdriver.remote.webelement import WebElement
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait # available since 2.4.0
from selenium.webdriver.support import expected_conditions as EC # available since 2.26.0

# ---------------
# Execution Hooks
# ---------------

@before_spec("<websearch>")
def before_spec():
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec("<websearch>")
def after_spec():
    context.driver.quit()

class Context(object):
    pass

context = Context()

# --------------------------
# Gauge step implementations
# --------------------------

@step('Visit <url>')
def test_visit_url(url):
    if not (url.startswith("http://") or url.startswith("https://")):
        context.driver.get("http://"+ url)
    else:
        context.driver.get(url)
    assert url in context.driver.current_url

@step('Look for <xpath>')
def test_find_element(xpath):
    try:
        e = context.driver.find_element_by_xpath(xpath)
    except:
        Messages.write_message('Could not find element with xpath "%s"' % xpath)
    assert isinstance(e, WebElement) is True
    return e

@step('Element <xpath> appears within <wait_time> seconds')
def test_wait_and_find_element(xpath, wait_time):
    try:
        e = WebDriverWait(context.driver, float(wait_time)).until(EC.presence_of_element_located((By.XPATH, xpath)))
    except:
        Messages.write_message('Could not find element with xpath "%s"' % xpath)
    assert isinstance(e, WebElement) is True
    return e


@step('Enter <search_terms> into <xpath>')
def test_enter_data_into_element(search_terms, xpath):
    e = test_find_element(xpath)
    e.send_keys(search_terms)
    assert e.get_attribute('value') == search_terms

@step('Click <xpath>')
def test_click_element(xpath):
    e = test_find_element(xpath)
    e.click()

2. specs/websearch.spec

# Web Search

Tags: websearch

    | search_terms                              |
    | MKS Instruments                           | 
    | RF Power Generator                        |
    | PECVD plasma enhanced vapour deposition   |

This is a demo specification for puppeting a Google Chrome instance which is used to search the web.

To execute this specification, run

    gauge specs

## User visits google.com

Tags: visit google

In this scenario, the user visits google.com and sees: the google logo (img), a search box (input), and a search button (input).

* Visit "google.com"
* Look for "//img[@alt='Google']"
* Look for "//input[@name='q' and @value='']"
* Look for "//input[@value='Google Search']"

## User searches using google.com

Tags: google search

In this scenario, the user navigates to google.com and performs a few searches.

* Visit "google.com"
* Enter <search_terms> into "//input[@name='q']"
* Click "//button[@value='Search' and @aria-label='Google Search']"
* Element "//div[@id='resultStats']" appears within "5" seconds

3. Run gauge

gauge --tags "websearch" specs

Expected behaviour:
The before_spec and after_spec hooks are executed, correctly instantiating the Selenium webdriver at the start of the spec and then closing the browser at the end of the spec.

Actual behaviour:
None of the tagged-execution hooks are run.

Workarounds:

1. Remove <search_terms> table at specification level. Remove any scenarios that use <search_terms>.

Updated specs/websearch.spec

# Web Search

Tags: websearch

This is a demo specification for puppeting a Google Chrome instance which is used to search the web.

To execute this specification, run

    gauge specs

## User visits google.com

Tags: visit google

In this scenario, the user visits google.com and sees: the google logo (img), a search box (input), and a search button (input).

* Visit "google.com"
* Look for "//img[@alt='Google']"
* Look for "//input[@name='q' and @value='']"
* Look for "//input[@value='Google Search']"

This is undesirable because it requires removing tests, or defining them in an inconvenient manner (e.g. hardcoded values instead of dynamic values from table).

2. Use untagged execution hooks:

# ---------------
# Execution Hooks
# ---------------

@before_spec()
def before_spec():
    context.driver = webdriver.Chrome()
    context.driver.implicitly_wait(10)

@after_spec()
def after_spec():
    context.driver.quit()

Unfortunately, now the before_spec and after_spec hooks will be run for ALL specs (undesirable for suites containing multiple disparate specs).

Version Information

• Gauge version: 0.8.5
• Gauge Python plugin version: 0.3.6
• Python version: 3.6.1 (conda env)
• getgauge package version, run pip3 show getgauge to get it: 0.2.0
• OS information: Windows 7 x64

[kashishm]

@afeique Thanks for logging this issue. This should be fixed in Gauge.
Closing this issue for now. Please track the progress on Gauge issue 708.

Feel free to reopen if this issue exists after the Gauge fix.