One-dimensional datatable transformer behavior in Cucumber 5 does not match Cucumber 4 behavior

[BuzMack]

Describe the bug
When trying to transform a one-dimensional DataTable (i.e. DataTable has only one row and many columns, or only one column and many rows) into a List, the resulting output differs from Cucumber 4 behavior.

Given two DataTables as follows:

Given table1
  | aaa | bbb | ccc |

Given table2
  | aaa |
  | bbb |
  | ccc |

Running the following code for both versions (4.4.0 or 5.5.0)

List<String> result1 = table1.asList(String.class);
List<String> result2 = table2.asList(String.class);

Using Cucumber 4 (version 4.4.0) has these results:

result1 ==> [aaa, bbb, ccc]
result2 ==> [aaa, bbb, ccc]

Using Cucumber 5 (version 5.5.0) has these results:

result1 ==> []
result2 ==> [aaa, bbb, ccc]

Additionally, when using a multi-dimensional table, asList(Type) yields a CucumberDataTableException

To Reproduce
Feature file used:

@DataTableTransforms
Feature: Test DataTable transforms

  Scenario: Show datatable transforms of one-dimensional tables
    Given show DataTable transforms for
      | aaa | bbb | ccc |
    Given show DataTable transforms for
      | aaa |
      | bbb |
      | ccc |

  Scenario: Show datatable transforms of two-dimensional tables
    Given show DataTable transforms for
      | aaa | bbb | ccc |
      | ddd | eee | fff |
    Given show DataTable transforms for
      | aaa | bbb |
      | ccc | ddd |
      | eee | fff |

Runner class DataTableTransform_Runner.class (comment or uncomment imports depending on version):

package com.expd.test.runners;

//imports for Cucumber 4.4.0
//import cucumber.api.CucumberOptions;
//import cucumber.api.testng.AbstractTestNGCucumberTests;

//imports for Cucumber 5.5.0
import io.cucumber.testng.AbstractTestNGCucumberTests;
import io.cucumber.testng.CucumberOptions;

import org.testng.annotations.DataProvider;

@CucumberOptions(
        tags = {"@DataTableTransforms"},
        features = {"src/feature"},
        glue = {"com.expd.test.datatabletests"},
        strict = true,
        plugin = {
                "pretty"
        }
)
public class DataTableTransform_Runner extends AbstractTestNGCucumberTests {
}

Step definition class DataTableTestSteps.class (comment or uncomment imports depending on version):

package com.expd.test.datatabletests;

//imports for Cucumber 4.4.0
//import cucumber.api.Scenario;
//import cucumber.api.java.After;
//import cucumber.api.java.Before;
//import cucumber.api.java.en.Given;

//imports for Cucumber 5.5.0
import io.cucumber.java.Scenario;
import io.cucumber.java.After;
import io.cucumber.java.Before;
import io.cucumber.java.en.Given;

import io.cucumber.datatable.DataTable;
import java.util.List;

public class DataTableTestSteps {
    protected Scenario scenario;

    @Before
    public void doBefore(Scenario scenario) {
        this.scenario = scenario;
    }

    @After
    public void doAfter(Scenario scenario) {
    }

    @Given("^show DataTable transforms for$")
    public void showTransforms(DataTable dataTable) {
        System.out.format("Input DataTable raw:%n%s", dataTable);
        System.out.format("DataTable.asLists(): %s%n", dataTable.asLists());
        System.out.format("DataTable.asLists(String.class): %s%n", dataTable.asLists(String.class));
        System.out.format("DataTable.asList(): %s%n", dataTable.asList());
        System.out.format("DataTable.asList(String.class): %s%n", dataTable.asList(String.class));
    }
}

TypeRegistration info... TypeRegistryConfiguration.class (only for version 4.4.0):

package com.expd.test.datatabletests;

import com.fasterxml.jackson.databind.ObjectMapper;
import cucumber.api.TypeRegistry;
import cucumber.api.TypeRegistryConfigurer;
import io.cucumber.cucumberexpressions.ParameterByTypeTransformer;
import io.cucumber.datatable.TableCellByTypeTransformer;
import io.cucumber.datatable.TableEntryByTypeTransformer;

import java.lang.reflect.Type;
import java.util.Locale;
import java.util.Map;

public class TypeRegistryConfiguration implements TypeRegistryConfigurer {

    @Override
    public Locale locale() {
        return Locale.ENGLISH;
    }

    @Override
    public void configureTypeRegistry(TypeRegistry typeRegistry) {
        Transformer transformer = new Transformer();
        typeRegistry.setDefaultDataTableCellTransformer(transformer);
        typeRegistry.setDefaultDataTableEntryTransformer(transformer);
        typeRegistry.setDefaultParameterTransformer(transformer);
    }

    private static class Transformer implements ParameterByTypeTransformer, TableEntryByTypeTransformer, TableCellByTypeTransformer {
        ObjectMapper objectMapper = new ObjectMapper();

        @Override
        public Object transform(String s, Type type) {
            return objectMapper.convertValue(s, objectMapper.constructType(type));
        }

        @Override
        public <T> T transform(Map<String, String> map, Class<T> aClass, TableCellByTypeTransformer tableCellByTypeTransformer) {
            return objectMapper.convertValue(map, aClass);
        }

        @Override
        public <T> T transform(String s, Class<T> aClass) {
            return objectMapper.convertValue(s, aClass);
        }
    }
}

TypeRegistration info... NewTypeRegistryConfiguration.class (only for version 5.5.0):

package com.expd.test.datatabletests;

import io.cucumber.java.DefaultDataTableCellTransformer;
import io.cucumber.java.DefaultDataTableEntryTransformer;
import io.cucumber.java.DefaultParameterTransformer;

import com.fasterxml.jackson.databind.ObjectMapper;

import java.lang.reflect.Type;

public class NewTypeRegistryConfiguration {
    ObjectMapper objectMapper = new ObjectMapper();

    @DefaultParameterTransformer
    @DefaultDataTableEntryTransformer
    @DefaultDataTableCellTransformer
    public Object transform(Object fromValue, Type toValueType) {
        return objectMapper.convertValue(fromValue, objectMapper.constructType(toValueType));
    }
}

When running the above with version 4.4.0, relevant output generated:

Input DataTable raw:
      | aaa | bbb | ccc |
DataTable.asLists(): [[aaa, bbb, ccc]]
DataTable.asLists(String.class): [[aaa, bbb, ccc]]
DataTable.asList(): [aaa, bbb, ccc]
DataTable.asList(String.class): [aaa, bbb, ccc]
Input DataTable raw:
      | aaa |
      | bbb |
      | ccc |
DataTable.asLists(): [[aaa], [bbb], [ccc]]
DataTable.asLists(String.class): [[aaa], [bbb], [ccc]]
DataTable.asList(): [aaa, bbb, ccc]
DataTable.asList(String.class): [aaa, bbb, ccc]


Input DataTable raw:
      | aaa | bbb | ccc |
      | ddd | eee | fff |
DataTable.asLists(): [[aaa, bbb, ccc], [ddd, eee, fff]]
DataTable.asLists(String.class): [[aaa, bbb, ccc], [ddd, eee, fff]]
DataTable.asList(): [aaa, bbb, ccc, ddd, eee, fff]
DataTable.asList(String.class): [aaa, bbb, ccc, ddd, eee, fff]
Input DataTable raw:
      | aaa | bbb |
      | ccc | ddd |
      | eee | fff |
DataTable.asLists(): [[aaa, bbb], [ccc, ddd], [eee, fff]]
DataTable.asLists(String.class): [[aaa, bbb], [ccc, ddd], [eee, fff]]
DataTable.asList(): [aaa, bbb, ccc, ddd, eee, fff]
DataTable.asList(String.class): [aaa, bbb, ccc, ddd, eee, fff]

When running with version 5.5.0, the relevant output is:

@DataTableTransforms
Scenario: Show datatable transforms of one-dimensional tables # src/feature/DataTableTransforms.feature:4
Input DataTable raw:
      | aaa | bbb | ccc |
DataTable.asLists(): [[aaa, bbb, ccc]]
DataTable.asLists(String.class): [[aaa, bbb, ccc]]
DataTable.asList(): [aaa, bbb, ccc]
DataTable.asList(String.class): []
  Given show DataTable transforms for                         # com.expd.test.datatabletests.DataTableTestSteps.showTransforms(io.cucumber.datatable.DataTable)
Input DataTable raw:
      | aaa |
      | bbb |
      | ccc |
DataTable.asLists(): [[aaa], [bbb], [ccc]]
DataTable.asLists(String.class): [[aaa], [bbb], [ccc]]
DataTable.asList(): [aaa, bbb, ccc]
DataTable.asList(String.class): [aaa, bbb, ccc]
  Given show DataTable transforms for                         # com.expd.test.datatabletests.DataTableTestSteps.showTransforms(io.cucumber.datatable.DataTable)



@DataTableTransforms
Scenario: Show datatable transforms of two-dimensional tables # src/feature/DataTableTransforms.feature:12
Input DataTable raw:
      | aaa | bbb | ccc |
      | ddd | eee | fff |
DataTable.asLists(): [[aaa, bbb, ccc], [ddd, eee, fff]]
DataTable.asLists(String.class): [[aaa, bbb, ccc], [ddd, eee, fff]]
DataTable.asList(): [aaa, bbb, ccc, ddd, eee, fff]
  Given show DataTable transforms for                         # com.expd.test.datatabletests.DataTableTestSteps.showTransforms(io.cucumber.datatable.DataTable)
      io.cucumber.datatable.CucumberDataTableException: 'java.util.List<java.lang.String>' could not transform
      | aaa | bbb | ccc |
      | ddd | eee | fff |

	at io.cucumber.datatable.DataTableType.transform(DataTableType.java:144)
	at io.cucumber.datatable.DataTableTypeRegistryTableConverter.toListOrNull(DataTableTypeRegistryTableConverter.java:163)
	at io.cucumber.datatable.DataTableTypeRegistryTableConverter.toList(DataTableTypeRegistryTableConverter.java:129)
	at io.cucumber.datatable.DataTable.asList(DataTable.java:181)
	at com.expd.test.datatabletests.DataTableTestSteps.showTransforms(DataTableTestSteps.java:43)
	at ✽.show DataTable transforms for(file:///E:/DevWorkspace/CucumberDataTableTransform/src/feature/DataTableTransforms.feature:13)
Caused by: io.cucumber.core.backend.CucumberInvocationTargetException
	at io.cucumber.java.Invoker.invoke(Invoker.java:34)
	at io.cucumber.java.JavaDefaultDataTableEntryTransformerDefinition.execute(JavaDefaultDataTableEntryTransformerDefinition.java:101)
	at io.cucumber.java.JavaDefaultDataTableEntryTransformerDefinition.lambda$new$0(JavaDefaultDataTableEntryTransformerDefinition.java:28)
	at io.cucumber.core.runner.CoreDefaultDataTableEntryTransformerDefinition$ConvertingTransformer.transform(CoreDefaultDataTableEntryTransformerDefinition.java:68)
	at io.cucumber.datatable.DataTableType.lambda$defaultEntry$1(DataTableType.java:136)
	at io.cucumber.datatable.DataTableType$TableEntryTransformerAdaptor.transform(DataTableType.java:258)
	at io.cucumber.datatable.DataTableType$TableEntryTransformerAdaptor.transform(DataTableType.java:240)
	at io.cucumber.datatable.DataTableType.transform(DataTableType.java:141)
	at io.cucumber.datatable.DataTableTypeRegistryTableConverter.toListOrNull(DataTableTypeRegistryTableConverter.java:163)
	at io.cucumber.datatable.DataTableTypeRegistryTableConverter.toList(DataTableTypeRegistryTableConverter.java:129)
	at io.cucumber.datatable.DataTable.asList(DataTable.java:181)
	at com.expd.test.datatabletests.DataTableTestSteps.showTransforms(DataTableTestSteps.java:43)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)

Expected behavior
The expectation is that a one-dimensional table with only one row and many columns transforms into a List of objects, and not into an empty List.

Additionally, the expectation is that asList() and asList(String.class) both produce the same output layout.

Context & Motivation

We have many step definitions defined which when simplified look like this:

@Given("^do something for$")
public void doSomething(List<String> table) {
    System.out.format("Input List<String>:%s%n", table);
}

When attempting to update the project to version 5.5.0, with only the simple NewTypeRegistryConfiguration class shown further above, all of these steps started encountering java.lang.IllegalArgumentException: Cannot deserialize instance of `java.lang.String` out of START_OBJECT token (and this is was a bit of a red herring).

Changing the step definitions to use a DataTable for the input parameter instead, and then transforming that table into our List showed where the actual difference was... namely the differing behavior of asList(Type itemType) between versions 4.4.0 and 5.5.0.

Since the output of asList() appeared to still behave as before, we attempted to transform the table into our List with the zero-parameter call, however quickly noticed that using a replaced String transformer did not have the desired result. Using the exact same steps as shown above, but adding this method to the step definition class:

    @DataTableType(replaceWithEmptyString = "[NONE]")
    public String listOfStringListsType(String cell) {
        return cell;
    }

Running the following scenario:

Scenario: Show datatable transforms with replacements of one-dimensional tables
  Given show DataTable transforms with replacements for
    | aaa |  | [NONE] | ddd |
  Given show DataTable transforms with replacements for
    | aaa    |
    |        |
    | [NONE] |
    | ddd    |

yielded this output... clearly showing that asList() does not use the transformer, whereas asList(String.class) does.

Input DataTable raw:
      | aaa |  | [NONE] | ddd |
DataTable.asLists(): [[aaa, null, [NONE], ddd]]
DataTable.asLists(String.class): [[aaa, null, , ddd]]
DataTable.asList(): [aaa, null, [NONE], ddd]
DataTable.asList(String.class): []

Input DataTable raw:
      | aaa    |
      |        |
      | [NONE] |
      | ddd    |
DataTable.asLists(): [[aaa], [null], [[NONE]], [ddd]]
DataTable.asLists(String.class): [[aaa], [null], [], [ddd]]
DataTable.asList(): [aaa, null, [NONE], ddd]
DataTable.asList(String.class): [aaa, null, , ddd]

Your Environment

• Versions used:
  • jackson-databind 2.9.8
  • cucumber libraries 4.4.0 and 5.5.0
  • junit 4.12

[mpkorstanje]

Thanks for the massive write up!

Given table1
  | aaa | bbb | ccc |

List<String> result1 = table1.asList(String.class);

As part of cucumber/common#540 this should have thrown an exception but but it looks like this case slipped through the cracks. An example of the exception would be:

io.cucumber.datatable.UndefinedDataTableTypeException: Can't convert DataTable to List<java.lang.String>.
There was a table cell converter but the table was too wide to use it.
Please reduce the table width or register a TableEntryTransformer or TableRowTransformer for java.lang.String.

The way to handle your current situation would be to transpose the table before transformation. You can do this using the @Transpose annotation or dataTable.transpose().asList(String.class).

Given table1
  | aaa | bbb | ccc |

@Given("table1")
public void table1(@Transpose List<String> dataTable) {
        
}

[BuzMack]

Nice... that suggestion along with the datatable update to 3.3.1 should solve my problem, thanks!