Configure what columns are selected from lookup [Zsolt]

[RoyMudie]

At the moment RDMP extracts all columns from lookup tables.
In particular hic_dataLoadRunID and hic_validFrom would not be extracted usually.
Please enable the user to select what columns are selected from a lookup.

https://jira.hic.dundee.ac.uk/browse/RDMPSUP-140

[tznind]

In particular hic_dataLoadRunID and hic_validFrom would not be extracted usually.

The ColumnDropper can be used to define system wide rules about not extracting given columns (i.e. dropping them from the dataset being extracted).

Please enable the user to select what columns are selected from a lookup.

A SupportingSQL object (shown in the picture) allows selecting any columns you want from an arbitrary table as a 'lookup table'. If this is crashing consistently with the screenshot message then we need to look into that. It looks very low level error.

[tznind]

A Lookup object itself is different and is used in the creation of code/description relationships.

Lookups are created as follows:

Is this about whether the lookups appear in the the word metadata report e.g. here we can see the full Lookup table is getting examined for the Word Metadata Report:

[tznind]

Ok so to confirm:

• You can extract whatever you want from a SupportingSql

For Lookup objects we can make the following changes

• Only write into metadata documents the code and description column(s) of the Lookup
• Only extract the code and description column(s) of the Lookup if it is included in an ExtractionConfiguration as a bundled lookup

Bear in mind that a Lookup can have multiple of each pk and desc column. For example where you have a test code AB1 that maps to 2 different tests - 1 in Tayside and 1 in Fife (so the join is TestCode+Healthboard => DrugDesc).

[tznind]

I've updated the inital description to remove references to SupportingSQL which is a red herring for this issue