Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
89 commits
Select commit Hold shift + click to select a range
3971c31
Add a feature to choose type of taxon tree in WB
CarolineDenis Jul 10, 2024
7ae8d32
TODO
CarolineDenis Jul 10, 2024
c8afd1d
List of available taxon trees to choose from when opening WB
CarolineDenis Jul 11, 2024
8fda5d9
comments
CarolineDenis Jul 11, 2024
69cf926
Remove comment
CarolineDenis Jul 11, 2024
01c8bd4
Don't filter fieldsData if not first call
CarolineDenis Jul 12, 2024
53f0add
Pass taxonTreeId with upload plan
CarolineDenis Jul 12, 2024
3282d1d
Add taxonId to uplaodPlan
CarolineDenis Jul 12, 2024
659304f
Use treeDefId to define which tree to look at in mapper
CarolineDenis Jul 12, 2024
c6619ca
Merge remote-tracking branch 'origin/issue-4829' into issue-4980
CarolineDenis Jul 16, 2024
eeac233
Remove comment
CarolineDenis Jul 16, 2024
dd675ba
Add treeId to upload plan
CarolineDenis Jul 16, 2024
670fc87
Fix filtering when taxonType is undefined, needed in QB
CarolineDenis Jul 16, 2024
64a956f
Merge branch 'issue-4829' into issue-4980
melton-jason Jul 16, 2024
fb9c289
Merge branch 'issue-4829' into issue-4980
acwhite211 Jul 31, 2024
3fc8ac1
Merge remote-tracking branch 'origin/production' into issue-4980
CarolineDenis Aug 2, 2024
e7e83ae
Start work on adding new column in WB for multiple trees
CarolineDenis Aug 2, 2024
f8dec7f
add handleTreeSelection
CarolineDenis Aug 5, 2024
22ccb87
Chnage variable name (duplicate)
CarolineDenis Aug 5, 2024
50ff95e
Merge branch 'production' into issue-4980
grantfitzsimmons Aug 6, 2024
d313c98
Merge branch 'production' into issue-4980
grantfitzsimmons Aug 19, 2024
fb477d0
init wb uploads handling taxon ranks with treedefid
acwhite211 Aug 20, 2024
4b12da5
simplify wb mutli-tree solution
acwhite211 Aug 20, 2024
57f9a09
fix taxon treedef scoping logic
acwhite211 Aug 20, 2024
c007b29
notes
acwhite211 Aug 20, 2024
9094d6b
fix find_treedef
acwhite211 Aug 20, 2024
7c052d2
fix mypy error
acwhite211 Aug 20, 2024
d786adf
Merge branch 'production' into issue-4980
grantfitzsimmons Aug 21, 2024
6abe879
allow for non-background wb uploads
acwhite211 Aug 21, 2024
65c364d
change TreeRecord ranks key type to include treedefid
acwhite211 Aug 22, 2024
7552b87
clearnup and fix unparsing unit test
acwhite211 Aug 22, 2024
4eb6730
reformulate and change taxonTreeId to treeId in schema
acwhite211 Aug 23, 2024
cc62e3f
remove adjust_upload_plan
acwhite211 Aug 23, 2024
e78e950
avoid schema adjustment for now
acwhite211 Aug 23, 2024
1ab212c
fix _get_treedef_id
acwhite211 Aug 26, 2024
03b6784
set treeId dynamically in unit tests
acwhite211 Aug 26, 2024
6dcf44c
Merge remote-tracking branch 'origin/production' into issue-4980
melton-jason Aug 28, 2024
94154e3
Implement choosing a specific treeDef in mappers
melton-jason Aug 28, 2024
4f657ef
Disable specific tree interface for all components save WB
melton-jason Aug 28, 2024
5ad3358
Pass tree definition id in API request
sharadsw Aug 28, 2024
004dda4
Fix tests
sharadsw Aug 28, 2024
47c9fdf
Lint code with ESLint and Prettier
sharadsw Aug 28, 2024
790baca
TreeRank NamedTuple
acwhite211 Aug 29, 2024
ca89ba2
Parse treeId in upload plan when creating lines
sharadsw Aug 30, 2024
d4af2d6
Fix typecheck
sharadsw Aug 30, 2024
9c0b163
Fix treeId parsing for collections with 1 tree
sharadsw Aug 30, 2024
4a4d58a
Lint code with ESLint and Prettier
sharadsw Aug 30, 2024
5ef3bfa
add rescope_tree_from_row in wb tree binding
acwhite211 Sep 3, 2024
e33c9cf
fix determinations naming bug
acwhite211 Sep 3, 2024
93810b0
fix unit tests
acwhite211 Sep 3, 2024
99afa31
better handle noRanksInRow
acwhite211 Sep 3, 2024
c4ac5a8
order by rankid
acwhite211 Sep 3, 2024
6103491
remove unneeded conditional in handle_no_ranks_in_row
acwhite211 Sep 3, 2024
508cbef
new rank key request formatting
acwhite211 Sep 4, 2024
8942c87
fix type error
acwhite211 Sep 4, 2024
79bbbcc
Modify upload plan request to avoid duplicate rank keys
sharadsw Sep 4, 2024
eb503a4
fix upload plan test
sharadsw Sep 4, 2024
655a793
reformulate rescope_tree_from_row
acwhite211 Sep 5, 2024
9babfb3
Change tree headers when rank name is ambiguous
sharadsw Sep 5, 2024
ca338dc
fix typecheck
sharadsw Sep 5, 2024
39b45ce
Lint code with ESLint and Prettier
sharadsw Sep 5, 2024
e215a77
fix RankColumn typecheck
acwhite211 Sep 5, 2024
3595c2b
TreeRankCell
acwhite211 Sep 5, 2024
270d06f
Fix mapping header being renamed in collections with only 1 tree
sharadsw Sep 5, 2024
833adb5
Fix upload plan when there are multiple ranks of the same tree in the…
sharadsw Sep 5, 2024
dd7e3f8
Merge remote-tracking branch 'origin/issue-4980' into issue-4980
sharadsw Sep 5, 2024
bc3e9f6
Lint code with ESLint and Prettier
sharadsw Sep 5, 2024
16671ba
update treerecord _to_match function and remove rescope_tree_from_row…
acwhite211 Sep 5, 2024
1a368d9
remove fullname hack
acwhite211 Sep 5, 2024
c84860f
Use tree name in header always
sharadsw Sep 6, 2024
67f874e
add back WorkBenchParseFailure for multipleRanksInRow and noRoot
acwhite211 Sep 6, 2024
3b75578
functional refactoring
acwhite211 Sep 6, 2024
2f1cfcc
remove unneeded exception
acwhite211 Sep 6, 2024
acb17a4
Merge branch 'issue-4980-func' into issue-4980
acwhite211 Sep 6, 2024
0432501
fix typecheck in WorkBenchParseFailure call
acwhite211 Sep 6, 2024
8a2ba88
formatting
acwhite211 Sep 6, 2024
fba9587
Merge branch 'issue-4980' of https://github.com/specify/specify7 into…
acwhite211 Sep 9, 2024
d2f7531
edit _filter_target_rank_columns and extract_treedef_id
acwhite211 Sep 9, 2024
e38d54c
remove base_treedef_id
acwhite211 Sep 9, 2024
f3e052a
scoped_ranks rewrite
acwhite211 Sep 10, 2024
7c313d3
misc
acwhite211 Sep 10, 2024
e56e82c
Move rank key functions to uploadPlanBuilder and uploadPlanParser
sharadsw Sep 10, 2024
10324c9
Lint code with ESLint and Prettier
sharadsw Sep 10, 2024
e8a9ed4
Add tree name in front of rank
CarolineDenis Sep 11, 2024
e00add2
catch multipleRanksForTreedef WorkBenchParseFailure
acwhite211 Sep 11, 2024
20bfd8a
edit accepted formatting to treedef_id~>rank_name
acwhite211 Sep 11, 2024
590579e
change extract_treedef_id to extract_treedef_name
acwhite211 Sep 11, 2024
c3f4862
avoid unit test failures, allow multiple ranks when only one treedef …
acwhite211 Sep 11, 2024
eb3c820
fix WorkBenchParseFailure column name
acwhite211 Sep 11, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -24,5 +24,6 @@ test('domain data is fetched and parsed correctly', async () =>
],
paleoContextChildTable: 'collectionobject',
referenceSymbol: '#',
treeSymbol: '$',
treeRankSymbol: '$',
treeDefinitionSymbol: '%',
}));
7 changes: 5 additions & 2 deletions specifyweb/frontend/js_src/lib/components/DataModel/schema.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,8 @@ type Schema = {
'Institution'
];
readonly referenceSymbol: string;
readonly treeSymbol: string;
readonly treeDefinitionSymbol: string;
readonly treeRankSymbol: string;
readonly fieldPartSeparator: string;
};

Expand Down Expand Up @@ -60,8 +61,10 @@ const schemaBase: Writable<Schema> = {

// Prefix for -to-many indexes
referenceSymbol: '#',
// Prefix for Tree Definitions
treeDefinitionSymbol: '%',
// Prefix for tree ranks
treeSymbol: '$',
treeRankSymbol: '$',
// Separator for partial fields (date parts in Query Builder)
fieldPartSeparator: '-',
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,14 @@ export function getTreeDefinitions<TREE_NAME extends AnyTree['tableName']>(
tableName
);

return typeof treeDefinitionId === 'number'
? specificTreeDefinitions.filter(
({ definition }) => definition.id === treeDefinitionId
)
: specificTreeDefinitions;
if (typeof treeDefinitionId === 'number') {
const resolvedDefinition = specificTreeDefinitions.find(
({ definition }) => definition.id === treeDefinitionId
);
return resolvedDefinition === undefined
? specificTreeDefinitions
: [resolvedDefinition];
} else return specificTreeDefinitions;
}

export function getTreeDefinitionItems<TREE_NAME extends AnyTree['tableName']>(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -94,8 +94,8 @@ test('Editing Data Object Formatter', () => {
</switch>
</format>
<aggregators>
<aggregator name='AccessionAgent' title='AccessionAgent' class='edu.ku.brc.specify.datamodel.Agent' default='true' separator='' ending='' format='AccessionAgent'/>
<aggregator name='AccessionAgent' title='AccessionAgent' class='edu.ku.brc.specify.datamodel.AccessionAgent' default='true' separator='' ending='' format='AccessionAgent'/>
<aggregator name='AccessionAgent' title='AccessionAgent' class='edu.ku.brc.specify.datamodel.Agent' default='true' separator='; ' ending='' format='AccessionAgent'/>
<aggregator name='AccessionAgent' title='AccessionAgent' class='edu.ku.brc.specify.datamodel.AccessionAgent' default='true' separator='; ' ending='' format='AccessionAgent'/>
</aggregators>
</formatters>"
`);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,6 @@ export type HtmlGeneratorFieldData = {
readonly isDefault?: boolean;
readonly isRelationship?: boolean;
readonly tableName?: keyof Tables;
readonly tableTreeDefName?: string;
};

type MappingLineBaseProps = {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,16 +16,16 @@ theories(valueIsToManyIndex, [
[[`${schema.referenceSymbol}2`], true],
[[`${schema.referenceSymbol}999`], true],
[['collectionobject'], false],
[[`${schema.treeSymbol}Kingdom`], false],
[[`${schema.treeRankSymbol}Kingdom`], false],
]);

theories(valueIsTreeRank, [
[[`${schema.referenceSymbol}1`], false],
[[`${schema.referenceSymbol}2`], false],
[[`${schema.referenceSymbol}999`], false],
[['collectionobject'], false],
[[`${schema.treeSymbol}Kingdom`], true],
[[`${schema.treeSymbol}County`], true],
[[`${schema.treeRankSymbol}Kingdom`], true],
[[`${schema.treeRankSymbol}County`], true],
]);

theories(getNumberFromToManyIndex, [
Expand All @@ -36,8 +36,8 @@ theories(getNumberFromToManyIndex, [
]);

theories(getNameFromTreeRankName, [
[[`${schema.treeSymbol}Kingdom`], 'Kingdom'],
[[`${schema.treeSymbol}County`], 'County'],
[[`${schema.treeRankSymbol}Kingdom`], 'Kingdom'],
[[`${schema.treeRankSymbol}County`], 'County'],
]);

theories(findDuplicateMappings, [
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -314,70 +314,60 @@ theories(getMappingLineData, [
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Phylum: {
optionLabel: 'Phylum',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Class: {
optionLabel: 'Class',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Order: {
optionLabel: 'Order',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Family: {
optionLabel: 'Family',
isRelationship: true,
isDefault: true,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subfamily: {
optionLabel: 'Subfamily',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Genus: {
optionLabel: 'Genus',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subgenus: {
optionLabel: 'Subgenus',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Species: {
optionLabel: 'Species',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subspecies: {
optionLabel: 'Subspecies',
isRelationship: true,
isDefault: false,
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
},
tableName: 'Taxon',
Expand Down Expand Up @@ -870,70 +860,60 @@ theories(getMappingLineData, [
isRelationship: true,
optionLabel: 'Class',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Family: {
isDefault: true,
isRelationship: true,
optionLabel: 'Family',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Genus: {
isDefault: false,
isRelationship: true,
optionLabel: 'Genus',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Kingdom: {
isDefault: false,
isRelationship: true,
optionLabel: 'Kingdom',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Order: {
isDefault: false,
isRelationship: true,
optionLabel: 'Order',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Phylum: {
isDefault: false,
isRelationship: true,
optionLabel: 'Phylum',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Species: {
isDefault: false,
isRelationship: true,
optionLabel: 'Species',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subfamily: {
isDefault: false,
isRelationship: true,
optionLabel: 'Subfamily',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subgenus: {
isDefault: false,
isRelationship: true,
optionLabel: 'Subgenus',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
$Subspecies: {
isDefault: false,
isRelationship: true,
optionLabel: 'Subspecies',
tableName: 'Taxon',
tableTreeDefName: 'Taxonomy',
},
},
selectLabel: localized('Taxon'),
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,20 @@ export const valueIsToManyIndex = (value: string | undefined): boolean =>
value?.slice(0, schema.referenceSymbol.length) === schema.referenceSymbol ||
false;

/**
* Returns whether a value is any special tree-related meta information
* (e.g. tree defintiion, tree rank)
*/
export const valueIsTreeMeta = (value: string | undefined): boolean =>
valueIsTreeDefinition(value) || valueIsTreeRank(value);

/** Returns whether a value is a tree definition name (e.x %Taxonomy) */
export const valueIsTreeDefinition = (value: string | undefined): boolean =>
value?.startsWith(schema.treeDefinitionSymbol) ?? false;

/** Returns whether a value is a tree rank name (e.x $Kingdom, $Order) */
export const valueIsTreeRank = (value: string | undefined): boolean =>
value?.startsWith(schema.treeSymbol) || false;
value?.startsWith(schema.treeRankSymbol) ?? false;

/**
* Returns index from a formatted -to-many index value (e.x #1 => 1)
Expand All @@ -40,6 +51,14 @@ export const valueIsTreeRank = (value: string | undefined): boolean =>
export const getNumberFromToManyIndex = (value: string): number =>
Number(value.slice(schema.referenceSymbol.length));

/**
* Returns tree definition name from a complete tree definition name
* (e.x %Taxonomy => Taxonomy)
* Opposite of formatTreeDefinition
*/
export const getNameFromTreeDefinitionName = (value: string): string =>
value.slice(schema.treeDefinitionSymbol.length);

/*
* BUG: in places where output of this function is displayed to the user,
* make sure to use tree rank title instead of name
Expand All @@ -51,7 +70,7 @@ export const getNumberFromToManyIndex = (value: string): number =>
*
*/
export const getNameFromTreeRankName = (value: string): string =>
value.slice(schema.treeSymbol.length);
value.slice(schema.treeRankSymbol.length);

/**
* Returns a formatted -to-many index from an index (e.x 1 => #1)
Expand All @@ -61,7 +80,7 @@ export const formatToManyIndex = (index: number): string =>
`${schema.referenceSymbol}${index}`;

// Meta fields
export const anyTreeRank = `${schema.fieldPartSeparator}any`;
export const anyTreeRank = `${schema.fieldPartSeparator}any` as const;
export const formattedEntry = `${schema.fieldPartSeparator}formatted`;

/**
Expand All @@ -75,14 +94,22 @@ export const formattedEntry = `${schema.fieldPartSeparator}formatted`;
*/
export const emptyMapping = '0';

/**
* Returns a complete tree definition name from a tree definition name
* (e.x Taxononomy => %Taxonomy)
* Opposite of getNameFromTreeDefinitionName
*/
export const formatTreeDefinition = (definitionName: string): string =>
`${schema.treeDefinitionSymbol}${definitionName}`;

/**
* Returns a complete tree rank name from a tree rank name
* (e.x Kingdom => $Kingdom)
* Opposite of getNameFromTreeRankName
*
*/
export const formatTreeRank = (rankName: string): string =>
`${schema.treeSymbol}${rankName}`;
`${schema.treeRankSymbol}${rankName}`;

// Match fields names like startDate_fullDate, but not _formatted
export const valueIsPartialField = (value: string): boolean =>
Expand Down Expand Up @@ -160,7 +187,7 @@ export const getGenericMappingPath = (mappingPath: MappingPath): MappingPath =>
mappingPath.filter(
(mappingPathPart) =>
!valueIsToManyIndex(mappingPathPart) &&
!valueIsTreeRank(mappingPathPart) &&
!valueIsTreeMeta(mappingPathPart) &&
mappingPathPart !== formattedEntry &&
mappingPathPart !== emptyMapping
);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ import {
parsePartialField,
valueIsPartialField,
valueIsToManyIndex,
valueIsTreeDefinition,
valueIsTreeRank,
} from './mappingHelpers';
import { getMappingLineData } from './navigator';
Expand Down Expand Up @@ -120,16 +121,19 @@ export function generateMappingPathPreview(
: 1;
const toManyIndexFormatted = toManyIndexNumber > 1 ? toManyIndex : undefined;

const [databaseFieldName, databaseTableOrRankName, databaseParentTableName] =
mappingPathSubset([baseTableName, ...mappingPath]);
const [
databaseFieldName,
databaseTableOrRankName,
databaseParentTableOrTreeName,
] = mappingPathSubset([baseTableName, ...mappingPath]);

// Attributes parts of filedLables to each variable or creates one if empty
const [
fieldName = camelToHuman(databaseFieldName),
tableOrRankName = camelToHuman(
getNameFromTreeRankName(databaseTableOrRankName)
),
parentTableName = camelToHuman(databaseParentTableName),
parentTableOrTreeName = camelToHuman(databaseParentTableOrTreeName),
] = mappingPathSubset(fieldLabels);

const isAnyRank = databaseTableOrRankName === formatTreeRank(anyTreeRank);
Expand Down Expand Up @@ -163,16 +167,20 @@ export function generateMappingPathPreview(
const tableNameFormatted =
tablesToHide.has(databaseTableOrRankName) &&
databaseFieldName !== formattedEntry
? [parentTableName || tableNameNonEmpty]
? [parentTableOrTreeName || tableNameNonEmpty]
: genericTables.has(databaseTableOrRankName)
? [parentTableName, tableNameNonEmpty]
? [parentTableOrTreeName, tableNameNonEmpty]
: [tableNameNonEmpty];

return filterArray([
...(valueIsTreeRank(databaseTableOrRankName)
? [isAnyRank ? parentTableName : tableOrRankName]
? [isAnyRank ? parentTableOrTreeName : tableOrRankName]
: tableNameFormatted),
fieldNameFormatted,
...(valueIsTreeRank(databaseTableOrRankName) &&
valueIsTreeDefinition(databaseParentTableOrTreeName)
? [parentTableOrTreeName]
: []),
toManyIndexFormatted,
])
.filter(Boolean)
Expand Down
Loading