Skip to content

#94 Added Testing Rules for the Lookup Resource #108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 37 commits into from
Jul 8, 2022
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
f388663
Updated odata-openapi commit
darnjo Dec 21, 2021
1b2ef56
Update odata-openapi submodule
darnjo Dec 21, 2021
ea9aa3e
Issue #94: Initial addition of Lookup resoruce tests
darnjo Dec 21, 2021
d5694de
Issue #94: added Cucumber BDD for Lookup Resource and stubs for gluecode
darnjo Dec 22, 2021
38a78ff
Merge branch 'main' into issue-94-add-testing-rules-for-lookup-resource
darnjo Dec 30, 2021
ed8a72e
Merge branch 'main' into issue-94-add-testing-rules-for-lookup-resource
darnjo Dec 31, 2021
a24de3b
Issue #94: Added Lookup Replication Tests
darnjo Jan 3, 2022
7a68230
Updated .gitignore
darnjo Jan 12, 2022
32d6c2c
Issue #94: Retrieve data using fetch api
darnjo Feb 13, 2022
8604a00
Issue #94: Added timestamp query replication strategy
darnjo Feb 13, 2022
7ced947
Issue #94: Added timestamp query replication strategy for asc
darnjo Feb 13, 2022
2e152c3
Issue #94: Add relevant annotation and metadata cache tests
darnjo Feb 14, 2022
e8a6850
Issue #94: Added required annotation and lookup name testing in the l…
darnjo Feb 14, 2022
0a58ece
Issue #94: WARN when empty LookupName items are found
darnjo Feb 14, 2022
114f66c
Issue #94: update to use TopAndSkip replication strategy instead
darnjo Feb 14, 2022
7152812
Issue #94: Changed fail to warn when lookup name not present. Changed…
darnjo Feb 15, 2022
669909c
Issue #94: separated the required annotation check from lookup name c…
darnjo Feb 16, 2022
d3b10fb
Issue #94: Lookup Resource Cleanup
darnjo Feb 23, 2022
37d840b
Issue #105: Add fast-fail behavior to all XML Metadata validation steps
darnjo Feb 23, 2022
d2607b7
Added better page handling and logging for sampling
darnjo Feb 25, 2022
a90bc83
Issue #94: move annotation reader to common utils
darnjo Feb 25, 2022
8662983
Issue #94: added JSON Serializer
darnjo Feb 26, 2022
db1b390
Issue #94: Lookups JSON serializer
darnjo Feb 26, 2022
4b233f8
Issue #94: Refactored utils into ODataUtils, separated functions
darnjo Feb 26, 2022
e159d34
Issue #94: Refactored ODataUtils to use Optional, and added comments
darnjo Feb 26, 2022
f2b435b
Issue #94: further conversion to optional
darnjo Feb 26, 2022
3c00eb9
Issue #94: created new json serializers
darnjo Apr 7, 2022
d68f9ec
#94: Intermediate commit
darnjo Apr 17, 2022
ced8292
#94: updated sampling strategy
darnjo Apr 17, 2022
b16864e
#107: Added support for OriginatingSystemName and OriginatingSystemID
darnjo Apr 17, 2022
f0b2a11
Updated Docker instructions in the README
darnjo Apr 17, 2022
2ace0d1
#106: updated docker README
darnjo Apr 17, 2022
370c425
Merge branch 'issue-94-add-testing-rules-for-lookup-resource' of http…
darnjo Apr 17, 2022
d7cedb5
Issue #94: annotated lookup field serializer
darnjo May 23, 2022
992f9bd
Issue #94: adding parallel stream logic
darnjo May 23, 2022
695acd2
Merge branch 'main' into issue-94-add-testing-rules-for-lookup-resource
darnjo Jul 8, 2022
365350f
#94: merge main
darnjo Jul 8, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -6,3 +6,4 @@ build/
.idea/
*.log
*.iml
.run/
20 changes: 0 additions & 20 deletions .run/Commander Acceptance Tests.run.xml

This file was deleted.

21 changes: 0 additions & 21 deletions .run/Feature_ DataDictionary 1.7 - Local Metadata.run.xml

This file was deleted.

61 changes: 32 additions & 29 deletions doc/Docker.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,39 @@
# RESO Commander and Docker
Both the command-line and automated testing tools can be run in a Docker container.
RESO automated testing tools and Commander utilities can both be run in a Docker containers.
The containers are slightly different in each case.

A [Dockerfile](./Dockerfile) has been provided to dockerize the application.
This can be used for CI/CD environments such as Jenkins or TravisCI. The following command will build an image for you:
### RESO Automated Testing Tools
A [GradleDockerfile](../GradleDockerfile) has been provided in order to prepare a Gradle
environment for the Commander. The container builds itself from the main branch of the source code, so you don't need
the entire repo checked out locally, just the file.

This can also be used in CI/CD environments such as Jenkins or TravisCI.

Run the RESO Certification tests in a Docker container locally by issuing one of the following commands.
Docker must be running on your local machine.

One way to do this is to build the container first and then run it:

```docker build --file GradleDockerfile -t web-api-commander-gradle .```

Once the container is built, you can use the Gradle commands normally with:
```docker run -it web-api-commander-gradle testWebApiCore_2_0_0 -DpathToRESOScript=/home/gradle/project/resoscripts/your.resoscript -DshowResponses=true```

You can also build the container on the fly:

```docker run --rm -it -v "$PWD":/home/gradle/project -v /path/to/your/resoscripts:/home/gradle/project/resoscripts -w /home/gradle/project -it $(docker build -f GradleDockerfile -q .) testWebApiCore_2_0_0 -DpathToRESOScript=/home/gradle/project/resoscripts/your.resoscript -DshowResponses=true```

Note that this will create a directory in your home directory for the project, and build artifacts and the log will be placed in that directory,
which is also where you will end up after runtime.

You may need to adjust the path separators if using Windows.


### Commander Utilities
A [Dockerfile](../Dockerfile) has also been provided to Dockerize the application for Commander utilities.

To run the Commander utilities, use the following commands:

### Commander Features Other Than Automated Web API Testing
```
$ docker build -t web-api-commander .
```
Expand All @@ -21,28 +49,3 @@ If you have input files you may need to mount your filesystem into the docker co
```
$ docker run -it -v $PWD:/app web-api-commander --validateMetadata --inputFile <pathInContainer>
```

### Automated Web API Testing

You may also run the tests in a Docker container locally by issuing one of the following commands.
Docker must be running on your local machine.

#### MacOS or Linux All-In-One Commands
```
cd ~; \
rm -rf commander-tmp/; \
mkdir commander-tmp; \
cd commander-tmp; \
git clone https://github.com/RESOStandards/web-api-commander.git; \
cd web-api-commander; \
docker run --rm -u gradle -v "$PWD":/home/gradle/project -v /path/to/your/resoscripts:/home/gradle/project/resoscripts -w /home/gradle/project gradle gradle testWebAPIServer_2_0_0_Core -DpathToRESOScript=/home/gradle/project/resoscripts/your.web-api-server.core.2.0.0.resoscript -DshowResponses=true
```

Note that this will create a directory in your home directory for the project, and build artifacts and the log will be placed in that directory,
which is also where you will end up after runtime.


#### Windows All-In-One WIP
```
cd C:\;mkdir commander-tmp;cd commander-tmp;git clone https://github.com/RESOStandards/web-api-commander.git;cd web-api-commander; docker run --rm -u gradle -v C:\current\path\web-api-commander:/home/gradle/project -v C:\path\to\your\resoscripts:/home/gradle/project/resoscripts -w /home/gradle/project gradle gradle testWebAPIServer_2_0_0_Core -DpathToRESOScript=/home/gradle/project/resoscripts/your.web-api-server.core.2.0.0.resoscript -DshowResponses=true
```
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="utf-8" ?>

<!--
>>> To be used as a template for Web API Server Core 1.0.2 Certification <<<
>>> To be used as a template for Web API Server Core 2.0.0 Certification <<<

NOTES:
* Anything marked REQUIRED should be filled in.
Expand Down
10 changes: 5 additions & 5 deletions src/main/java/org/reso/certification/codegen/DDLProcessor.java
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ private String buildInsertLookupsStatement() {
standardFieldMap.forEach((standardName, referenceStandardField) -> {
String inserts = buildLookupValueInserts(referenceStandardField);
if (inserts.length() > 0) {
markupMap.putIfAbsent(referenceStandardField.getLookupStandardName(),
markupMap.putIfAbsent(referenceStandardField.getLookupName(),
(markupMap.keySet().size() > 0 ? ", " : EMPTY_STRING) + PADDING + inserts);
}
});
Expand All @@ -219,16 +219,16 @@ private String buildInsertLookupsStatement() {
private String buildLookupValueInserts(ReferenceStandardField standardField) {
StringBuilder content = new StringBuilder();

if (getEnumerations().get(standardField.getLookupStandardName()) != null) {
if (getEnumerations().get(standardField.getLookupName()) != null) {
AtomicReference<String> fieldHash = new AtomicReference<>();

//iterate through each of the lookup values and generate their edm:EnumType content
getEnumerations().get(standardField.getLookupStandardName()).forEach(lookup -> {
getEnumerations().get(standardField.getLookupName()).forEach(lookup -> {

// key is the sha256 of the following values
fieldHash.set(sha256()
.hashString(
standardField.getLookupStandardName()
standardField.getLookupName()
+ lookup.getLookupDisplayName()
+ lookup.getLookupValue(), StandardCharsets.UTF_8)
.toString());
Expand All @@ -237,7 +237,7 @@ private String buildLookupValueInserts(ReferenceStandardField standardField) {
.append(content.length() > 0 ? ", " : EMPTY_STRING).append("\n")
.append(PADDING).append("(")
.append("\"").append(fieldHash.get()).append("\"")
.append(", ").append("\"").append(standardField.getLookupStandardName()).append("\"")
.append(", ").append("\"").append(standardField.getLookupName()).append("\"")
.append(", ").append("\"").append(lookup.getLookupDisplayName()).append("\"")
.append(", ").append("\"").append(lookup.getLookupDisplayName()).append("\"")
.append(", ").append("\"").append(lookup.getLookupValue()).append("\"")
Expand Down
28 changes: 14 additions & 14 deletions src/main/java/org/reso/certification/codegen/EDMXProcessor.java
Original file line number Diff line number Diff line change
Expand Up @@ -198,11 +198,11 @@ private String buildEnumTypeMarkup() {
standardFieldsMap.forEach((resourceName, standardFieldMap) -> {
standardFieldMap.forEach((standardName, referenceStandardField) -> {
if (referenceStandardField.isSingleEnumeration()) {
markupMap.putIfAbsent(referenceStandardField.getLookupStandardName(), buildSingleEnumTypeMarkup(referenceStandardField));
markupMap.putIfAbsent(referenceStandardField.getLookupName(), buildSingleEnumTypeMarkup(referenceStandardField));
}

if (referenceStandardField.isMultipleEnumeration()) {
markupMap.putIfAbsent(referenceStandardField.getLookupStandardName(), buildMultipleEnumTypeMarkup(referenceStandardField));
markupMap.putIfAbsent(referenceStandardField.getLookupName(), buildMultipleEnumTypeMarkup(referenceStandardField));
}
});
});
Expand Down Expand Up @@ -241,11 +241,11 @@ private String buildNavigationPropertyMarkup(String resourceName) {
private String buildSingleEnumTypeMarkup(ReferenceStandardField standardField) {
StringBuilder content = new StringBuilder();

if (getEnumerations().get(standardField.getLookupStandardName()) != null) {
content.append("<EnumType Name=\"").append(standardField.getLookupStandardName()).append("\">");
if (getEnumerations().get(standardField.getLookupName()) != null) {
content.append("<EnumType Name=\"").append(standardField.getLookupName()).append("\">");

//iterate through each of the lookup values and generate their edm:EnumType content
getEnumerations().get(standardField.getLookupStandardName()).forEach(lookup -> {
getEnumerations().get(standardField.getLookupName()).forEach(lookup -> {
content
.append("<Member Name=\"").append(lookup.getLookupValue()).append("\">")
.append(EDMXTemplates.buildDisplayNameAnnotation(lookup.getLookupDisplayName()))
Expand All @@ -258,9 +258,9 @@ private String buildSingleEnumTypeMarkup(ReferenceStandardField standardField) {
} else {
content
.append("<!-- TODO: implement if you are using the single-valued enumeration \"")
.append(standardField.getLookupStandardName()).append("\" -->")
.append("<EnumType Name=\"").append(standardField.getLookupStandardName()).append("\">")
.append("<Member Name=\"Sample").append(standardField.getLookupStandardName()).append("EnumValue").append("\"/>")
.append(standardField.getLookupName()).append("\" -->")
.append("<EnumType Name=\"").append(standardField.getLookupName()).append("\">")
.append("<Member Name=\"Sample").append(standardField.getLookupName()).append("EnumValue").append("\"/>")
.append("</EnumType>");
}
return content.toString();
Expand All @@ -269,11 +269,11 @@ private String buildSingleEnumTypeMarkup(ReferenceStandardField standardField) {
private String buildMultipleEnumTypeMarkup(ReferenceStandardField standardField) {
StringBuilder content = new StringBuilder();

if (getEnumerations().get(standardField.getLookupStandardName()) != null) {
content.append("<EnumType Name=\"").append(standardField.getLookupStandardName()).append("\">");
if (getEnumerations().get(standardField.getLookupName()) != null) {
content.append("<EnumType Name=\"").append(standardField.getLookupName()).append("\">");

//iterate through each of the lookup values and generate their edm:EnumType content
getEnumerations().get(standardField.getLookupStandardName()).forEach(lookup -> {
getEnumerations().get(standardField.getLookupName()).forEach(lookup -> {
content
.append("<Member Name=\"").append(lookup.getLookupValue()).append("\">")
.append(EDMXTemplates.buildDisplayNameAnnotation(lookup.getLookupDisplayName()))
Expand All @@ -285,8 +285,8 @@ private String buildMultipleEnumTypeMarkup(ReferenceStandardField standardField)
content.append("</EnumType>");
} else {
content
.append("<!-- TODO: implement if you are using the multi-valued enumeration \"").append(standardField.getLookupStandardName()).append("\" -->")
.append("<EnumType Name=\"").append(standardField.getLookupStandardName()).append("\">")
.append("<!-- TODO: implement if you are using the multi-valued enumeration \"").append(standardField.getLookupName()).append("\" -->")
.append("<EnumType Name=\"").append(standardField.getLookupName()).append("\">")
.append(EDMXTemplates.buildDDWikiUrlAnnotation(standardField.getWikiPageUrl()))
.append(EDMXTemplates.buildDescriptionAnnotation(standardField.getDefinition()))
.append("<Member Name=\"Sample").append(standardField.getStandardName()).append("EnumValue").append("\"/>")
Expand Down Expand Up @@ -394,7 +394,7 @@ public static String buildEnumTypeMultiMember(ReferenceStandardField field) {
if (!field.getLookup().toLowerCase().contains("lookups")) return EMPTY_STRING;
return ""
+ "<Property Name=\"" + field.getStandardName()
+ "\" Type=\"Collection(" + RESO_NAMESPACE + ".enums." + field.getLookupStandardName() + ")\">"
+ "\" Type=\"Collection(" + RESO_NAMESPACE + ".enums." + field.getLookupName() + ")\">"
+ buildDisplayNameAnnotation(field.getDisplayName())
+ buildDDWikiUrlAnnotation(field.getWikiPageUrl())
+ buildDescriptionAnnotation(field.getDefinition())
Expand Down
Loading