Update to version v1.0.12

Update node and Python Libs

Author: tabdunabi

CHANGELOG.md

@@ -5,6 +5,12 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.0.12] - 2023-06-07
+
+### Changed
+
+- Node and python library updates
+
 ## [1.0.11] - 2023-04-18
 
 ### Changed

README.md

@@ -1,6 +1,6 @@
 # Document Understanding Solution
 
-DUS leverages the power of Amazon Textract, Amazon Comprehend , Amazon Comprehend Medical Amazon Elasticsearch and Amazon Kendra to provide digitization , domain-specific data discovery, redaction controls , structural component extraction and other document processing & understanding capaibilities.
+DUS leverages the power of Amazon Textract, Amazon Comprehend , Amazon Comprehend Medical Amazon OpenSearch Service and Amazon Kendra to provide digitization , domain-specific data discovery, redaction controls , structural component extraction and other document processing & understanding capabilities.
 
 ![img](./images/DUS_with_kendra.png)
 
@@ -84,7 +84,7 @@ The solution is set up to reserve lambda concurrency quota. This is both to limi
 ## Development Deploy
 
 The instructions below cover installation on Unix-based Operating systems like macOS and Linux.
-You can use a AWS Cloud9 enviornment or EC2 instance (recommended: t3.large or higher on Amazon Linux platform) to deploy the solution
+You can use a AWS Cloud9 environment or EC2 instance (recommended: t3.large or higher on Amazon Linux platform) to deploy the solution
 
 ### Requirements
 
@@ -160,7 +160,7 @@ The solution is set up to reserve lambda concurrency quota. This is both to limi
 - `yarn deploy:show` : displays the url of the client app
 - `yarn destroy` : tears down the CloudFormation backend and client stacks
 
-### Development Deploy Workflow and stacknaming
+### Development Deploy Workflow and stack naming
 
 The `package.json` script node `stackname` sets the stackname for the deploy commands. Throughout development it has been imperative to maintain multiple stacks in order to allow client app development and stack architecture development to work without creating breaking changes. When a new stackname is merged into develop it should have the most up to date deployments.
 
@@ -197,11 +197,11 @@ Run `yarn license-report` to generate a license report for all npm packages. See
 
 ### Classic Mode
 
-This is first release of the DUS solution. The major services included in this mode include Amazon Elasticsearch, Amazon Textract, Amazon Comprehend and Amazon Comprehend Medical that allow digitization, information extraction and indexing in DUS.
+This is first release of the DUS solution. The major services included in this mode include Amazon OpenSearch Service, Amazon Textract, Amazon Comprehend and Amazon Comprehend Medical that allow digitization, information extraction and indexing in DUS.
 
 ### Kendra-Enabled Mode
 
-In the Classic version, DUS supports searching/indexing of documents using Amazon Elasticsearch
+In the Classic version, DUS supports searching/indexing of documents using Amazon OpenSearch Service
 In the kendra enabled mode, Amazon Kendra is added as an additional capability and can be used for exploring features such as Semantic Search, Adding FAQs and Access Control Lists.
 Simply set the ` enableKendra: "true"` in package.json
 _Note:_ Amazon Kendra Developer edition is deployed as a part of this deployment.
@@ -226,13 +226,13 @@ In Kendra mode, you can also upload the corresponding access control list under
 
 ## Cost
 
-- As you deploy this sample application, it creates different resources (Amazon S3 bucket, Amazon SQS Queue, Amazon DynamoDB table, Elasticsearch (and potenitally Amazon Kendra) clsuter(s) and AWS Lambda functions etc.). When you analyze documents, it calls different APIs (Amazon Textract , Amazon Comprehend and Amazon Comprehend Medical) in your AWS account. You will get charged for all the API calls made as part of the analysis as well as any AWS resources created as part of the deployment. To avoid any recurring charges, delete stack using "yarn destroy".
+- As you deploy this sample application, it creates different resources (Amazon S3 bucket, Amazon SQS Queue, Amazon DynamoDB table, OpenSearch Service (and potentially Amazon Kendra) cluster(s) and AWS Lambda functions etc.). When you analyze documents, it calls different APIs (Amazon Textract , Amazon Comprehend and Amazon Comprehend Medical) in your AWS account. You will get charged for all the API calls made as part of the analysis as well as any AWS resources created as part of the deployment. To avoid any recurring charges, delete stack using "yarn destroy".
 
 - The CDK Toolkit stacks that are created during deploy of this solution are not destroyed when you tear down the solution stacks. If you want to remove these resources, delete the S3 bucket that contains `staging-bucket` in the name, and then delete the `CDKToolkit` stack.
 
 - You are responsible for the cost of the AWS services used while running this reference
   deployment. The solution consists of some resources that have to be paid by the hour/size
-  such as Amazon Elasticsearch, Amazon Kendra and Amazon S3 while others are serverless technologies where
+  such as Amazon OpenSearch Service, Amazon Kendra and Amazon S3 while others are serverless technologies where
   costs are incurred depending on the number of requests.
   The approximate cost for the solution for 100 documents/day comes under $20/day for the Classic Mode and under $80/day for Kendra-Enabled Mode. For accurate and most up-to-date pricing information, refer [AWS Pricing](https://aws.amazon.com/pricing/)
 
@@ -258,4 +258,4 @@ The intended use is for users to use this application as a reference architectur
 
 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.
 
-The searchable PDF functionality is inluded as a pre-compiled jar binary. See the following README for more information: `source/lambda/pdfgenerator/README.md`
+The searchable PDF functionality is included as a pre-compiled jar binary. See the following README for more information: `source/lambda/pdfgenerator/README.md`

deployment/document-understanding-cicd/index.js

@@ -62,7 +62,7 @@ exports.handler = async (event, context) => {
 
         Logger.log(
           Logger.levels.ROBUST,
-          "adding access permissions to Log Groups to Amazon ElasticSearch"
+          "adding access permissions to Log Groups to Amazon OpenSearch Service"
         );
 
         const _repo = process.env.CODECOMMIT_REPO;

deployment/document-understanding-cicd/package-lock.json

@@ -182,6 +182,7 @@ Resources:
                 ruby: 2.7
               commands:
                 - echo "This buildspec is based on image - aws/codebuild/amazonlinux2-x86_64-standard:4.0"
+                - n 16
                 - node --version
                 - echo "Installing jq package"
                 - wget -O jq https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64
@@ -192,7 +193,7 @@ Resources:
                 - python get-pip.py
                 - echo "Installing moto for running tests"
                 - pip install moto==2.3.2
-                - npm install -g cdk@1.158.0
+                - npm install -g cdk@1.203.0
                 - cdk --version
                 - npm install -g typescript
                 - tsc --version

deployment/document-understanding-solution.template

@@ -0,0 +1,21 @@
+[node-yarnoutdated]
+aws-amplify=4.3.46
+aws-cdk=1.203.0
+date-fns=1.30.1
+next=12.2.4
+next-redux-wrapper=5.0.0
+query-string=7.1.3
+react=17.0.2
+react-pdf=4.2.0
+redux-actions=2.6.5
+dotenv=16.0.1
+eslint=8.33.0
+eslint-config-prettier=8.6.0
+inquirer=9.0.2
+node-fetch=3.2.10
+ramda=0.28.0
+react-webcam=7.0.1
+sass=1.54.0
+@babel/helper-validator-identifier=7.18.6
+typescript=4.9.5
+aws-sdk=2.1392.0

source/.viperlightignore

@@ -26,7 +26,7 @@ export const lensTrack = id => lensPath(['entities', 'tracks', id])
 const kendraModeDescription = `
 In this track, you can search through multiple documents and find information faster
 and more efficiently, using  traditional search based technologies such as Amazon 
-Elasticsearch Service (Amazon ES) as well using state-of-the-art machine learning and natural language enterprise search
+OpenSearch Service as well using state-of-the-art machine learning and natural language enterprise search
 with Amazon Kendra.
 
 Once you select a document, you'll be able to interact with Amazon Textract,
@@ -36,7 +36,7 @@ of document digitization and extraction of sensitive information.
 For demo purposes, the solution is pre-loaded with data related to certain medical conditions such as Diabetes and Kidney Disease. Ask a question related to these topics or use the suggested queries in the search bar and explore the different search capabilities.
 `
 const classicModeDescription = `In this track, you can search through multiple documents and find information faster
-and more efficiently, using Amazon Elasticsearch.
+and more efficiently, using Amazon OpenSearch Service.
 
 Once you select a document, you'll be able to interact with Amazon Textract,
 Amazon Comprehend, and Amazon Comprehend Medical to experience the power

source/app/store/entities/tracks/data.js

@@ -1,4 +1,4 @@
-docutils==0.19
+docutils==0.20.1
 jmespath==1.0.1
 python_dateutil==2.8.2
 boto3==1.24.37 

source/lambda/boto3/requirements.txt

@@ -38,7 +38,7 @@ def generateDocumentID(bucketName, s3Client):
 def downloadDocument(bucketName, documentKey, filename, s3Client):
 
     # create a local file
-    datafile = open('/tmp/' + filename, 'wb')
+    datafile = open('/tmp/' + filename, 'wb') # nosec B108
 
     # system error
     if datafile == None:
@@ -58,7 +58,7 @@ def downloadDocument(bucketName, documentKey, filename, s3Client):
 def uploadDocument(bucketName, documentKey, filename, s3Client):
 
     # open local file
-    datafile = open('/tmp/' + filename, 'rb')
+    datafile = open('/tmp/' + filename, 'rb') # nosec B108
 
     if datafile == None:
         print("UploadDocument: failed to open local file: " + filename)

source/lambda/documentbulkprocessor/lambda_function.py

@@ -39,7 +39,7 @@ def get_mime_type(request):
     Utilizes magic number checking via the 'filetype' library to determine if the files are of a valid type.
     """
     client = boto3.client('s3')
-    local_path = f"/tmp/{request['objectName'].rsplit('/',1)[-1]}"
+    local_path = f"/tmp/{request['objectName'].rsplit('/',1)[-1]}" # nosec B108
     client.download_file(request['bucketName'],
                          request['objectName'], local_path)
 

source/lambda/documentprocessor/lambda_function.py

@@ -1,6 +1,6 @@
 requests-aws4auth==0.9
 elasticsearch==7.9.1
-requests==2.28.1
+requests==2.31.0
 idna==3.3
 chardet==5.1.0
 certifi==2022.12.07

source/lambda/elasticsearch/requirements.txt

@@ -48,28 +48,28 @@
     "test-client-build": "next build app"
   },
   "dependencies": {
-    "@aws-cdk/aws-apigateway": "1.158.0",
-    "@aws-cdk/aws-cloudformation": "1.158.0",
-    "@aws-cdk/aws-cognito": "1.158.0",
-    "@aws-cdk/aws-dynamodb": "1.158.0",
-    "@aws-cdk/aws-ec2": "1.158.0",
-    "@aws-cdk/aws-elasticsearch": "1.158.0",
-    "@aws-cdk/aws-events": "1.158.0",
-    "@aws-cdk/aws-iam": "1.158.0",
-    "@aws-cdk/aws-lambda": "1.158.0",
-    "@aws-cdk/aws-lambda-event-sources": "1.158.0",
-    "@aws-cdk/aws-logs": "1.158.0",
-    "@aws-cdk/aws-s3": "1.158.0",
-    "@aws-cdk/aws-s3-deployment": "1.158.0",
-    "@aws-cdk/aws-s3-notifications": "1.158.0",
-    "@aws-cdk/aws-sns": "1.158.0",
-    "@aws-cdk/aws-sqs": "1.158.0",
-    "@aws-cdk/core": "1.158.0",
+    "@aws-cdk/aws-apigateway": "1.203.0",
+    "@aws-cdk/aws-cloudformation": "1.203.0",
+    "@aws-cdk/aws-cognito": "1.203.0",
+    "@aws-cdk/aws-dynamodb": "1.203.0",
+    "@aws-cdk/aws-ec2": "1.203.0",
+    "@aws-cdk/aws-elasticsearch": "1.203.0",
+    "@aws-cdk/aws-events": "1.203.0",
+    "@aws-cdk/aws-iam": "1.203.0",
+    "@aws-cdk/aws-lambda": "1.203.0",
+    "@aws-cdk/aws-lambda-event-sources": "1.203.0",
+    "@aws-cdk/aws-logs": "1.203.0",
+    "@aws-cdk/aws-s3": "1.203.0",
+    "@aws-cdk/aws-s3-deployment": "1.203.0",
+    "@aws-cdk/aws-s3-notifications": "1.203.0",
+    "@aws-cdk/aws-sns": "1.203.0",
+    "@aws-cdk/aws-sqs": "1.203.0",
+    "@aws-cdk/core": "1.203.0",
     "@babel/helper-validator-identifier": "^7.18.6",
-    "@types/node": "^18.6.1",
-    "aws-amplify": "^4.3.28",
-    "aws-cdk": "1.158.0",
-    "aws-sdk": "^2.1354.0",
+    "@types/node": "20.2.5",
+    "aws-amplify": "^4.3.46",
+    "aws-cdk": "1.203.0",
+    "aws-sdk": "^2.1391.0",
     "classnames": "^2.3.1",
     "date-fns": "^1.30.1",
     "dotenv": "^16.0.1",
@@ -102,7 +102,7 @@
     "serialize-javascript": "^6.0.0",
     "short-uuid": "^4.2.0",
     "susy": "^3.0.7",
-    "typescript": "^4.7.4"
+    "typescript": "4.9.5"
   },
   "devDependencies": {
     "autoprefixer": "^10.4.7",
@@ -115,5 +115,11 @@
     "eslint-plugin-react-hooks": "^4.6.0",
     "legally": "^3.5.10",
     "prettier": "^2.8.3"
+  },
+  "overrides": {
+    "fast-xml-parser": "4.2.4"
+  },
+  "resolutions": {
+    "fast-xml-parser": "4.2.4"
   }
 }