Added translation support for lists #33
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We noticed that list elements were not being translated, so this simple fix will allow for that
updated mock so that li elements were also being returned
larasakarya
approved these changes
Jun 21, 2023
athan29
approved these changes
Jun 21, 2023
kyoncal
added a commit
that referenced
this pull request
Jun 25, 2023
* amplify pipeline * amp pipeline * pipeline * i wish the pipeline worked * aaaaaaaaa * pipeline * Merging dev into translation storage branch to fix amplify pipeline (#21) * Backend full testing (#19) * Added initial Test files and refactored AWSBlogPost to be more testable. Changed the checkstyle to remove checks for class design since it interferes with test classes. When testing, it was apparent that some classes are poorly designed. They need to be refactored. * FIxed the bug where some blog posts would repeat translated elements by making a new request object everytime (probably java being smart and optimizing some code away) * Used explicit mock instead of annotations to initialize mock * Added first integration test * Added Jacoco for Unit Testing Reports * Added Baisc Unit Tests for every class with logic, and an integration test. * Removed unecessary jacoco part in build.gradle * Refactored AWSBlogPosts and Text classes to make more sense * fixed checkstyle * Pipeline (#20) * adding skeleton readme.md * Changes node versions for pipeline --------- Co-authored-by: kyonc2022 <91875365+kyonc2022@users.noreply.github.com> Co-authored-by: Tezzish <31407433+Tezzish@users.noreply.github.com> * ppipeline * Updated graphql schema to match amplify * schema fixed * pipeline check * frontend fix * frontend trying fix * frontend push fix * unnecessary prints deleted * checking for valid url added here * unnecessary comment deleted * importing requests * trying to fix the lambda * frontend double html issue fix * checking for a fix * adding functionality for the storage of translations * frontend fix * no * check * trying my best * frontend fix * lambda fix * working environment + new lambda template * redid amplify * quick fix * conflict fixes * merge fix * merge fixx * amplify * merge fix * merge fix * deleting unnecessary function * amplify? * fix * fix * delete old func * merge fix * still old function * fix * bug * fix * fix * fix * new function + cleanup * a * added comment to check * amplify things * lambda library fixed * wrong import removed * LHS working (#26) * Cloud scape integration (#28) * Added Basic UI elements Application now has basic cloudScape React Elements in the form * functioning components + file refactoring Additionally started implementation of alerts * Added Alerts and Loading * Added Logo * Logo isn't displayed over errors * CloudScape works on original LHS side * Pipeline fix got the correct Jest configuration set up. This is the testing framework we will use * Set up Testing Configuration * adding translation storages to s3 bucket and dynamoDB * new lambda added to trigger step function * url fix * working lambda * working lambda * fixing the lambda to invoke step functions * lambda change, returns html files * lambda now returns the authors and the title of the blog post as well * lambda fix * lhs lambda done * re-adding translation storage lambda as it was deleted * adding script to convert dataset to csv * accidental lambda deletion fixed * step function trigger lambda updated according to graohql query * updated lambda * comments added to lambda * passing new lambda args * updated invoker step function lambda to return the result of the step function * step function invoker complete * frontend needs to be updated, commented out for now * step func invoker updated * updated lambda according to query * steo invoker works fully * updated graphql schema * graphwl fixed * conflicts fixed * graphwl fixed * revert * after amplify pull functions were restored, one lambda had to be updated since wrong version was present * schema updated * commented out a query for now * mini step function fix * missing import added * fixing amplify * adding translation storage lambda to amplify + layer * fixing package-lock.json for pipeline * running npm install to get package.json and package-lock.json to be in sync * frontend needs to be worked on, commented out for now * restored lambdas * setup ready * Added RHS reconstructin function Note that we still need to translate and replace list elements <il> (and possibly other text elements) * url string fixed * newDB table * Changed logo after client feedback (#30) * Allow the RatingIds to be autogenerated by appSync (#32) * amplify setup complete * storingTranslation function updated * unnecessary comments deleted * trying to fix translation storage function * database query fixed for new db * fully working frontend * build * Added translation support for lists (#33) * Added translation support for lists We noticed that list elements were not being translated, so this simple fix will allow for that * Fixed failing text case updated mock so that li elements were also being returned * merge fix * frontend fix for visuals * frontend final fix complete * trying to fix node pipeline * Added rating, missing id from return * fixing ratings * added id to be returned * Cdk init (#35) * Added rating to graphql schema * Added the star elements to the front-end * Made the star element appear only after completion of translation * added jacoco proper report to UserConfigFunction * updated package-lock.json * Added generated graphql queries for new API that includes rating * Added logic to send and mutate the rating to the backend using the star rating * hotfix * hotfix ridiculous typo * Updated graphql queries to reflect the queries actually occuring * made functions async to deal with uncaught promise * Fixed visibility bug for ratingId variable that prevented the graphql call from going through. * initialize CDK project * Added the function to creating getblogpostcontent * Added a handler for getcontentblogfunction and changed the name of the stack * Added a TypeScript version of an AppStack the uses CDK. Modified front-end resources to work with the generated Amplify app. **NOTE** This will not longer automatically create a back-end environment when adding GitHub as a code source in Amplify.. * Added url and s3url fields to blogpost in graphql schema and refactored input names * Minor Semantic change to schema and changed repo owner from kyon to ishan * Added a build script to export GetBlogContent function * Added App.js fixed merge conflicts * update app.js and package lock to allow pipeline to pass * Remove java cdk directory * updated aws-exports * Removed Unnecessary Amplify file and readded build back-end * Readded back-end to amplify.yml * Revert the change that removed a GraphQL element * Added missing translationModel * Added amplify init to amplify yaml * Revert changes done to amplify yaml and remove aws graphql annotations on rating * Removed Team Provider so that back-end environment can be built automatically in amplify * Revert "Removed Team Provider so that back-end environment can be built automatically in amplify" This reverts commit c738766. * Empty team-provider-info.json * Added name for current environment * remove backend from amplify yaml * Added auto-generated graphql to the main schema * Updated queries.js * update translationmodel queries * Added the creation of basic DynamoDB queries and mutations * trying to fix frontend * working frontend + backend without ratings * Ratings and content display * updated package-lock for pipeline fix * Removed second 'export default App' * Github pipeline fix 0 * updated css * css visuals and translation functionality successful * schema.graphql fix * css + rating fix frontend * removed hardcoding of languages and models * Pipeline errors and function additions * pipeline typo * returned checkstyle warnings * Added requirements and tests to functions * removed json from requiremnts.txt * removed copy and time from requirements.txt * Javascript pipeline * Initial translationStorage tests * test revert * fixing python pipeline * python pipeline...again * pytests pipeline * reformated pipelines & reset src\aws_exports.js * removed print satements * Pipeline name * removed print satements v2 * fixed bucket name * Frontend documentation (#34) * Added comments to App.js and App.css * Added Frontend README, Deleted Documents This commit adds the necessary README file to the frontend so that future developers can understand how to continue development * minor comments to components * Removed 'Documents' from root README * Improved comments following merge Also deleted unnecessary files * Added app landing page * Further refactoring of components * Removed warnings, finalized frontend * Added imports to fix failings tests * Code cleanup and documentation (#38) * Fixed carriage return issue with build.sh in each python lambda. * Added an exports file so users can change that instead of messing with code when trying to load this onto their environment. * Updated READMEs * Added convinient buildall bash scripts * Modified buildall script to include gradle and fixed getting environment variable in python functions. * Minor Semantic Changes and README * Added dynamic links to AWS service dependancies * Updated README for UserConfigFunction * Get rid of unnecessary files * Cleaned up code * Modified Testing and code clean up * Update cdk-init-ts-stack.ts Changed node 18 to node 16 * Update README.md Added forgotten step * readmes added for the lambdas (#39) * readmes added for the lambdas * readme updated * readme updated * readme * readme complete * Ml model (#40) * creating script to convert dataset to CSV * adding data * adding code for tokenizer and preprocessing * changing from tensorflow to pytorch + tokenization and other preprocessing * adding truncation and padding as well as unk tokens to the tokenizer * saving tokenizer + other minor changes * using a ByteLevelBPEtokenizer instead as this is more fitting for turkish * using GPT2Tokenizer to create tokens for our data * setting up dataset and config for GPT2 * adding changes to training * adding BERT to generate encodings before applying gpt2 * Using just gpt2 for training * adding bart implementation as it finally works for seq2seq * removing older models that don't work * adding train.py script for training on SageMaker * adding tokenizer script to save tokenized datasets on s3 * adding project structure to be like what sagemaker expects * adding loss graph and changes to sagemaker files * adding model and changing the training files * adding model deployment notebook * changing the value of the s3 bucket for deployment * adds the integration for our model on the app --------- Co-authored-by: Lara Sakarya <L.Sakarya@student.tudelft.nl> Co-authored-by: larasakarya <72052108+larasakarya@users.noreply.github.com> Co-authored-by: Tezzish <31407433+Tezzish@users.noreply.github.com> Co-authored-by: Ishan <ishanpahwa14@gmail.com> Co-authored-by: Athanasios Christopoulos <A.Christopoulos@student.tudelft.nl> Co-authored-by: PabloLacombe <77081540+PabloLacombe@users.noreply.github.com> Co-authored-by: Pablo Lacombe <pablo.lacombe@hotmail.com> Co-authored-by: Athanasios Christopoulos <77352412+athan29@users.noreply.github.com>
larasakarya
added a commit
that referenced
this pull request
Jun 27, 2023
* amplify pipeline * amp pipeline * pipeline * i wish the pipeline worked * aaaaaaaaa * pipeline * Merging dev into translation storage branch to fix amplify pipeline (#21) * Backend full testing (#19) * Added initial Test files and refactored AWSBlogPost to be more testable. Changed the checkstyle to remove checks for class design since it interferes with test classes. When testing, it was apparent that some classes are poorly designed. They need to be refactored. * FIxed the bug where some blog posts would repeat translated elements by making a new request object everytime (probably java being smart and optimizing some code away) * Used explicit mock instead of annotations to initialize mock * Added first integration test * Added Jacoco for Unit Testing Reports * Added Baisc Unit Tests for every class with logic, and an integration test. * Removed unecessary jacoco part in build.gradle * Refactored AWSBlogPosts and Text classes to make more sense * fixed checkstyle * Pipeline (#20) * adding skeleton readme.md * Changes node versions for pipeline --------- Co-authored-by: kyonc2022 <91875365+kyonc2022@users.noreply.github.com> Co-authored-by: Tezzish <31407433+Tezzish@users.noreply.github.com> * ppipeline * Updated graphql schema to match amplify * schema fixed * pipeline check * frontend fix * frontend trying fix * frontend push fix * unnecessary prints deleted * checking for valid url added here * unnecessary comment deleted * importing requests * trying to fix the lambda * frontend double html issue fix * checking for a fix * adding functionality for the storage of translations * frontend fix * no * check * trying my best * frontend fix * lambda fix * working environment + new lambda template * redid amplify * quick fix * conflict fixes * merge fix * merge fixx * amplify * merge fix * merge fix * deleting unnecessary function * amplify? * fix * fix * delete old func * merge fix * still old function * fix * bug * fix * fix * fix * new function + cleanup * a * added comment to check * amplify things * lambda library fixed * wrong import removed * LHS working (#26) * Cloud scape integration (#28) * Added Basic UI elements Application now has basic cloudScape React Elements in the form * functioning components + file refactoring Additionally started implementation of alerts * Added Alerts and Loading * Added Logo * Logo isn't displayed over errors * CloudScape works on original LHS side * Pipeline fix got the correct Jest configuration set up. This is the testing framework we will use * Set up Testing Configuration * adding translation storages to s3 bucket and dynamoDB * new lambda added to trigger step function * url fix * working lambda * working lambda * fixing the lambda to invoke step functions * lambda change, returns html files * lambda now returns the authors and the title of the blog post as well * lambda fix * lhs lambda done * re-adding translation storage lambda as it was deleted * adding script to convert dataset to csv * accidental lambda deletion fixed * step function trigger lambda updated according to graohql query * updated lambda * comments added to lambda * passing new lambda args * updated invoker step function lambda to return the result of the step function * step function invoker complete * frontend needs to be updated, commented out for now * step func invoker updated * updated lambda according to query * steo invoker works fully * updated graphql schema * graphwl fixed * conflicts fixed * graphwl fixed * revert * after amplify pull functions were restored, one lambda had to be updated since wrong version was present * schema updated * commented out a query for now * mini step function fix * missing import added * fixing amplify * adding translation storage lambda to amplify + layer * fixing package-lock.json for pipeline * running npm install to get package.json and package-lock.json to be in sync * frontend needs to be worked on, commented out for now * restored lambdas * setup ready * Added RHS reconstructin function Note that we still need to translate and replace list elements <il> (and possibly other text elements) * url string fixed * newDB table * Changed logo after client feedback (#30) * Allow the RatingIds to be autogenerated by appSync (#32) * amplify setup complete * storingTranslation function updated * unnecessary comments deleted * trying to fix translation storage function * database query fixed for new db * fully working frontend * build * Added translation support for lists (#33) * Added translation support for lists We noticed that list elements were not being translated, so this simple fix will allow for that * Fixed failing text case updated mock so that li elements were also being returned * merge fix * frontend fix for visuals * frontend final fix complete * trying to fix node pipeline * Added rating, missing id from return * fixing ratings * added id to be returned * Cdk init (#35) * Added rating to graphql schema * Added the star elements to the front-end * Made the star element appear only after completion of translation * added jacoco proper report to UserConfigFunction * updated package-lock.json * Added generated graphql queries for new API that includes rating * Added logic to send and mutate the rating to the backend using the star rating * hotfix * hotfix ridiculous typo * Updated graphql queries to reflect the queries actually occuring * made functions async to deal with uncaught promise * Fixed visibility bug for ratingId variable that prevented the graphql call from going through. * initialize CDK project * Added the function to creating getblogpostcontent * Added a handler for getcontentblogfunction and changed the name of the stack * Added a TypeScript version of an AppStack the uses CDK. Modified front-end resources to work with the generated Amplify app. **NOTE** This will not longer automatically create a back-end environment when adding GitHub as a code source in Amplify.. * Added url and s3url fields to blogpost in graphql schema and refactored input names * Minor Semantic change to schema and changed repo owner from kyon to ishan * Added a build script to export GetBlogContent function * Added App.js fixed merge conflicts * update app.js and package lock to allow pipeline to pass * Remove java cdk directory * updated aws-exports * Removed Unnecessary Amplify file and readded build back-end * Readded back-end to amplify.yml * Revert the change that removed a GraphQL element * Added missing translationModel * Added amplify init to amplify yaml * Revert changes done to amplify yaml and remove aws graphql annotations on rating * Removed Team Provider so that back-end environment can be built automatically in amplify * Revert "Removed Team Provider so that back-end environment can be built automatically in amplify" This reverts commit c738766. * Empty team-provider-info.json * Added name for current environment * remove backend from amplify yaml * Added auto-generated graphql to the main schema * Updated queries.js * update translationmodel queries * Added the creation of basic DynamoDB queries and mutations * trying to fix frontend * working frontend + backend without ratings * Ratings and content display * updated package-lock for pipeline fix * Removed second 'export default App' * Github pipeline fix 0 * updated css * css visuals and translation functionality successful * schema.graphql fix * css + rating fix frontend * removed hardcoding of languages and models * Pipeline errors and function additions * pipeline typo * returned checkstyle warnings * Added requirements and tests to functions * removed json from requiremnts.txt * removed copy and time from requirements.txt * Javascript pipeline * Initial translationStorage tests * test revert * fixing python pipeline * python pipeline...again * pytests pipeline * reformated pipelines & reset src\aws_exports.js * removed print satements * Pipeline name * removed print satements v2 * fixed bucket name * Frontend documentation (#34) * Added comments to App.js and App.css * Added Frontend README, Deleted Documents This commit adds the necessary README file to the frontend so that future developers can understand how to continue development * minor comments to components * Removed 'Documents' from root README * Improved comments following merge Also deleted unnecessary files * Added app landing page * Further refactoring of components * Removed warnings, finalized frontend * Added imports to fix failings tests * testing * check done * adding frontend tess * trying to make testing work * trying to make testing work * Code cleanup and documentation (#38) * Fixed carriage return issue with build.sh in each python lambda. * Added an exports file so users can change that instead of messing with code when trying to load this onto their environment. * Updated READMEs * Added convinient buildall bash scripts * Modified buildall script to include gradle and fixed getting environment variable in python functions. * Minor Semantic Changes and README * Added dynamic links to AWS service dependancies * Updated README for UserConfigFunction * Get rid of unnecessary files * Cleaned up code * Modified Testing and code clean up * Update cdk-init-ts-stack.ts Changed node 18 to node 16 * Update README.md Added forgotten step * trying to fix frontend * getBlogContent testing * pipeline fix * test_replace_text_with_translation Wrote all tests for replace_text_with_translation function * frontend test * testing checkingUrl, stepFunctionInvoker * frontend testing finalized * pipeline change * pipeline python * pipeline python * pipeline python * small test changes * pipeline * pipeline * pipeline * frontend back * frontend back * frontend back * pip * frontend back * frontend back * asfsa * checking url tests are fixed * stepfunction invoker tests are fixed * tests fixed * Co-authored-by: Athanasios Christopoulos <athan29@users.noreply.github.com> * tests fixed * testing working --------- Co-authored-by: Lara Sakarya <L.Sakarya@student.tudelft.nl> Co-authored-by: larasakarya <72052108+larasakarya@users.noreply.github.com> Co-authored-by: kyonc2022 <91875365+kyonc2022@users.noreply.github.com> Co-authored-by: Tezzish <31407433+Tezzish@users.noreply.github.com> Co-authored-by: username <k.s.caldera@student.tudelft.nl> Co-authored-by: Ishan <ishanpahwa14@gmail.com> Co-authored-by: PabloLacombe <77081540+PabloLacombe@users.noreply.github.com> Co-authored-by: Pablo Lacombe <pablo.lacombe@hotmail.com> Co-authored-by: Paby <P.O.Lacombe@student.tudelft.nl>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We noticed that list elements were not being translated, so this simple fix will allow for that