Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PR into Main #80

Merged
merged 168 commits into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
168 commits
Select commit Hold shift + click to select a range
629cd33
watchdog lib basics and flake8 update
Yudi-G Jun 6, 2024
85ab868
runs with command
Yudi-G Jun 6, 2024
8aa65c0
added examples
Yudi-G Jun 6, 2024
305e3d3
flake8 config changes
Yudi-G Jun 6, 2024
1a7483b
linting fix
Yudi-G Jun 6, 2024
3ff14ee
codecov yaml addition
Yudi-G Jun 6, 2024
e8ff36b
Merge pull request #23 from COS301-SE-2024/feature/codecovyml
Yudi-G Jun 6, 2024
7d5b75f
Update lint-tests.yaml
Yudi-G Jun 6, 2024
3f53b38
PDF OCR added
u21598012 Jun 6, 2024
8c47878
encoding preprocessor added
u21598012 Jun 6, 2024
f064288
DOCX OCR added
u21598012 Jun 6, 2024
74912de
xlsx OCR added
u21598012 Jun 6, 2024
7190a46
Did a simple setup of Angular and Electron
Nevin-Thomas Jun 7, 2024
c517cc9
Update lint-tests.yaml
Yudi-G Jun 7, 2024
a31ca71
added filewatcher test
Yudi-G Jun 7, 2024
0b8faff
filewatcher linting
Yudi-G Jun 7, 2024
cdad6b5
filewatcher linting
Yudi-G Jun 7, 2024
6ca21b8
Update lint-tests.yaml, continue on error
Yudi-G Jun 7, 2024
5a8bb9f
Fixed Excel OCR
u21598012 Jun 7, 2024
205f221
Basic virus scanning
u21598012 Jun 7, 2024
cac7fc0
Merge pull request #27 from COS301-SE-2024/Yudi-G-patch-1
Yudi-G Jun 7, 2024
2cc16be
gitleaks fix
Yudi-G Jun 7, 2024
88e9828
Merge pull request #34 from COS301-SE-2024/fix/developGitleaks
Yudi-G Jun 7, 2024
8571e0d
chore: Update Angular and Electron dependencies
paul-nhlapo Jun 8, 2024
1510d50
Deactivated virus scanning
u21598012 Jun 8, 2024
126c297
Lemmatization
u21598012 Jun 8, 2024
aebaa69
Implemented Inbox Component
Yeshlen Jun 9, 2024
e37f023
Fixed the error on inbox.component.ts file
Nevin-Thomas Jun 9, 2024
0a36d2a
Creating the violations page (still in progress)
Nevin-Thomas Jun 9, 2024
f8caeb5
Add new routes for home, report, and upload document components
paul-nhlapo Jun 9, 2024
460a9f6
Added sanitization
u21598012 Jun 9, 2024
a2fc163
Separation of functionality
u21598012 Jun 9, 2024
91fa78d
Regex Layer
u21598012 Jun 9, 2024
49ea47b
Landing page, report page and upload page
paul-nhlapo Jun 9, 2024
f240336
Add new routes for home, report, and upload document components and m…
paul-nhlapo Jun 9, 2024
ea15bf0
some progress on the Violatons page
Nevin-Thomas Jun 10, 2024
a077528
update on Violations page
paul-nhlapo Jun 10, 2024
f10cd3e
update to Violations page
paul-nhlapo Jun 10, 2024
9026de0
concurrent folder monitoring
Yudi-G Jun 10, 2024
abcfa98
added notes to file watcher
Yudi-G Jun 10, 2024
6f9a363
added function to scan dir
Yudi-G Jun 10, 2024
2d8f653
added function to scan dir
Yudi-G Jun 10, 2024
7e82c08
Update lint-tests.yaml
Yudi-G Jun 10, 2024
8cc7642
Update lint-tests.yaml
Yudi-G Jun 10, 2024
ca22084
Update lint-tests.yaml
Yudi-G Jun 10, 2024
a075ab8
Merge pull request #45 from COS301-SE-2024/Yudi-G-patch-1
Yudi-G Jun 10, 2024
ba83d79
I sorted out the violation folder by using the ng generate command
Nevin-Thomas Jun 10, 2024
762f084
Violation buttons loads different documents according to file type
Nevin-Thomas Jun 10, 2024
68d5085
Adding flowbite
paul-nhlapo Jun 10, 2024
f418fa3
Merge branch 'feature/new-frontend' of https://github.com/COS301-SE-2…
paul-nhlapo Jun 10, 2024
20afaa6
Update dependencies and add Tailwind CSS for styling , Fixed some co…
paul-nhlapo Jun 10, 2024
973862b
This commit adds lazy loading for images in the app component to impr…
paul-nhlapo Jun 10, 2024
9648bea
switch
u21598012 Jun 11, 2024
3d593fb
General CSS styling changes
Yeshlen Jun 11, 2024
dea9e68
feat: Add lazy loading for images in app component and update routing…
paul-nhlapo Jun 11, 2024
57e9523
Removed file creation
u21598012 Jun 15, 2024
9d302cf
Merge pull request #48 from COS301-SE-2024/feature/text_extraction
u21598012 Jun 15, 2024
c8c3b5f
Merge branch 'feature/new-frontend' of https://github.com/COS301-SE-2…
Yeshlen Jun 15, 2024
0d2b07b
Added Fast API
Yeshlen Jun 16, 2024
2839943
Frontend with API
Yeshlen Jun 20, 2024
55cc9c9
Backend endpoint integration
u21598012 Jun 20, 2024
79a2dfd
File removal post processing
u21598012 Jun 20, 2024
fabe0ef
Analysis is displayed
Yeshlen Jun 20, 2024
06a25ef
Merge pull request #52 from COS301-SE-2024/integration/first
u21598012 Jun 20, 2024
c2af5ef
In
u21598012 Jun 20, 2024
bec788b
Path successfully watched
u21598012 Jun 20, 2024
e84e983
started file watcher timer
Yudi-G Jun 21, 2024
95bd56c
timer working
Yudi-G Jun 21, 2024
979f5c9
testing file watcher
Yudi-G Jun 22, 2024
1ca5afd
testing file watcher, new
Yudi-G Jun 22, 2024
53f5d0c
test for file watcher editing
Yudi-G Jun 22, 2024
b4f7bef
added threading to file watcher
Yudi-G Jun 22, 2024
a8eca0a
modified test file
Yudi-G Jun 22, 2024
d14983f
modified test file and added notes to file watcher
Yudi-G Jun 22, 2024
6b622d3
Created the manual page
paul-nhlapo Jun 22, 2024
17cd562
Virus scan initialized
u21598012 Jun 23, 2024
b8bf036
Regex Layer unit test
u21598012 Jun 23, 2024
e586734
pytest fix
Yudi-G Jun 23, 2024
66652b6
pytest fix
Yudi-G Jun 23, 2024
0a09832
Removed the .MAUI app
paul-nhlapo Jun 23, 2024
3e9a61a
pytest fix
Yudi-G Jun 23, 2024
d12c9c9
File monitor integrated
u21598012 Jun 23, 2024
f81e367
Merge branch 'integration/file_monitor' of https://github.com/COS301-…
u21598012 Jun 23, 2024
c1af8cd
Path change
u21598012 Jun 23, 2024
bc4a62f
chore: update @angular-devkit/build-angular and @angular/cli to versi…
paul-nhlapo Jun 23, 2024
b2487e6
chore: Update @angular dependencies to version 18.0.4
paul-nhlapo Jun 23, 2024
942c249
chore: Update @types/node dependency to version 20.14.8
paul-nhlapo Jun 23, 2024
60d8817
fix testing
Yudi-G Jun 23, 2024
54eddae
Requirements
u21598012 Jun 23, 2024
b93c92c
Merge branch 'integration/file_monitor' of https://github.com/COS301-…
u21598012 Jun 23, 2024
296de63
Update python-app-test.yml
Yudi-G Jun 23, 2024
be25c55
Merge pull request #59 from COS301-SE-2024/Yudi-G-patch-2
Yudi-G Jun 23, 2024
f486b2b
fix testing
Yudi-G Jun 23, 2024
2fc13c8
fix testing
Yudi-G Jun 23, 2024
7752fd6
fitz fix
u21598012 Jun 23, 2024
88fd0f0
Merge branch 'integration/file_monitor' of https://github.com/COS301-…
u21598012 Jun 23, 2024
a75e54e
fitz removal
u21598012 Jun 23, 2024
ea41674
fitz removal
u21598012 Jun 23, 2024
ceace8e
fix testing
Yudi-G Jun 23, 2024
bf59cdf
fitz removal
u21598012 Jun 23, 2024
cb56e88
agnostic fix
u21598012 Jun 23, 2024
0eb5569
fixed tests
u21598012 Jun 23, 2024
5704e6c
fixed tests
u21598012 Jun 23, 2024
9246109
update to the latest packages
paul-nhlapo Jun 23, 2024
9770f12
poppler
Yudi-G Jun 23, 2024
2f33281
poppler
Yudi-G Jun 23, 2024
54ed5bd
tesseract actions
Yudi-G Jun 23, 2024
8d557ff
tesseract actions
Yudi-G Jun 23, 2024
a806690
test fix
u21598012 Jun 23, 2024
401689d
lint fixes
Yudi-G Jun 23, 2024
18d98f2
Just adjusted the back button
Nevin-Thomas Jun 23, 2024
51a4a05
Create frontendTests.yml
Yudi-G Jun 23, 2024
1df63c7
Merge pull request #60 from COS301-SE-2024/Yudi-G-patch-2
Yudi-G Jun 23, 2024
9da3414
workflow frontend tests
Yudi-G Jun 23, 2024
9552db1
Updating the package.json and updating the help page to use a carosel
paul-nhlapo Jun 23, 2024
49c1f80
workflow frontend tests
Yudi-G Jun 23, 2024
b69e314
Frontend Unit Tests
Yeshlen Jun 23, 2024
23ea739
Some minor changes
Nevin-Thomas Jun 23, 2024
f96e4db
workflow frontend tests
Yudi-G Jun 23, 2024
994e21e
Hot fixes
paul-nhlapo Jun 23, 2024
e2c4705
Merge branch 'tests/frontend' of https://github.com/COS301-SE-2024/GD…
Nevin-Thomas Jun 23, 2024
06282ad
workflow frontend tests
Yudi-G Jun 23, 2024
fce1010
Merge remote-tracking branch 'origin/tests/frontend' into tests/frontend
Yudi-G Jun 23, 2024
ba9abf6
workflow frontend tests
Yudi-G Jun 23, 2024
7ef1685
workflow frontend tests
Yudi-G Jun 23, 2024
26bf837
workflow frontend tests
Yudi-G Jun 23, 2024
08d80e1
Fixed the size limit
paul-nhlapo Jun 23, 2024
1cd0e1e
workflow frontend tests
Yudi-G Jun 23, 2024
2c7c388
workflow frontend tests
Yudi-G Jun 23, 2024
3205fb5
workflow frontend tests
Yudi-G Jun 23, 2024
31c533e
Merge pull request #55 from COS301-SE-2024/HotFix
Yeshlen Jun 23, 2024
f1b66b7
workflow frontend tests
Yudi-G Jun 23, 2024
97dbbea
Frontend Unit Tests
Yeshlen Jun 23, 2024
1b3402c
Removed pycache
Yeshlen Jun 23, 2024
7e49692
Removed pycache
Yeshlen Jun 23, 2024
20f54e7
Merge pull request #58 from COS301-SE-2024/integration/file_monitor
Yeshlen Jun 23, 2024
6ac694c
cosmetic hot fix
u21598012 Jun 23, 2024
99fade7
chrome in workflows
Yudi-G Jun 23, 2024
a084694
chrome in workflows
Yudi-G Jun 23, 2024
76c9c5a
chrome in workflows
Yudi-G Jun 23, 2024
4bff9dd
Frontend Unit Tests
Yeshlen Jun 23, 2024
afb5a00
Merge branch 'develop' into tests/frontendWorking
Yudi-G Jun 23, 2024
068cdf3
Merge pull request #64 from COS301-SE-2024/tests/frontendWorking
Yudi-G Jun 23, 2024
53cfe48
no fitz
Yudi-G Jun 23, 2024
5b17c0c
Update python-app-test.yml
Yudi-G Jun 23, 2024
c3b1998
file service removal w/ comments
Yudi-G Jun 23, 2024
cec30fb
Update lint-tests.yaml
Yudi-G Jun 23, 2024
d4764e1
Updating the new develop
paul-nhlapo Jun 24, 2024
3a688ed
Upload CSS Fix
Yeshlen Jun 24, 2024
4e90375
image edit
paul-nhlapo Jun 24, 2024
cccb169
Update README.md
Yeshlen Jun 24, 2024
73971d1
checking
Yudi-G Jun 24, 2024
9e40f9a
Merge pull request #78 from COS301-SE-2024/test/backendEntry
Yudi-G Jun 24, 2024
dbc40af
edited yaml
Yudi-G Jun 24, 2024
8a59e8b
edited yaml
Yudi-G Jun 24, 2024
4a0933a
Merge pull request #79 from COS301-SE-2024/test/backendEntry
u21598012 Jun 24, 2024
dfbe88b
Demo 2 Documentation
Yeshlen Jun 24, 2024
c841473
Update README.md
Yeshlen Jun 24, 2024
41d6021
Update README.md
Yeshlen Jun 24, 2024
ea5a7fa
Update README.md
Yeshlen Jun 24, 2024
d6740c9
Demo 2 Arch Diagram
Yeshlen Jun 24, 2024
5926db8
Update README.md
Yeshlen Jun 24, 2024
bcb204d
Update README.md
Yeshlen Jun 24, 2024
f183a84
Update README.md
Yeshlen Jun 24, 2024
4970a61
Update README.md
Yeshlen Jun 24, 2024
0d4e277
Update README.md
Yeshlen Jun 24, 2024
0ef42b3
Update README.md
u21598012 Jun 24, 2024
d6ecf36
Merge branch 'main' into develop
Yudi-G Jun 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .codecov.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
coverage:
status:
patch:
default:
target: 60%
2 changes: 1 addition & 1 deletion .github/configs/.flake8
Original file line number Diff line number Diff line change
@@ -1,2 +1,2 @@
[flake8]
extend-ignore = W292, W291, E302, W293, E501
extend-ignore = W292, W291, E302, W293, E501, E261, E262
38 changes: 38 additions & 0 deletions .github/workflows/frontendTests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# This workflow will do a clean installation of node dependencies, cache/restore them, build the source code and run tests across different versions of node
# For more information see: https://docs.github.com/en/actions/automating-builds-and-tests/building-and-testing-nodejs

name: Node.js CI

on:
push:
pull_request:

jobs:
build:

runs-on: ubuntu-latest

strategy:
matrix:
node-version: [20.x]

steps:
- uses: actions/checkout@v4
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v3
with:
node-version: ${{ matrix.node-version }}
cache: 'npm'
cache-dependency-path: gnd-app/package-lock.json

- name: Install Chrome
run: |
sudo apt-get update
sudo apt-get install -y google-chrome-stable

- run: |
cd gnd-app
npm ci
- run: |
cd gnd-app
npm test -- --watch=false --browsers=ChromeHeadless
15 changes: 11 additions & 4 deletions .github/workflows/lint-tests.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,7 @@

- name: Super-linter
uses: super-linter/super-linter@v6.5.1 # x-release-please-version
continue-on-error: true
env:
# To report GitHub Actions status checks
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -40,13 +41,18 @@
VALIDATE_NATURAL_LANGUAGE: false
VALIDATE_PYTHON_FLAKE8: false
VALIDATE_PYTHON_MYPY: false
VALIDATE_GITLEAKS: false
VALIDATE_CSHARP: false


VALIDATE_CSS: false
VALIDATE_JSON: false
VALIDATE_HTML: false
VALIDATE_JAVASCRIPT_STANDARD: false
VALIDATE_TYPESCRIPT_STANDARD: false
VALIDATE_CHECKOV: false

# doing this to use the config
run-flake8-lint:
name: Run flake8 lint
continue-on-error: true
runs-on: ubuntu-latest

permissions:
Expand All @@ -68,5 +74,6 @@
pip install flake8

- name: Run flake8
continue-on-error: true
run: |
flake8 --config=.github/configs/.flake8
flake8 --config=.github/configs/.flake8
15 changes: 14 additions & 1 deletion .github/workflows/python-app-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,19 +44,32 @@ jobs:
uses: actions/setup-python@v3
with:
python-version: "3.10"

- name: Install NLTK and Download Resources
run: |
python -m pip install nltk
python -c "import nltk; nltk.download('stopwords')"
python -c "import nltk; nltk.download('punkt')"
- name: Install dependencies
run: |
sudo apt-get install poppler-utils
sudo apt-get install tesseract-ocr
python -m pip install --upgrade pip
pip install flake8 pytest pytest-cov
ls -a
cd backend
if [ -f requirements.txt ]; then pip install -r requirements.txt; fi

- name: Test with pytest
run: |
ls -a
cd backend/Document_parser
pytest --cov=. --cov-report=xml

- name: Integration test
run: |
ls -a
cd backend
python backend_entry_int_test.py

- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v4.0.1
Expand Down
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,7 @@
# Ignore bin and obj folders
bin/
obj/
/__pycache__/
**/__pycache__/
venv/
myenv/

3 changes: 0 additions & 3 deletions GDPR Non-compliant Detector/.gitignore

This file was deleted.

Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.

This file was deleted.

Binary file not shown.
Binary file not shown.
Binary file not shown.
14 changes: 0 additions & 14 deletions GDPR Non-compliant Detector/App.xaml

This file was deleted.

12 changes: 0 additions & 12 deletions GDPR Non-compliant Detector/App.xaml.cs

This file was deleted.

16 changes: 0 additions & 16 deletions GDPR Non-compliant Detector/AppShell.xaml

This file was deleted.

10 changes: 0 additions & 10 deletions GDPR Non-compliant Detector/AppShell.xaml.cs

This file was deleted.

76 changes: 0 additions & 76 deletions GDPR Non-compliant Detector/GDPR Non-compliant Detector.csproj

This file was deleted.

This file was deleted.

27 changes: 0 additions & 27 deletions GDPR Non-compliant Detector/GDPR Non-compliant Detector.sln

This file was deleted.

Loading
Loading