Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

B/google api #113

Merged
merged 52 commits into from
Oct 28, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
c372791
initial commit. Added files and module references.
Whatsuup Oct 8, 2023
c5f5bf5
Changed image api interface
Whatsuup Oct 8, 2023
0f65d99
Error fixes
Whatsuup Oct 8, 2023
8284a3c
added file and function structure
Whatsuup Oct 8, 2023
9c72b36
async trait fix
Whatsuup Oct 10, 2023
cc1edff
manual rollback of interface name
Whatsuup Oct 10, 2023
354e813
interface update and docs
Whatsuup Oct 10, 2023
8dcb536
constructor implementations and fixes
Whatsuup Oct 10, 2023
d7bb344
must use change and push
Whatsuup Oct 10, 2023
9981f7b
added project id to info struct und .env
Whatsuup Oct 10, 2023
ae0350f
added test data
Whatsuup Oct 24, 2023
b558757
added google-auth dependency
Whatsuup Oct 24, 2023
808a3dc
.env change
Whatsuup Oct 24, 2023
f94a9a8
implemented api request + added basic test
Whatsuup Oct 24, 2023
77e6279
implemented image_validation
Whatsuup Oct 24, 2023
4643272
removed unnecessary structs. Removed json parser as it is no longer n…
Whatsuup Oct 24, 2023
7d98c66
Added more error types
Whatsuup Oct 24, 2023
e14de5f
implemented google_api_handler.rs
Whatsuup Oct 24, 2023
caec8e0
formatting and error fixes
Whatsuup Oct 24, 2023
0eaaf69
added image_evaluation docs
Whatsuup Oct 25, 2023
0448a73
added api_request docs
Whatsuup Oct 25, 2023
49d2a72
added google_api_handler docs
Whatsuup Oct 25, 2023
93db336
added module docs
Whatsuup Oct 25, 2023
1d0020d
fmt, clippy
Whatsuup Oct 25, 2023
c8c4a10
added image_evaluation tests
Whatsuup Oct 25, 2023
d5bf087
added api_request tests
Whatsuup Oct 25, 2023
ab7f8e0
working on google_api_handler tests
Whatsuup Oct 25, 2023
3c45b0f
working on google_api_handler tests
Whatsuup Oct 25, 2023
fd8de53
some requested changes
Whatsuup Oct 25, 2023
25813a9
secret file path
worldofjoni Oct 25, 2023
a4a0348
const -> static
Whatsuup Oct 26, 2023
6c2420d
removed io operations out of component
Whatsuup Oct 26, 2023
85ee56f
Changed token lifetime to 30sec. Rollback if tokens can be reused.
Whatsuup Oct 26, 2023
d0b1c78
fixed test issues
Whatsuup Oct 26, 2023
eabeace
fixed test issues 2
Whatsuup Oct 26, 2023
47fe4a4
push before merge with backend
Whatsuup Oct 26, 2023
bbf666b
Merge branch 'backend' into b/google_api
Whatsuup Oct 26, 2023
dd1d483
cargo lock
Whatsuup Oct 26, 2023
59aa861
database fixes
Whatsuup Oct 26, 2023
f1dce2c
fixed request test
Whatsuup Oct 26, 2023
c003e02
fixed handler test
Whatsuup Oct 26, 2023
4e53c2d
fmt
Whatsuup Oct 26, 2023
027e423
fixed docs
Whatsuup Oct 26, 2023
be737b3
added google_api initialisation
Whatsuup Oct 26, 2023
9d22e07
correct env var setting
worldofjoni Oct 27, 2023
f29edfc
set google project id var
worldofjoni Oct 27, 2023
190d7fd
quote env var
worldofjoni Oct 27, 2023
ce7cc1f
better quotation
worldofjoni Oct 27, 2023
8ab9764
still different quotes
worldofjoni Oct 27, 2023
032bdc7
base64 json
worldofjoni Oct 27, 2023
8d67ce4
flush file
worldofjoni Oct 27, 2023
4061256
fixed error messages
worldofjoni Oct 28, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions .github/workflows/rust.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ env:
SMTP_USERNAME: ${{ vars.SMTP_USERNAME }}
SMTP_PASSWORD: ${{ secrets.SMTP_PASSWORD }}
ADMIN_EMAIL: ${{ vars.ADMIN_EMAIL }}
GOOGLE_CLIENT_JSON_BASE64: ${{ secrets.GOOGLE_CLIENT_JSON_BASE64 }}
GOOGLE_PROJECT_ID: ${{ vars.GOOGLE_PROJECT_ID }}


jobs:
Expand Down Expand Up @@ -103,6 +105,11 @@ jobs:
run: |
cargo install sqlx-cli --no-default-features --features native-tls,postgres
cargo sqlx database setup
- name: Prepare secret file
run: |
echo "$GOOGLE_CLIENT_JSON_BASE64" >> secret.base64
base64 --decode secret.base64 > secret.json
echo "SERVICE_ACCOUNT_JSON=$(realpath secret.json)" >> $GITHUB_ENV
- name: Run tests
run: cargo test --verbose

Expand Down Expand Up @@ -135,6 +142,11 @@ jobs:
run: |
cargo install sqlx-cli --no-default-features --features native-tls,postgres
cargo sqlx database setup
- name: Prepare secret file
run: |
echo "$GOOGLE_CLIENT_JSON_BASE64" >> secret.base64
base64 --decode secret.base64 > secret.json
echo "SERVICE_ACCOUNT_JSON=$(realpath secret.json)" >> $GITHUB_ENV
- name: Generate code coverage
run: |
cargo tarpaulin --all-targets --skip-clean --workspace --timeout 120 --out xml
Expand Down
9 changes: 8 additions & 1 deletion backend/.env.example
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
# copy this file to .env and set your own values
# optional environment variabls (when default values are available) are commented out
# optional environment variables (when default values are available) are commented out

# --- connection to database ---
DATABASE_URL=postgres://postgres_user@localhost/mensa_app
Expand All @@ -15,6 +15,13 @@ ADMIN_EMAIL=
#FULL_PARSE_SCHEDULE=
#UPDATE_PARSE_SCHEDULE=

# --- google api ---
Whatsuup marked this conversation as resolved.
Show resolved Hide resolved
SERVICE_ACCOUNT_JSON=path_to_json
GOOGLE_PROJECT_ID=PROJECT_ID
Whatsuup marked this conversation as resolved.
Show resolved Hide resolved
# Order: adult,spoof,medical,violence,racy
# Mapping: 0=Unknown,1=VeryUnlikely,2=Unlikely,3=Possible,4=Likely,5=VeryLikely
IMAGE_ACCEPTANCE_VALUES=0,0,0,0,0

# --- mensa parser ---
#CLIENT_TIMEOUT=
#MENSA_BASE_URL=https://www.sw-ka.de/de/hochschulgastronomie/speiseplan/
Expand Down
5 changes: 4 additions & 1 deletion backend/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,7 @@ target/

# test coverage
lcov.info
*.profraw
*.profraw

# private test data
src/layer/data/image_validation/test/test-client.json
135 changes: 131 additions & 4 deletions backend/Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 2 additions & 0 deletions backend/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ lazy_static = "1.4.0"
colored = "2.0.4"
image = "0.24.0"
tower-http = { version = "0.4.4", features = ["fs"] }
serde = "1.0.188"
google-jwt-auth = "0.0.2"
Whatsuup marked this conversation as resolved.
Show resolved Hide resolved
hyper = { version = "0.14" }
mime = "0.3.17"
hmac = "0.12.1"
Expand Down
5 changes: 5 additions & 0 deletions backend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ Alternatively, you can also just build yourself and run the binary. See [below](
### Environment variables
To pass configuration options to the backend application environment variables are used.
The following options are available:

| Name | Description | Default / Required |
| ---------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
| `LOG_CONFIG` | Configure which messages are logged. For more information on the used syntax, see [here](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/filter/struct.EnvFilter.html#directives). You may want to set this to `warn,mensa_app_backend=trace` to enable all messages we produce. | `warn,mensa_app_backend=info` |
Expand All @@ -50,6 +51,9 @@ The following options are available:
| `SMTP_PASSWORD` | Password to access the SMTP server. | required |
| `FULL_PARSE_SCHEDULE` | [Cron](https://cron.help/)-**like** schedule for when to run a full parsing to get the meal plans for the next three weeks. **A sixth, first _digit_ specifying the seconds is necessary!** | `0 0 2 * * *` |
| `UPDATE_PARSE_SCHEDULE` | Schedule for when to update the mealplan for the current day. Same format as `FULL_PARSE_SCHEDULE` | `0 */15 10-15 * * *` |
| `SERVICE_ACCOUNT_JSON` | Path to the service account json that can be downloaded during the key generation for the service account in the google console. | required |
| `GOOGLE_PROJECT_ID` | The `project_id` can be copied from the google (console). It should be the same project from which the account comes. | required |
| `IMAGE_ACCEPTANCE_VALUES` | Comma separated list of five integers from 0 to 5. These values determine which images will be accepted and which not. More info about levels and categories are in the documentation. | `1,1,1,1,1` |
| `CLIENT_TIMEOUT` | Timeout in ms for requesting the webpage containing the meal plan. | `6000` |
| `MENSA_BASE_URL` | Base URL where meal plans are requested. It excludes the canteens name, which will be appended later on. | `https://www.sw-ka.de/de/hochschulgastronomie/speiseplan/` |
| `CANTEENS` | Comma (`,`) separated list of canteens which should be requested and parsed. These are appended on the `MENSA_BASE_URL`. | `mensa_adenauerring,mensa_gottesaue,mensa_moltke,mensa_x1moltkestrasse,mensa_erzberger,mensa_tiefenbronner,mensa_holzgarten` |
Expand All @@ -60,6 +64,7 @@ The following options are available:
| `IMAGE_DIR` | Path to folder where images will be stored. Backslashes (`\`) need to be escaped (`\\`). | required |
| `MAX_IMAGE_WIDTH` and `MAX_IMAGE_HEIGHT` | Maximum width and height stored for stored images. Uploaded images will be scaled accordingly. | `1920` and `1080` |


## Building the backend

### Run the backend
Expand Down
2 changes: 1 addition & 1 deletion backend/src/interface/api_command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -69,5 +69,5 @@ pub enum CommandError {
ImageStorageError(#[from] image_storage::ImageError),
/// Error while image verification.
#[error("Image could not be verified: {0}")]
ImageValidationError(#[from] image_validation::ImageError),
ImageValidationError(#[from] image_validation::ImageValidationError),
}
Loading
Loading