Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/meditor 937 upgrade and simplify meditor #69

Merged
merged 33 commits into from
Jan 3, 2025
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
3b1176f
feature: move legacy API endpoints into app
Nov 22, 2024
7321ce0
feature: remove legacy API from proxy
Nov 22, 2024
1d5a999
feature: upgrade NGINX to 1.27.2
Nov 22, 2024
6e80b53
chore: cleanup docker-compose
Nov 22, 2024
c6d7877
feature: upgrade app to latest Node/Next versions
Nov 26, 2024
52d1c48
feature: install and configure next-auth
Dec 5, 2024
942a581
feature: switch session from legacy API to NextAuth
Dec 9, 2024
6c9474c
chore: cleanup Next build errors
Dec 9, 2024
bb101d4
feature: move all auth endpoints to NextJS and ensure it's working wi…
Dec 11, 2024
dc441d8
chore: remove legacy-api from the project
Dec 11, 2024
1f3fc31
feature: refactor API to use authentication and assertions
Dec 11, 2024
3c6babd
chore: refactor app to simplify error handling
Dec 11, 2024
ba48d7d
feature: centralize db and queue connections
Dec 12, 2024
5ecb56e
feature: migrate remaining HttpExceptions
Dec 17, 2024
7812f0a
feature: fix state badges
Dec 17, 2024
8b34c94
feature: use getSession hook instead of passing user around
Dec 17, 2024
11651f8
feature: use the api error message instead of generic errors
Dec 17, 2024
f0b2cb4
chore: remove stray console log
Dec 18, 2024
6fc25c7
feature: create user account on sign in to support email notifications
Dec 18, 2024
73660e4
feature: update tests to use new versions of Jest and TS
Dec 18, 2024
c54689b
chore: fix build
Dec 18, 2024
690f096
feature: switch to Caddy from NGINX
Dec 18, 2024
4b7d817
chore: nextauth secret property misnamed
Dec 19, 2024
1ac04df
chore: remove buildspec.yml as we aren't using CodeBuild
Dec 19, 2024
4549552
chore: addressing PR comments
Dec 19, 2024
97abd90
Merge branch 'main' into feature/MEDITOR-937-upgrade-and-simplify-med…
Dec 19, 2024
faa5fa9
chore: unused tests as we're using a 3rd party library
Dec 19, 2024
67003d2
chore: update CI workflow to use Node 22
Dec 20, 2024
cae9cae
feature: address PR feedback
Jan 2, 2025
0c6349f
feature: consolidate userCanAccessModel checks
Jan 2, 2025
24d7c0f
chore: use assert
Jan 2, 2025
6945a5c
chore: use assert
Jan 2, 2025
6f2d50b
feature: only bind hostname if running in docker
Jan 3, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .env.example
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mEditor can now be run without Docker.

.env updates include NextAuth ENV vars and local URLs

Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
NODE_ENV=development

NEXTAUTH_URL=http://localhost/meditor/api/auth
NEXTAUTH_URL_INTERNAL=http://meditor_app:3000/meditor/api/auth
AUTH_SECRET=ThisSecretEncryptsTheAuthJWTToken

# include in your .env to disable email notifications, if disabled you will see the email in the logs instead of an actual
# email being sent
DISABLE_EMAIL_NOTIFICATIONS=true
Expand All @@ -8,6 +12,11 @@ AUTH_HOST=urs.earthdata.nasa.gov
AUTH_CLIENT_ID=ask-someone-for-this
AUTH_CLIENT_SECRET=ask-someone-for-this

DB_CONNECTION=mongo # will be used to switch between database providers
DB_HOST=localhost
DB_PORT=27017
DB_DATABASE=meditor

MAIL_HOST=ndc-relay.ndc.nasa.gov
# Uncomment if you are planning to use Cognito for login
# follow the steps for creating a client at: https://www.npmjs.com/package/passport-cognito-oauth2
Expand Down
7 changes: 0 additions & 7 deletions buildspec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,13 +25,6 @@ batch:
REPOSITORY_NAME: meditor/meditor_docs
PACKAGE_PATH: packages/docs
ignore-failure: false
- identifier: legacy_api
debug-session: false
env:
variables:
REPOSITORY_NAME: meditor/meditor_server
PACKAGE_PATH: packages/legacy-api
ignore-failure: false
- identifier: notebook_viewer
debug-session: false
env:
Expand Down
77 changes: 0 additions & 77 deletions docker-compose.override.yml
Copy link
Collaborator Author

@joncarlson joncarlson Dec 11, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docker has introduced "develop" parameters that only take effect when developing locally, making a separate override compose unnecessary

This file was deleted.

66 changes: 55 additions & 11 deletions docker-compose.yml
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most of this is merging the two docker compose files together, noting where there are new items or updates

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll need your help to get this running. I must have something misconfigured, but a docker system prune + rebuild and a few minutes of searching around yielded nothing.

Original file line number Diff line number Diff line change
@@ -1,13 +1,21 @@
services:
notebookviewer:
depends_on:
- legacy-api
container_name: meditor_notebookviewer
build: packages/notebook-viewer
environment:
- FLASK_ENV=development
env_file:
- .env
volumes:
- ./packages/notebook-viewer:/usr/src/app

proxy:
container_name: meditor_proxy
build: packages/proxy
restart: on-failure
ports:
- "127.0.0.1:80:8080"
depends_on:
- legacy-api
- nats
- app
- database
Expand All @@ -16,36 +24,72 @@ services:
notifier:
depends_on:
- nats
container_name: meditor_notifier
build: packages/notifier
restart: on-failure
env_file:
- .env
volumes:
- ./packages/notifier:/usr/src/app
- /usr/src/app/node_modules

app:
depends_on:
- legacy-api
container_name: meditor_app
build: packages/app
restart: on-failure
env_file:
- .env
ports:
- "127.0.0.1:4000:4000"
develop:
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

New develop section for the app, makes volume mounting unnecessary. If you do docker-compose up --watch or docker compose watch these take effect

watch:
- action: sync
path: ./packages/app
target: /usr/src/app
ignore:
- node_modules/
- .next/
- action: rebuild
path: package.json

docs:
container_name: meditor_docs
build: packages/docs
env_file:
- .env
ports:
- "127.0.0.1:3030:3030"
depends_on:
- legacy-api
- app

legacy-api:
depends_on:
- database
- nats

database:
image: ${REGISTRY}mongo:4.1.4
container_name: meditor_database
volumes:
- ./mongo-data:/data/db
ports:
- "127.0.0.1:27018:27017"

nats:
image: ${REGISTRY}nats-streaming:0.15.1
container_name: meditor_nats
volumes:
- ./nats-data:/nats/data
ports:
- "127.0.0.1:4222:4222"
- "127.0.0.1:8222:8222"

monitor:
image: ${REGISTRY}portainer/portainer:1.20.0
container_name: meditor_monitor
command: -H unix:///var/run/docker.sock
volumes:
- /var/run/docker.sock:/var/run/docker.sock
- monitor-data:/data

agent:
image: ${REGISTRY}portainer/agent:1.1.2
container_name: meditor_agent
environment:
AGENT_CLUSTER_ADDR: tasks.agent
volumes:
Expand Down
3 changes: 3 additions & 0 deletions packages/app/.env.development
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
DB_NAME=meditor
HOST='http://localhost:3000/meditor'
NEXT_PUBLIC_API_BASE_PATH='/meditor/api'
NEXT_PUBLIC_IMAGE_UPLOAD_URL='https://lb.gesdisc.eosdis.nasa.gov/images/upload'
NEXT_PUBLIC_NOTEBOOKVIEWER_URL='/meditor/notebookviewer/'
NEXTAUTH_URL=http://localhost:3000/meditor/api/auth
NEXTAUTH_URL_INTERNAL=http://localhost:3000/meditor/api/auth
3 changes: 3 additions & 0 deletions packages/app/.env.production
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
DB_NAME=meditor
HOST='https://lb.gesdisc.eosdis.nasa.gov/meditor'
NEXT_PUBLIC_API_BASE_PATH='/meditor/api'
NEXT_PUBLIC_IMAGE_UPLOAD_URL='https://lb.gesdisc.eosdis.nasa.gov/images/upload'
NEXT_PUBLIC_NOTEBOOKVIEWER_URL='/meditor/notebookviewer/'
NEXTAUTH_URL=https://lb.gesdisc.eosdis.nasa.gov/meditor/api/auth
NEXTAUTH_URL_INTERNAL=https://meditor_app:3000/meditor/api/auth
5 changes: 4 additions & 1 deletion packages/app/.eslintrc.json
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Auto-updated from ESLint upgrade

Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
{
"extends": "next/core-web-vitals"
"extends": "next/core-web-vitals",
"rules": {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding a couple rules to ignore here, both personal bias so please share any thoughts:

  • Turning off exhaustive dependency warnings from "useEffects". We intentionally don't include exhaustive dependencies (see useEffects for first render).
  • Making links to pages a warning instead of an error. The only real place we use this is the breadcrumb to return to the homepage. For some reason, that route does not retrigger getServerSideProps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the flat file configuration was deprecated in ESLint 9. Here's what works for me:

  • change the file name to eslint.config.js
  • use the config (I left some stuff commented out that Next recommended, since we might not want it)
import { FlatCompat } from '@eslint/eslintrc'

const compat = new FlatCompat({
    // import.meta.dirname is available after Node.js v20.11.0
    baseDirectory: import.meta.dirname,
})

const eslintConfig = [
    ...compat.config({
        // extends: ['next/core-web-vitals', 'next/typescript'],
        extends: ['next/core-web-vitals'],
    }),
    {
        rules: {
            'react-hooks/exhaustive-deps': 'off',
            '@next/next/no-html-link-for-pages': 1,
        },
    },
]

export default eslintConfig

"react-hooks/exhaustive-deps": "off"
}
}
2 changes: 1 addition & 1 deletion packages/app/Dockerfile
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
ARG REGISTRY
FROM ${REGISTRY}node:16.11.1
FROM ${REGISTRY}node:22.11.0

ENV NPM_CONFIG_LOGLEVEL warn
ARG node_env
Expand Down
15 changes: 15 additions & 0 deletions packages/app/auth/db.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,21 @@ class UsersDb {
})
.toArray()
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These two new queries support NextAuth. Creating a user account on sign in and retrieving the mEditor user (to get at the user's roles)

async getMeditorUserByUid(uid: string): Promise<Document> {
const user = await this.#db
// the mEditor user record is a generic, dynamic document, like any other model's document (see the "Users" model)
.collection<Document>(this.#USERS_COLLECTION)
.findOne(
{
id: uid,
'x-meditor.deletedOn': { $exists: false }, // filter out deleted users
},
{ sort: { 'x-meditor.modifiedOn': -1 } }
)

return makeSafeObjectIDs(user)
}
}

const db = new UsersDb()
Expand Down
21 changes: 0 additions & 21 deletions packages/app/auth/http.ts
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused file

This file was deleted.

Loading
Loading