Skip to content

Initial Task List and Scope

Jump to bottom
Alex Romanova edited this page Feb 13, 2023 · 48 revisions

Overview

The following is a high-level list of tasks that need to be considered and implemented at some point (they are not prioritized). The goal of this list is to help define the scope of the tasks, technologies, and patterns we need to use.

This list needs to be turned into Issues and, very quickly, this list should become obsolete.

Tasks

Category Interested Assigned
Front end Eakam, Mario, Tymyr Eakam
Back end Stef, Taimoor, Eakam, Tymur Taimoor
Data Taimoor, Chen-Yuan Chen-Yuan
Authentication Stefan Frunza Stefan
Notifications Taimoor Taimoor
DNS Alex, Won Won
Certificates Denes, Won Denes
Development Alex, Won Won
Testing Stef, Eakam Eakam
Deployment Stef Stef, Dave
Documentation Alex Alex
Queue Denes

Challenges

Category Due Challenge Status
Certificates Milestone 0.2 Should be able to issue (staging) a cert with manual control and no DNS verification DONE
Authentication Milestone 0.2 Setting up docker container for test SAML idp, see what I can do with SAMLIFY.
Notifications Milestone 0.2 Ensure that we are able to successfully send test notifications
Data Milestone 0.2 Have seeding work for tables other than User table.

Web

  • We'll need some page designs very quickly. Start looking at existing DNS/Cert UIs (e.g., AWS, CloudFlare, Azure, Google Cloud, Namecheap, etc). How do they solve this UX? Learn from them, and let's not waste time inventing new UI.
  • Use the Remix Blue Stack as a basis for our initial setup. It does most of what we need, and we'll swap out or remove some pieces later on
  • Setup Chakra UI with Remix. See also the Chakra UI Remix Example
  • Use ChakraUI for all our components vs. creating our own
  • Use colorScheme='red' for ChakraUI components to match Seneca's red colour theme, see https://chakra-ui.com/docs/components/button/usage#usage
  • Landing page with a way to login
  • All other pages should require a user to be authenticated
  • User Dashboard page with a way to create or manage subdomains and certificates
    • User can create new subdomains (A, AAAA, CNAME)
    • User can view existing subdomains
    • User can update existing subdomains
    • User can delete existing subdomains
    • User can renew a subdomain for an additional period of time (e.g., add another 6 months)
    • User can create a wildcard certificate
    • User can view wildcard certificate and private key
    • User can get instructions on how to use the certificate
    • User can revoke wildcard certificate
    • User can renew wildcard certificate
  • Admin Dashboard with overall stats (e.g., how many users, subdomains, certs) and a way to query/filter/view/modify all user data
  • Include link to Seneca's Acceptable Use Policy (e.g., can't use submdomains/certs for commercial use, etc)

Server

  • Consider swapping Remix's Express server for Satellite. This may require some tweaks to Satellite to make it usable outside of Telescope. We might also decide to port a bunch of best-practices from Satellite over to the server.ts code used in Remix instead (i.e., to not bother pulling in unnecessary dependencies like elastic, redis, etc).
  • Use Satellite's logger (Pino)
  • Use server-side cookie based sessions to simplify scaling in production. See https://remix.run/docs/en/v1/utils/sessions
  • Include healthcheck route, which can be used in Docker to figure out if the server is OK
  • Figure out the best solution to doing background tasks. The system needs to send notifications and perform other background tasks like removing domains/certs after they expire. This shouldn't happen on the main thread. Many systems that do this (Telescope for example) use a queue and workers based on BullMQ and Redis.

Data

  • Create an initial Prisma schema for MySQL. See also https://www.prisma.io/docs/concepts/components/prisma-schema
    • User data from SAML response (username, email, name, etc)
    • Subdomains for a user (username, subdomain, description, ports list, course, date created, when it expires)
    • Certificates for a user (username, subject, certificate in PEM format, private key in PEM format encrypted, when it's valid from/to)
  • Figure out how to use MySQL via docker compose in development
  • Figure out the best way to encrypt private keys for secure storage in the database
  • Domain records should include a Description of how it is being used, Port numbers, Course code (if any)

Auth

Notifications

  • Use nodemailer to send email notifications to users. For example, when a domain or certificate is going to expire and should be renewed.
  • Configure nodemailer to use an OAuth-based email account on staging and production

DNS

  • Enforce a configurable (i.e., env var) limit on the number of subdomains each user can create. Write tests to make sure this works.
  • Decide if we should detect and ban "bad words" in domain names (e.g., use something like https://github.com/web-mech/badwords)
  • Configure AWS JS SDK v3 to use our IAM User's credentials and Region in dev/staging/production
  • Configure the Route53 client to use our Hosted Zone ID (i.e., get it from the environment)
  • Create a "dns" module that we can use on the server-side (e.g., app/lib/dns.server.ts or something) to work with DNS records in various parts of the app. Specifically, it will need to:
    • Use the ChangeResourceRecordSetsCommand to create, update, or delete resource records in the hosted zone.
    • Write unit tests for creating, updating, and deleting resource records against a mock route53 container
    • Use the GetChangeCommand to determine when DNS records have been propagated (i.e., are ready to be used vs just requested).
    • Write unit test for querying the state of a record with AWS against a mock route53 container
  • Domains should expire after a configurable (i.e., env var) amount of time. Default to 6 months.
  • Send an email before a domain is going to expire so users can log in and renew. Amount of time should be configurable (i.e., env var)
  • Deal with, and write tests for, various rate limiting issues with Route53

Certs

  • Provide detailed info page showing how to use certificate we generate (i.e., DNS Challenge)
  • Provide detailed info page showing how to setup an HTTP Challenge to auto-renew a certificate we don't generate, but uses one of our domains
  • Provide a way to generate a wildcard cert for the user's top-level subdomain. The DNS for the subdomain must be active in Route53 first
  • Provide a way to get the existing cert + private key after it has been created (e.g., 1 month later when they have forgotten)
  • Provide a way to renew an existing cert
  • Provide a way to revoke an existing cert
  • Send notifications to users before certs will expire. Time before expiry should be configurable (i.e., env var)
  • Deal with, and write tests for, various rate limiting issues with Let's Encrypt

Development

Testing

  • Adopt an aggressive testing philosophy. When you review new code, always ask "how will we test this?" and "where are the tests?" If we don't have tests, we can't trust the code, and should be skeptical. NOTE: tests don't always need to happen first or at the same time; but there needs to be a plan to add them.
  • Use Jest for unit testing. See also https://sergiodxa.com/articles/test-remix-loaders-and-actions for Remix specific bits on loaders, etc.
  • Use Playwright for e2e testing
  • Use GitHub Actions for CI to check eslint, prettier, unit tests, e2e tests, etc.
  • Add test coverage info so we know how we're doing

Deploy

  • Adopt a deploy-first mentality: if it's hard to deploy, we won't do it, therefore we won't be able to test it for real. Focus on making things easy to deploy.
  • Dockerize the Remix Web App/Server
  • Use GitHub Actions to publish image on every push to main and tag. Could use CDOT's private registry or some public one?
  • Use GitHub Actions for CD to deploy to staging server
  • Use docker swarm for staging and production deployments
  • Use webhooks (e.g., https://github.com/adnanh/webhook) to trigger updates on staging and production from CD
  • Explore use of SolarWinds APM for integration with Seneca monitoring infrastructure. See https://github.com/solarwindscloud/solarwinds-bindings-node. Need to confirm with ITS if they want.
  • Create top-level, static web page for senecastudent.ca that explains what all subdomains on the server are for, give contact info, etc. Could redirect to this page.
  • Figure out a name for the service (i.e., {name}.senecacollege.ca prod and {name}-dev.senecacollege.ca staging). Starchart is our technology name, but we need a product/service name that the Seneca community will use.
  • Deal with log rotation so we don't fill-up the staging/prod disk.
  • Don't log more than necessary in production (e.g., errors)
  • Figure out how we're going to do HTTPS in staging/production: does Seneca handle this, or do we need to?

Docs

  • Adopt a "docs matter" philosophy. We want this project to succeed long term, even when all of the current devs have moved on. To do that, we have to capture our knowledge in docs. When people make big changes, always ask "do we need to update or add docs for this?" and "where are the docs?" Docs can come later vs. at the same time, but it needs to be included in our planning.
  • Start to organize our documentation in the Wiki so it's going to be easy for new devs to join us later on (proper links between docs, hierarchy)
  • Create a CONTRIBUTING.md to help new devs get involved. Keep it updated
  • Create and maintain our README.md to help people understand what the project is, now to use it, etc.
  • Create docs for deploying and maintaining the servers in staging and production

Home

Winter 2023

Telescope

Planning

Technologies

Developer documentation

Don't like this sidebar? Let me know what to improve on!

Clone this wiki locally