Multiple APIs in a single app

[miqueiasfernandesaka]

Is your feature request related to a problem? Please describe.

I would like suggest a way to join Architect backend managing with microservices pattern. Providing a way to manage many API Gateway's with it Lambda Functions. Including set up environment variables to each Lambda/API Gateway, resources, etc.

Describe the solution you'd like
I think something looks like "AWS Amplify Framework":

arc add api

And after this, the "wizard" settings of it resources access.

Describe alternatives you've considered
I've search about some plugins, but nothing looks like this.

[ryanblock]

Curious to hear more about this. Why would one API not suffice? Why would multiple Arc apps not suffice?

[ryanblock]

Following up here with a lengthy conversation we had in the Architect Discord starting on 2021/11/06:

Chudbreeder

Hey all, Can you create more then one websocket endpoint with an arc project?

ryan

depends on what you mean by endpoint. @ws provisions a single websocket api, in which you can define an arbitrary number of websocket events; there are by default 3 (connect, default (like a catchall), disconnect), but you can define as many custom events as you like
similar to the arc model for @http, which provisions a single http api with an arbitrary number of http routes. one api for @http, one api for @ws; any additional apis would have to be built with a plugin

Chudbreeder

ryan thanks, yeah I am looking to end up with multiple wss://domains. This isn't mandatory and can be managed with one, just curious if it is possible to separate concerns if I was able to have two separate frontend apps (say an admin and a public app) connect to two totally isolated domains.

ryan

sounds like two separate apps to me

Chudbreeder

yeah two frontend apps for sure, but was hoping to keep one serverless backend for them both. would be nice to benefit from the shared code in arc across these two websockets endpoints as well as not needing a whole new stack for each one. Really been thinking about backends as just a collection of functions lately. but it is what it is.

ryan

Ok well, knowing how arc works, how would you expect multiple APIs to work in an arc file? How would you handle file path conflicts if each had a get / handler?

Chudbreeder

easiest way , off the top of my head, is how http does it by allowing me to specify a source file. though it would be nice if they could just be named and placed into folders based on that.

ryan

That would force users to use the verbose format, which right now is an opt-in. Also still fuzzy to me how one would even expect to define a second API
What would the minimal arc for that look like?

Chudbreeder

Sorry for the delay. So just to make sure we aren't mixing things up I am still talking about just multiple websocket endpoints, a full on API. but I was thinking it could look something like this

@app
myapp

@ws
public # default to (ws/public/(connect,disconnect,default))
admin # (ws/admin/(connect,disconnect,default))
super-secret # optional more verbose way
  root whatever/ws/dir/you/want

ryan

Hmm, that may misunderstand how APIGv2 works? https://docs.aws.amazon.com/apigateway/latest/developerguide/websocket-api-develop-routes.html
That said you can achieve the same effect by adding with to those private routes

Chudbreeder

Sorry not following "adding with to those private routes", but if there is a way to achieve 2 ws://endpoints with 1 arc project that was all I was looking for.
I do get how its working and maybe this just overcomplicated arc's goals. I was looking at it as an abstraction of complex CF/SAM. I think it is fair to say with cloudformation or SAM I could declare multiple AWS::ApiGatewayV2::Api's and connect them to different AWS::Serverless::Function's.
As I said before this isn't really a deal breaker, another endpoint is just a small measure of security, and we could certainly architect things to just use one. My team is weighing Arc vs just writing SAM and the pushback I am getting is "well what if we needed to..." kinda stuff. I guess the answer to that can always be macros/plugins.

ryan

typoed on my phone
i meant to say you can achieve the same effect by adding the private routes to your public api (behind auth)
but per that link and how you described it in arc format above, i ws/public/$routes + ws/private/$routes implies the same API, and that isn't how API Gateway v2 works. to your point, yes, you can definitely declare multiple APIs in the same SAM document. and you do not need arc to manage that, either. my suggestion is still that for the cleanest boundaries in your project: multiple arc projects
for shared code, i've really been getting a lot from npm i https://url/to/git/repo.git, flexible and nicer imo for private shared code

Chudbreeder

Yup i am getting that, again for me i didn't want a boundary 🙂 I wanted 2 ws endpoints within the same shared code space.
to your first point though adding private routes to the public api. When talking about WS you mean using a single connect/disconnect/default but using socket auth to determine who can listen/send what events?
and yeah we have looking to like a million ways to share code haha :). The downside to repos is when you're working on shared code you need to swap over + npm link or get into submodules, which is a whole pit of its own.

ryan

yeah i am not a fan of submodules, for sure

Chudbreeder

Dude its a hot mess

ryan

the worst
When talking about WS you mean using a single connect/disconnect/default but using socket auth to determine who can listen/send what events?

Chudbreeder

yeah that was the original plan 🙂 but then i got thinking, which is alway dangerous, maybe i could separate it for a slight security bonus

ryan

maybe, but then you're just playing the security through obscurity game

Chudbreeder

public folks would ALL know the admin ws endpoint. though honestly it wouldnt be that hard to find in our situation as it s really more of 2 public WS enpoints, just delivered to different applications.

ryan

you still need to auth access

Chudbreeder

yeah exactly
and the benefits are almost none performance wise with a AWS ws anyway.

ryan

another thing you can do if you want to share code is put multiple arc projects in the same repo

Chudbreeder

oh that's interesting. though I imagine IAM sharing across them might be a bit tricky?

ryan

multiple arc projects in the same repo is technically no different than multiple arc projects in multiple repos, doesn't influence iam one way or another. just may make sharing certain things easier
but yes, with multiple projects, if you want to share resources across projects, you may have to do a bit of iam work

Chudbreeder

gotcha. an a slightly random tangent, its been maybe 6 or 8 months since i looked into this. There is still not good way to authenticate a websocket connection on connect, right? I think at the time it was cookies, hijacking the Sec-WebSocket-Protocol header or using basic auth like wss://[token]@endpoint and expecting user to be a token.
this also maybe something you havent cared enough about to look into 🙂 but if so its been something thats always annoyed me, needing to connect to the socket then send a message like auth, just extra bytes 🙂

ryan

in begin we issue a nonce with the html response valid for like 5 seconds i think, and the ws connection passes the token and then we consider that client authorized to that account

Chudbreeder

passes a token as a query param?

ryan

yes
this is the entirety of how begin.com authorizes websockets
...
i wrote that like 2 years ago and haven't updated it since. been super solid.

Chudbreeder

I gotcha, yeah a quickly expiring token is not a bad idea, limits the concern of url sniffing with query params. I will have to think about that a bit for us, as we would need an endpoint to grad that token. so need to weigh the cost of that lambda request vs just another socket message. Good stuff to think about though

ryan

there's no token issuance endpoint
the nonce is included with your authorized html responses
just a simple jwe

Chudbreeder

yup gotcha. expectation here is that the user will instantly connect to the socket. Just thinking in our cases it maybe a while since that happenes after they get into the system

ryan

meaning websocket-only? no progressive enhancement? and what if the ws connection breaks?

Chudbreeder

oh for a second i thought auth-token was in arc functions not shared. so thats a custom thing you all have created. I was just saying in our setup a user could authenticate and then 5 minutes later reach the point where they need to connect to the socket. We also have to connect/disconnect often as the users will be in the app for a crazy long time. Like anywhere from 2 to 12 hours a session. And we dont wanna take on all that connection time WS cost. so we will need to be dropping them and pulling them back occasionally.

ryan

yeah, it's just a jwe parser, nothing special.
what is the app, out of curiosity?

Chudbreeder

We do work with Twitch, building interactive apps that sit along streams. Our main application allows viewers to manipulate video games as streamers play them. So a twitch stream or marathon can go like heck even 24 hours sometimes. and people will sit in there with the app open and just troll people all night haha

ryan

ah

Chudbreeder

yeah, it could get very expensive quick

ryan

def

Chudbreeder

Its so silly thats browsers dont send headers with websocket connections :/ such a painful oversight.
Anyway, thanks for walking through all this, we will stick with single WS for now and I am gonna keep pushing arc for us haha

ryan

no sweat, thanks for telling us more about the use case, always helpful to hear