@blurfx @krapie @window9u Could you share your thoughts on migration and any considerations?

@devleejb

I completely agree that migrating the CodePair backend to Go aligns well with Yorkie contributors' expertise, making collaboration more natural. If this proposal moves forward, I’m excited to join the migration team and contribute to building a robust Go-based backend!

Once a new repository for the Go-based backend is created, I suggest we discuss key decisions like:

1. HTTP Framework: Should we use Gin, Echo, or native middleware?
2. MongoDB Integration: Options include the official mongo-go-driver or tools like mgm for simplifying operations.

@window9u

Thank you for your answer!

1. HTTP Framework: Should we use Gin, Echo, or native middleware?

I understand that yorkie uses native middleware. Is that correct? If so, we can use it as a valuable reference.

2. MongoDB Integration: Options include the official mongo-go-driver or tools like mgm for simplifying operations.

yorkie also uses mongo-go-driver. Additionally, we reference the database architecture in yorkie to support other options (e.g., MySQL, and more).

I think fiber is a good option for the http framework. It's very fast and has some nice features.

@devleejb

HTTP Framework

As far as I know, yes, Yorkie uses native middleware. However, this decision likely stems from its need to handle more complex logic, such as RPC connections and health checks. Here's an example of Yorkie code:

mux := http.NewServeMux()
mux.Handle(v1connect.NewYorkieServiceHandler(newYorkieServer(yorkieServiceCtx, be), opts...))
mux.Handle(v1connect.NewAdminServiceHandler(newAdminServer(be, tokenManager), opts...))
mux.Handle(v1connect.NewClusterServiceHandler(newClusterServer(be, opts...)))
mux.Handle(grpchealth.NewHandler(healthChecker))
mux.Handle(httphealth.NewHandler(healthChecker))

For CodePair, if the logic mainly involves simple CRUD operations, using a framework could be a good choice to streamline development. But I’ve only worked with native HTTP handlers, so I don’t have much experience with frameworks. I trust @blurfx’s recommendation would be more informed in this case.

MongoDB Integration

Since Yorkie uses the official mongo-go-driver, it’s a dependable choice for CodePair as well. I also agree with adding support for relational databases as an option, as it could increase flexibility and scalability.

Additionally, I’ve been interested in exploring semantic search using vector databases for CodePair. When considering other database options, it might be valuable to account for this use case. I’ve heard that PostgreSQL offers excellent scalability and extensions for features like vector search, making it worth considering for future-proofing the architecture.

I've used some go http frameworks like net/http, fasthttp, and fiber before and had no problem adapting to any of the three libraries/frameworks. the three of them provides a similar developer experience to most web frameworks. If you have a experience with net/http in go, or express or koa in node.js, I think you're probably already familiar with it.

Database:

I'm curious why you chose MongoDB for your existing CodePair, and what you've liked and disliked about it.
Changing databases is a quite large task and MongoDB is relatively scalable than other databases, so I think we can keep it, but as @window9u said, PostgreSQL might be a good option. there are many (third-party) extensions that can do full text search or message queue, etc...

@blurfx I chose MongoDB to utilize Yorkie’s existing database. To clarify, I am referring to supporting a new database type like yorkie (MongoDB, In-Memory), not replacing the production database.

I misread that. I understand now.

@blurfx Could you recommend a framework? Actually, I’ve been considering using Yorkie’s architecture to accommodate contributions from Yorkie contributors.

@devleejb I don't know the workload of CodePair server, but I can suggest the following options:

Performance - Fiber
Ecosystem - Echo
Minimal, lightweight, (not framework) - chi

I personally prefer fiber but is not compatible net/http interface so there might be issues with using third-party packages.

So I think echo would be great with a large ecosystem and a well-documented and built-in features.

@blurfx @devleejb
I think it would be good to create a few backend scaffolding PRs and compare them.

@devleejb @hackerwins @devleejb @window9u

I have experimentally ported the backend to a Go server application using both echo and net/http for comparison, with the help of Cursor(Claude).

• Echo-based Go server
• Net/http-based Go server

Please note that some modules might be missing or not fully implemented.

@krapie
Thank you for your hard work! It’s truly amazing!
I think echo works really well for codepair. I’ll take a closer look at the code this weekend.
If you have any suggestions, please feel free to share them!

I’ll put together a migration plan (including target APIs, timeline, validation, etc.).

I’m aiming to release version v0.2 along with the Go migration. :)

@sejongk I’m also curious about your thoughts.

I have outlined the goals, tasks, considerations, and discussion points for this migration (CodePair Document).
If you’re interested in this issue, please add your name to the Participants field.

Feel free to join the discussion!
Support from Gophers is highly appreciated.

If there are no further discussions, I’d like to move on to the next tasks. :)

CC. @blurfx How about using echo so we can take advantage of its large ecosystem? I also looked into chi, but I’d rather use something with a bigger community around it.

@devleejb Great. I think echo is a good way to get things done quickly because a lot of features comes from the framework's built-in features or community.

@kokodak @blurfx @krapie @window9u @sihyeong671
Is anyone interested in implementing scaffolding? I’m short on time these days.
I think we can start with @krapie 's scaffolding.

@devleejb
Can I try this on? And I'll ask questions if I get stuck.

@window9u Of course! Feel free to reach out any time when you’re stuck.

I will be busy this week, but I think I'll be able to start working on it actively next week. There is something I can help with?

@window9u Could you create a draft PR later? It would be better for tracking the implementation progress, and others can easily contribute to that branch.

@krapie
Sure! I'll create it once I start writing the code. For now, I'm focusing on understanding the previous context.

I will be busy this week, but I think I'll be able to start working on it actively next week. There is something I can help with?

@blurfx Could you help set up an incremental deployment process? As we discussed earlier, I believe we can leverage Nginx within a Docker container.

If you’re interested in working on it, feel free to ask me. I’ll assign the task to you!

• Discussions @devleejb
  • Framework vs. net/http
  • Yorkie Intelligence (LangChain)
  • Documentation (eg, swagger)
  • Deployment Method
  • Testing Method (Verify whether the Go API supports same behavior)
• Implement scaffolding (Go) @window9u
  • Project Initialization (Framework, CORS, Swagger, Envrironment...)
  • Auth middleware
  • Dummy Database Layer (Including database connection)
  • Dummy Service Layer
  • Dummy Handler Layer
• Implement scaffolding (Python) @sihyeong671
  • Project Initialization (Framework, CORS, Swagger, Envrironment...)
• Set up incremental deploy @blurfx
• Migrate APIs (https://codepair-api.yorkie.dev/api)
  • GET /users
  • PUT /users
  • GET /auth/login/github
  • POST /auth/refresh
  • POST /workspaces
  • GET /workspaces
  • GET /workspaces/{worskapce_slug}
  • POST /workspaces/{workspace_id}/invite-token
  • POST /workspaces/join
  • GET /workspaces/{workspace_id}/users
  • PUT /workspaces/{workspace_id}/documents/{document_id}
  • GET /workspaces/{workspace_id}/documents/{document_id}
  • GET /workspaces/{workspace_id}/documents
  • POST /workspaces/{workspace_id}/documents
  • POST /workspaces/{workspace_id}/documents/{document_id}/share-token
  • GET /documents/share
  • POST /check/name-conflict
  • POST /check/yorkie
  • POST /intelligence
  • POST /intelligence/{feature}
  • POST /files
  • GET `/files/{file_name}
  • POST /files/export-markdown
  • GET /settings
• Testing
• Deploy
• Deprecate NestJS backend

I will be busy this week, but I think I'll be able to start working on it actively next week. There is something I can help with?

@blurfx Could you help set up an incremental deployment process? As we discussed earlier, I believe we can leverage Nginx within a Docker container.

Sure. I'll try it soon.

I'll take python Implement scaffolding, please assign me that issue