How to deal with the growing count of directories

[h1z3y3]

As services count is growing, the directories count is also increasing.

Any idea to deal with it in an elegant way? @asim

Maybe we can flatten directories by different categories.

[asim]

I think it will probably be fine until we hit 100 services, then we may need to think further. It just depends on what problem we're trying to fix. At the moment its a very easy to understand structure.

[h1z3y3]

LOL, it'll be soon ~
Now it is 57/100 😜

[asim]

One thing we should think about is how to quickly generate these CRUD services like contact, note, etc. I think if we can turn everything into a service then it brings high level primitives to the platform as first class citizens. Somebody should not have to program the logic of lists or todos on top of the DB. We should offer them as first class services.

But they're effectively all just variations of the same concept with a different name and different proto. So if we could I guess write some sort of generator that would take a proto and write the code or the Go domain and generate based on that it would be good.

What do you think?

[h1z3y3]

This is exactly what I want to say after working on several services.
These services contain almost same CURD handler, except the store key.

What's more , micro new xxx gives simple .proto and simple handler with Call(), Stream(), PingPong().
These three methods could be useless, 99% service will delete them.
So I have to modify the .proto file and the handler file with my own method, like Read(), Update(), 'Create()', etc.

It could be better If the generator can generate handler by we given .proto file.

I want the process just is like:

1. I only need to create a .proto file, with service functions and some message definations.
2. Run generation command, like micro new xxx --with-proto my.proto
3. It helps generate all the handler functions and one domain directory, and handler functions don't have any logic codes, only call the domain layer
4. Then the domain layer is where our logic code is.

I think what we need to do is extending the micro new command

[asim]

I agree with you. I'm inclined to think rather than trying to modify micro new, since its got a lot related to it, we create a new command micro gen. Its a clean space to create a new thing. So then micro gen [service] --proto=my.proto will actually generate a real service. Whereas micro new is just pure boilerplate/template.

[h1z3y3]

Awesome! I can't wait for it.
If there could be some works on it I'd glad to contribute.

[asim]

I'm happy to accept it as a contribution if you have experience dealing with proto like that. Otherwise we'll need to look for inspiration from elsewhere.

[h1z3y3]

Actually I don't have too much experience in implementing it, but I used similar tool-chains in go-zero (https://go-zero.dev/en/) and read some source codes.
It's more likes a DSL(https://en.wikipedia.org/wiki/Domain-specific_language) in go-zero but I think we're going to the original .proto file.

See more: https://go-zero.dev/en/goctl-api.html

[asim]

One thing I have always imagined is moving to pure Go for the service definition. It always felt off to use proto. It's so tightly coupled to grpc and isn't natively Go. I think there is an opportunity to define services elsewhere and generate a proto from it if needed.

This is why I started creating micro.mu files. I think cuelang.org is a good dsl for such a purpose. The question just becomes. How do you define the service.

[h1z3y3]

I get it, so the micro.mu file is defined to generate entire service, right? That would be awesome.

I've learned much from you, thank you very much, I'm going to deep in the cuelang and learn about it.

[asim]

Yes the ".mu" file will become central to everything. Much like how in the rails world rack apps had a .ru file. If we're able to process it like a real dsl then it can be really powerful.