Feature request: add robots.txt & sitemap.xml generation for packit

[renhiyama]

No description provided.

[Tholkappiar]

Hello Maintainers ,

want some clarification on this feature , i know "the primary use of "Packit" is to optimize and bundle code for faster and more efficient ", can you kindly explain this feature and how Packit is related to this feature ?

[renhiyama]

Primary use of "Packit" is to generate code based on different conditions met. This idea gives interested devs access to low level API access (useful for monkey patching). The default readers & writers (read the docs first btw) generates code (and transpiles the necessary code too) to make it work.

please read src/cli/cmds/utils/Packit.js code to learn more about the default readers and writers that generate code for any reejs project.

I would suggest adding robots.txt and sitemap.xml to the "static" writer. Static writer runs after pages & api writers have done their jobs, but static writer writes code before pages writer, so you are able to access /__reejs/cache/cache.json and others before /* resolving to your custom 404 page. Remember, the nodejs server framework(s) basics says: the earlier you attach the router, the more priority it has to resolve.
So the pages writer writes content to the end of the code, just before the finish writer, which generates code to host the server.

DISCLAIMER: Packit ISN'T FAST than others. It feels and is fast only because of the underlying transpilers, readers & writers attached w/ it. Reejs uses sucrase transpiler instead of transpilers like swc, which seem to be resonable and faster for our given field of usage. YOU CAN make Packit SLOW if you make use of such transpilers that are slower than the ones we use. Packit is just a cool handler that knows how to do the things as they're meant to be done. if-else conditions, file caching, URL Imports and cherry-picked underlying dependencies makes Packit fast.

---

The writers are friendly towards you, they can share, modify (and DELETE!) variables for your context. This feature is the reason why writers are ran serially instead of parallely. The pages writer generates pages' router links - which can be used in static writer if you use it from the helpers object available inside the writer. Make use of it and generate sitemap.xml & robots.txt and write it to public directory, and register a route for it too. I would suggest reading from reecfg.json to declare robots.txt . Feel free to add your own opinions and options to reecfg.json file.

[Tholkappiar]

Oh.. Thanks for the clean explanation! Going to learn something new ! Can you assign this feat to me ?

[renhiyama]

Done

[renhiyama]

@Tholkappiar any updates?

[Tholkappiar]

Unassign this issue from me , I will work on this issue later !

[sumanthmadishetty]

Can i Take this?

[renhiyama]

@sumanthmadishetty feel free to work on this. There's not much contributors so dw about someone else taking up the job