For consideration: Suggest LLM_RULES.md file in your readme

[jon-eden-tsw]

Hey,

Great utility, many thanks!

Something I've been doing that you may want to include in the readme is the suggestion that people bake a set of repo specific rules into their repos.

I have a LLM_RULES.md file that I have at the base of my repo. It does things like prompt the LLM to use British English, UK date formats as well as other things like provision of some specifications on certain approaches for storybook, component structure etc.

Cheers

Jon

[yamadashy]

Hi,
Thank you for your suggestion!

This is a very interesting idea that I hadn't thought of before. Indeed, if we could incorporate something like this, it would allow us to give instructions to LLMs without needing to input custom instructions for each LLM individually.

Based on your proposal, here's what I'm envisioning:

We could add a setting in the config like this:

{
  "output": {
    "ruleFilePath": "LLM_RULES.md"
  }
}

Currently, when we include output.headerText, it adds the following to the output:

Additional User-Provided Header:
(headerText content)

With this new feature, it would add something like this:

Additional User-Provided Rules:
(LLM_RULES.md content)

What do you think about this approach?

[yamadashy]

Hi @jon-eden-tsw,

Thank you so much for your suggestion and for sharing your approach with LLM_RULES.md!

I've taken your idea and implemented it in a way that aligns with Repopack's existing configuration structure. Instead of a specific LLM_RULES.md file, I've added a new instructionFilePath option to our configuration. This allows users to specify any file (which could indeed be named LLM_RULES.md if they prefer) containing project-specific instructions or rules for AI analysis.

I've created a pull request that implements this feature:

• Add support for project instruction file #46

This implementation provides flexibility for users to include repo-specific rules, coding standards, or any other relevant information for AI analysis, much like your LLM_RULES.md approach.

What are your thoughts on this approach?

Thank you again for your contribution to improving Repopack.

[jon-eden]

Hey,

Yeah great move with that files path - likely that a few others may have done this before.

Now if only we could ensure that the LLMs actually work with the rules!

Cheers

Jon

[yamadashy]

Thank you for your feedback! I was actually planning to test various input patterns for this feature.

Once I can confirm that Claude and other AI models prioritize and follow the instructions in this file, I intend to merge this pull request and proceed with a release.

[yamadashy]

I've been testing various approaches, including explicit instructions like:

1. New code must be written in JS.
2. Add as many comments as possible

However, getting consistent adherence is tricky. I'm still exploring better methods to improve this.

I'll keep working on it and update when I make progress.

[jon-eden]

Yeah even with explicit instructions it does seem hit and miss sometimes on getting the results to be consistent. I suppose that super clean code consistency helps. Any differing approaches will serve to confuse the LLM.

I suspect that there is probably no way to absolutely force the LLM to do something one way - for now!

[yamadashy]

I've decided to experimentally introduce this Custom Instruction feature.
The reason for this is that i've already tested it with several feature implementation proposals, and it seems to be working well in enhancing AI's understanding and suggestions for the project.

I plan to continuously evaluate and improve this feature moving forward.
Your feedback and observations on its effectiveness will be valuable as we refine this approach.

[yamadashy]

I've implemented a similar feature in latest release v0.1.35.

I've added support for custom instruction files, which allows users to provide project-specific rules for AI analysis.

For details, please see: https://github.com/yamadashy/repopack/releases/tag/v0.1.35

I'd appreciate your feedback if you try it out!