Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: #35 Update tokenization logic causing issues for larger codebases #68

Merged
merged 3 commits into from
Oct 21, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/release-drafter.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ categories:
labels:
- chore
- maintenance
- refactor
- title: ⛔️ Deprecated
labels:
- deprecated
Expand Down
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -51,4 +51,5 @@ examples/readme-edgecase.md
readmeai/conf/templates
readmeai/conf/prompts.toml
readmeai/settings/prompts.toml
readmeai/md_builder/data/badges.json
readmeai/markdown/data/badges.json
templates/
75 changes: 31 additions & 44 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,70 +1,57 @@
# Contributing Guidelines
Welcome to our open source project! We appreciate your interest in contributing to our project. Please take a moment to review these guidelines before making a contribution to ensure a smooth and successful collaboration.

## Ways to Contribute
There are many ways to contribute to our project. You can:
Thanks for your interest in contributing to *README-AI*! Please review these guidelines to ensure a smooth process.

- Report issues and bugs
- Suggest new features or improvements
- Submit bug fixes or code contributions
## Make Valuable Contributions

## Code of Conduct
We expect all contributors to adhere to our code of conduct to ensure a respectful and inclusive environment for everyone. Our code of conduct can be found in the [CODE_OF_CONDUCT](./CODE_OF_CONDUCT.md) file.
Strive to make **useful**, **creative**, and **high quality** contributions.

## How to Contribute
**Useful:** Solve common problems, use cases, exceptions, or issues.

Contributions are always welcome! Please follow these steps:
**Creative:** Innovative and helping us all learn new things.

1. Fork the project repository. This creates a copy of the project on your account that you can modify without affecting the original project.
2. Clone the forked repository to your local machine using a Git client like Git or GitHub Desktop.
3. Create a new branch with a descriptive name (e.g., `new-feature-branch` or `bugfix-issue-123`).
```sh
git checkout -b new-feature-branch/add-some-feature
```
4. Make changes to the project's codebase.
5. Commit your changes to your local branch with a clear commit message that explains the changes you've made.
```sh
git commit -m 'Implemented new feature.'
```
6. Push your changes to your forked repository on GitHub using the following command
```sh
git push origin new-feature-branch/add-some-feature
```
7. Create a [pull request](https://github.com/eli64s/README-AI/pulls) to the original repository.
**High Quality:** Well-written, structured, and explained.

Open a new pull request to the original project repository. In the pull request, describe the changes you've made and why they're necessary.
The project maintainers will review your changes and provide feedback or merge them into the main branch.
## Follow The Code of Conduct

## Reporting Issues and Bugs
Review and adhere to our [CODE_OF_CONDUCT](./CODE_OF_CONDUCT.md).

If you find an issue or bug in our project, please report it by opening a new issue in the project's issue tracker. Please include a clear description of the issue or bug, as well as any relevant code or error messages.
## Ways to Contribute

## Suggesting New Features or Improvements
There are many ways to contribute to *README-AI*! You can:

If you have an idea for a new feature or improvement to our project, please suggest it by opening a new issue in the project's issue tracker. Please include a clear description of the feature or improvement you are suggesting, as well as any relevant examples or use cases.
- Simply start a discussion by asking a question or making a suggestion.
- Look for opportunities to make processes more efficient to improve the user experience.
- Find ways to make code more readable and easier to understand.
- Find unhandled exceptions and bugs when running the program.
- Write unit test for the program.
- Brainstorm new CLI options and features that would be useful to users.
- i.e. `--language` option to specify the language of the README.
- What is the best way to implement README file generation in any language selected?

## Submitting Bug Fixes or Code Contributions
These are just a few exmaples, there are many more ways to contribute to *README-AI*.

If you would like to contribute code to our project, please follow these guidelines:
## Submitting Changes

- Before making a contribution, please discuss your proposed changes with the project maintainers to ensure they align with the project's goals and direction.
- Please write clear, well-documented code that adheres to the project's coding style and standards.
- Please write unit tests for any new code contributions or changes to ensure they are working correctly and do not break existing functionality.
- Please ensure your code is properly formatted and free of any linting errors or warnings.
- Please commit your changes with clear and descriptive messages that explain what you have done.
- Please ensure your changes are rebased onto the latest version of the main project branch before submitting a pull request.
- Please ensure your pull request includes a clear description of the changes you have made and why they are necessary or beneficial.
1. Fork the repository and clone it locally.
2. Create a new branch with a descriptive name i.e <code>feature/new-feature-name</code> or <code>bugfix-issue-123</code>.
3. Make focused changes with clear commits.
4. Open a pull request document the changes you've made and why they're necessary.
5. Respond to code reviews from maintainers.

## Code Reviews
## Code Quality Expectations

All contributions to our project will be reviewed by project maintainers before they are merged into the main project branch. Reviews may include feedback on the code, suggestions for improvement, or requests for additional testing or documentation.
- Clear, well-documented code
- Include tests for new code
- Follow project style standards
- Rebase onto latest main branch

## Attribution

Contributors to our project will be acknowledged in the project's README.md file and AUTHORS.md file.

## License

By contributing to our project, you agree to license your contributions under the project's open source license. The project's license can be found in the LICENSE.md file.
By contributing to our project, you agree to license your contributions under the project's open source license. The project's license can be found in the [LICENSE](./LICENSE.md) file.

Thank you for contributing to our project!
4 changes: 2 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ help:

.PHONY: clean
clean: format
bash scripts/clean.sh
bash scripts/clean.sh clean

.PHONY: format
format:
Expand All @@ -31,7 +31,7 @@ lint:

.PHONY: reqs
reqs:
pip install -r requirements.txt
pip install -r requirements.txt

.PHONY: conda
conda:
Expand Down
95 changes: 48 additions & 47 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
<div align="right">

[![CN](https://img.shields.io/badge/CN-🇨🇳-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-zh-CN.md)
[![DE](https://img.shields.io/badge/DE-🇩🇪-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-de.md)
[![FR](https://img.shields.io/badge/FR-🇫🇷-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-fr.md)
[![CN](https://img.shields.io/badge/CN-🇨🇳-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-zh-CN.md)
[![DE](https://img.shields.io/badge/DE-🇩🇪-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-de.md)
[![FR](https://img.shields.io/badge/FR-🇫🇷-white?style=plastic)](https://github.com/eli64s/readme-ai/blob/main/docs/README-fr.md)
</div>

<div align="center">
Expand All @@ -12,7 +12,7 @@
<br>README-AI
</h1>
<h3>◦ Generate beautiful and informative <i>README</i> files</h3>
<h3>◦ Developed with OpenAI's GPT language model APIs</h3>
<h3>◦ Powered by OpenAI's GPT language model APIs</h3>
<br>
<p align="center">
<img src="https://img.shields.io/badge/Markdown-000000.svg?&logo=Markdown&logoColor=white" alt="Markdown" />
Expand Down Expand Up @@ -44,7 +44,7 @@

- [📖 Table of Contents](#-table-of-contents)
- [🔭 Overview](#-overview)
- [🤖 Demos](#-demos)
- [🤖 Demo](#-demo)
- [💫 Features](#-features)
- [👩‍💻 Usage](#-usage)
- [📦 Installation](#-installation)
Expand Down Expand Up @@ -74,13 +74,13 @@ Dev tool to auto-generate detailed README.md files from the command-line using O

Streamlines documentation creation and maintenance, enhancing developer productivity. *README-AI* aims to improve the adoption of open-source software, enabling all skill levels and disciplines to understand, utilize, and contribute to projects more effectively.

**⚠️ Disclaimer**

This project is under development with an opinionated setup. It's vital to review the text generated by the OpenAI API to ensure it accurately represents your codebase.
<div style="border-left: 4px solid #ffa500;padding: 5px 10px;">
⚠️ Note: This project is under development with an opinionated setup. It's vital to review the text generated by the OpenAI API to ensure it accurately represents your codebase.
</div>

---

## 🤖 Demos
## 🤖 Demo

***Command-Line Interface***

Expand Down Expand Up @@ -113,7 +113,7 @@ This project is under development with an opinionated setup. It's vital to revie
<td>
<h4><i>Project Slogan and Badges</i></h4>
<p>
‣ A slogan to highlight your poject is generated by <a href="https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L58">prompting</a> OpenAI's GPT engine.
‣ A slogan to highlight your poject is generated by <a href="https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L70">prompting</a> OpenAI's GPT engine.
</p>
<p>
‣ Codebase dependencies and metadata are visualized using <a href="https://shields.io/">Shields.io</a> badges.
Expand All @@ -128,11 +128,11 @@ This project is under development with an opinionated setup. It's vital to revie
</table>
<table>
<p>
‣ Use the CLI option <code>--badges</code> to select the style of badges for your README! 3 styles are currently supported:
</p>
‣ Use the CLI option <code>--badges</code> to select the style of badges for your README! <br>
‣ 6 styles are currently supported: <i>flat (default), flat-square, plastic, for-the-badge, social, square</i>. Find a few examples below.
<tr>
<td>
<h4 style="text-align:left;">1. Shieldsio default badge style</h4>
<h4 style="text-align:left;">1. Shieldsio <em>flat</em> badge style</h4>
<p style="text-align:left;">Command: none as its the default style for <em>readme-ai</em></p>
<div style="text-align:center;">
<img src="https://raw.githubusercontent.com/eli64s/readme-ai/main/examples/imgs/badges-shieldsio-default.png" alt="badges-shieldsio-default" />
Expand All @@ -142,7 +142,7 @@ This project is under development with an opinionated setup. It's vital to revie
<tr>
<td>
<h4 style="text-align:left;">2. Shieldsio <em>for-the-badge</em> style</h4>
<p style="text-align:left;">Command: <code>--badges shields</code></p>
<p style="text-align:left;">Command: <code>--badges for-the-badge</code></p>
<div style="text-align:center;">
<img src="https://raw.githubusercontent.com/eli64s/readme-ai/main/examples/imgs/badges-shieldsio-flat.png" alt="badges-shieldsio-flat" />
</div>
Expand Down Expand Up @@ -492,34 +492,38 @@ poetry install

<br>

***Command-Line Arguments***
***Command-Line Arguments*** "flat",
"flat-square",
"style=for-the-badge",
"plastic",
"square",

To generate a *README.md* file, use the `readmeai` command in your terminal, along with the arguments below.

| Short Flag | Long Flag | Description | Type | Status |
|------------|------------------|-----------------------------------------------------|-------------|--------------|
| `-k` | `--api-key` | Your language model API secret key. | String | Optional |
| `-b` | `--badges` | Select 'shields' or 'square' to change badge style. | String | Optional |
| `-e` | `--emojis` | Add emojis to your README.md file heading sections | Boolean | Optional |
| `-f` | `--offline-mode` | Run offline without calling the OpenAI API. | Boolean | Optional |
| `-m` | `--model` | Large language model engine (gpt-3.5-turbo) | String | Optional |
| `-o` | `--output` | The output path for your README.md file. | Path/String | Optional |
| `-r` | `--repository` | The URL or path to your code repository. | URL/String | Required |
| `-t` | `--temperature` | The temperature (randomness) of the model. | Float | Optional |
| `-l` | `--language` | The language of text to write README in. | String | Coming Soon! |
| `-s` | `--style` | The README template style to build. | String | Coming Soon! |
| Short Flag | Long Flag | Description | Type | Status |
|------------|-----------------|----------------------------------------------------------------|-------------|--------------|
| `-k` | `--api-key` | Your language model API secret key. | String | Optional |
| `-b` | `--badges` | [Styles](https://shields.io/badges) - `flat (default), flat-square, plastic, for-the-badge, social, sqaure` | String | Optional |
| `-e` | `--emojis` | Use ```false``` to remove emojis from README default template. | Boolean | Optional |
| `-f` | `--offline` | Use ```true``` to run README-AI without an API key. | Boolean | Optional |
| `-m` | `--model` | Default large language model is the `gpt-3.5-turbo` engine. | String | Optional |
| `-o` | `--output` | The output path for your README.md file. | Path/String | Optional |
| `-r` | `--repository` | The URL or path to your code repository. | URL/String | Required |
| `-t` | `--temperature` | The temperature (randomness) of the model responses. | Float | Optional |
| `-l` | `--language` | The language of text to write README in. | String | Coming Soon! |
| `-s` | `--style` | The README template style to build. | String | Coming Soon! |

<br>

***Custom Settings***

To customize the README file generation process, you can modify the project's [configuration file:](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml)

- [*api*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L2) - OpenAI language model API configuration settings.
- [*git*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L13) - Default git repository settings used if no repository is provided.
- [*paths*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L18) - Directory paths and files used by the *readme-ai* application.
- [*prompts*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L28) - Large language model prompts used to generate the README file.
- [*md*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml#L61) - Dynamic Markdown section code templates used to build the README file.
- [*api*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml) - OpenAI language model API configuration settings.
- [*git*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml) - Default git repository settings used if no repository is provided.
- [*paths*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml) - Directory paths and files used by the *readme-ai* application.
- [*prompts*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml) - Large language model prompts used to generate the README file.
- [*md*](https://github.com/eli64s/readme-ai/blob/main/readmeai/settings/config.toml) - Dynamic Markdown section code templates used to build the README file.

---

Expand Down Expand Up @@ -570,9 +574,11 @@ poetry run python3 -m readmeai.cli.commands -o readme-ai.md -r https://github.co

***Using Streamlit***

Use the app directly in your browser via Streamlit Community Cloud.
Generate a README files directly in your browser on Streamlit Cloud!

> 🎈 Navigate at [readmeai.streamlit.app](https://readmeai.streamlit.app/)

- [🛸 Take me to *readme-ai* on Streamlit!](https://readmeai.streamlit.app/)
Application may be unavailable at times as it's hosted on a free tier.

---

Expand All @@ -596,9 +602,9 @@ bash scripts/test.sh
- [*Streamlit Community Cloud - readmeai*](https://readmeai.streamlit.app/)
- [ ] Refactor our large language model engine to enable more robust README generation.
- [ ] Explore [LangChain 🦜️🔗](https://python.langchain.com/docs/get_started/introduction) as an alternative to using the OpenAI API directly.
- [ ] Explore [LlamaIndex 🦙](https://gpt-index.readthedocs.io/en/stable/index.html) framework and Retrieval augmented generation (RAG) paradigm.
- [ ] Explore [LlamaIndex 🦙](https://gpt-index.readthedocs.io/en/stable/index.html) framework and Retrieval Augmented Generation (RAG) paradigm.
- [ ] Building template system to create README files for specific use-cases (data, mobile, web, etc.)
- [ ] Add support for generating README files in any language (i.e. CN, ES, FR, JA, KO, RU).
- [ ] Design README output templates for a variety of use-cases (i.e. data, web-dev, minimal, etc.)
- [ ] Develop GitHub Actions script to automatically update the README file when new code is pushed.

---
Expand All @@ -611,17 +617,12 @@ bash scripts/test.sh

## 🤝 Contributing

Looking to contribute to *readme-ai*? Here is what you can do to help:

- 🐍 Look for opportunities to make the code more efficient and readable.
- 🤖 Exception handling and bug fixes during README generation.
- 📝 Improve documentation and add more examples to the README.
- 🔡 Add support for generating README files in any language (i.e. CN, ES, FR, JA, KO, RU).
- 🎨 Create new templates for different use-cases (i.e. data, web-dev, minimal, etc.).
- The README is constructed in sections, defined in the [config.toml] file.
- Follow the existing format to get started.

[Contributing Guidelines](https://github.com/eli64s/readme-ai/blob/main/CONTRIBUTING.md)
- 👋 Say hello anytime!
- [Discussions](https://github.com/eli64s/readme-ai/discussions)
- 🐛 Found a bug?
- [Open an Issue](https://github.com/eli64s/readme-ai/issues)
- 🧚‍♀️ Want to contribute?
- [Contributing Guidelines](https://github.com/eli64s/readme-ai/blob/main/CONTRIBUTING.md)

---

Expand Down
4 changes: 2 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,8 @@ build-backend = "poetry.core.masonry.api"

[tool.poetry]
name = "readmeai"
version = "0.4.028"
description = "Automated README file generator 🧚‍♀️"
version = "0.4.029"
description = "Automated README.md file generator tool, powered by AI 🧚"
authors = ["Eli <0x.eli.64s@gmail.com>"]
license = "MIT"
readme = "README.md"
Expand Down
Loading
Loading