Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom Robots File Parser #601

Closed
wants to merge 4 commits into from
Closed

Custom Robots File Parser #601

wants to merge 4 commits into from

Commits on Sep 2, 2024

  1. fix malformed url in robots

    @addie9800
    addie9800 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    7cacb55 View commit details
    Browse the repository at this point in the history

  2. draft CustomRobotFileParser

    @addie9800
    addie9800 committed Sep 2, 2024
    Configuration menu
    Copy the full SHA
    01034ee View commit details
    Browse the repository at this point in the history

Commits on Sep 3, 2024

  1. Change to CustomRobotFileParser

    @addie9800
    addie9800 committed Sep 3, 2024
    Configuration menu
    Copy the full SHA
    033679a View commit details
    Browse the repository at this point in the history

  2. Merge branch 'add-robots-text-parser' into custom-robots-file-parser

    @addie9800
    addie9800 authored Sep 3, 2024
    Configuration menu
    Copy the full SHA
    fe1c4d9 View commit details
    Browse the repository at this point in the history