Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix robots noindex directive check when non-utf8 chars present #43

Merged
merged 6 commits into from
Apr 22, 2022

Conversation

cicirello
Copy link
Owner

Summary

The check for a robots noindex directive in an html file was failing with exception if any non-utf8 characters were present. This is now fixed to properly handle this case. This PR also includes a couple other miscellaneous improvements, including refactoring robots.txt parsing logic.

Closing Issues

Closes #42

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Improvements to existing code, such as refactoring or optimizations (non-breaking)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • I have read the CONTRIBUTING document.
  • My code follows the code style of this project.
  • My change requires a change to the documentation.
  • I have updated the documentation accordingly.
  • I have added tests to cover my changes.
  • All new and existing tests passed.

@cicirello cicirello self-assigned this Apr 22, 2022
@cicirello cicirello merged commit 41eec24 into master Apr 22, 2022
@cicirello cicirello deleted the fix-non-utf8-chars branch April 22, 2022 15:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Req: [non UTF8 character error message console log]
1 participant