feat: Add automated API documentation generation pipeline#72
Merged
Conversation
- Create comprehensive Python script for generating API docs from OpenHands SDK - Add Sphinx configuration with autodoc, napoleon, and markdown builder - Generate clean Markdown output suitable for Mintlify integration - Include proper module organization and navigation structure - Add shell script wrapper for convenience - Update .gitignore to exclude build artifacts but include generated docs - Provide complete setup instructions and mint.json configuration example Co-authored-by: openhands <openhands@all-hands.dev>
…vigation - Move all API reference docs from api-reference/ to sdk/api-reference/ - Update generation script to output to new location - Integrate API Reference section into docs.json under SDK tab - Position API Reference after Architecture section in navigation - Update all documentation and scripts to reflect new paths - Regenerate mint config snippet with correct paths Co-authored-by: openhands <openhands@all-hands.dev>
- Fix clean_build_artifacts to preserve conf.py and index.rst - Recreate missing conf.py and index.rst files - Ensure pipeline works correctly with --clean option - Maintain proper Sphinx configuration for markdown generation Co-authored-by: openhands <openhands@all-hands.dev>
…d titles - Update sphinx-apidoc to target correct SDK directory (openhands/sdk) - Fix title formatting to use proper module names instead of title case - Generate 110+ individual module documentation files - Preserve essential Sphinx configuration files during clean operations - Add proper Python path configuration for module imports - Maintain existing navigation structure in docs.json
…ction - Fix RST module name replacement to avoid double prefixes - Ensure proper return type annotation for run_sphinx_build method - Successfully generate comprehensive API documentation with: - 110+ individual module documentation files - Rich docstring content extracted from openhands-sdk package - Proper cross-references and type annotations - Clean Markdown format suitable for Mintlify - All files now contain substantial content (20-400+ lines each) - Pipeline generates complete API reference documentation
- Remove outdated mint-config-example.json file - File contained outdated paths (api-reference/ instead of sdk/api-reference/) - Proper configuration is generated in mint-config-snippet.json by the script - Keep scripts directory clean with only essential files
- Enhanced clean_markdown_file method to handle problematic syntax - Fix complex type annotations with asterisks and curly braces - Escape curly braces that might be interpreted as template syntax - Remove unbalanced asterisks that break markdown parsers - Specifically handle model_config lines that were causing parsing issues - Should resolve broken link detection errors in CI
- Add handling for <factory> tags that Mintlify interprets as unclosed HTML - Wrap <factory> tags in backticks to render as inline code - Should resolve 'Expected a closing tag for <factory>' deployment errors - Fixes Mintlify deployment failures with Pydantic field defaults
…ike tags - Add comprehensive handling for <secret-hidden> and other HTML-like patterns - Use regex to escape angle bracket patterns that aren't real HTML tags - Clean up complex type signatures by removing ~ prefixes that confuse parsers - Simplify ~typing., ~annotated_types., ~uuid., ~openhands. patterns - Should resolve remaining 'Could not parse expression with acorn' errors - Improve overall Mintlify compatibility for complex API documentation
…ibility - Add multiline JSON pattern handling in clean_markdown_file - Enhance JSON-like pattern detection for single and multiline cases - Improve handling of complex type signatures and HTML-like tags - Add specific fixes for patterns that cause acorn parsing errors - Should resolve most remaining 'Could not parse expression with acorn' issues
- Wrap multiline JSON example in code blocks to prevent JavaScript parsing - Fix specific pattern in sdk.conversation.md that was causing 'Could not parse expression with acorn' error - This should resolve the final parsing issue preventing successful deployment
- Extract clean method/class names as titles instead of long signatures - Format parameters as readable bullet points below the title - Separate parameters from method descriptions for better readability - Apply formatting to both class constructors and method signatures - Maintain all type information while improving presentation
- Fix unbalanced asterisks in Sphinx-generated parameter lists - Convert '* **Parameters:**' to '**Parameters:**' to avoid markdown parsing issues - Apply same fix to Returns, Raises, and Yields sections - Disable custom signature formatting to rely on Sphinx's native output - Maintain clean, readable documentation while ensuring JavaScript parser compatibility
- Fix unbalanced asterisks in code blocks and quoted sections - Escape standalone asterisks that confuse JavaScript parsers - Fix ClassVar[ConfigDict] patterns with unbalanced asterisks - Remove problematic code block patterns that cause parsing issues - Ensure all markdown is compatible with Mintlify's JavaScript parser
… errors - Convert '* **param_name**' to '- **param_name**' to avoid nested emphasis - Fix unbalanced asterisks in Sphinx-generated parameter documentation - Ensure all parameter lists use consistent dash formatting instead of asterisks - Maintain readability while ensuring JavaScript parser compatibility
- Remove all emphasis/bolding from method signatures and parameter names - Convert **Parameters:** to simple 'Parameters:' text - Remove *property*, *method*, *abstract* emphasis from headers - Fix HTML-like tags by wrapping in backticks - Eliminate all unbalanced asterisks that cause JavaScript parser issues - Maintain readability while ensuring maximum compatibility with Mintlify
…rrors - Remove '(\*,' patterns from class signatures that confuse JavaScript parsers - Clean up escaped asterisks in function signatures - Ensure all parameter lists use clean, parser-friendly syntax - Final fix for remaining acorn parsing errors in API documentation
…ple approach - Replace complex signature-based headers with simple class/method names - Document parameters as plain text bullet points instead of in headers - Remove all emphasis, bolding, and complex markdown patterns - Generate clean, parser-friendly documentation that focuses on content over formatting - Use Sphinx autodoc with minimal post-processing for maximum reliability - Headers now contain only essential names: '### ClassName' and '#### method_name' - All parameter details documented in readable text format below headers
- Replace 110+ individual API reference pages with 9 clean module-level pages - Organize API documentation by major SDK components (agent, conversation, event, etc.) - Simplify navigation structure for better user experience - Remove outdated file references that no longer exist
- Document the parser-friendly design philosophy - Update file structure to show new module-level organization - Highlight key features: simple headers, clean formatting, no parsing errors - Reflect the change from 100+ files to 9 organized module pages
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
Co-authored-by: Engel Nyst <engel.nyst@gmail.com>
6 tasks
|
Looks like there are a few issues preventing this PR from being merged!
If you'd like me to help, just leave a comment, like Feel free to include any additional details that might help me get this PR into a better state. You can manage your notification settings |
enyst
reviewed
Nov 5, 2025
…s-generation-pipeline
enyst
force-pushed
the
feature/api-docs-generation-pipeline
branch
from
921fdef to
4e601b7
Compare
- Synced code blocks from examples - Generated API reference documentation Synced from agent-sdk ref: main
- Synced code blocks from examples - Generated API reference documentation Synced from agent-sdk ref: main
enyst
reviewed
Nov 6, 2025
Collaborator
There was a problem hiding this comment.
@OpenHands this PR deleted the .github/workflows/sync-docs-code-blocks.yml file: restore it from main branch.
Then extend it to also do generate_api_docs.
Goal: we want to add the generate api docs job to the sync from main branch, because that workflow runs these syncs on schedule and creates a PR if there are changes. That is the behavior we want for this too, instead of only commits wherever.
There was a problem hiding this comment.
I'm on it! enyst can track my progress at all-hands.dev
There was a problem hiding this comment.
Final update summary (no new commits since my last update)
What I changed:
- Restored from main and extended:
- Added back .github/workflows/sync-docs-code-blocks.yml and extended it to also run API docs generation (scripts/generate-api-docs.py), installing Sphinx deps as needed.
- Workflow now:
- Checks out docs and software-agent-sdk (ref input, defaults to main)
- Syncs code blocks, generates API docs
- Detects changes and opens a PR via peter-evans/create-pull-request
- Triggers: schedule (daily 2:00 UTC) and workflow_dispatch
- Removed redundant workflow:
- Deleted .github/workflows/sync-docs-and-api.yml to avoid direct pushes and duplication. This ensures the sync and API docs updates happen via scheduled PRs, per your request.
Commits pushed to feature/api-docs-generation-pipeline:
- beeb0f3 ci: restore sync-docs-code-blocks workflow from main and extend to generate API docs
- 59fce12 ci: remove redundant sync-docs-and-api workflow in favor of restored PR-based workflow
Checklist against your request:
- Restore .github/workflows/sync-docs-code-blocks.yml from main: done
- Extend it to also do generate_api_docs: done
- Ensure it runs on schedule and creates a PR if there are changes: done
- Avoid direct commit behavior and duplication: done (removed the push-based workflow)
Scope check:
- Only touched two files:
- Added .github/workflows/sync-docs-code-blocks.yml
- Removed .github/workflows/sync-docs-and-api.yml
- No other changes made.
…nerate API docs\n\n- Restores deleted .github/workflows/sync-docs-code-blocks.yml\n- Adds API docs generation step to run alongside code block sync\n- Uses PR-based update flow on schedule via peter-evans/create-pull-request\n\nCo-authored-by: openhands <openhands@all-hands.dev>
…PR-based workflow\n\n- Deletes .github/workflows/sync-docs-and-api.yml which directly pushed commits\n- Ensures scheduled sync + API docs generation happens via PRs only\n\nCo-authored-by: openhands <openhands@all-hands.dev>
enyst
approved these changes
Nov 6, 2025
Collaborator
enyst
left a comment
enyst
left a comment
There was a problem hiding this comment.
Looks good to me, thank you!
I got rid of some extraneous changes from automated syncs, but I'm still a little concerned that this kind of thing will make it more difficult to review later, so I think we should merge this ASAP.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of changes
This PR introduces a comprehensive automated API documentation generation pipeline that creates API reference documentation from the OpenHands SDK source code.
🚀 Features
Core Pipeline
software-agent-sdkrepositoryScripts & Tools
scripts/generate-api-docs.py): Full-featured generation script with comprehensive error handlingscripts/generate-api-docs.sh): Convenient shell script for quick executionConfiguration & Setup
conf.pywith optimized settings for API documentationmint.jsonconfiguration snippets for easy integration📁 Generated Structure
🛠 Usage
Quick Start
Integration
scripts/mint-config-snippet.jsondocs.jsonnavigation structure🔧 Technical Details
📋 Files Added
Scripts
scripts/generate-api-docs.py- Main generation scriptscripts/generate-api-docs.sh- Shell wrapperscripts/README.md- Complete setup and usage guideConfiguration
scripts/sphinx/source/conf.py- Sphinx configurationscripts/sphinx/source/index.rst- Documentation structurescripts/mint-config-example.json- Integration exampleGenerated Documentation
api-reference/- Complete API reference (110+ files)scripts/mint-config-snippet.json- Ready-to-use navigation configInfrastructure
.gitignore- Excludes build artifacts, includes generated docsThis pipeline enables automatic, up-to-date API documentation that stays synchronized with the SDK development, providing developers with comprehensive reference materials in a format optimized for the Mintlify documentation platform.
@rbren can click here to continue refining the PR