-
Notifications
You must be signed in to change notification settings - Fork 500
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[PROPOSAL] Overall Documentation Structure/Restructure #419
Comments
Amazing breakdown! I don't see much here about development docs. Will that be covered separately in the OpenSeach and OpenSeach dashboards developer guides? |
Great callout! I think it would make sense to add them in the developer guides but we can also link them in the main overview. What do you think? |
@ashwin-pc: Still learning everything of course. Could you describe the distinction between the overall OpenSearch documentation and the "development docs". I assume the development docs you are referring to are these: https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#developer-guide. If so, should we make a distinction between "Development" and "Contribution"? Either way, adding the links to the various guides in the main overview make sense to me. I think we could extend that principle to all external links. |
On the troubleshooting section:
I'm conflicted about making that section it's own. Would it be better to have individual "Troubleshooting" subsections within each section. For example, the security plugin docs could include a page for troubleshooting TLS, SAML, and security admin. Furthermore, we could make "Common Issues" it's own top-level page, to account for issues that might not fit into the Use Case sections. Any thoughts? |
By dev docs I meant documentation that new contributors can use to familiarize themselves with the internals of our systems. Looking at Andrews list, I see a lot that helps new users understand OpenSeach but for new contributors, there are still a lot of gaps. For example, for OpenSeach Dashboards the only section I see is a link to the dev guide. Which makes sense for an OpenSeach user. But if someone wishes to contribute to Dashboards, the existing documentation is quite sparse. And the developer guide doesn't do much more than give us setup instructions and the style guide to follow. My comments are biased to Dashboards but I think the same applies to OpenSeach. |
I might be wrong, but I don't think we have a good developer guide. In dashboards we have a single developer doc and that doesn't say much. It's just installation instructions and a style guide. We need a whole new set of docs for that. Ideally generated directly from the code. |
IMO, it would be great if docs could be moved into main repo like Elastic does it. It is very useful to have docs live with the code so that contributors can add/update/remove the relevant docs in the same pull request. It can be a requirement that changes are documented before a pull request is merged. |
This is interesting. I feel like making /doc (or /docs) a top-level directory in the "main" repository a good mechanism to signal the importance of documentation but I'm not sure how it would work given the distributed nature of the repos in the OpenSearch project specifically. |
My first instinct is why not both? We can have a general troubleshooting section where people can go to in addition to individual "Troubleshooting" subsections within each section. The content only needs to be written once and can them be discovered in either use flow (e.g. problem -> troubleshooting -> specific topic OR problem -> specific topic -> troubleshooting). |
This makes sense to me. The question then becomes: Where should they live in the above structure? |
So the general troubleshooting section would just link off to the individual troubleshooting sections, where people can find the actual content? |
Maybe a top-level section labeled "Contribution". Contribution: This section provides instructions and best practices for developing the OpenSearch project.
|
Ultimately, this is going to depend on how we define what the docs site's purpose is. This is how we're currently organizing the information: Repos: What is this thing? Documentation: What is this thing? So if we want to include developer guide content in OpenSearch documentation, then I'm not sure what information we'd put in the repos themselves, and we don't want duplicated information anywhere. My thinking is if first-time contributors are looking to contribute to the projects, then they're on the repos already, so why not stick the developer guide information like how to contribute and more in-depth explanations of systems in the repo's README? If the docs site's purpose is to document how to use OpenSearch, then it doesn't make much sense to document how to contribute to OpenSearch on the docs site. |
That is in line with how other Open Source tools operate, keep contribution instructions in the code base where contribution occurs. I do think there is value in having a hub in which we route users towards our repos though. For example, Elastic has a Community Contribution Program, a hub for users which to contribute to Elastic. They gate their instructions behind a web app that requires you to sign-up, which I don't love. However, the page and app does encourage contributors by promising recognition for those that make significant contributions. |
So where would the details for building say a plugin go? To build a plugin today all we have is a blog post. But it does not for example contains the details of how to generate one, what are the different functional libraries available to a plugin developer, different key concepts that one should know before building one etc. I guess what i am looking for in the documentation is something like this: Its not really a guide on how contribute to Android for example but how to build on top of it. And most of the contents are also very relevant to someone who's contributing. The actual style guide and contribution specific details can and should remain in the repo, but guides, api, and other core concepts make more sense here |
Right, and that's why I'm saying we should decide what the purpose of the OpenSearch docs site is, and whether that should just be how to use OpenSearch VS everything OpenSearch-related. I would say that if users are looking at the plugin's repo to download, clone, do whatever with a plugin, they should be able to look at that repo's README to get the information they need, rather than jump through another hoop to the docs site. I agree that guides and APIs should be in the OpenSearch documentation, but I think it makes more sense to have breakdowns on what libraries are available in a plugin/what to know when thinking about contributing to a plugin within the repo, where they would actually make contributions and where the code actually lives. I like that Android development link because it educates people on how to use Android to build custom apps. But they don't mention anything about contributing to Android, which is at an entirely different link: https://source.android.com/setup/contribute |
I'd like to call out what I feel is a gap in our existing docs - In the situation where someone might be learning Docker and OpenSearch at the same time, I found the amount of guidance just a bit short. We should probably provide a generous dose of help for some basic docker familiarity. Some examples: Installation and Configuration -
|
Regarding contribution - from someone who just enters OpenSearch as a developer (not a user), there is a huge gap of How OpenSearch Works: Internal Architecture / Design
A lot of excellent content in that area has been contributed as blog posts. I think it makes sense to have a section in the docs about the architecture and design of OpenSearch. Here is an example of an issue I opened which has been closed since there is not such place to add that content to: #426 |
@asafm I've added a "How OpenSearch Works" section to the proposal let me know what you think. |
@nateynateynate I think we could include some of this in the install guides for OpenSearch via Docker (under "Installation and Configuration" section) and link out to Docker content for any command deep dives. What do you think? |
@keithhc2 So, where are we landing on the structure? I can make an update if we feel we're aligned. |
I really like this. It's long over due and it's solving some long-standing concerns I've had. Overall feedback: There are a lot of sections. I think, as a user, I might get overwhelmed by this list. Laterally, I also don't like how several dissimilar things are is bundled together ("Analytics, Dashboards, and Machine Learning" seems especially potpourri). I would suggest an additional layer of organization. Maybe three big areas - "Setting up & running OpenSearch," "Using OpenSearch for your data" and "OpenSearch internals & contribution" and then a few other general ones (maybe Overview, external links, API reference). Other nits and picks: "Security" section: I would definitely break this up. This seems to cover the footprint of the security plugin which bundles a lot dissimilar functionality. I would say that access control needs its own category. Encryption might also be a good area to cover. (SSL is listed as a query language which is amusingly incorrect). "Search and Query" section: needs work. SSL is encryption - should it be PPL? No mention of DQL. SQL is listed twice. "Clients and tools" section: break this up. Languages and clients are apples and oranges. I think this historical more than anything. |
Perfect. I sure would like to not have any reason to have to visit a different website to get the practical information needed. I'll take it. |
Shouldn't we have a similar section for dashboards? Btw this is what I was referring to when I mentioned developer docs being different from the developer guide on the repository |
I agree. Below is how I see it. Clients: This section will provide light documentation and links to download clients. Tools: This section will provide documentation for tools that are part of the OpenSearch project. Topics in this section include: OpenSearch CLI |
This is a big change and there are lots of |
This is a great callout, thanks @dlvenable I believe there was previous work done to ensure there was redirecting from old docs (e.g., when the fork happened) so I imagine that can be extended to this restructuring. I'll add above to make sure we have it called out. |
@brijos I've updated to reflect your input - let me know if that works! |
I'm struggling to find any information on DQL in the documentation. When in the openseach UI in I added my comment to this issue because I saw @stockholmux mention lack of DQL documentation in one of his comments. |
Thanks @passbt I've added it explicit under query languages as well as the Dashboards section! |
@macrakis I've moved k-NN and search plugins under "Optimization" (which itself is under "Search and Query"). What's your thought here - is this the right taxonomy? |
Here's a potential reorder for the major side-nav topics, as discussed by @hdhalter and I.
Any thoughts? |
I think "Developer Guides" should be "Contributing", be fairly high level, and we should kick any actual developer guides out of the doc and move those entirely in READMEs. REST API Reference is probably just "APIs" and has a tree of APIs that are all versioned. |
Hi all, thanks for all your thoughtful feedback! I invite you to take a look at the recent version of the documentation at https://opensearch.org/docs/latest/. We've not only created A TON of content over the last year, but have restructured the TOC. We've arranged the topics into logical categories based on use case, rather than plugin. We've also added a feedback mechanism where visitors can give us direct feedback on the content or structure. Feel free to give it a try. But this is only the beginning! We have a lot more enhancements in store. We are continuing to add content where there are gaps, we are improving the content that is there, and we are further categorizing the content. Also, we will decouple the content that doesn't follow the OpenSearch versioning. We are excited to continue to helping to make the community successful in using OpenSearch, and I'll keep this issue open as we continue to enhance the documentation. |
@hdhalter Do you think it's time for me to close this issue? Specifically since https://opensearch.org/docs/latest/ addresses the bulk of the proposal. I might suggest that it is time to close on the 2022 proposal and open a 2023+ proposal/plan. Thoughts before I close? |
Sure, @ahopp , I don't mind if we close this. Good idea to start a new one with the changes that are in flight and planned. |
Background
Since the launch of OpenSearch, there has been some great progress in documenting the project. So far, the Open Distro documentation has been updated and moved to OpenSearch, and there have been continued additions to content coverage on OpenSearch plugin features, OpenSearch search, fundamental OpenSearch language clients content, OpenSearch APIs (e.g., CAT, bulk, DSL), and other ad hoc content based on basis address community requests (e.g., updating security content, updating docker configs, and more).
Despite the significant progress, we haven’t established a plan we can work backwards from specific set of needs. Additionally, the current documentation is still structured in a manner better suited for a peripheral product (e.g. Open Distro).
Proposal
To resolve this, I am recommending an update to the basic hierarchy of content as well as the overall content subjects I think should be covered by the documentation site. In this proposal I tried to consider the following:
While I framed this as a table of contents and described what sections I think we should have in our documentation, I also want to acknowledge that there are many more decisions to be made (e.g. how we title the content, what interlinks we have, what navigation looks like throughout, etc.). Those topics are important and need to be discussed. However, the structure and documentation content seemed to be the most composable, logical place to start. Once we feel good about the initial organization schemes and structures, we should review proposals for labeling, navigation, search, etc.
Finally, while there is some importance in ordering (e.g. overview should probably be high on the list), I’ve focused on coverage for this proposal. I think we can make efforts optimizing the ordering once we have aligned on the content.
Recommended Structure
Below is the general proposal for the section and structure of the content needed for the OpenSearch project.
What are remaining open questions?
While I tried to focus on what documentation users would be looking for based on feedback on the forums, GitHub, and looking at examples of other OSS projects, I know there is still a big opportunity for optimization. Please poke holes, provide feedback, identify gaps, and make proposals! From there we can use this issue as a living roadmap we all can add to and plan our documentation efforts.
The text was updated successfully, but these errors were encountered: