Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROPOSAL] Overall Documentation Structure/Restructure #419

Closed
ahopp opened this issue Feb 21, 2022 · 37 comments
Closed

[PROPOSAL] Overall Documentation Structure/Restructure #419

ahopp opened this issue Feb 21, 2022 · 37 comments
Assignees
Labels
2 - In progress Issue/PR: The issue or PR is in progress. Usability
Milestone

Comments

@ahopp
Copy link

ahopp commented Feb 21, 2022

Background

Since the launch of OpenSearch, there has been some great progress in documenting the project. So far, the Open Distro documentation has been updated and moved to OpenSearch, and there have been continued additions to content coverage on OpenSearch plugin features, OpenSearch search, fundamental OpenSearch language clients content, OpenSearch APIs (e.g., CAT, bulk, DSL), and other ad hoc content based on basis address community requests (e.g., updating security content, updating docker configs, and more).

Despite the significant progress, we haven’t established a plan we can work backwards from specific set of needs. Additionally, the current documentation is still structured in a manner better suited for a peripheral product (e.g. Open Distro).

Proposal

To resolve this, I am recommending an update to the basic hierarchy of content as well as the overall content subjects I think should be covered by the documentation site. In this proposal I tried to consider the following:

  • The structure should be flexible enough to allow the OpenSearch project documentation to change as the overall project changes. It will also need to be able to be reviewed and refreshed periodically.
  • The structure should be suitable for all readers. While we can’t optimize for every type of user, we should endeavor to support all.
  • Not all content will be reviewed in the same context so the structure scheme should allow for some repetition.
  • We should structure the documents to support both information-oriented reference material and goal-oriented how-to material.
  • The goal should be to reduce as much cognitive load in looking for content as possible
  • EDIT: Old links should redirect to the new structure where ever possible (thanks @dlvenable)

While I framed this as a table of contents and described what sections I think we should have in our documentation, I also want to acknowledge that there are many more decisions to be made (e.g. how we title the content, what interlinks we have, what navigation looks like throughout, etc.). Those topics are important and need to be discussed. However, the structure and documentation content seemed to be the most composable, logical place to start. Once we feel good about the initial organization schemes and structures, we should review proposals for labeling, navigation, search, etc.

Finally, while there is some importance in ordering (e.g. overview should probably be high on the list), I’ve focused on coverage for this proposal. I think we can make efforts optimizing the ordering once we have aligned on the content.

Recommended Structure

Below is the general proposal for the section and structure of the content needed for the OpenSearch project.

  • Overview: As the name implies, this section provides an overview of OpenSearch and OpenSearch Dashboards including concepts, paths to community content (ex. roadmaps, current design docs, etc.), and release/end-of-life information. Topics in this section include;
    • Project Overview and Concepts
    • First steps and Learning Progressions
    • Roadmap
    • Design documents/RFCs
    • Releases
    • End of Life
    • FAQ
  • Installation and Configuration: This section will demonstrate how to install and configure OpenSearch and OpenSearch Dashboards including install requirements and examples for common configurations. It will also include content specific to upgrading from common systems or versions. Topics in this section include;
    • Installation Overview and Concepts
    • Planning
    • Installing on
    • Upgrading
    • Configuration
    • Configuring multiple node types
  • Manage, Monitor, and Tune: This section will focus on methods, instructions, and guidelines to run OpenSearch and OpenSearch Dashboards. Topics in this section include;
    • Dev Tools
    • Stack Management
    • Performance Analyzer
  • Migration: This section will provide content on migrating to OpenSearch and OpenSearch Dashboards from various common sources (ex. Open Distro, older version of OpenSearch, Elasticsearch, etc.).Topics in this section include;
    • Overview and concepts
    • From Open Distro
    • From OSS Elasticsearch
    • From Elasticsearch
    • From Other Databases
  • Security: This section will provide both an overviews of security (include key concepts) and configuration options/guidance including authorization, authentication, and auditing. Topics in this section include;
    • Overview and concepts
    • How-to/tutorials
    • Configuration
    • Authentication
    • Access control
    • Auditing
  • Indexes and ingestion: This section will be all about the ins and outs of the OpenSearch index including reindexing, aliases, and overall index management. Topics in this section include;
    • Overview and concepts
    • Index templates
    • Units and data types
    • Indexing APIs
    • Indexing back pressure
    • Reindexing
    • Index aliases
    • Data Streams
    • Index Management
    • Ingestion Nodes
  • Search and Query: This section will provide an overview of search and query concepts as well as provide information on query options across OpenSearch and OpenSearch Dashboards. Topics in this section include;
    • Overview and concepts
    • Search (e.g. Query DSL, Search Templates, Autocomplete, etc.)
    • Query languages (e.g. DQL, DSL, PPL, SQL, etc.)
    • Cross Cluster Search
    • Optimization (e.g., k-NN, search plugins)
    • Aggregations
  • Availability and Recovery: This section will cover all the guidance needed for snapshots, cluster replication (and cross cluster replication), and recovery methods. It will also include information to optimize for availability. Topics in this section include;
    • Overview
    • Snapshots
    • Cluster Replication
    • Cross Cluster Replication
  • OpenSearch Dashboards: This section will have the information needed to use OpenSearch and OpenSearch Dashboards for analytics, visualization, and dashboarding. Topics in this section include;
    • Overview and concepts
    • Exploring (e.g., discovery, data views, filtering, DQL)
    • Visualizing (e.g., creating visualizations, creating dashboards)
    • Sharing (e.g., sharing dashboards, creating reports)
    • Detecting (e.g, alerting, anomaly detection)
    • Best Practices / Considerations
  • Observability: This section will provide everything you need to know to understand logs, metrics, uptime data, application traces, etc. This will include an overview of observability features as well as details on trace and log analytics. Topics in this section include;
    • Overview and concepts
    • Event analytics
    • Operational panels
    • Piped Processing Language
    • Notebooks
    • Trace Analytics
    • Log Analytics
  • Clients: This section will provide documentation and links to download clients.
    • Supported language clients (e.g., Python, JavaScript, Java)
    • Install Instructions for clients (e.g., pip, Gradle dependencies)
  • Tools: This section will provide documentation for tools that are part of the OpenSearch project. Topics in this section include;
    • OpenSearch CLI
    • Ingestion tools (e.g. Data Prepper, Logstash, Fluentd, Tika, etc.)
    • Data collection agents (e.g. Beats, Fluent Bit, etc.)
    • Analytics tools (e.g. Grafana, Power BI, Tableau, etc.)
    • OpenTelemetry
    • OpenSearch Benchmark
  • Troubleshooting: This section will focus on resolving common challenges such as troubleshooting configuration issues and fixing security issues. Topics in this section include;
    • Common issues
    • securityadmin.sh
    • TLS
    • SAML
    • OpenID
  • REST API Reference: This section will provide a breakdown of the REST API and demonstrate what can be called to configure or access OpenSearch functions.
  • How OpenSearch Works: This section will provide an overview internal architecture and include aggregated general educational material on how core OpenSearch components work in OpenSearch.
    • Ex. How does OpenSearch work?
    • Ex. How does replication work?
    • Ex. How does the Shard Allocator (Balancer) work?
    • Ex. How do plugins work?
  • External links: This section will provide a path to external links and associated documents not part of the core reference documentation. Topics in this section include;
    • Javadoc
    • OpenSearch developer guide
    • Dashboards developer guide

What are remaining open questions?
While I tried to focus on what documentation users would be looking for based on feedback on the forums, GitHub, and looking at examples of other OSS projects, I know there is still a big opportunity for optimization. Please poke holes, provide feedback, identify gaps, and make proposals! From there we can use this issue as a living roadmap we all can add to and plan our documentation efforts.

@ashwin-pc
Copy link
Member

Amazing breakdown! I don't see much here about development docs. Will that be covered separately in the OpenSeach and OpenSeach dashboards developer guides?

@ahopp
Copy link
Author

ahopp commented Feb 22, 2022

Amazing breakdown! I don't see much here about development docs. Will that be covered separately in the OpenSeach and OpenSeach dashboards developer guides?

Great callout! I think it would make sense to add them in the developer guides but we can also link them in the main overview. What do you think?

@Naarcha-AWS
Copy link
Collaborator

Naarcha-AWS commented Feb 22, 2022

@ashwin-pc: Still learning everything of course. Could you describe the distinction between the overall OpenSearch documentation and the "development docs". I assume the development docs you are referring to are these: https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#developer-guide.

If so, should we make a distinction between "Development" and "Contribution"? Either way, adding the links to the various guides in the main overview make sense to me. I think we could extend that principle to all external links.

@Naarcha-AWS
Copy link
Collaborator

On the troubleshooting section:

Troubleshooting: This section will focus on resolving common challenges such as troubleshooting configuration issues and fixing security issues. Topics in this section include;

    Common issues
    securityadmin.sh
    TLS
    SAML
    OpenID

I'm conflicted about making that section it's own. Would it be better to have individual "Troubleshooting" subsections within each section. For example, the security plugin docs could include a page for troubleshooting TLS, SAML, and security admin.

Furthermore, we could make "Common Issues" it's own top-level page, to account for issues that might not fit into the Use Case sections.

Any thoughts?

@ashwin-pc
Copy link
Member

ashwin-pc commented Feb 22, 2022

@ashwin-pc: Still learning everything of course. Could you describe the distinction between the overall OpenSearch documentation and the "development docs". I assume the development docs you are referring to are these: https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md#developer-guide.

If so, should we make a distinction between "Development" and "Contribution"? Either way, adding the links to the various guides in the main overview make sense to me. I think we could extend that principle to all external links.

By dev docs I meant documentation that new contributors can use to familiarize themselves with the internals of our systems. Looking at Andrews list, I see a lot that helps new users understand OpenSeach but for new contributors, there are still a lot of gaps.

For example, for OpenSeach Dashboards the only section I see is a link to the dev guide. Which makes sense for an OpenSeach user. But if someone wishes to contribute to Dashboards, the existing documentation is quite sparse. And the developer guide doesn't do much more than give us setup instructions and the style guide to follow.

My comments are biased to Dashboards but I think the same applies to OpenSeach.

@ashwin-pc
Copy link
Member

Amazing breakdown! I don't see much here about development docs. Will that be covered separately in the OpenSeach and OpenSeach dashboards developer guides?

Great callout! I think it would make sense to add them in the developer guides but we can also link them in the main overview. What do you think?

I might be wrong, but I don't think we have a good developer guide. In dashboards we have a single developer doc and that doesn't say much. It's just installation instructions and a style guide. We need a whole new set of docs for that. Ideally generated directly from the code.

@mattweber
Copy link

IMO, it would be great if docs could be moved into main repo like Elastic does it. It is very useful to have docs live with the code so that contributors can add/update/remove the relevant docs in the same pull request. It can be a requirement that changes are documented before a pull request is merged.

@ahopp
Copy link
Author

ahopp commented Feb 22, 2022

IMO, it would be great if docs could be moved into main repo like Elastic does it. It is very useful to have docs live with the code so that contributors can add/update/remove the relevant docs in the same pull request. It can be a requirement that changes are documented before a pull request is merged.

This is interesting. I feel like making /doc (or /docs) a top-level directory in the "main" repository a good mechanism to signal the importance of documentation but I'm not sure how it would work given the distributed nature of the repos in the OpenSearch project specifically.

@CEHENKLE @dblock what are your thoughts?

@ahopp
Copy link
Author

ahopp commented Feb 22, 2022

I'm conflicted about making that section it's own. Would it be better to have individual "Troubleshooting" subsections within each section. For example, the security plugin docs could include a page for troubleshooting TLS, SAML, and security admin.

Furthermore, we could make "Common Issues" it's own top-level page, to account for issues that might not fit into the Use Case sections.

Any thoughts?

My first instinct is why not both? We can have a general troubleshooting section where people can go to in addition to individual "Troubleshooting" subsections within each section. The content only needs to be written once and can them be discovered in either use flow (e.g. problem -> troubleshooting -> specific topic OR problem -> specific topic -> troubleshooting).

@ahopp
Copy link
Author

ahopp commented Feb 22, 2022

I might be wrong, but I don't think we have a good developer guide. In dashboards we have a single developer doc and that doesn't say much. It's just installation instructions and a style guide. We need a whole new set of docs for that. Ideally generated directly from the code.

This makes sense to me. The question then becomes: Where should they live in the above structure?

@keithhc2
Copy link
Contributor

My first instinct is why not both? We can have a general troubleshooting section where people can go to in addition to individual "Troubleshooting" subsections within each section. The content only needs to be written once and can them be discovered in either use flow (e.g. problem -> troubleshooting -> specific topic OR problem -> specific topic -> troubleshooting).

So the general troubleshooting section would just link off to the individual troubleshooting sections, where people can find the actual content?

@Naarcha-AWS
Copy link
Collaborator

I might be wrong, but I don't think we have a good developer guide. In dashboards we have a single developer doc and that doesn't say much. It's just installation instructions and a style guide. We need a whole new set of docs for that. Ideally generated directly from the code.

This makes sense to me. The question then becomes: Where should they live in the above structure?

Maybe a top-level section labeled "Contribution".


Contribution: This section provides instructions and best practices for developing the OpenSearch project.

  • Contribution pipeline and review overview
  • OpenSearch development guide
  • Dashboard's development guide
  • Security Plugin development guide

@keithhc2
Copy link
Contributor

keithhc2 commented Feb 22, 2022

By dev docs I meant documentation that new contributors can use to familiarize themselves with the internals of our systems. Looking at Andrews list, I see a lot that helps new users understand OpenSeach but for new contributors, there are still a lot of gaps.

For example, for OpenSeach Dashboards the only section I see is a link to the dev guide. Which makes sense for an OpenSeach user. But if someone wishes to contribute to Dashboards, the existing documentation is quite sparse. And the developer guide doesn't do much more than give us setup instructions and the style guide to follow.

Ultimately, this is going to depend on how we define what the docs site's purpose is. This is how we're currently organizing the information:

Repos:

What is this thing?
How do get started with development and build it from source?
How do I contribute changes or bug fixes?

Documentation:

What is this thing?
How do I install it?
How do I get started with it?
Reference stuff: supported operations, configuration, tuning, best practices, REST API, etc.

So if we want to include developer guide content in OpenSearch documentation, then I'm not sure what information we'd put in the repos themselves, and we don't want duplicated information anywhere. My thinking is if first-time contributors are looking to contribute to the projects, then they're on the repos already, so why not stick the developer guide information like how to contribute and more in-depth explanations of systems in the repo's README? If the docs site's purpose is to document how to use OpenSearch, then it doesn't make much sense to document how to contribute to OpenSearch on the docs site.

@Naarcha-AWS
Copy link
Collaborator

By dev docs I meant documentation that new contributors can use to familiarize themselves with the internals of our systems. Looking at Andrews list, I see a lot that helps new users understand OpenSeach but for new contributors, there are still a lot of gaps.
For example, for OpenSeach Dashboards the only section I see is a link to the dev guide. Which makes sense for an OpenSeach user. But if someone wishes to contribute to Dashboards, the existing documentation is quite sparse. And the developer guide doesn't do much more than give us setup instructions and the style guide to follow.

Ultimately, this is going to depend on how we define what the docs site's purpose is. This is how we're currently organizing the information:

Repos:

What is this thing? How do get started with development and build it from source? How do I contribute changes or bug fixes?

Documentation:

What is this thing? How do I install it? How do I get started with it? Reference stuff: supported operations, configuration, tuning, best practices, REST API, etc.

So if we want to include developer guide content in OpenSearch documentation, then I'm not sure what information we'd put in the repos themselves, and we don't want duplicated information anywhere. My thinking is if first-time contributors are looking to contribute to the projects, then they're on the repos already, so why not stick the developer guide information like how to contribute and more in-depth explanations of systems in the repo's README? If the docs site's purpose is to document how to use OpenSearch, then it doesn't make much sense to document how to contribute to OpenSearch on the docs site.

That is in line with how other Open Source tools operate, keep contribution instructions in the code base where contribution occurs. I do think there is value in having a hub in which we route users towards our repos though. For example, Elastic has a Community Contribution Program, a hub for users which to contribute to Elastic. They gate their instructions behind a web app that requires you to sign-up, which I don't love. However, the page and app does encourage contributors by promising recognition for those that make significant contributions.

@ashwin-pc
Copy link
Member

Ultimately, this is going to depend on how we define what the docs site's purpose is. This is how we're currently organizing the information:

Repos:

What is this thing? How do get started with development and build it from source? How do I contribute changes or bug fixes?

Documentation:

What is this thing? How do I install it? How do I get started with it? Reference stuff: supported operations, configuration, tuning, best practices, REST API, etc.

So if we want to include developer guide content in OpenSearch documentation, then I'm not sure what information we'd put in the repos themselves, and we don't want duplicated information anywhere. My thinking is if first-time contributors are looking to contribute to the projects, then they're on the repos already, so why not stick the developer guide information like how to contribute and more in-depth explanations of systems in the repo's README? If the docs site's purpose is to document how to use OpenSearch, then it doesn't make much sense to document how to contribute to OpenSearch on the docs site.

So where would the details for building say a plugin go? To build a plugin today all we have is a blog post. But it does not for example contains the details of how to generate one, what are the different functional libraries available to a plugin developer, different key concepts that one should know before building one etc.

I guess what i am looking for in the documentation is something like this:
https://developer.android.com/docs

Its not really a guide on how contribute to Android for example but how to build on top of it. And most of the contents are also very relevant to someone who's contributing.

The actual style guide and contribution specific details can and should remain in the repo, but guides, api, and other core concepts make more sense here

@keithhc2
Copy link
Contributor

Ultimately, this is going to depend on how we define what the docs site's purpose is. This is how we're currently organizing the information:
Repos:
What is this thing? How do get started with development and build it from source? How do I contribute changes or bug fixes?
Documentation:
What is this thing? How do I install it? How do I get started with it? Reference stuff: supported operations, configuration, tuning, best practices, REST API, etc.
So if we want to include developer guide content in OpenSearch documentation, then I'm not sure what information we'd put in the repos themselves, and we don't want duplicated information anywhere. My thinking is if first-time contributors are looking to contribute to the projects, then they're on the repos already, so why not stick the developer guide information like how to contribute and more in-depth explanations of systems in the repo's README? If the docs site's purpose is to document how to use OpenSearch, then it doesn't make much sense to document how to contribute to OpenSearch on the docs site.

So where would the details for building say a plugin go? To build a plugin today all we have is a blog post. But it does not for example contains the details of how to generate one, what are the different functional libraries available to a plugin developer, different key concepts that one should know before building one etc.

I guess what i am looking for in the documentation is something like this: https://developer.android.com/docs

Its not really a guide on how contribute to Android for example but how to build on top of it. And most of the contents are also very relevant to someone who's contributing.

The actual style guide and contribution specific details can and should remain in the repo, but guides, api, and other core concepts make more sense here

Right, and that's why I'm saying we should decide what the purpose of the OpenSearch docs site is, and whether that should just be how to use OpenSearch VS everything OpenSearch-related. I would say that if users are looking at the plugin's repo to download, clone, do whatever with a plugin, they should be able to look at that repo's README to get the information they need, rather than jump through another hoop to the docs site.

I agree that guides and APIs should be in the OpenSearch documentation, but I think it makes more sense to have breakdowns on what libraries are available in a plugin/what to know when thinking about contributing to a plugin within the repo, where they would actually make contributions and where the code actually lives.

I like that Android development link because it educates people on how to use Android to build custom apps. But they don't mention anything about contributing to Android, which is at an entirely different link: https://source.android.com/setup/contribute

@nateynateynate
Copy link
Member

I'd like to call out what I feel is a gap in our existing docs - In the situation where someone might be learning Docker and OpenSearch at the same time, I found the amount of guidance just a bit short. We should probably provide a generous dose of help for some basic docker familiarity. Some examples:

Installation and Configuration -

  • Using docker cp to pull out yml files, make changes, and then docker cp them back in. A corollary - it was also not quite clear where exactly the config file lives inside of the docker container, which I had to execute an interactive shell to figure out. I realize the difficulty is low for that, but we should at least have it properly documented.

  • Attaching new volumes to containers to prepare them as a snapshot repository through edits to docker-compose.yml prior to bringing up containers. New volumes cannot be attached to containers that have already been started.

@asafm
Copy link

asafm commented Mar 1, 2022

Regarding contribution - from someone who just enters OpenSearch as a developer (not a user), there is a huge gap of How OpenSearch Works: Internal Architecture / Design

  • How does replication works?
  • How does the Shard Allocator (Balancer) works?
  • How do plugins work?

A lot of excellent content in that area has been contributed as blog posts.
Today to learn those fields, I have to google, find all sorts of blogs posts, and piece it together.

I think it makes sense to have a section in the docs about the architecture and design of OpenSearch.

Here is an example of an issue I opened which has been closed since there is not such place to add that content to: #426

@ahopp
Copy link
Author

ahopp commented Mar 1, 2022

@asafm I've added a "How OpenSearch Works" section to the proposal let me know what you think.

@ahopp
Copy link
Author

ahopp commented Mar 1, 2022

I'd like to call out what I feel is a gap in our existing docs - In the situation where someone might be learning Docker and OpenSearch at the same time, I found the amount of guidance just a bit short. We should probably provide a generous dose of help for some basic docker familiarity. Some examples:

@nateynateynate I think we could include some of this in the install guides for OpenSearch via Docker (under "Installation and Configuration" section) and link out to Docker content for any command deep dives. What do you think?

@ahopp
Copy link
Author

ahopp commented Mar 1, 2022

I agree that guides and APIs should be in the OpenSearch documentation, but I think it makes more sense to have breakdowns on what libraries are available in a plugin/what to know when thinking about contributing to a plugin within the repo, where they would actually make contributions and where the code actually lives.

I like that Android development link because it educates people on how to use Android to build custom apps. But they don't mention anything about contributing to Android, which is at an entirely different link: https://source.android.com/setup/contribute

@keithhc2 So, where are we landing on the structure? I can make an update if we feel we're aligned.

@asafm
Copy link

asafm commented Mar 1, 2022

@asafm I've added a "How OpenSearch Works" section to the proposal let me know what you think.

@ahopp Looks good 👍

@stockholmux
Copy link
Member

I really like this. It's long over due and it's solving some long-standing concerns I've had.

Overall feedback: There are a lot of sections. I think, as a user, I might get overwhelmed by this list. Laterally, I also don't like how several dissimilar things are is bundled together ("Analytics, Dashboards, and Machine Learning" seems especially potpourri). I would suggest an additional layer of organization. Maybe three big areas - "Setting up & running OpenSearch," "Using OpenSearch for your data" and "OpenSearch internals & contribution" and then a few other general ones (maybe Overview, external links, API reference).

Other nits and picks:

"Security" section: I would definitely break this up. This seems to cover the footprint of the security plugin which bundles a lot dissimilar functionality. I would say that access control needs its own category. Encryption might also be a good area to cover. (SSL is listed as a query language which is amusingly incorrect).

"Search and Query" section: needs work. SSL is encryption - should it be PPL? No mention of DQL. SQL is listed twice.

"Clients and tools" section: break this up. Languages and clients are apples and oranges. I think this historical more than anything.

@nateynateynate
Copy link
Member

@nateynateynate I think we could include some of this in the install guides for OpenSearch via Docker (under "Installation and Configuration" section) and link out to Docker content for any command deep dives. What do you think?

Perfect. I sure would like to not have any reason to have to visit a different website to get the practical information needed. I'll take it.

@ashwin-pc
Copy link
Member

@asafm I've added a "How OpenSearch Works" section to the proposal let me know what you think.

Shouldn't we have a similar section for dashboards?

Btw this is what I was referring to when I mentioned developer docs being different from the developer guide on the repository

@brijos
Copy link

brijos commented Mar 1, 2022

"Clients and tools" section: break this up. Languages and clients are apples and oranges. I think this historical more than anything.

I agree. Below is how I see it.

Clients: This section will provide light documentation and links to download clients.
Clients include: Python, JavaScript, Java, etc.

Tools: This section will provide documentation for tools that are part of the OpenSearch project. Topics in this section include:

OpenSearch CLI
Ingestion tools (e.g. Data Prepper, Logstash, Fluentd, Tika, etc.)
Data collection agents (e.g. Beats, Fluent Bit, etc.)
Analytics tools (e.g. Grafana, Power BI, Tableau, etc.)
OpenTelemetry
OpenSearch Benchmark

@dlvenable
Copy link
Member

This is a big change and there are lots of https://opensearch.org/docs links out on the internet. I think it would be valuable to support redirecting old links to new links. Perhaps not every link could get a redirect, but a lot of the major pages should. As a user of documentation, I find it highly frustrating to follow an old link and then have to rework my way into the correct page.

@ahopp
Copy link
Author

ahopp commented Mar 3, 2022

support redirecting old links to new links

This is a great callout, thanks @dlvenable I believe there was previous work done to ensure there was redirecting from old docs (e.g., when the fork happened) so I imagine that can be extended to this restructuring. I'll add above to make sure we have it called out.

@ahopp
Copy link
Author

ahopp commented Mar 3, 2022

I agree. Below is how I see it.

@brijos I've updated to reflect your input - let me know if that works!

@passbt
Copy link

passbt commented Mar 3, 2022

I'm struggling to find any information on DQL in the documentation. When in the openseach UI in Discover page the default is syntax is DQL. If I click the DQL link in the search bar, I'm taken to this url, which gives me no relevant information that I'm aware of. It would be very helpful to link to a page with examples or update this page to include examples of how to use the DQL search bar. For now, I'm left searching using Lucene, because I find syntax documentation for Lucene.

I added my comment to this issue because I saw @stockholmux mention lack of DQL documentation in one of his comments.

@ahopp
Copy link
Author

ahopp commented Mar 3, 2022

Thanks @passbt I've added it explicit under query languages as well as the Dashboards section!

@ahopp
Copy link
Author

ahopp commented Mar 3, 2022

@macrakis I've moved k-NN and search plugins under "Optimization" (which itself is under "Search and Query"). What's your thought here - is this the right taxonomy?

@hdhalter hdhalter added this to the Anytime milestone May 21, 2022
@hdhalter hdhalter added the 1 - Backlog Issue: The issue is unassigned or assigned but not started label May 31, 2022
@Naarcha-AWS
Copy link
Collaborator

Here's a potential reorder for the major side-nav topics, as discussed by @hdhalter and I.

  • Overview
  • Installation and configuration
  • Migration
  • Security
  • Index and Ingestion
  • OpenSearch Dashboards
  • Observability
  • Search and Query
  • Availability and Recovery
  • Manage, Monitor, Tune
  • ML
  • Clients
  • Tools
  • REST API reference
  • Developer guides (for contribution)

Any thoughts?

@hdhalter hdhalter added 2 - In progress Issue/PR: The issue or PR is in progress. and removed 1 - Backlog Issue: The issue is unassigned or assigned but not started labels Jul 1, 2022
@dblock
Copy link
Member

dblock commented Sep 29, 2022

I think "Developer Guides" should be "Contributing", be fairly high level, and we should kick any actual developer guides out of the doc and move those entirely in READMEs.

REST API Reference is probably just "APIs" and has a tree of APIs that are all versioned.

@Naarcha-AWS Naarcha-AWS modified the milestones: 2022-Q3, 2022-Q4 Oct 10, 2022
@bbarani bbarani self-assigned this Feb 13, 2023
@hdhalter
Copy link
Contributor

Hi all, thanks for all your thoughtful feedback! I invite you to take a look at the recent version of the documentation at https://opensearch.org/docs/latest/. We've not only created A TON of content over the last year, but have restructured the TOC. We've arranged the topics into logical categories based on use case, rather than plugin. We've also added a feedback mechanism where visitors can give us direct feedback on the content or structure. Feel free to give it a try.

But this is only the beginning! We have a lot more enhancements in store. We are continuing to add content where there are gaps, we are improving the content that is there, and we are further categorizing the content. Also, we will decouple the content that doesn't follow the OpenSearch versioning.

We are excited to continue to helping to make the community successful in using OpenSearch, and I'll keep this issue open as we continue to enhance the documentation.

@ahopp
Copy link
Author

ahopp commented May 4, 2023

@hdhalter Do you think it's time for me to close this issue? Specifically since https://opensearch.org/docs/latest/ addresses the bulk of the proposal. I might suggest that it is time to close on the 2022 proposal and open a 2023+ proposal/plan.

Thoughts before I close?

@hdhalter
Copy link
Contributor

hdhalter commented May 4, 2023

Sure, @ahopp , I don't mind if we close this. Good idea to start a new one with the changes that are in flight and planned.

@hdhalter hdhalter closed this as completed May 4, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2 - In progress Issue/PR: The issue or PR is in progress. Usability
Projects
None yet
Development

No branches or pull requests