A New Documentation Working Group

[ericsnekbytes]

Summary

As discussed in the JupyterLab weekly meeting, this issue proposes a new Jupyter Documentation working group, to coordinate docs work across the entire Jupyter ecosystem (Notebook, Lab, Server, etc.).

Why make a working group?

Repeated and recent docs breakages have made clear the need for focused and coordinated work across subprojects, to ensure that users are not turned away from the Jupyter ecosystem because of bad early experiences from untended documentation. Distributed docs efforts per subproject also produce content with disparate structure and quality, even when common information is being explained.

By coordinating docs work across the ecosystem and regularly focusing efforts on docs quality, we can achieve better consistency and quality across subprojects, help prevent docs bugs and errors due to core code drift, and help the community by increasing discovery and access to information they need, and helping to identify information that needs to be added.

Possible major focuses:

• Content (writing docs)
• Discovery/community engagement (getting users info they need)
• Feedback (what needs to be documented? what info do people need?)

Potential early work targets:

• Characterize documentation needs across subprojects (what needs to be added, improved, made consistent across each)
• Look at the wishlist issue and forum post for tasks to bring in
  • Add a footer to docs pages, etc.
• Capture/store (possibly automate) readthedocs traffic stats/CSVs for last 90 days on a regular basis to get rough metrics on what’s important to users
• Add some easy CI tests to ensure commands in the tutorial run without erroring (would have helped prevent the docs from breaking this time as mentioned above), other CI tests
• “Get Help” effort spanning multiple subprojects, to consistently communicate to users how they can get help from Jupyter stakeholders/developers/resources, on each subproject's documentation site
• Document insights from recent issues/bugs (such as the Homebrew bug)
• And more!

What can I do?

If you're interested in working on docs efforts, leave a comment below to express your interest. When we have enough interest, we can start next steps in officially forming the working group.

---

Resources

The jupyter/docs repo holds the draft group charter, meeting minutes, etc.
https://github.com/jupyter/docs

---

First meeting

(This section will be kept updated with current planning around the first meeting)

A first meeting date/schedule has been (tentatively) decided! It will be weekly, on Wednesday at 11am EST, starting October 4th (2 weeks from this post). This is an open call, so (anyone) please join if you're interested :)

Agenda:

• Discuss: Documentation Working Group charter
• Begin early docs work / planning
• Plan for next steps / formal recognition of the working group
  • Create a team compass and adapt Jupyter resources (code of conduct/governance, etc.)

---

Mentions: @chbrandt @JasonWeill
See also:

• Jupyter Forum post
• New Documentation Working Group proposal jupyter/jupyter#699
• New Documentation Working Group proposal jupyterhub/team-compass#681

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[willingc]

Thanks @ericsnekbytes for kicking this off. Have you announced this issue on Discourse? Given that it involves docs across all of Jupyter, it would be helpful to spread the word. Thanks!

[ericsnekbytes]

Good idea, I'll make a post there :)

[fcollonval]

Thanks a lot @ericsnekbytes for starting this.

[chbrandt]

Thanks for tagging me, @ericsnekbytes . I'm surely in.

[tobkin]

Subscribe me

[chbrandt]

Hey @ericsnekbytes, would you like to have a first discussion on the first steps?

For instance, as you pointed out, the mapping (or characterization) of subprojects docs and estimating how to dettach them would be an important move. That would allow us to understand better the impact for the developers before anything else.

[ericsnekbytes]

Hi all! Back from vacation now :) I'm going to suggest a weekly meeting time and a tentative first meeting date for this new group in the Notebook/Lab calls today (probably in ~2 weeks to give people a chance to learn about it and attend). Thanks to everyone who has commented/expressed interest.

I've updated the description with the documentation wishlist and will be adding a link to the forum post too.

[ericsnekbytes]

A first meeting date/schedule has been (tentatively) decided! It will be weekly, on Wednesday at 11am EST, starting October 4th (2 weeks from this post). This is an open call, so (anyone) please join if you're interested :)

I will keep the description at the top of the page updated with first-meeting plans/agenda (more details there), chime in if you have something to add :)

[yuvipanda]

Hello from the JupyterHub project (although only speaking for myself) :)

First, as someone who steps into writing JupyterLab code very ocassionally, let me say I'm very excited about the momentum behind this working group! Running JupyterLab has a lot of moving pieces, and knowing 'what is where?' is often difficult. I think this will help a lot in guiding new people in, and thank you @ericsnekbytes for setting it up (also, wonderful username :D)

All the components discussed here seem to be related to JupyterLab primarily, along with the server bits needed to run that (jupyter server, RTC, etc). I don't think there's any participation from the JupyterHub community here at this point - I only knew of this because @willingc pinged me in jupyterhub/mybinder.org-deploy#2732 (thanks Carol!). So the primary question I have is - what is the scope of this? Is JupyterHub (and by extension, binder) in or out of scope here?

My gut feel from reading through this issue and looking at the folks involved, is that JupyterHub / Binder is out of scope for here, at least for now. Given that, I have a specific, small ask - can this be renamed "the JupyterLab documentation working group" or something similar? I completely agree that a documentation WG would be very useful, and don't want to add any stopping energy to the momentum y'all have got! So a quick rename would help clarify scope and let y'all continue.

And in the future, I'd love for this to be expanded and JupyterHub also become in scope! However, I'd like that to be socialized into the JupyterHub community via our own processes, such as:

1. The JupyterHub Team Compass repo
2. The JupyterHub Collaboration Cafe, the equivalent of the JupyterLab team meeting.

With work done there and participation, this could (and I hope it does!) expand into a "Jupyter Documentation Working Group". However, doing this organizational work itself is a lot of effort, and I can't volunteer myself to be here on behalf of the JupyterHub team, nor can I volunteer anyone else from the JupyterLab or JupyterHub communities(given lack of response to the issue Carol opened). I don't want to impinge on the momentum that exists in the JupyterLab / Jupyter Server side for this work, and hope the rename will clarify things for now.

How does that sound? :)

[krassowski]

It would be very valuable to have someone from JupyterHub join the meetings, not only to provide feedback on how and when to cross-link to hub docs, but also because (in my opinion) hub has one of the better docs in Jupyter ecosystem, at least in some aspects.

PS. I believe that we advised on subsequent JupyterLab calls that before starting the group initiators join calls of other Jupyter subprojects to announce the intent and consult, but I do not know if this advice was acted on and what the conversation looked like in subproject meetings.

[ericsnekbytes]

I want to be clear that the mission of this Docs Working Group is not to control or dictate anything to the subprojects.

The Docs Working Group should exist as a support/helper and advisory group to all the subprojects in Jupyter. Here are some additional details to clarify the WG's scope and role (tentative and subject to future feedback):

The WG's scope and role:

• Help write docs (inside the bounds of each subproject's governance)
• Develop recommendations and guidance:
  • For communicating common information and concepts across the Jupyter ecosystem
  • For style and best practices when authoring documentation media (including technical advice such as tools-usage and markdown guidance that helps support high level goals, like interlinking/cross-connection between subprojects)
  • ...because we want to encourage consistency across the Jupyter ecosystem and between different projects
• Community engagement
  • To help connect users with information
  • To gather feedback from the community about what needs to be documented/what information they need
• Help support and improve all aspects of documentation across the Jupyter ecosystem
  • This means, for members of the group, also taking on any related tasks that will be beneficial to docs and support the Docs Working Group's goals (such as writing PRs for unit tests related to documentation code, automations for gathering ReadTheDocs traffic stats, etc.)

One reason for starting a WG is to provide a place for consistent, focused effort to be spent specifically on docs across the whole ecosystem.

In other words, the WG exists in part to provide capacity and resources to the subprojects (some of which are already suffering from a lack of resources/capacity, and more specifically to work on docs in particular).

Reflecting on the above, a goal/mission for the group might boil down to:

Goals/Mission

• Help support and improve all aspects of documentation across the Jupyter ecosystem
• Make high quality documentation that is clear, comprehensive, and serves the varying needs of Jupyter's diverse community
• Engage with the community to help users get the information they need

---

I would prefer to keep the name the same, as it reflects the Docs Working Group's goal of providing resources/capacity/guidance to all subprojects in the Jupyter ecosystem.

(And to be clear, the docs working group is not here to dictate to JupyterHub or any other subproject, any guidance or help offered by the docs working group would not supersede or affect JupyterHub's internal guidance and processes at all)

I will echo @krassowski 's thoughts that it would be very valuable to have someone from JupyterHub in the process (hopefully at least to provide input and feedback during this process as we attempt to get the docs working group started).

What would it take for you to be comfortable with this working group? Personally, I would want you to feel like the docs working group is here to help support your subproject with its own mission as a partner.

I would also appreciate suggestions from others about how best to connect with other subprojects and stakeholders to bring them into this discussion. I have attended the Notebook, Lab, Server, and ipywidgets meetings to gather support and volunteers, but there are many people and subprojects within Jupyter and more to go for this discussion to be "complete". Please spread the word to your circle!

[yuvipanda]

Thanks @ericsnekbytes!

What would it take for you to be comfortable with this working group?

This is actually a great question, and perhaps is the core of the issue. Speaking for myself and not the whole jupyterhub community, here's some that would help!

• Move discussion out of the jupyterlab org into perhaps the jupyter org, so there is wider visibility and clarity who this is intended for. Optionally post in the jupyterhub/team-compass repo to advertise this location.
• Attend the next JupyterHub Collaboration Cafe (https://blog.jupyter.org/online-collaboration-caf%C3%A9-launch-jupyterhub-team-meetings-to-become-more-collaborative-spaces-b713edadf15) and invite people to come! I think even right now, very few people in the JupyterHub community know about this - the discussion is happening in a jupyterlab org issue, and the only link to this was from a mybinder.org-deploy issue.

I think these two would go a long way in including the JupyterHub community more in this discussion!

I would also appreciate suggestions from others about how best to connect with other subprojects and stakeholders to bring them into this discussion. I have attended the Notebook, Lab, Server, and ipywidgets meetings to gather support and volunteers, but there are many people and subprojects within Jupyter and more to go for this discussion to be "complete".

Absolutely, the ecosystem is very large and completeness is fleeting. And bringing people together is also a lot of work (which is why I didn't offer to do it :D), so I appreciate you doing this :) Hopefully the prior suggestions will help bring more folks from the JupyterHub community onto this, and expand how much of the Jupyter ecosystem it covers.

[ericsnekbytes]

Thanks @yuvipanda for those suggestions, I will be attending the next JupyterHub Collaboration Cafe, and have made issues in other repos that point to this discussion. (Apologies for the delay, I've been furiously writing Lab docs for the plugin system that I've been trying to finish this week).

There was no consensus on a clear Jupyter-wide repo to move to (governance? jupyter/jupyter?), so logistically it was easier to point to this issue from those repos...we can still move this discussion if it's needed (and with someone who has proper permissions). I've linked to those new ones in the issue description here. Thanks!

[willingc]

Thanks @yuvipanda and @ericsnekbytes for the continued discussion.

As someone who championed documentation for many years, I do think that if this is a Project-wide effort, and the Docs Workgroup should happen in a common Jupyter space.

While JupyterLab is an important project, it's not the only project; JupyterHub/Binder projects serve a different set of users and developers.

While I commend the energy to improve documentation project-wide, I do think that it would be valuable to have a common meta-Jupyter space for discussions, meeting notes, and principles.

One group, accessibility, needs visibility and input to documentation as well.

cc/ Exec Team (@blink1073 @fperez @ellisonbg @jasongrout @Ruv7 @afshin)

[willingc]

P.S. Thanks @ericsnekbytes for working hard on the JupyterLab plugin docs ✨ 🥇

[rpwagner]

(Sorry for any spurious notifications about edits, I thought the conversation was moved to the governance repo.)

First, I am very happy to folks willing to put effort into this activity. From the security perspective, one of the repeated pieces of feedback we've heard from groups deploying Jupyter components (JupyterLab, JupyterHub, etc.) who aren't active in the community is that it's challenging to get a unified view of "what is Jupyter" and how the components work together.

Next, I will second @yuvipanda's comments about ensuring this is a Jupyter-wide activity and not just within a specific Subproject. This WG has the opportunity to improve on that feedback I mentioned, provided the documentation takes a holistic approach.

[ericsnekbytes]

As someone who championed documentation for many years, I do think that if this is a Project-wide effort, and the Docs Workgroup should happen in a common Jupyter space.

I would be happy moving this discussion to another repo in the Jupyter org, does someone have permissions? Do we know which repo is the best place? Perhaps jupyter/governance? I haven't moved it yet only for these kind of logistical reasons.

While JupyterLab is an important project, it's not the only project; JupyterHub/Binder projects serve a different set of users and developers.

@willingc This is an important point to hit. The Docs Working Group should serve as a resource/helper to all the subprojects, and each will have different needs. As I mentioned above, this group should hook into existing subproject's meetings/governance to provide docs-writing-help and assistance to provide that tailored approach. I also suspect there will be big benefits for users/readers just from taking a coordinated approach across the whole Jupyter ecosystem, which should result in more consistency and context in each of the subproject documentation sites (as mentioned by @rpwagner ).

One group, accessibility, needs visibility and input to documentation as well.

This is another important point, and it should be one of the main goals of this working group: Making sure that Jupyter docs serve the varying needs of Jupyter's diverse community.

We know the Jupyter community is full of people from different fields (Sciences, Technology, Finance, Engineering, Etc), some who may not be professional programmers, and who have varying levels of experience with many of the processes, tools and paradigms being used inside Jupyter products. We should strive to open the documentation up to as many of these people as possible through a variety of means.

one of the repeated pieces of feedback we've heard from groups deploying Jupyter components (JupyterLab, JupyterHub, etc.) who aren't active in the community is that it's challenging to get a unified view of "what is Jupyter" and how the components work together.

@rpwagner This is a major motivating factor for me starting this proposal, as this feedback mirrors my own experience with Jupyter! I want other people to have a clearer picture of what Jupyter is, so they don't go through the same cycle of confusion I experienced :)

[krassowski]

Do we know which repo is the best place? Perhaps jupyter/governance?

I think this should go to jupyter/governance at this stage, but I do not know. I opened jupyter/governance#184 to suggest clarifying this with a solution I think is good but maybe there are better ideas.

I would be happy moving this discussion to another repo in the Jupyter org, does someone have permissions? [...] I haven't moved it yet only for these kind of logistical reasons.

The process is quite cumbersome:

• a) [ ] create a new temporary repository under jupyterlab organization; I think it could be private
• b) [ ] move this issue to that repository
• c) [ ] transfer the temporary repository to jupyter organization
• d) [ ] someone who has admin rights on jupyter needs to accept the transfer
• e) [ ] transfer the issue from the temporary repository to the governance repository

[willingc]

For simplicity, we may want to just use the jupyter/jupyter repo for the docs workgroup. Its purpose was docs across the project, and it seems a natural place to have discussion. Another alternative would be to create/use jupyter/docs. cc/ @ericsnekbytes @krassowski

The more I think about this. Perhaps creating jupyter/docs makes the most sense.

[willingc]

I created jupyter/docs to get the Work Group discussions started in a common space. The repo can be renamed later, if desired. @ericsnekbytes, I gave you write privileges and @krassowski, I granted you maintainer privileges.

[jtpio]

The more I think about this. Perhaps creating jupyter/docs makes the most sense.

Should https://github.com/jupyter/jupyter be focused on just hosting the jupyter metapackage then?

There is already some good documentation content in https://github.com/jupyter/jupyter, which I guess could be moved to https://github.com/jupyter/docs?

[ericsnekbytes]

Reminder to everyone here that the first meeting is scheduled for tomorrow, October 4th at 11am EST (half an hour before the Notebook weekly call). Bring your thoughts/ideas! :)

I am adding some preliminary info to the jupyter/docs repo, and will be adding a PR for a charter/mission statement for the group soon. Thanks @willingc for setting that up :)

[willingc]

@jtpio I think that your suggestion to move the doc content from jupyter/jupyter to jupyter/docs makes good sense. I can't remember why we never did a separate docs repo.

[trallard]

I'd be interested in forming part of this working group.

Thank you all for all the ground work done so far.

[ericsnekbytes]

Weekly meeting is starting now! :)

[Ruv7]

Hi all - thanks for the mention @willingc - I appreciate it because I've been out of the office for a few weeks and may have missed this otherwise. Happy to see this work taking off!

A heads up that @marthacryan and I are actively working on documentation for processes related to establishing new working groups and standing committees. Please reach out if you have any questions about processes. We expect to have something to share in the next week or two but in the meantime I would advise the group to take a look at the charters for existing working groups/standing committees:

• 1st Working Group: Jupyter Community Building (JCB)
• 1st Standing Committee: Diversity, Equity, and Inclusion (DEI)
• 2nd Working Group (pending approval): Jupyter Media Strategy (JMS)

[ericsnekbytes]

Reminder to everyone that the weekly meeting starts in 1 hour (8AM PST, Oct 11 2023), please join!

@Ruv7 Thanks for those links!

[ivanov]

I'm just learning about this working group, or proposed working group, thanks to the cross-post to jupyter/jupyter#699.

I want to say I'm relieved to hear this:

I want to be clear that the mission of this Docs Working Group is not to control or dictate anything to the subprojects.

and also wanted to second @jtpio's point about the wealth of docs on https://github.com/jupyter/jupyter which I've been sprucing up in the last three days and is back down to zero open PRs (from 22), and 60 fewer open issues (and counting).

I would be happy to help transfer this issue to jupyter/governance, but I gave up my admin bit here just last week, so some one else will have to do that (or temporarily elevate me admin again).

[ericsnekbytes]

The time has come (after much preparing!) to approach the Executive Council for official recognition of the Docs Working Group. Any community members (and group founders) who can attend the next EC meeting (Thu Dec. 14 at 10AM PST, zoom link) are welcome and appreciated! Please see this issue on the EC team compass for more info.

[ericsnekbytes]

Pinging founding members @chbrandt @willingc @fcollonval @krassowski @bollwyvl @ivanov @RRosio

[willingc]

@ericsnekbytes I have a meeting conflict for the EC meeting. Thanks for all of the organizational work that you have been doing.

[ericsnekbytes]

Hi all, here with a reminder that Docs Working Group meetings have resumed (after the holiday break), and one is being held now if you can join (8am PST). Cheers!

[willingc]

@ericsnekbytes Hope the meeting went well. Would it be possible (or did I miss) a calendar invite for this standing meeting? Thanks! P.S. Best email: willingc@gmail.com

[ericsnekbytes]

@willingc The Docs Working Group weekly meeting is 8am PST on Wednesdays, you should be able to see it on the public community calendar (let me know if not, I can send an email or otherwise modify the calendar event) :)

[ericsnekbytes]

The Jupyter Docs Working Group is official now :D

Thank you! to everyone who participated here in the proposal, and the charter acceptance process 🚀 Now let's go build some docs :)