Update install instructions for final BuildStream 2.0 release

[ssssam]

See commits for rationale for the various changes.

[gtristan]

Overall, I like the install from source instructions here, however it looks like this is unfortunately again growing into a lot of overlap with the user facing instructions at: https://buildstream.build/install.html

Lets keep the user facing install paths published at https://buildstream.build/install.html and have the version specific developer/packager facing instructions at https://docs.buildstream.build/master/main_install.html. This makes the two pages serve obviously different purposes, and makes the reading of either of these less crowded and intense.

Maybe it would be helpful to have a little map:

                            Main page (A)
                             /       \
                           /           \
            Source (B-1), (B-2)       Docker (C)

(A): https://buildstream.build/install.html
(B-1): https://docs.buildstream.build/master/main_install.html
(B-2): https://docs.buildstream.build/1.6/main_install.html
(C): https://gitlab.com/BuildStream/buildstream-docker-images/-/blob/master/USING.md

While sub-pages might link back to the main user facing install page on the main website (A), we should always assume (A) as a starting point, and (A) should always be our answer for anyone's question of "How do I install BuildStream"

Introduction

I think the statement:

BuildStream is packaged in some Linux distributions. If your distro has an up-to-date package we recommend using that.

Can go away, this is already mentioned where it is better suited, at: https://buildstream.build/install.html

It could be good to have a forward in main_install.rst saying something like:

This page explains how to build and install this version of BuildStream from sources. If you are trying to install BuildStream on your computer it is recommended to try following the general purpose installation instructions here first, and return here only if you need to build and install from source.

Installation from PyPI

I think this can all go away and is not relevant here, and is already explained where it is better suited at: https://buildstream.build/install.html

Docker images

I think the whole "Using docker images" section here can also go away completely.

If there are user facing images ready to be used by users, we should trust https://gitlab.com/BuildStream/buildstream-docker-images/-/blob/master/USING.md to document how to use the latest BuildStream 1 and BuildStream 2 images respectively, and link to that page from the user facing documentation at https://buildstream.build/install.html (without going into a whole detailed shpiel about it there either)...

And of course we should touch up https://gitlab.com/BuildStream/buildstream-docker-images/-/blob/master/USING.md a bit to make sure it at least tells you the registry and image names which are useful to use, and posts some useful invocation like:

docker run -priviliged -it -v `pwd`:/build <bstimagename:latest> /bin/bash

Build/run dependencies

When discussing installing dependencies, here we are talking about the dependencies needed to install from source, so it is unnecessary to split up dependencies as we currently have them down (redundantly on a per-distro basis on top of it), with statements like: "Run this command" and then if that is not enough, also "Run this other command too".

Also, this would look so much more concise if it were simply a condensed table with two columns:

Distribution	Command
Arch	sudo pacman -S bubblewrap fuse3 git lzip patch python python-pip gcc
Debian	sudo apt-get install git lzip patch bubblewrap fuse3 python3-pip python3 g++ python3-dev

Otherwise right now, it looks like you have to scroll through pages of documentation just to install the distro dependencies needed to build and install BuildStream.

Checking out git

When talking about checking out git (which are the only installs we need to talk about on this page I guess), it is unnecessary to talk about Upgrading separately.

If we have instructed the user how to checkout a useful tag/branch which contains what they want, then they already have the ammunition they need go ahead and install a different or newer version later.

Install instructions

When discussing the actual installation instructions, it is worth noting:

• pip3 install --user . as a default just to conveniently install
• python3 setup.py install it is important to note that this is supported because this is the regular distro packager path, which does not involve downloading dependencies from the non-existing internet

We should also have a simple reminder here that depending on where you have decided to install buildstream, you may need to adjust $PATH or activate your virtual environment in order to use it.

This last part will allow us to nuke that whole Post-install setup section which merely talks about $PATH

Virtual environments

It is also worth noting as part of install instructions that you can use a python virtual environment if you want to have multiple versions of BuildStream installed on the same machine.

An interesting question is whether to duplicate virtual environment instructions on the main page (using PyPI packages) and here on the source installation page (installing from git).

If we continue to link to this page from the main page in order to discuss virtual environments, we should use an encoded link anchor in the reStructuredText for this:

.. _install-virtual-environment:

Because the main website has a link to: https://docs.buildstream.build/master/main_install.html#installing-in-virtual-environments which only happens to work because we didn't yet haphazardly rename the title of a section in our docs.

Completions

We should restore the section about completions as this is an important detail for distro packagers and also users.

For this we can merely link: https://github.com/apache/buildstream/blob/master/src/buildstream/data/bst

We can safely say (like the file says) where it should be installed, and we can even just paste the contents of that script and inform users that they can just append that text to ~/.bash_completion (this completion script never changes so it's quite safe).

---

After trimming out all that fat above, this would leave us with a more manageable structure of only:

• Introduction
• Dependencies for building/running this version of buildstream
• Installing buildbox
  • Instructions about binaries
  • Link(s) to how to build buildbox from sources in upstream buildbox
• Cloning the right git tag/branch
• Installation instructions (from the checkout)
  • Virtual environments
• Completions

[ssssam]

I'm taking another look at this today, thanks for the detailed guidance.

On dependencies and package lists, I will move them into a table, but we should still separate the install-time requirements (gcc, pip, python3-devel) from the runtime requirements. Let's try a table with 2 columns.

About a suitable 'install' command for distro packagers, it's useful to give some guidance on this. The python3 setup.py install command will work (as long as Bst doesn't change buildsystem), but it may not be what distro package guidelines recommend now. did some quick research to satisfy my own curiousity:

• Fedora recommends use of a %pyproject_install macro, which runs pip install --no-deps --no-index --use-pep517 . (link)
• Debian doesn't give any strong recommendation, the existing buildstream package uses pybuild via debhelper, which autodetects one of multiple backends. Running pybuild --detect in buildstream.git reports distutils, so they do indeed run setup.py directly, although there is a pyproject backend that may become default in future which runs python3 -m build and post-processes the generated .whl.

All this to highlight there's no "one size fits all" distro packager command, they should use whatever the Python guidelines for that distro recommends, and that can be either running setuptools directly or any PEP517 compatible frontend.

[ssssam]

Updated following the above guidance.

[ssssam]

Last 2 review comments now addressed in 7925769