Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add pex --docs and several pex3 docs options. #2365

Merged
merged 5 commits into from
Feb 15, 2024
Merged

Add pex --docs and several pex3 docs options. #2365

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
63651e8
Add `pex --docs` and several `pex3 docs` options.
jsirois Feb 15, 2024
002ea38
Fix changelog PR entry.
jsirois Feb 15, 2024
3cc81e1
It's hard to get used to minor bumps!
jsirois Feb 15, 2024
8d8f081
Fix unguarded typing imports.
jsirois Feb 15, 2024
6de7530
Mention the Python version policy.
jsirois Feb 15, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 30 additions & 0 deletions CHANGES.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,35 @@
# Release Notes

## 2.2.0

This release adds tools to interact with Pex's new embedded offline
documentation. You can browse those docs with `pex --docs` or, more
flexibly, with `pex3 docs`. See `pex3 docs --help` for all the options
available.

This release also returns to [SemVer](https://semver.org/) versioning
practices. Simply, you can expect 3 things from Pex version numbers:

+ The first component (the major version) will remain 2 as long as
possible. Pex tries very hard to never break existing users and to
allow them to upgrade without fear of breaking. This includes not
breaking Python compatibility. In Pex 2, Python 2.7 is supported as
well as Python 3.5+ for both CPython and PyPy. Pex will only continue
to add support for new CPython and PyPy releases and never remove
support for already supported Python versions while the major version
remains 2.
+ The second component (the minor version) will be incremented whenever
a release adds a feature. Since Pex is a command line tool only (not
a library), this means you can expect a new subcommand, a new option,
or a new allowable option value was added. Bugs might also have been
fixed.
+ The third component (the patch version) indicates only bugs were
fixed.

You can expect the minor version to get pretty big going forward!

* Add `pex --docs` and several `pex3 docs` options. (#2365)

## 2.1.164

This release moves Pex documentation from https://pex.readthedocs.io to
Expand Down
28 changes: 25 additions & 3 deletions pex/bin/pex.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
)
from pex.common import die, is_pyc_dir, is_pyc_file, safe_mkdtemp
from pex.dependency_manager import DependencyManager
from pex.docs.command import serve_html_docs
from pex.enum import Enum
from pex.inherit_path import InheritPath
from pex.interpreter_constraints import InterpreterConstraints
Expand All @@ -50,7 +51,7 @@

if TYPE_CHECKING:
from argparse import Namespace
from typing import Dict, Iterable, Iterator, List, Optional, Set, Text, Tuple
from typing import Dict, Iterable, Iterator, List, NoReturn, Optional, Set, Text, Tuple

import attr # vendor:skip

Expand Down Expand Up @@ -90,6 +91,13 @@ def __call__(self, parser, namespace, values, option_str=None):
sys.exit(0)


class OpenHtmlDocsAction(Action):
def __call__(self, *args, **kwargs):
# type: (...) -> NoReturn
try_(serve_html_docs(open_browser=True))
sys.exit(0)


def configure_clp_pex_resolution(parser):
# type: (ArgumentParser) -> None
group = parser.add_argument_group(
Expand Down Expand Up @@ -762,8 +770,22 @@ def configure_clp():
"--help-variables",
action=PrintVariableHelpAction,
nargs=0,
help="Print out help about the various environment variables used to change the behavior of "
"a running PEX file.",
help=(
"Print out help about the various environment variables used to change the behavior "
"of a running PEX file."
),
)
parser.add_argument(
"--docs",
"--help-html",
dest="open_html_docs",
action=OpenHtmlDocsAction,
nargs=0,
help=(
"Open a browser to view the embedded documentation for this Pex installation. For "
"more flexible interaction with the embedded documentation, you can use this Pex "
"installation's `pex3` script. Try `pex3 docs --help` to get started."
),
)

return parser
Expand Down
258 changes: 57 additions & 201 deletions pex/cli/commands/docs.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,220 +1,76 @@
# Copyright 2024 Pex project contributors.
# Licensed under the Apache License, Version 2.0 (see LICENSE).

import errno
import json
import logging
import os
import re
import signal
import subprocess
import sys
import time
from textwrap import dedent

from pex import docs
from pex.cli.command import BuildTimeCommand
from pex.commands.command import try_open_file
from pex.common import safe_open
from pex.result import Error, Result
from pex.typing import TYPE_CHECKING
from pex.variables import ENV
from pex.version import __version__

if TYPE_CHECKING:
from typing import Optional, Union

import attr # vendor:skip
else:
from pex.third_party import attr


logger = logging.getLogger(__name__)


SERVER_NAME = "Pex v{version} docs HTTP server".format(version=__version__)
SERVER_DIR = os.path.join(ENV.PEX_ROOT, "docs", "server", __version__)


@attr.s(frozen=True)
class Pidfile(object):
_PIDFILE = os.path.join(SERVER_DIR, "pidfile")

@classmethod
def load(cls):
# type: () -> Optional[Pidfile]
try:
with open(cls._PIDFILE) as fp:
data = json.load(fp)
return cls(url=data["url"], pid=data["pid"])
except (OSError, IOError, ValueError, KeyError) as e:
logger.warning(
"Failed to load {server} pid file from {path}: {err}".format(
server=SERVER_NAME, path=cls._PIDFILE, err=e
)
)
return None
from pex.docs.command import HtmlDocsConfig, register_open_options, serve_html_docs
from pex.docs.server import SERVER_NAME, Pidfile
from pex.docs.server import shutdown as shutdown_docs_server
from pex.result import Ok, Result, try_

@staticmethod
def _read_url(
server_log, # type: str
timeout, # type: float
):
# type: (...) -> Optional[str]

# The permutations of Python versions, simple http server module and the output it provides:
# 2.7: Serving HTTP on 0.0.0.0 port 46399 ... -mSimpleHttpServer
# 3.5: Serving HTTP on 0.0.0.0 port 45577 ... -mhttp.server
# 3.6+: Serving HTTP on 0.0.0.0 port 33539 (http://0.0.0.0:33539/) ... -mhttp.server
class Docs(BuildTimeCommand):
"""Interact with the Pex documentation.

start = time.time()
while time.time() - start < timeout:
with open(server_log) as fp:
for line in fp:
if line.endswith(os.linesep):
match = re.search(r"Serving HTTP on 0.0.0.0 port (?P<port>\d+)", line)
if match:
port = match.group("port")
return "http://localhost:{port}".format(port=port)
return None
With no arguments, ensures a local documentation server is running and then opens a browser
to view the local docs.
"""

@classmethod
def record(
cls,
server_log, # type: str
pid, # type: int
timeout=5.0, # type: float
):
# type: (...) -> Optional[Pidfile]
url = cls._read_url(server_log, timeout)
if not url:
return None

with safe_open(cls._PIDFILE, "w") as fp:
json.dump(dict(url=url, pid=pid), fp, indent=2, sort_keys=True)
return cls(url=url, pid=pid)

url = attr.ib() # type: str
pid = attr.ib() # type: int

def alive(self):
# type: () -> bool
# TODO(John Sirois): Handle pid rollover
try:
os.kill(self.pid, 0)
return True
except OSError as e:
if e.errno == errno.ESRCH: # No such process.
return False
raise

def kill(self):
# type: () -> None
os.kill(self.pid, signal.SIGTERM)


@attr.s(frozen=True)
class LaunchResult(object):
url = attr.ib() # type: str
already_running = attr.ib() # type: bool


def launch_docs_server(
document_root, # type: str
port, # type: int
timeout=5.0, # type: float
):
# type: (...) -> Union[str, LaunchResult]

pidfile = Pidfile.load()
if pidfile and pidfile.alive():
return LaunchResult(url=pidfile.url, already_running=True)

# Not proper daemonization, but good enough.
log = os.path.join(SERVER_DIR, "log.txt")
http_server_module = "http.server" if sys.version_info[0] == 3 else "SimpleHttpServer"
env = os.environ.copy()
# N.B.: We set up line buffering for the process pipes as well as the underlying Python running
# the http server to ensure we can observe the `Serving HTTP on ...` line we need to grab the
# ephemeral port chosen.
env.update(PYTHONUNBUFFERED="1")
with safe_open(log, "w") as fp:
process = subprocess.Popen(
args=[sys.executable, "-m", http_server_module, str(port)],
env=env,
cwd=document_root,
preexec_fn=os.setsid,
bufsize=1,
stdout=fp.fileno(),
stderr=subprocess.STDOUT,
def add_extra_arguments(cls, parser):
register_open_options(parser)
parser.add_argument(
"--no-open",
dest="open",
default=True,
action="store_false",
help="Don't open the docs; just ensure the docs server is running and print its info.",
)

pidfile = Pidfile.record(server_log=log, pid=process.pid, timeout=timeout)
if not pidfile:
try:
os.kill(process.pid, signal.SIGKILL)
except OSError as e:
if e.errno != errno.ESRCH: # No such process.
raise
return log

return LaunchResult(url=pidfile.url, already_running=False)


def shutdown_docs_server():
# type: () -> bool

pidfile = Pidfile.load()
if not pidfile:
return False

logger.info(
"Killing {server} {url} @ {pid}".format(
server=SERVER_NAME, url=pidfile.url, pid=pidfile.pid
kill_or_info = parser.add_mutually_exclusive_group()
kill_or_info.add_argument(
"-k",
"--kill-server",
dest="kill_server",
default=False,
action="store_true",
help="Shut down the {server} if it is running.".format(server=SERVER_NAME),
)
kill_or_info.add_argument(
"--server-info",
dest="server_info",
default=False,
action="store_true",
help="Print information about the status of the {server}.".format(server=SERVER_NAME),
)
)
pidfile.kill()
return True


class Docs(BuildTimeCommand):
"""Interact with the Pex documentation."""

def run(self):
# type: () -> Result
html_docs = docs.root(doc_type="html")
if not html_docs:
# TODO(John Sirois): Evaluate if this should fall back to opening latest html docs instead of just
# displaying links.
return Error(
dedent(
"""\
This Pex distribution does not include embedded docs.

You can find the latest docs here:
HTML: https://docs.pex-tool.org
PDF: https://github.com/pex-tool/pex/releases/latest/download/pex.pdf
"""
).rstrip()
)
if self.options.server_info:
pidfile = Pidfile.load()
if pidfile and pidfile.alive():
return Ok(
"{server} serving {info}".format(server=SERVER_NAME, info=pidfile.server_info)
)
return Ok("No {server} is running.".format(server=SERVER_NAME))

if self.options.kill_server:
server_info = shutdown_docs_server()
if server_info:
return Ok("Shut down {server} {info}".format(server=SERVER_NAME, info=server_info))
return Ok("No {server} was running.".format(server=SERVER_NAME))

# TODO(John Sirois): Consider trying a standard pex docs port, then fall back to ephemeral if:
# 2.7: socket.error: [Errno 98] Address already in use
# 3.x: OSError: [Errno 98] Address already in use
# This would allow for cookie stickiness for light / dark mode although the furo theme default of detecting
# the system default mode works well in practice to get you what you probably wanted anyhow.
result = launch_docs_server(html_docs, port=0)
if isinstance(result, str):
with open(result) as fp:
for line in fp:
logger.log(logging.ERROR, line.rstrip())
return Error("Failed to launch {server}.".format(server=SERVER_NAME))
launch_result = try_(
serve_html_docs(
open_browser=self.options.open, config=HtmlDocsConfig.from_options(self.options)
)
)
if self.options.open:
return Ok()

logger.info(
return Ok(
(
"{server} already running at {url}"
if result.already_running
else "Launched {server} at {url}"
).format(server=SERVER_NAME, url=result.url)
"{server} already running {info}"
if launch_result.already_running
else "Launched {server} {info}"
).format(server=SERVER_NAME, info=launch_result.server_info)
)
return try_open_file(result.url)
Loading
Loading