Skip to content

Commit

Permalink
Add a brief explanation about repository peculiarities
Browse files Browse the repository at this point in the history
Also make the language about upstream/downstream focus more on the actual location of the source code (Google source repository vs GitHub).

Link this doc from CONTRIBUTING.md

PiperOrigin-RevId: 306535153
  • Loading branch information
GMNGeoffrey authored and copybara-github committed Apr 14, 2020
1 parent 1726929 commit d56d8ee
Show file tree
Hide file tree
Showing 2 changed files with 32 additions and 20 deletions.
7 changes: 7 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,13 +34,20 @@ use GitHub pull requests (PRs) for this purpose. Consult
[GitHub Help](https://help.github.com/articles/about-pull-requests/) for more
information on using pull requests.

## Peculiarities

We use a GitHub integration to import PRs into our upstream (Google internal)
source code management. Once it is approved internally, each PR will be merged
into the master branch as a single commit by the same tooling. The description
will match the PR title followed by the PR description. Accordingly, please
write these as you would a helpful commit message. Please also keep PRs small
(focused on a single issue) to streamline review and ease later culprit-finding.

Our documentation on
[repository management](https://github.com/google/iree/blob/master/docs/repository_management.md)
has more information on some of the oddities in our repository setup and
workflows.

## Community Guidelines

This project follows
Expand Down
45 changes: 25 additions & 20 deletions docs/repository_management.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,22 @@
# IREE Repository Management

Due to the process by which we synchronize this GitHub project with our internal
Google source code repository, there are some oddities in our workflows and
processes. We aim to minimize these, and especially to minimize their impact on
external contributors, but they are documented here for clarity and
transparency. If any of these things are particularly troublesome or painful for
your workflow, please reach out to us so we can prioritize a fix.

## Dependencies

As a project which brings together compiler, runtime and graphics systems,
dependency management is somewhat complex. We use git submodules for C++
dependencies and, where possible, language specific external package management
for other types of dependencies (i.e. Python).

In addition, the IREE Open Source project is actually downstream from its
"source of truth" within Google -- where dependencies are managed entirely
differently. This imposes constraints on repository management tasks that may
not be obvious.
In addition, dependencies are managed entirely different in the Google internal
source code repository. This imposes constraints on repository management tasks
that may not be obvious.

Shortcut commands (read below for full documentation):

Expand All @@ -30,15 +36,15 @@ Shortcut commands (read below for full documentation):

Currently, the two most challenging projects to manage as dependencies are
TensorFlow and LLVM. Both are typically pinned to specific versions that are
integrated upstream at a cadence up to many times per day. Further, because LLVM
does not ship with Bazel BUILD files, IREE "borrows" the BUILD files from
TensorFlow (for building with Bazel). Just to make it more interesting, since
TensorFlow does not ship with CMakeLists, IREE uses overlay CMakeLists.txt to
build subsets of TensorFlow needed for the compiler (when built with CMake).
While these externally managed build files are written to be moderately generic,
they can and do break and require manual intervention at times (i.e. there is no
guarantee that updating to a new commit of either will not require some manual
work on the build files).
integrated in the Google source repository at a cadence up to many times per
day. Further, because LLVM does not ship with Bazel BUILD files, IREE "borrows"
the BUILD files from TensorFlow (for building with Bazel). Just to make it more
interesting, since TensorFlow does not ship with CMakeLists, IREE uses overlay
CMakeLists.txt to build subsets of TensorFlow needed for the compiler (when
built with CMake). While these externally managed build files are written to be
moderately generic, they can and do break and require manual intervention at
times (i.e. there is no guarantee that updating to a new commit of either will
not require some manual work on the build files).

The only combination which is expected to work is the llvm-project commit noted
in the `LLVM_COMMIT` setting in
Expand All @@ -53,17 +59,16 @@ source of truth prior to accepting code that depends on it.
#### Adding dependencies

In general, adding dependencies will require coordination with Google engineers
to make the dependency available in both upstream and downstream. It is
important to ask on the mailing list prior to expecting to contribute such
changes.
to make the dependency available both on GitHub and in the Google source
repository. It is important to ask on the mailing list prior to expecting to
contribute such changes.

#### Pushing dependency changes

When working on a development branch, feel free to stage changes however makes
sense. However, when sending a PR, note that the upstream systems ignore any
submodule version updates when merging the commit. Our source of truth for
versions is in the `SUBMODULE_VERSIONS` file in the repository root. Here is an
example:
sense. However, when sending a PR, note that our integration systems will
overwrite the submodule version updates from the `SUBMODULE_VERSIONS` file in
the repository root. Here is an example:

```text
6ec136281086b71da32b5fb068bd6e46b78a5c79 third_party/abseil-cpp
Expand Down

0 comments on commit d56d8ee

Please sign in to comment.