Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Clearly document the prelude file #1674

Closed
davidsantiago opened this issue Aug 22, 2016 · 20 comments
Closed

Clearly document the prelude file #1674

davidsantiago opened this issue Aug 22, 2016 · 20 comments
Labels
P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) stale Issues or PRs that are stale (no activity for 30 days) team-Documentation Documentation improvements that cannot be directly linked to other team labels team-Starlark-Integration Issues involving Bazel's integration with Starlark, excluding builtin symbols type: documentation (cleanup)

Comments

@davidsantiago
Copy link
Contributor

davidsantiago commented Aug 22, 2016

I just came across the concept of the bazel prelude while trying to resolve another issue (#1673), and I could not find any explanation of how the prelude is supposed to work anywhere. Searching bazel.io turns up nothing, and the google groups have only two passing mentions. It does appear to be used in another bazel project, as the Skydoc documentation instructs users to insert a snippet into their prelude. I think it would be helpful to get an explanation of what the prelude is for and how to use it effectively.

@katre katre added type: documentation (cleanup) P3 We're not considering working on this, but happy to review a PR. (No assignee) labels Aug 22, 2016
@ahyangyi
Copy link

ahyangyi commented Aug 29, 2016

Not really related but I wish could be fixed together: in-bazel error messages of BUILD files are messy when prelude is used.

Steps to reproduce:

  • Get a working workspace that uses prelude.
  • Find a BUILD file, add the following lines:
cc_library(

You get error messages that look like:

ERROR: /your/path/to/your/build/file/BUILD:51:9: syntax error at 'newline': expected ,
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:1:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:2:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:3:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:4:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:5:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:6:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:7:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:8:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
ERROR: /your/path/to/your/prelude_bazel/prelude_bazel:9:6: file '@//path/to/some/bazel/file/loaded/by/prelude.bzl' was not correctly loaded. Make sure the 'load' statement appears in the global scope in your file.
...

which is extremely messy, and I doubt the users will notice that the correct diagnosis is in the first line of hundreds lines of error messages.

@laurentlb
Copy link
Contributor

@ahyangyi Thanks, we've now fixed the error messages.
Please file a new bug if you still see confusing messages.

@ahyangyi
Copy link

Thank you, @laurentlb !

@mfarrugi
Copy link

What are some use cases for a prelude? Is this a feature bazel is trying to get rid of?

I have seen adding load statements for language rules, but I would rather be able to put those rules into global scope in a WORKSPACE instead.

Is it free to have an unused load in a BUILD file?

@laurentlb
Copy link
Contributor

The prelude file is not very well supported. Inside Google, we rarely use it (we used it a few times to assist us during a migration). The prelude file doesn't play well with incremental builds: if you modify it (or any of its dependencies), every package will have to be reloaded.

To automatically removed unused load statements from you BUILD files, try buildozer (https://github.com/bazelbuild/buildtools/tree/master/buildozer):
buildozer 'fix unusedLoads' path/to/BUILD

@alandonovan
Copy link
Contributor

alandonovan commented May 11, 2020

Laurent, could you confirm my understanding of the prelude?

  • The prelude is an optional file whose contents are effectively textually inserted at the start of every BUILD (but not .bzl) file loaded by Bazel. It is thus a place to stuff workspace-global definitions.
  • The location of the prelude is fixed, but the contents may be edited by users.
  • Bazel's prelude is found at //tools/build_rules:prelude_bazel. By default there is no such file.
  • Exoblaze uses the prelude to define various rules such as cc_binary, using built-in implementations.
  • Blaze does not look for a prelude.
  • In the future, implicit textual insertion will be replaced by implicit load-like semantics. Few users should notice the effects of that change.
  • The prelude was never intended to be a feature. It was designed as a crutch to aid the Bazel team with their migrations. Ideally it would be removed.

Any advice on where this should be documented? Perhaps where we explain the default environment of a BUILD file?

@laurentlb
Copy link
Contributor

All correct. We should recommend using only load statements in the prelude.

In the website, we can have a new page under Reference > BUILD files.

@alandonovan
Copy link
Contributor

alandonovan commented May 11, 2020

The prelude should not have its own page---we do not want to draw attention to it. It should be a footnote in a page describing the predeclared environment of a BUILD file: Starlark core operators, functions (glob, select ...), build rules, and so on.

Where in the source code would one add such a page? //site/_layouts/...?

@laurentlb
Copy link
Contributor

The documentation of BUILD files is here: https://docs.bazel.build/versions/master/build-ref.html#BUILD_files

@alandonovan
Copy link
Contributor

That page is the introduction to the core concepts. It is not the appropriate place for reference material on the predeclared environment of a BUILD file.

Where would one add such a page?

@laurentlb
Copy link
Contributor

We can create a new page under Reference > BUILD files with all the semantics of BUILD files?

@alandonovan
Copy link
Contributor

alandonovan commented May 12, 2020

Yes! But where in the repository would the page's file reside?

@laurentlb
Copy link
Contributor

@ahyangyi
Copy link

In the future, implicit textual insertion will be replaced by implicit load-like semantics. Few users should notice the effects of that change.

I would add that the current implicit textual insertion creates confusing error messages in case of bugs... which can only be solved when you replace it with implicit load-like semantics.

Hence, a change like that would definitely be noticed by me, but it's for the better!

@alandonovan
Copy link
Contributor

Hence, a change like that would definitely be noticed by me, but it's for the better!

I meant to say "noticed by Starlark logic". But it's good to know the change will make at least one person happy. :)

@alandonovan
Copy link
Contributor

alandonovan commented May 15, 2020

I was about to send a CL to add this new file, but I realized it would be a case of "all the existing X are bad, let me create another one", where X in this case is "pages documenting the environment". This is a step in the wrong direction.

We desperately need to carefully reorganize our API reference docs (ideally without breaking URLs). The problem is that they are partly generated by a program, so this is actually a somewhat subtle logic change not a doc edit. We cannot entrust the task of organizing docs to our doc tooling (e.g. stardoc). It can generate parts of the docs but a human editor must decide how they are stitched together. Also, every document should explain how to edit it, even if that means editing .java code.

Ideally we would have one page with a complete index or ToC, organized into several logical sections, with each section being an explanation of the grouping and a set of links to a single page that explains all functions in that section fully.

  1. Core Starlark built-ins: None, True, False, len, etc.
  2. Common predeclared built-ins (select, depset, ...)
  3. BUILD environment (native.package, native.glob, ... + all native.rules).
    Explain (or link to explanation) that BUILD files are "data" not "code" (no if/for/def/f(**kwargs).
    Explain that the native package is opened in BUILD files. Each name can be link to native.foo.
    Explain the prelude.
  4. .bzl predeclared built-ins (rule, aspect, + hundreds of providers and suchlike)
  5. native.xyz functions (package, glob, exports_files, etc).
    Can only be called from a BUILD thread.
  6. native rules (these are also in the native package, and thus the BUILD environment.

Core built-ins are specified at https://github.com/bazelbuild/starlark/blob/master/spec.md#built-in-constants-and-functions. That doc needs a table of contents. (The one at go.starlark.net from which it was copied has one; not sure how it got lost.) They are also documented (from Java code) at https://docs.bazel.build/versions/master/skylark/lib/globals.html, but this doc also contains lots of things that don't belong. Also, we should call them predeclared built-ins, not globals.

Common functions (select, depset) are included here, along with core built-ins and .bzl things:
https://docs.bazel.build/versions/master/skylark/lib/globals.html

The native package is documented here
https://docs.bazel.build/versions/master/skylark/lib/native.html
'package' is currently missing but that will be fixed by CL 311541013.
This doc also needs a better explanation that native.foo is accessed as foo in a BUILD file but as native.foo in a .bzl file.

Built-ins common to BUILD and .bzl files: select, depset, ...
We don't have a good page for this. They are documented here, but mixed up with providers and all the .bzl stuff: https://docs.bazel.build/versions/master/skylark/lib/globals.html

This doc is a mix of native.x functions plus select (but no depset). Makes no sense.
https://docs.bazel.build/versions/master/be/functions.html

We should prefer fewer, larger pages, over many smaller ones (especially many non-intuitive groupings---I spent half my time clicking to find the seemingly arbitrary page on which a given function appears.) Browsers are quite capable of rendering big docs, they work best on slower networks, and users can jump around more easily.

@brandjon
Copy link
Member

A couple corrections to the above:

It is thus a place to stuff workspace-global definitions.

Ever since c93ef7b, it's been local to a repo, so that the prelude definitions don't bleed between your dependencies and your main repo.

In the future, implicit textual insertion will be replaced by implicit load-like semantics. Few users should notice the effects of that change.

#11940.

@aiuto
Copy link
Contributor

aiuto commented Aug 28, 2020

cc: @floriographygoth @gregestren @jin

@brandjon brandjon added team-Starlark-Integration Issues involving Bazel's integration with Starlark, excluding builtin symbols and removed team-Starlark labels Feb 15, 2021
@brandjon brandjon changed the title Feature request: Documentation on the prelude Clearly document the prelude file Feb 15, 2021
@brandjon brandjon added P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) and removed P3 We're not considering working on this, but happy to review a PR. (No assignee) labels Oct 26, 2022
Copy link

Thank you for contributing to the Bazel repository! This issue has been marked as stale since it has not had any activity in the last 1+ years. It will be closed in the next 90 days unless any other activity occurs. If you think this issue is still relevant and should stay open, please post any comment here and the issue will no longer be marked as stale.

@github-actions github-actions bot added the stale Issues or PRs that are stale (no activity for 30 days) label Mar 17, 2024
Copy link

This issue has been automatically closed due to inactivity. If you're still interested in pursuing this, please post @bazelbuild/triage in a comment here and we'll take a look. Thanks!

@github-actions github-actions bot closed this as not planned Won't fix, can't repro, duplicate, stale Jun 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P4 This is either out of scope or we don't have bandwidth to review a PR. (No assignee) stale Issues or PRs that are stale (no activity for 30 days) team-Documentation Documentation improvements that cannot be directly linked to other team labels team-Starlark-Integration Issues involving Bazel's integration with Starlark, excluding builtin symbols type: documentation (cleanup)
Projects
None yet
Development

No branches or pull requests

9 participants