docs

docs/modules/ROOT/nav.adoc

@@ -33,6 +33,7 @@
 ** xref:android/kotlin.adoc[]
 ** xref:pythonlib/intro.adoc[]
 *** xref:pythonlib/dependencies.adoc[]
+*** xref:pythonlib/publishing.adoc[]
 * xref:comparisons/why-mill.adoc[]
 ** xref:comparisons/maven.adoc[]
 ** xref:comparisons/gradle.adoc[]

docs/modules/ROOT/pages/pythonlib/publishing.adoc

@@ -1,14 +1,40 @@
-= Publishing Python Projects
+= Python Packaging & Publishing
 :page-aliases: Publishing_Python_Projects.adoc
 
 include::partial$gtag-config.adoc[]
 
 This page will discuss common topics around publishing your Python projects for others to use
+include::partial$example/pythonlib/publishing/1-publish-module.adoc[]
 
-== Building packages locally
+== Advanced Packaging
 
-include::partial$example/pythonlib/publishing/1-publish-module.adoc[]
+Behind the scenes, Mill delegates most Python packaging tasks to other tools,
+and only takes care of configuring them with information it has on your build.
+
+By default, it will:
+
+* create a synthetic `pyproject.toml` file from its own metadata
+
+* use `setuptools` to package the module
+
+* first create a source distribution and then use that to build a wheel (instead of building a wheel directly)
+
+While this should be sufficient for most projects, sometimes you need a little
+customization.
+
+=== Customizing the `pyproject.toml` and other build files
+
+If you're happy to use a PEP-518-compliant `pyproject.toml` to describe how to
+package your published project, but would like some customization, you can amend
+the `pyproject` task with your own metadata.
+
+You can also include additional files in the packaging process by adding them to
+`buildFiles`. You can then reference these in your `pyproject.toml` file.
+
+TODO: example
 
-== Uploading packages to PyPI
+=== Changing the packaging process entirely
 
-// include::partial$Publishing_Footer.adoc[]
+In case customizing of `pyproject` is too cumbersome, or you cannot use it for
+some reason, you can always override the `sdist` and `wheel` tasks with your own
+packaging implementation. Publishing with `__publish` will still work as usual.

example/pythonlib/publishing/1-publish-module/build.mill

@@ -1,9 +1,12 @@
+// All packaging and publishing functionality is defined in `PublishModule`.
+// Start by extending it.
+
 import mill._, pythonlib._
 
 object `package` extends RootModule with PythonModule with PublishModule {
 
+  // information about dependencies will be included in the published package
   def pythonDeps = Seq("jinja2==3.1.4")
-  def moduleDeps = Seq(other)
 
   def publishMeta = PublishMeta(
     name = "test2-jodersky",
@@ -13,20 +16,75 @@ object `package` extends RootModule with PythonModule with PublishModule {
     authors = Seq(Developer("John Doe", "jdoe@example.org"))
   )
 
+  // the version under which the package will be published
   def publishVersion = "0.0.3"
 
-  object other extends PythonModule with PublishModule {
+}
+
+// You'll need to define some metadata in the `publishMeta` and `publishVersion`
+// tasks. This metadata is roughly equivalent to what you'd define in a
+// https://todo.org[`pyproject.toml` file].
+//
+// You'll also need to create a `readme` file, which will be bundled in the
+// final package and serves as the landing page seen on PyPI. By default, Mill
+// assumes a file starting with the string `readme` (in any capitalization), but
+// you can override it to whatever you please.
 
-    def publishMeta = PublishMeta(
-      name = "test2-other-jodersky",
-      description = "an nested example package",
-      requiresPython = ">= 3.12",
-      license = License.MIT,
-      authors = Seq(Developer("John Doe", "jdoe@example.org"))
-    )
+// [NOTE]
+// ====
+// The version of your package is not included in `publishMeta`, but
+// rather in its own `publishVersion` task. This is done so that you can easily
+// override the task to automate the version, such as deriving it from source
+// control.
+// ====
 
-    def publishVersion = "0.0.3"
+// == Building packages locally ==
+//
+// You can build a source distribution or wheel by running the following tasks:
 
-  }
+/** Usage
+mill show sdist
+> ".../out/sdist.dest/dist/test2_jodersky-0.0.3.tar.gz"
 
-}
+mill show wheel
+> ".../out/wheel.dest/dist/test2_jodersky-0.0.3-py3-none-any.whl"
+*/
+
+
+// These files can then be `pip-installed` by other projects, or, if you're
+// using Mill, you can include them in your xref:unmanagedWheels task. Usually
+// however, you'd want to publish them to a package index such as PyPI or your
+// organization's internal package repository.
+
+// == Uploading your packages to PyPI (or other repository)
+//
+// Uploading your packages to PyPI can be done by running `mill __.publish`.
+//
+// Mill uses https://todo[`twine`] to upload packages, and respects its
+// configuration. However, you can also configure it directly via command line
+// arguments and environment variables:
+//
+// [source,bash]
+// ----
+// # export MILL_TWINE_USERNAME=<username, not necessary for PyPI>
+// export MILL_TWINE_PASSWORD=<apitoken>
+// mill __.publish --repositoryUrl=https://test.pypi.org/legacy/
+// ----
+//
+// [NOTE]
+// ====
+// Mill does not transitively upload all your packages, hence we
+// recommended to use `mill __.publish`, instead of `mill <module>.publish`.
+// While it's technically possible to upload packages of individual Mill modules
+// by calling their `publish` tasks separately, you'd usually want to ensure all
+// your dependencies are also published.
+// ====
+//
+// === Check before uploading
+//
+// Twine has a nice feature to check your artifacts before uploading them. You
+// can also do this with Mill, by running:
+/** Usage
+mill __.checkPublish
+> ... PASSED
+*/

pythonlib/src/mill/pythonlib/PublishModule.scala

@@ -137,7 +137,7 @@ trait PublishModule extends PythonModule {
     // we use setup tools by default, which can only work with a single source directory, hence we
     // flatten all source directories into a single hierarchy
     val flattenedSrc = T.dest / "src"
-    for (dir <- sources(); if os.exists(dir.path)) {
+    for (dir <- (sources() ++ resources()); if os.exists(dir.path)) {
       for (path <- os.list(dir.path)) {
         os.copy.into(path, flattenedSrc, mergeFolders = true, createFolders = true)
       }