Use intro-installed.json instead of intro-install-plan.json to map files

[lithomas1]

Currently, meson uses intro-install-plan.json to map files. However, for some reason, meson doesn't list installed directories, and also will list files that will not be installed.

For example, on Windows, when doing python -m build, the build will fail on the mapping step, as it cannot find the pdb file(which is listed in the install-plan.json file).
FWIW, meson will also not find this file, but it will skip it instead of erroring out, and the installed.json file will not contain it. e.g.

File 'pandas\\_libs\\window\\indexers.cp38-win_amd64.pdb' not found, skipping

This probably makes sense, as meson-python is building in release mode, so I would presume meson is deleting/not building the .pdb files which are used for debugging.

In addition, intro-install-plan.json will miss directories installed by install_subdir.
(I am installing into the python platlibdir like install_subdir('somedir', install_dir: get_option('python.platlibdir') / 'pandas/...')

I am currently using this as an alternative to py.install_sources and listing every file, which would create a ton of boilerplate code.

So, I think meson-python should switch to mapping files from installed.json or a combination of installed.json and install-plan.json

My code can be found here https://github.com/lithomas1/pandas/tree/meson-poc.

[rgommers]

Hmm, I wasn't aware of install_subdir, that could actually be quite useful. The get_option('python.platlibdir') is a little unfortunate, but still saves a lot of boilerplate indeed.

So, I think meson-python should switch to mapping files from installed.json or a combination of installed.json and install-plan.json

I don't know enough about this, but from https://mesonbuild.com/IDE-integration.html#install-plan it looks like intro-installed.json contains full absolute paths. It looks like there's at least a small issue with generated files, see the double forward slashes here:

{"/home/rgommers/code/scipy/build/scipy/__config__.py": "/home/rgommers/code/scipy/build-install/lib/python3.10/site-packages//scipy/__config__.py"

[lithomas1]

Maybe someone from the meson side knows why there's the discrepancy between intro-install-plan and intro-installed?

cc @eli-schwartz

[FFY00]

Install install_subdir missing from install-plan is a major issue, but it might be simple to fix on the Meson side. install-plan listing pdb files which are not going to be installed is in a similar situation, albeit not as bad.

[rgommers]

Install install_subdir missing from install-plan is a major issue, but it might be simple to fix on the Meson side

Is there a Meson issue for this yet?

[lithomas1]

mesonbuild/meson#10639

[rgommers]

mesonbuild/meson#10765 fixed install_subdir. In the test I'm adding for it in gh-145 it looks like:

{
   "install_subdirs":{
      "/home/rgommers/code/tmp/meson-python/tests/packages/scipy-like/mypkg/submod":{
         "destination":"{py_platlib}/mypkg/submod",
         "tag":null
      }
   }
}

Traceback shows that this is a bug in meson-python now:

  File "/home/rgommers/anaconda3/envs/scipy-dev/lib/python3.10/site-packages/mesonpy/__init__.py", line 551, in build
    self._install_file(whl, counter, origin, destination)
  File "/home/rgommers/anaconda3/envs/scipy-dev/lib/python3.10/site-packages/mesonpy/__init__.py", line 511, in _install_file
    if self._is_elf(os.fspath(origin)):
  File "/home/rgommers/anaconda3/envs/scipy-dev/lib/python3.10/site-packages/mesonpy/__init__.py", line 386, in _is_elf
    with open(file, 'rb') as f:
IsADirectoryError: [Errno 21] Is a directory: '/tmp/build-via-sdist-gsjwt6mz/mypkg-2.3.4/mypkg/submod'

[rgommers]

Update:

• The install_subdir fix is now merged and tested in this repo. It needs Meson master/0.64.0 0.63.3 to actually work though.
• The fix for the pdb files issue is BUG: Don't add debug files as targets when not building in debug mode meson#10800

[rgommers]

It looks like both issues are fixed now, so I'll close this (@lithomas1 please correct me if I'm wrong). The pdb files one will be in Meson 0.64.0