installation: add mechanism for symlinking mlir-opt when creating venv

[superlopuh]

For the last couple of years, I've had a custom-built llvm bin folder in my path. This has mostly been OK but sometimes resulted in weird build errors. With this mechanism, we can have the best of both worlds, with having a normal mlir-opt in the path except for when the venv is activated:

~/Developer/xdslproject/xdsl sasha/misc/env *129 !1 ❯ which mlir-opt           
/opt/homebrew/opt/llvm/bin/mlir-opt
~/Developer/xdslproject/xdsl sasha/misc/env *129 ❯ uv run which mlir-opt  
/Users/sasha/Developer/xdslproject/xdsl/.venv/bin/mlir-opt

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 91.29%. Comparing base (293b747) to head (a56a947).
Report is 2 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #3660   +/-   ##
=======================================
  Coverage   91.29%   91.29%           
=======================================
  Files         466      466           
  Lines       58353    58353           
  Branches     5623     5623           
=======================================
  Hits        53275    53275           
  Misses       3629     3629           
  Partials     1449     1449           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[jorendumoulin]

this would mess with my autoenv setup https://github.com/hyperupcall/autoenv that also uses .env files (but those .env files aren't valid makefiles so including those results in errors)

maybe just supply the argument when running make venv?

make venv XDSL_MLIR_OPT=/custom-llvm/bin/

[compor]

Maybe some minor confusing (?) points:

1. we ignore .env since it is supposed to be a per-user file, and this change subverts it to be explicitly included along with potentially other vars
2. based on 1) can we just keep the XDSL_MLIR_OPT check for symlinking? do we need to tie it with .env? One could still use it and maybe export it with export $(cat .env | xargs) or similar.
3. what happens if I also have export in the shell the variable XDSL_MLIR_OPT along with another path in .env?
   What takes precedence? I can't tell off the top of my head.
4. UV_ENV_FILE allows the use of several .env files (different names) and with this we are associating our mlir-opt variable to only be in .env.

[alexarice]

I think I mentioned this at the time, but for tools like tblgen-to-py it would be very useful to have access to an entire llvm repo instead of just an mlir-opt binary. Wonder if this would be worth revisiting if we're messing around with this sort of stuff.

[compor]

I think I mentioned this at the time, but for tools like tblgen-to-py it would be very useful to have access to an entire llvm repo instead of just an mlir-opt binary. Wonder if this would be worth revisiting if we're messing around with this sort of stuff.

do you need the repo or an installation directory? if it is the latter, doesn't llvm-config cover most needs?

[superlopuh]

All great points. There were a few things that I was thinking about and I think I should revisit:

1. the var should only hold the address to the llvm bin, not mlir-opt, allowing for other tools
2. the .env thing is a separate thing altogether, let's have that discussion in another PR

Joren:

The whole point is that I would like to just make venv and for the right thing to happen, and not call it with an extra argument.

Chris:

1. not sure what you mean here :)
2. let's have the .env discussion in a different PR
3. great point, no idea :)
4. I didn't manage to get UV_ENV_FILE to do anything, another reason to do this separately maybe

[superlopuh]

Actually maybe before I do anything, could anyone get UV_ENV_FILE to do anything at all? Maybe I'm just not holding it right

[compor]

Actually maybe before I do anything, could anyone get UV_ENV_FILE to do anything at all? Maybe I'm just not holding it right

I tried it, I think the idea there is to make the environment specified in the .env files available in commands like uv run which in turn means that they can be accessed from os.environ and friends - not in the shell of the current activated venv.

[alexarice]

I think I mentioned this at the time, but for tools like tblgen-to-py it would be very useful to have access to an entire llvm repo instead of just an mlir-opt binary. Wonder if this would be worth revisiting if we're messing around with this sort of stuff.

do you need the repo or an installation directory? if it is the latter, doesn't llvm-config cover most needs?

repo, in particular the tablegen files in the repo

[superlopuh]

what is llvm-config?

[superlopuh]

Ah then I think we would need to agree on a mechanism and then add more things to it, if you need a repo can also add the repo address

[superlopuh]

I've added just the ability to put mlir-opt in the venv, will open a new PR for .env things

[math-fehr]

Looks good to me, I would just change the variable name and point to mlir-opt directly instead.

Should this be documented in the readme as well?

[math-fehr]

I would probably rename this MLIR_OPT_PATH and point directly to mlir-opt instead?