Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

remove GeneratorSettings.cache #2703

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

WebFreak001
Copy link
Member

instead the cache is a property of PackageManager

This way dub lib users don't have to try to figure out where the real cache path is and can just continue using the generator and describe APIs like in previous versions.

Customizing cache path is no longer easily possible and now must be done through creating a custom PackageManager and use that on new Project() to pass into the generators. If needed, APIs to modify m_dirs on startup could be added to Dub subclasses in the future.

Also for how the other code was setup, this looks like a more reasonable place to put the code, as well as PackageManager being a good place to handle path modification / generation.

Sadly the GeneratorSettings public API has already been released in a tag and thus this is a breaking change, that for example reggae also uses: https://github.com/atilaneves/reggae/blob/2302be1809edde6e1ac3552ee595970601f960ad/src/reggae/dub/interop/dublib.d#L70

However, at least for reggae, the migration would be trivial, since it already instantiates a custom PackageManager: https://github.com/atilaneves/reggae/blob/2302be1809edde6e1ac3552ee595970601f960ad/src/reggae/dub/interop/dublib.d#L208 and I suppose other apps wanting to control how DUB builds are not using new Dub either and instead create custom PackageManager instances.

For the usual dub-as-a-library user this PR allows users to start a DUB build, reusing the default DUB cache directory, without hardcoding or copying the DUB cache paths or SpecialDirs implementation from DUB itself. So I think this is definitely a better API than burdening the users with figuring out where the cache should go for builds if they just want to call dub build through the API.

Note: there was metadata_cache.json that still used the old non-central cache path? I moved that into the central cache path now, but maybe @Geod24 knows why this was not yet in the central cache? That part might still need a change.

instead the cache is a property of PackageManager

This way dub lib users don't have to try to figure out where the real
cache path is and can just continue using the generator and describe
APIs like in previous versions.

Customizing cache path is no longer easily possible and now must be done
through creating a custom PackageManager and use that on `new Project()`
to pass into the generators. If needed, APIs to modify m_dirs on
startup could be added to Dub subclasses in the future.
@WebFreak001 WebFreak001 requested a review from Geod24 September 26, 2023 22:39
@github-actions
Copy link

github-actions bot commented Sep 26, 2023

✅ PR OK, no changes in deprecations or warnings

Total deprecations: 14

Total warnings: 0

Build statistics:

 statistics (-before, +after)
-executable size=5357768 bin/dub
-rough build time=82s
+executable size=5357832 bin/dub
+rough build time=81s
Full build output
DUB version 1.33.1, built on Aug 26 2023
LDC - the LLVM D compiler (1.34.0):
  based on DMD v2.104.2 and LLVM 16.0.6
  built with LDC - the LLVM D compiler (1.34.0)
  Default target: x86_64-unknown-linux-gnu
  Host CPU: skylake-avx512
  http://dlang.org - http://wiki.dlang.org/LDC


  Registered Targets:
    aarch64     - AArch64 (little endian)
    aarch64_32  - AArch64 (little endian ILP32)
    aarch64_be  - AArch64 (big endian)
    amdgcn      - AMD GCN GPUs
    arm         - ARM
    arm64       - ARM64 (little endian)
    arm64_32    - ARM64 (little endian ILP32)
    armeb       - ARM (big endian)
    avr         - Atmel AVR Microcontroller
    bpf         - BPF (host endian)
    bpfeb       - BPF (big endian)
    bpfel       - BPF (little endian)
    hexagon     - Hexagon
    lanai       - Lanai
    loongarch32 - 32-bit LoongArch
    loongarch64 - 64-bit LoongArch
    mips        - MIPS (32-bit big endian)
    mips64      - MIPS (64-bit big endian)
    mips64el    - MIPS (64-bit little endian)
    mipsel      - MIPS (32-bit little endian)
    msp430      - MSP430 [experimental]
    nvptx       - NVIDIA PTX 32-bit
    nvptx64     - NVIDIA PTX 64-bit
    ppc32       - PowerPC 32
    ppc32le     - PowerPC 32 LE
    ppc64       - PowerPC 64
    ppc64le     - PowerPC 64 LE
    r600        - AMD GPUs HD2XXX-HD6XXX
    riscv32     - 32-bit RISC-V
    riscv64     - 64-bit RISC-V
    sparc       - Sparc
    sparcel     - Sparc LE
    sparcv9     - Sparc V9
    spirv32     - SPIR-V 32-bit
    spirv64     - SPIR-V 64-bit
    systemz     - SystemZ
    thumb       - Thumb
    thumbeb     - Thumb (big endian)
    ve          - VE
    wasm32      - WebAssembly 32-bit
    wasm64      - WebAssembly 64-bit
    x86         - 32-bit X86: Pentium-Pro and above
    x86-64      - 64-bit X86: EM64T and AMD64
    xcore       - XCore
   Upgrading project in /home/runner/work/dub/dub/
    Starting Performing "release" build using /opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/ldc2 for x86_64.
    Building dub 1.34.0+commit.26.g00a52893: building configuration [application]
source/dub/dependency.d(917,18): Deprecation: scope variable `this` assigned to non-scope parameter `oth` calling `opEquals`
source/dub/dependency.d(920,30): Deprecation: scope variable `this` assigned to non-scope parameter `a` calling `doCmp`
source/dub/dependency.d(921,27): Deprecation: scope variable `this` assigned to non-scope parameter `b` calling `doCmp`
source/dub/dependency.d(939,26): Deprecation: scope variable `this` assigned to non-scope parameter `oth` calling `opEquals`
source/dub/internal/configy/Exceptions.d(130,34): Deprecation: reference to local variable `buffer` assigned to non-scope anonymous parameter
source/dub/internal/configy/Exceptions.d(134,34): Deprecation: reference to local variable `buffer` assigned to non-scope anonymous parameter
source/dub/internal/configy/Exceptions.d(248,27): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(250,27): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(283,27): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(286,27): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(323,31): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(325,31): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(332,31): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
source/dub/internal/configy/Exceptions.d(335,31): Deprecation: `@safe` function `formatMessage` calling `formattedWrite`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/write.d(537,34):        which calls `std.format.spec.FormatSpec!char.FormatSpec.writeUpToNextSpec!(void delegate(in char[]) @safe).writeUpToNextSpec`
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        which wouldn't be `@safe` because of:
/opt/hostedtoolcache/dc/ldc2-1.34.0/x64/ldc2-1.34.0-linux-x86_64/bin/../import/std/format/spec.d(258,33):        scope variable `this` assigned to non-scope parameter `e` calling `put`
     Linking dub
STAT:statistics (-before, +after)
STAT:executable size=5357832 bin/dub
STAT:rough build time=81s

@rikkimax
Copy link
Contributor

I take it that there is no possibility of adding a deprecated workaround to give existing users some time to upgrade?

@WebFreak001
Copy link
Member Author

I don't think there is any way really that wouldn't make the code very ugly, like keeping the old stuff in as second branch (with a lot of copy-pasted code)

The API has been released for over 6 months now already so I don't exactly like entirely removing it either.

I opted for implementing it this way, since it looked like pretty obviously @Geod24 wanted to get rid of the public access for the cache dir entirely, naming the getter "cachePathDontUse"

Nice thing with this PR is at least we are removing that cachePathDontUse method since it's no longer needed and I personally think like putting the path handling inside PackageManager is the better idea here than some global methods inside generator.d that need to pass in the user-specified cache directory. Although from a user perspective having an API to change the cache directory is nice as well.

I realized this was an issue because I didn't specify any .cache to the GeneratorSettings (just using code from the old API basically) and it would assert(false) inside TargetGenerator since it also exposed the cache file path since a few months ago and thus started making old code break.

However actually fixing the cache path to use what DUB uses isn't possible since cachePathDontUse is not public and also pretty obviously doesn't look like it should be made public. I don't know entirely what other plans Mathias has here with the cache code, so I don't want to just expose it either.

Alternatively to this PR I could also make it actually use the cachePathDontUse function as a default when no cache path is specified, which I think from a user's point of view would be just as valid of a fix. This would also move it from the command line, where private/package APIs really shouldn't be used, somewhere into the implementation so it might no longer need to be "don't use".

Copy link
Member

@Geod24 Geod24 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I never manage to figure out a way to make it "nice". This approach is a bit of a slippery slope as other places that may need the cache may end up depending on PackageManager. Instinctively, I think that the cache part should be linked to the Package, but currently the class is not ready for this.

@@ -121,11 +121,13 @@ class ProjectGenerator
protected {
Project m_project;
NativePath m_tempTargetExecutablePath;
PackageManager m_packageMan;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's the reason I didn't do it in the first place - Having the PackageManager in the generator is a bit icky.

@Geod24
Copy link
Member

Geod24 commented Jan 17, 2024

@WebFreak001 : See #2764
Generator really shouldn't know about the PackageManager. How would you like to proceed ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants