dmd.vsoptions: Prepare for LDC & use latest VS version found via COM #11089

kinke · 2020-05-01T13:52:47Z

No description provided.

kinke · 2020-05-01T13:53:00Z

dlang-bot · 2020-05-01T13:57:24Z

Thanks for your pull request and interest in making D better, @kinke! We are looking forward to reviewing it, and you should be hearing from a maintainer soon.
Please verify that your PR follows this checklist:

My PR is fully covered with tests (you can see the coverage diff by visiting the details link of the codecov check)
My PR is as minimal as possible (smaller, focused PRs are easier to review than big ones)
I have provided a detailed rationale explaining my changes
New or modified functions have Ddoc comments (with Params: and Returns:)

Please see CONTRIBUTING.md for more information.

If you have addressed all reviews or aren't sure how to proceed, don't hesitate to ping us with a simple comment.

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#11089"

kinke · 2020-05-01T16:37:38Z

src/dmd/root/filename.d

@@ -820,7 +820,7 @@ nothrow:
        }
        else version (Windows)
        {
-            return name.toCStringThen!((cstr) => cstr.toWStringzThen!((wname)
+            return name.toWStringzThen!((wname)


C null-termination was superfluous here.

kinke · 2020-05-01T16:38:34Z

src/dmd/root/filename.d

@@ -1002,26 +1002,15 @@ nothrow:
                // First find out how long the buffer has to be.
                const fullPathLength = GetFullPathNameW(&wname[0], 0, null, null);
                if (!fullPathLength) return null;
-                auto fullPath = new wchar[fullPathLength];
+                auto fullPath = (cast(wchar*) mem.xmalloc_noscan((fullPathLength + 1) * wchar.sizeof))[0 .. fullPathLength + 1];
+                scope(exit) mem.xfree(fullPath.ptr);


One less leak.

kinke · 2020-05-01T16:39:53Z

src/dmd/root/filename.d

-                // Find out size of the converted string
-                const retLength = WideCharToMultiByte(
-                    codepage, 0 /*flags*/, &fullPath[0], fullPathLength, null, 0, null, null);
-                auto ret = new char[retLength];


This previously meant that the result could not be free'd via FileName.free() (not allocated via mem.xmalloc).

kinke · 2020-05-01T16:48:09Z

src/dmd/vsoptions.d

+    return null;
+}
+
+char* buildPath(const(char)*[] fragments...)


This being here has more to do with how this PR has evolved in ldc-developers/ldc#3415. Maybe it should replace the FileName.buildPath() implementation, which inefficiently uses N-1 buffers (incl. N-2 of those leaking) when specifying N fragments.

Edit: Done.

rainers

LGTM, most of my concerns of intermediate versions have been resolved already.

Apart from the minor nits, I'm a bit uncomfortable with merging these non-trivial changes into stable just a couple of days before release. Can you target master and cherry pick if you want it in the next LDC release?

src/dmd/vsoptions.d

rainers · 2020-05-02T05:49:17Z

src/dmd/vsoptions.d

+        if (VSInstallDir is null)
+        {
+            wchar[260] buffer = void;
+            // VS Build Tools 2017 (default installation path)


Is this necessary? Not detected via COM? What about VS2019?

See the commit msg. I don't remember precisely, been too long, and don't have any Build Tools installation available.

Visual D detects the Build Tools by iterating over the keys in HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall, see https://github.com/dlang/visuald/blob/master/nsis/visuald.nsi#L1642

Heh so there is a way... IIRC, I grepped the entire registry for some trace to the install dir but couldn't find anything.

I think adding that would be overkill / I'm not looking forward to implementing it ;). I'd hope that it can be found via COM nowadays; vswhere.exe has been bundled since VS 2017 v15.2 (but not with v15.0 and v15.1), so I don't know whether the COM interface was only introduced in v15.2 too.

src/dmd/vsoptions.d

rainers · 2020-05-02T06:11:21Z

src/dmd/vsoptions.d

@@ -635,18 +722,46 @@ const(char)* detectVSInstallDirViaCOM()
        return null;
    scope(exit) instances.Release();

+    static struct WrappedBString


Nit: ok, but seems a bit excessive for dealing with releasing two strings. scope(exit) and a swap function might do the trick, too.

Yeah it might be excessive, but after having to explain the previous variant to you, I went with this (eliminating 4 scope(exit) lines...).

src/dmd/vsoptions.d

kinke · 2020-05-02T11:07:51Z

Can you target master and cherry pick if you want it in the next LDC release?

Of course I can; I thought the COM change would be welcome for DMD too.

Needed for LDC, which uses UTF-8 throughout, on Windows hosts too. Also expose this module to C++, for usage in LDC.

…a COM Also try to exclude VS installations without Visual C++. Also try to detect a VC++ Build Tools 2017 installation in its default installation directory; that's from LDC, and IIRC, there was no other way to detect it automatically (nothing in the registry).

rainers · 2020-05-02T13:11:02Z

Of course I can; I thought the COM change would be welcome for DMD too.

It is welcome, but not urgent. I don't remember anyone complaining.

@weak

Dentcho Bankov (1): Fix for #3374 Ernesto Castellotti (1): AVR: Add predefined version AVR and emit TLS globals as regular ones (#3420) Hiroki Noda (8): Add --fno-plt option to avoid PLT external calls (#3443) Remove bitrig support (#3445) [RISCV] getTargetCPU should returns generic-rvXX, not generic Remove RISCV_LLVM_DEV Sync .editorconfig with dmd (#3449) Remove unused LINUX_DISTRIBUTION_IS_REDHAT definition from CMake (#3470) [Trivial] Remove unused parameter attribute (#3477) [Trivial] refactor doLTO in writeModule (#3506) Jacob Carlborg (3): Add test runner template for iOS Setup CI pipeline for iOS on Bitrise Update to run iOS device testing on iPhone 6S 12.0 (#3450) Johan Engelen (4): Fix PGO instrumentation crash for final switch(enum) statements together with -release. (#3512) Fix partial discarding of function hash info for PGO (#3511) Add missing `UnrolledLoopStatement` to PGO region count instrumentation and calculation. Also add extra precautions to catch this problem earlier in the future. (#3524) Copy LLVM's ThreadSanitizer (TSan) library to LDC's libraries like we do for ASan, and set the correct linker flags (#3522) Martin Kinkelin (206): CI: Adapt to new ubuntu:rolling image (focal) (#3412) Upgrade frontend & libs to early v2.092.0 (dlang/dmd@abf2b56326) druntime: Fix deprecated syntax for GDC-style inline asm dmd.root.filename: Restore C++ absolute() / ext() versions Fix frontend C++ header regressions dmd-testsuite: Fix and adapt tests to new implicit compiler output checks Don't crash after inline IR parsing failure dmd-testsuite: Fix and adapt some more tests for LDC Prevent errors for unsupported pragmas and `-ignore -v` output Improve check for multiple entry point definitions dmd-testsuite: Minimally adapt fail_compilation/needspkgmod.d Fix new pragma([printf,scanf]) for targets with non-pointer va_list Restore compilability with ltsmaster via deprecated body keyword etc. Fix new deprecations in D parts Adopt DMD's MSVC toolchain detection Revise dmd.vsoptions, switching to wide APIs Don't skip MSVC setup in a preset environment if the target arch doesn't match Small dmd.vsoptions refactoring Refactor narrow <-> wide string conversions Merge upstream stable (dlang/dmd@e37e1be58c) Consolidate object.__va_list alias for SysV x64 and non-Apple ARM Prevent redundant type semantic for va_list ABI: Switch to front-end isHFVA() check druntime: Fix core.stdc.stdarg regression on Win64 AArch64: Populate a struct's argTypes Bump bundled dub to v1.21.0-beta.1 druntime: Merge upstream stable (incl. non-Apple AArch64 support for core.{stdc.stdarg,vararg}) druntime: Fix core.internal.vararg.aarch64 compile error Sync with dlang/dmd#11089 Merge upstream stable (dlang/dmd@2dc6c9d619) Refactoring: Introduce BaseBitcastABIRewrite Refactoring: Introduce TargetABI::isExternD() Refactoring: Introduce ArgTypesRewrite Fix AArch64 ABI, based on front-end toArgTypes machinery Make IR type determination for captured lazy params TargetABI-agnostic Shippable CI: Don't ignore dmd-testsuite-debug failures any longer dmd-testsuite: Adapt a few tests for non-x86 archs Shippable CI: Disable the 3 gdb lit-tests too, like the other gdb tests Improve detection of host real_t semantics (#3414) Don't use host C runtime for printing compile-time reals, use LLVM dmd-testsuite: Cherry-pick dlang/dmd#11075 Revert to host C runtime for printing decimal strings CMake: Exclude rt.memset from druntime druntime: Fix unguarded use of `assumeUsed` in core.thread.fiber README.md: Add FreeBSD install cmdline Fix casting (static and dynamic) arrays to vectors (#3419) Fix return statements: Don't access memory from destructed temporaries (#3426) Merge upstream stable (dlang/dmd@c333b99182) (#3429) Bump bundled dub to v1.21.0 final (#3433) gccbuiltins: Add AMD GCN and Nvidia PTX (#3411) Add proper support for -checkaction=halt (#3431) Merge upstream stable (dlang/dmd@9921b6ca25) (#3434) Emulate @weak functions on Windows and don't emit COMDATs for ELF anymore druntime: Revise @weak usages druntime: Fix and include core.stdcpp.* CMake: Bump LDC version to v1.22.0 Fix ICE for *captured* parameters *not* passed on the LLVM level Try to get rid of a possibly related LDC-specific front-end mod wrt. tuple params GCC-style asm: Add support for indirect input operands (#3438) druntime: Enable stdcpp integration tests on Windows druntime: Revise upstream diff a bit & speed-up core.math.ldexp for x87 real (#3440) druntime: Adopt fast ldexp implementation from libmir (#3446) Merge upstream stable (dlang/dmd@2f5f5299aa) (#3452) CirrusCI FreeBSD: Add bootstrap step to test self-compiled compiler druntime: Revise core.stdc.math for FreeBSD [FreeBSD] druntime: Switch to libexecinfo for backtrace Phobos: Fix tilde expansion for HOME=/ Cross-module inlining: Enable emission into multiple object files Fix: Prevent regular definitions from being weakened to available_externally Android: Default to cortex-a8 CPU for ARMv7-A (#3439) Switch to a white list of archs wrt. -m32/-m64 in cc cmdlines (#3460) Azure CI: Work around debuginfo/strings_cdb.d regression on Win32 with MSVC upgrade (#3465) Windows hosts: Handle LDC_VSDIR set to non-existing path (#3466) CirrusCI: Add prebuilt package for FreeBSD CirrusCI FreeBSD: Build LDC itself with full LTO (like Azure) CirrusCI FreeBSD: Switch from ports-ldc to official package [WebAssembly] druntime: Add grow and size memory intrinsics Merge v2.092.1+ (dlang/dmd@171aa05706) [Android] druntime: Handle TLS init issues wrt. alignment Android: Default to bfd linker for adjacent .tdata/.tbss sections Android: Bump used NDK from r21 to r21d Changelog: Add v1.22 (#3468) CI: Bump some host LDC versions etc. Cirrus & Circle CI: No multilib for Ubuntu 20.04+ jobs CI: Link libstdc++ statically for libldc-jit.so of prebuilt Linux packages (#3474) Upgrade frontend & libs to early v2.093.0 (dlang/dmd@eee9185638) Add CLI option -vtemplates dmd-testsuite: Adapt/disable some new tests Try to restore compilability with ltsmaster druntime: Get rid of superfluous LLVM_atleast template instantiations Phobos: Relax some new std.complex unittests for double-precision real Shippable CI: Work around linkability issue for bootstrap compiler druntime: Enable core.stdcpp.utility integration test on Windows Fix naked DMD-style asm emission for non-Mac x86 Darwin targets Bitrise CI: Test cross-compilability of druntime & Phobos to iOS/x86_64 Azure CI Mac: Add prebuilt druntime & Phobos for iOS/x86_64 (simulator) Shippable CI: Get rid of obsolete dmd-testsuite (release) workaround Shippable CI: Somewhat speed-up building of dub and dlang tools CMake: Use -linkonce-templates when compiling ldc2/ldmd2/ldc-build-runtime... all-at-once -betterC: Don't use unsupported EH for handling clean-ups (#3482) Refactoring: Specialize IrAggr into IrStruct and IrClass Frontend: Get rid of unused backend-reserved `Symbol` Frontend: Revise backend-reserved `code` alias Frontend: Get rid of unused backend-reserved aliases Frontend: Fix backend-reserved `type` alias Move TypeInfo_Struct emission to new ir/irstruct.cpp Merge upstream stable (dlang/dmd@bafab33f27) Get rid of superfluous buildClassDtor() helper Revise struct/class/interface TypeInfo generation, incl. getting rid of GENERATE_OFFTI CMake: Set up Visual D when using the Visual Studio generator (#3494) Merge upstream stable (dlang/dmd@642fc2e6a9) Fix issue #3501: Don't re-define struct TypeInfos (#3502) Bump bundled dub to v1.22.0 Raise min LLVM version to 6.0 (#3493) Fix issue #3496 - missing IR declarations for some fwd-declared functions (#3503) Fix ICE wrt. inline IR and empty parameter types tuple (#3509) CI: Bump LDC-LLVM to v10.0.1 (#3513) Windows hosts: Fix linker cmdline length limitation via response files Simplify IR names of virtual function pointers (#3534) Refactor response file stuff shared by ldmd and ldc druntime: Declare libstdc++ type_info class as abstract druntime: Use core.stdcpp.typeinfo.type_info class in rt.dwarfeh dmd-testsuite: Link against libc++ for CppRuntime_Clang targets Add `-gdwarf` CLI option to emit DWARF debuginfos for MSVC targets (#3533) Merge upstream stable (dlang/dmd@69f0a91fc4) (#3538) Extend -platformlib to Posix targets Phobos: Enable proper single- and double-precision std.math.ldexp overloads (#3539) Azure CI: Bump host clang to v10.0.1 on Linux (#3540) Merge v2.093.1+ (dlang/dmd@92778612c0) (#3541) druntime: Add @noplt function UDA (nonlazybind) Changelog: Add v1.23 (#3542) Travis CI: Add arm64 job (#3469) Add a GitHub Sponsors button (#3544) Merge upstream stable (dlang/dmd@f602545af9) (#3550) Travis CI: Try to work around out-of-memory errors (arm64) (#3552) Windows: Stabilize cdb-based CodeView lit-tests (#3555) Fix potentially wrong context pointers when calling delegate literals (#3554) Upgrade frontend & libs to v2.094.0-beta.1 Fix little upstream C++ header regression Fix another little C++ header regression druntime: Disable most rt/util/typeinfo.d unittests Fix another frontend C++ interop regression Silence C++ compiler error/warning wrt. narrowing Populate new Target::architectureName for JSON output Merge upstream stable (dlang/dmd@05c9d0fb38) Add support for -cov=ctfe Add support for -vtemplates=list-instances Re-add support for -HC without value README.md: Bump min LLVM version Adapt lit-test codegen/gh3094.d dmd-testsuite: Adapt new tests and revise LDC-specifics for fail_compilation -cov=ctfe: Initialize _d_cover_data with CTFE coverage data druntime: Fix -betterC linking error regression Adapt to llvm::cl::parser<bool> being final for LLVM < 9 Merge upstream stable (dlang/dmd@3855618290) Refactor IRScope handling LLVM 11: Adapt debuginfos wrt. static array and vector lengths Add support for LLVM 11 Azure CI: Upgrade to LDC-LLVM v11.0.0-rc2 Fix LLVM < 8 compatibility (no llvm::CallBase) Use LLVM 10+ setFunctionAttributes() Adapt lit-test plugins/addFuncEntryCall for LLVM 11 Revise IRScope refactoring Hide new LLVM 11 CLI options Adapt dynamic-compile/JIT stuff for LLVM 11 Fix alignment issue when casting vector rvalue to static array Disable lit-test instrument/xray_link.d for Mac and LLVM 11+ Azure CI: Bump LDC-LLVM to v11.0.0-rc2+ Azure CI: Bump LDC-LLVM to ~v11.0.0-rc3 Windows: Add *some* support for -fsanitize=fuzzer and enable ASan tests FreeBSD: Disable TSan tests Bump bundled dub to v1.23.0 druntime: Fix memory leak regression for main thread Align with upstream by employing VectorArrayExp for `<vector>.array` dmd-testsuite: Adapt 2 tests for LDC Restore compilability with GDC host compilers Try to restore compilability with ltsmaster CI: Bump LDC-LLVM to v11.0.0-rc4 Phobos: Minimally relax std.internal.math.errorfunction unittest for 32-bit MSVC (LLVM 11 regression) CI: Bump LDC-LLVM to v11.0.0-rc4+ with AArch64 regression fix Don't enforce (normal) emission for pragma(inline, true) function templates, and don't cull lambdas (#3570) Merge upstream stable (dlang/dmd@f2192d943c) CMake: Remove LDC_TARGET_PRESET (#3571) CMake: Tweak build order for D executables (LDC_LINK_MANUALLY=OFF only) CMake: Split up DDMD_DFLAGS into DFLAGS_{BASE,BUILD_TYPE,LDC} CMake: Don't enforce COMPILE_D_MODULES_SEPARATELY for ldc2-unittest Fix issue with add-multilib-section.sh Make LDC_LINK_MANUALLY=OFF actually usable for Posix hosts & use for CI Fix -Xcc=-Wl,... - disallow comma-separated shortcut for -Xcc Azure CI: Use 4 threads for Windows jobs Fix ThreadSanitizer support by removing detaching of main thread upon program termination (#3572) Merge upstream stable (dlang/dmd@80b7e82194) druntime: Cherry-pick macOS AArch64 support (dlang/druntime#3226) Traverse full chain of nested aggregates when resolving a nested variable (#3558) Fix build problem after #3575 (#3584) CI: Bump LDC-LLVM to v11.0.0 final Cirrus CI: Somewhat reduce YAML redundancy druntime: Use LLVM intrinsic for core.bitop.byteswap(ushort) README.md: Add Bitrise badge Merge upstream stable (v2.094.1) (#3588) README.md: Update Shippable badge logo Azure CI: Include macOS arm64 libraries in prebuilt Mac package (#3583) Merge upstream stable (dlang/dmd@cf518229cf) (#3593) Changelog: Add v1.24 druntime: Fix stdcpp integration test for Ubuntu 20.10 Cirrus CI: Adapt to Ubuntu rolling being 20.10 with LLVM 11 now Rob Rau (1): Fix wrong dcompute address space loads and stores (#3428) Roberto Rosmaninho (1): Add -output-mlir and prepare for MLIR emission (#3313) looked-at-me (1): Refactor and improve string literal emission (#3492)

kinke requested a review from ibuclaw as a code owner May 1, 2020 13:52

kinke force-pushed the vsoptions branch from 43d6346 to 68d1164 Compare May 1, 2020 13:57

kinke force-pushed the vsoptions branch 3 times, most recently from e3e2a57 to de6f2ec Compare May 1, 2020 16:11

kinke commented May 1, 2020

View reviewed changes

kinke force-pushed the vsoptions branch 3 times, most recently from c48a444 to c333451 Compare May 1, 2020 18:59

rainers reviewed May 2, 2020

View reviewed changes

kinke force-pushed the vsoptions branch from c333451 to 5618b60 Compare May 2, 2020 12:28

kinke added 3 commits May 2, 2020 14:31

dmd.vsoptions: Prepare for UTF-8 support

b18cf2e

Needed for LDC, which uses UTF-8 throughout, on Windows hosts too. Also expose this module to C++, for usage in LDC.

Refactor narrow <-> wide string conversions

386891a

kinke force-pushed the vsoptions branch from 5618b60 to 4ccad33 Compare May 2, 2020 12:31

kinke requested review from Geod24, MartinNowak and WalterBright as code owners May 2, 2020 12:31

dlang-bot added the Severity:Bug Fix label May 2, 2020

kinke changed the base branch from stable to master May 2, 2020 12:31

kinke changed the title ~~[stable] dmd.vsoptions: Prepare for LDC & use *latest* VS version found via COM~~ dmd.vsoptions: Prepare for LDC & use *latest* VS version found via COM May 2, 2020

dmd.root.filename: Refactor buildPath()

de029b4

kinke force-pushed the vsoptions branch from 4ccad33 to de029b4 Compare May 2, 2020 12:32

rainers approved these changes May 3, 2020

View reviewed changes

rainers added the Merge:auto-merge label May 3, 2020

dlang-bot merged commit f05dea9 into dlang:master May 3, 2020

kinke deleted the vsoptions branch May 3, 2020 10:48

kinke added a commit to kinke/ldc that referenced this pull request May 3, 2020

Sync with dlang/dmd#11089

a1817b7

kinke mentioned this pull request Jun 18, 2020

dmd.root.filename: Fix FileName.canonicalPath() regression on Windows #11293

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

dmd.vsoptions: Prepare for LDC & use latest VS version found via COM #11089

dmd.vsoptions: Prepare for LDC & use latest VS version found via COM #11089

kinke commented May 1, 2020

kinke commented May 1, 2020

dlang-bot commented May 1, 2020 •

edited

Loading

kinke May 1, 2020

kinke May 1, 2020

kinke May 1, 2020 •

edited

Loading

kinke May 1, 2020 •

edited

Loading

rainers left a comment

rainers May 2, 2020

kinke May 2, 2020

rainers May 2, 2020

kinke May 2, 2020 •

edited

Loading

rainers May 2, 2020

kinke May 2, 2020 •

edited

Loading

kinke commented May 2, 2020

rainers commented May 2, 2020

dmd.vsoptions: Prepare for LDC & use *latest* VS version found via COM #11089

dmd.vsoptions: Prepare for LDC & use *latest* VS version found via COM #11089

Conversation

kinke commented May 1, 2020

kinke commented May 1, 2020

dlang-bot commented May 1, 2020 • edited Loading

Bugzilla references

Testing this PR locally

kinke May 1, 2020

Choose a reason for hiding this comment

kinke May 1, 2020

Choose a reason for hiding this comment

kinke May 1, 2020 • edited Loading

Choose a reason for hiding this comment

kinke May 1, 2020 • edited Loading

Choose a reason for hiding this comment

rainers left a comment

Choose a reason for hiding this comment

rainers May 2, 2020

Choose a reason for hiding this comment

kinke May 2, 2020

Choose a reason for hiding this comment

rainers May 2, 2020

Choose a reason for hiding this comment

kinke May 2, 2020 • edited Loading

Choose a reason for hiding this comment

rainers May 2, 2020

Choose a reason for hiding this comment

kinke May 2, 2020 • edited Loading

Choose a reason for hiding this comment

kinke commented May 2, 2020

rainers commented May 2, 2020

dmd.vsoptions: Prepare for LDC & use latest VS version found via COM #11089

dmd.vsoptions: Prepare for LDC & use latest VS version found via COM #11089

dlang-bot commented May 1, 2020 •

edited

Loading

kinke May 1, 2020 •

edited

Loading

kinke May 1, 2020 •

edited

Loading

kinke May 2, 2020 •

edited

Loading

kinke May 2, 2020 •

edited

Loading