Translate _d_newarray{U,iT,T} to a single template

[teodutu]

This achieves the following:

• Convert _d_newarray{U,iT,T} to a single template _d_newarrayT that handles arrays of elements that either have an init symbol or are zero-initialised.
• Move compiler lowering to the semantic phase
• Store lowered expression in NewExp.lowering
• Copy or move array utils functions as templates from rt.lifetime.d to core.internal.array.utils.d. Some cannot be removed from rt.lifetime.d because they're still used by non-template hooks and it's impossible to get a type from a TypeInfo argument at compile time as required by templates.

This is still a draft because some tests are failing locally and I don't see how they're related to my changes. I just wanted to see if they also fail when running the tests remotely.

[dlang-bot]

Thanks for your pull request, @teodutu!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

Testing this PR locally

If you don't have a local development environment setup, you can use Digger to test this PR:

dub run digger -- build "master + dmd#15299"

[ibuclaw]

Is OVERRIDE_MEMALLOC being considered here? I notice that newitem and newclass already seem to be dead. Potentially at the cost of compile-time.

[teodutu]

OVERRIDE_MEMALLOC

@ibuclaw, I will take care of _d_newclass and _d_newitemT in the future but don't see any _d_newarray* hook defined under OVERRIDE_MALLOC here [1]. Is there any other place?

[1]

dmd/compiler/src/dmd/root/rmem.d

Lines 208 to 279 in 646fc3d

	static if (OVERRIDE_MEMALLOC)
	{
	// Override the host druntime allocation functions in order to use the bump-
	// pointer allocation scheme (`allocmemoryNoFree()` above) if the GC is disabled.
	// That scheme is faster and comes with less memory overhead than using a
	// disabled GC alone.
	extern (C) void* _d_allocmemory(size_t m_size) nothrow
	{
	return allocmemory(m_size);
	}
	private void* allocClass(const ClassInfo ci) nothrow pure
	{
	alias BlkAttr = GC.BlkAttr;
	assert(!(ci.m_flags & TypeInfo_Class.ClassFlags.isCOMclass));
	BlkAttr attr = BlkAttr.NONE;
	if (ci.m_flags & TypeInfo_Class.ClassFlags.hasDtor
	&& !(ci.m_flags & TypeInfo_Class.ClassFlags.isCPPclass))
	attr |= BlkAttr.FINALIZE;
	if (ci.m_flags & TypeInfo_Class.ClassFlags.noPointers)
	attr |= BlkAttr.NO_SCAN;
	return GC.malloc(ci.initializer.length, attr, ci);
	}
	extern (C) void* _d_newitemU(const TypeInfo ti) nothrow;
	extern (C) Object _d_newclass(const ClassInfo ci) nothrow
	{
	const initializer = ci.initializer;
	auto p = mem.isGCEnabled ? allocClass(ci) : allocmemoryNoFree(initializer.length);
	memcpy(p, initializer.ptr, initializer.length);
	return cast(Object) p;
	}
	version (LDC)
	{
	extern (C) Object _d_allocclass(const ClassInfo ci) nothrow
	{
	if (mem.isGCEnabled)
	return cast(Object) allocClass(ci);
	return cast(Object) allocmemoryNoFree(ci.initializer.length);
	}
	}
	extern (C) void* _d_newitemT(TypeInfo ti) nothrow
	{
	auto p = mem.isGCEnabled ? _d_newitemU(ti) : allocmemoryNoFree(ti.tsize);
	memset(p, 0, ti.tsize);
	return p;
	}
	extern (C) void* _d_newitemiT(TypeInfo ti) nothrow
	{
	auto p = mem.isGCEnabled ? _d_newitemU(ti) : allocmemoryNoFree(ti.tsize);
	const initializer = ti.initializer;
	memcpy(p, initializer.ptr, initializer.length);
	return p;
	}
	// TypeInfo.initializer for compilers older than 2.070
	static if(!__traits(hasMember, TypeInfo, "initializer"))
	private const(void[]) initializer(T : TypeInfo)(const T t)
	nothrow pure @safe @nogc
	{
	return t.init;
	}
	}

[teodutu]

So far the DMD and phobos tests are passing. core/internal/array/duplication.d calls _d_newarrayU. When I tried to split _d_newarrayT into _d_newarrayU that only allocates memory and _d_newarrayT that initialises it, I broke some stuff in the GC. That's why there's commented code added in the PR. I'll try to fix these errors before converting to a full PR.

The builkite failure seems unrelated. I tested it locally and it gives the same error when using either dmd-2.104.1 or my branch. Other completely different PRs are also failing this test, so I think it has nothing to do with these changes.

[teodutu]

dup() still uses the old hook [1]. I'll leave those for a later PR as this one is already large enough as it is. In addition, getting this merged will allow me to start working on _d_arraym{iT,T}X as well [2].

[1]

dmd/druntime/src/core/internal/array/duplication.d

Line 11 in 396fc8d

private extern (C) void[] _d_newarrayU(const scope TypeInfo ti, size_t length) pure nothrow;

[2] https://github.com/dlang/dmd/blob/396fc8d477ca66e20f6b4fec4d532f2aee053ba3/druntime/src/rt/lifetime.d#L1135C19-L1158

[RazvanN7]

Up until this point we have not put into changelogs the templatization of druntime hooks. However, if people find it useful we could add it. @WalterBright @ibuclaw what do you think?

I, personally, don't see any harm in putting such info in the changelog, except that it might be more verbose with no benefit for the majority of the readers (having a custom runtime should be a niche use case).

[teodutu]

@ryuukk, @RazvanN7, I created PR #15729 with a changelog entry about this. We can discuss this further there.

[schveiguy]

@RazvanN7 Having a changelog helps knowing what you have to do for each version of the compiler you want to support with your custom runtime.

[schveiguy]

This PR introduced a regression #22517