[dcompute] add support for OpenCL image I/O #3835

thewilsonator · 2021-09-20T08:24:36Z

No description provided.

tests/codegen/dcompute_cl_images.d

dmd/id.d

JohanEngelen · 2021-09-20T10:58:02Z

gen/dcompute/targetOCL.cpp

+
+      auto sd = ptr->type->isTypeStruct();
+      auto mod = sd ? sd->sym->getModule() : nullptr;
+      if (mod && mod->md && mod->md->id == Id::opencl) {


So this is where you hardcode that the module name must be opencl. Should this not be ldc.opencl ? That way, we can ship LDC with that hardcoded module.
Or am I misunderstanding things?

Should this not be ldc.opencl?

Alas it must be, the SPIR-V backend depends on exclusively the type e.g. opencl.image1d_ro_t and I don't know how to force the name of a type.

I think it would work if you put module opencl; at the top of the file, while still having it in ldc/opencl.d.

true, druntime/src/ldc would have to be on the import path in addition to druntime/src.

I'll do that in a separate PR, because only a few of the types are needed to test the code generation and I've not finished the file yet, I still need to figure out which combinations of {1d,2d,3d}, {ro,wo,rw}, {array, msaa, depth, buffer,(none)} are valid and the msaa ones are only valid with OpenGL sharing or something.

(Also I still don't know how submodule update PRs work.)

true, druntime/src/ldc would have to be on the import path in addition to druntime/src.

That would be a big change (because then suddenly import attributes; would 'work').
I would expect the type names to be generated from the name specified by module <name>, rather than the how it is imported using import <name>.
Zooming out a little, I think requiring that the type name in D is exactly how LLVM wants it internally is very brittle. For example, if LLVM expects a different type name, then all user code would have to change aswell (perhaps solvable with a bunch of stable alias types in LDC's opencl.d?). I'd say first priority is getting things to work, but soon after there needs to be some thought behind the future/longer term stability of this functionality working.

I'll do that in a separate PR

ok

That would be a big change (because then suddenly import attributes; would 'work')

Indeed. We could have it along side object.d, I currently have it under dcompute/source

requiring that the type name in D is exactly how LLVM wants it internally is very brittle

It may be brittle, but it is exceedingly unlikely to change (SPIR (n.b. not SPIR-V) is a standard after all) unless we change backends (which I would love to do, but there is not official SPIR-V that is part of the LLVM project) in which case they would probably all be i8 addrspace(1)* to intrinsic functions which would be completely handled by the dcompute library.

perhaps solvable with a bunch of stable alias types in LDC's opencl.d?

Of course, there's no point calling it an GlobalPointer!image1d_ro_t when you can alias it to ReadOnlyImage!1.
The only mildly annoying concern is the conditionally declared types that depend on if OpenGL image sharing is enabled.

Should this not be ldc.opencl?

Yeah, that would be much nicer indeed IMO, keeping extra LDC stuff in its package.

I don't know how to force the name of a type.

The name is currently initialized and then untouched AFAICT in

ldc/ir/irtypeaggr.cpp

Line 206 in f9ba172

LLStructType::create(gIR->context(), ad->toPrettyChars())),

. You could override it in

ldc/ir/irtypestruct.cpp

Lines 49 to 55 in f9ba172

IrTypeStruct *IrTypeStruct::get(StructDeclaration *sd) {

IF_LOG Logger::println("Building struct type %s @ %s", sd->toPrettyChars(),

sd->loc.toChars());

LOG_SCOPE;

auto t = new IrTypeStruct(sd);

getIrType(sd->type) = t;

(before returning for opaque structs, as all of these magic structs are opaque at the moment).

Also I still don't know how submodule update PRs work.

You push a branch onto the LDC repo and then reference it here in an LDC PR. After merging the PR here, the druntime branch needs to be merged into druntime's ldc branch (and must be a fast-forward merge, i.e., the druntime branch must branch off the current ldc head).

You could override it before returning for opaque structs

Thanks, will do some investigation tomorrow about this.

as all of these magic structs are opaque at the moment

correct, and I don't see that changing any time soon.

You push a branch onto the LDC repo and then reference it here in an LDC PR. After merging the PR here, the druntime branch needs to be merged into druntime's ldc branch (and must be a fast-forward merge, i.e., the druntime branch must branch off the current ldc head).

THANKS!!! will bookmark this.

You could use isFromLDC_OpenCL(sd) for the if-condition now, that would simplify things (and restrict it to ldc.opencl only).

gen/dcompute/targetOCL.cpp

tests/codegen/dcompute_cl_images.d

thewilsonator · 2021-09-26T03:20:14Z

Any final comments? It would be good to get this in before v1.28 is tagged.

kinke · 2021-09-26T04:19:59Z

gen/dcompute/targetOCL.cpp

@@ -38,6 +39,7 @@

 namespace {
 class TargetOCL : public DComputeTarget {
+    bool usedImage;


Seems like you read from it but don't set it.

fixed in 1f24be6 stackoverflow tells me that s.rfind("needle", 0) == 0 is how you're supposed to do startsWith on a string prior to C++20.

thewilsonator · 2021-09-27T03:32:13Z

Hmm, I seem to have a merge conflict that I can't figure out how to fix.

thewilsonator · 2021-09-27T04:15:27Z

Hmm, the druntime commit seems to be

-e1ba20fec08568c9f7d81ceedbac84d78cfdcd34
+e3738e482e9854cf7eb05c8f79abe5405fd1a8a9

instead of

-06fdf5464471d9877bce2aeadc80d527b7e8db1d
+e3738e482e9854cf7eb05c8f79abe5405fd1a8a9

hence the merge conflict. I've no idea why its doing that though or how to fix it.

thewilsonator · 2021-09-27T04:23:30Z

~~Yay, it worked!~~ Nope the submodule changes are wrong.

kinke · 2021-09-27T05:02:49Z

That's probably caused by master pointing to a newer druntime hash (I've merged upstream stable recently) than when this branch was branched off - rebase this branch in that case (and yes, it's going to complain about a druntime conflict, just commit your change, your druntime branch is fine).

thewilsonator · 2021-09-27T05:08:31Z

Yay! I've still got some fixes to do on the druntime bit.

kinke · 2021-09-27T05:17:48Z

You could try making ldc.opencl a .di - it has opaque structs and aliases only, shouldn't need to be compiled (no init symbols etc., neither for GlobalPointer!… instantiations as they are most likely zero-initialized).

thewilsonator · 2021-09-27T05:28:56Z

That fixes it locally.

thewilsonator · 2021-09-28T00:38:31Z

Ubuntu 16.04 x64 ltsmaster fails with:

Unable to checkout 'f2a6ae9802aceef9e958ac0d62b0128b85c1aeae' in submodule path 'runtime/druntime'

is that a problem with ltsmaster CI?

kinke · 2021-09-28T02:12:32Z

I've just retried it (thought you might have pushed the druntime commit a bit too late after pushing here), but it failed identically. Might have to do with an old git of that vanilla Ubuntu 16 image and some strange --depth 50 shallow-cloning behavior. The job is new, so no history of this problem so far.

kinke · 2021-09-28T02:15:57Z

You could try removing the --depth bit in

ldc/.cirrus.yml

Line 186 in f9ba172

git submodule update --init --depth $CIRRUS_CLONE_DEPTH

thewilsonator · 2021-09-28T03:52:07Z

Hmm, didn't work.

kinke · 2021-09-28T13:58:54Z

You've patched the wrong line.

thewilsonator · 2021-09-28T21:50:33Z

Thanks, that seems to have fixed it.

For unknown reasons this can result in "Unable to checkout 'hash' in submodule path 'runtime/druntime'"

thewilsonator · 2021-09-29T05:18:13Z

I think this is good to go.

ir/irtypestruct.cpp

kinke · 2021-09-30T06:41:07Z

[As for druntime, I suppose you could get rid of @compute(CompileFor.deviceOnly) as it's a .di anyway.]

thewilsonator · 2021-10-10T09:55:09Z

[As for druntime, I suppose you could get rid of @compute(CompileFor.deviceOnly) as it's a .di anyway.]

I think it serves a purpose as documentation, its probably not necessary but does no harm. Anyway this is good to go from my perspective.

gen/dcompute/targetOCL.cpp

thewilsonator · 2021-10-10T23:50:09Z

Good to go?

thewilsonator · 2021-10-12T12:23:59Z

Thanks!

thewilsonator force-pushed the cl-images-2 branch from 88d22cf to e0f55eb Compare September 20, 2021 09:37

thewilsonator commented Sep 20, 2021

View reviewed changes

tests/codegen/dcompute_cl_images.d Outdated Show resolved Hide resolved

JohanEngelen reviewed Sep 20, 2021

View reviewed changes

tests/codegen/dcompute_cl_images.d Outdated Show resolved Hide resolved

JohanEngelen reviewed Sep 20, 2021

View reviewed changes

dmd/id.d Show resolved Hide resolved

JohanEngelen reviewed Sep 20, 2021

View reviewed changes

gen/dcompute/targetOCL.cpp Show resolved Hide resolved

JohanEngelen reviewed Sep 21, 2021

View reviewed changes

tests/codegen/dcompute_cl_images.d Show resolved Hide resolved

thewilsonator force-pushed the cl-images-2 branch from 89e2e9c to 0b66b0f Compare September 26, 2021 03:18

kinke reviewed Sep 26, 2021

View reviewed changes

thewilsonator force-pushed the cl-images-2 branch 3 times, most recently from f6f2cdc to 1002c90 Compare September 27, 2021 04:00

thewilsonator force-pushed the cl-images-2 branch from 1002c90 to e8df267 Compare September 27, 2021 04:23

thewilsonator force-pushed the cl-images-2 branch 2 times, most recently from b162c2b to a5b78ed Compare September 27, 2021 05:07

thewilsonator force-pushed the cl-images-2 branch from a5b78ed to 52bef9e Compare September 27, 2021 05:54

thewilsonator force-pushed the cl-images-2 branch from 3241c9f to 00464a6 Compare September 28, 2021 20:30

remove --depth from cirrus ubuntu ltsmaster for submodule update

c409a85

For unknown reasons this can result in "Unable to checkout 'hash' in submodule path 'runtime/druntime'"

thewilsonator force-pushed the cl-images-2 branch from 00464a6 to b9cee9d Compare September 28, 2021 22:05

kinke reviewed Sep 30, 2021

View reviewed changes

ir/irtypestruct.cpp Outdated Show resolved Hide resolved

thewilsonator force-pushed the cl-images-2 branch from b9cee9d to 44b32e1 Compare October 6, 2021 02:10

kinke reviewed Oct 10, 2021

View reviewed changes

gen/dcompute/targetOCL.cpp Outdated Show resolved Hide resolved

[dcompute] add support for OpenCL image I/O

ecac84a

thewilsonator force-pushed the cl-images-2 branch from 44b32e1 to ecac84a Compare October 10, 2021 10:03

kinke merged commit b09d4be into ldc-developers:master Oct 11, 2021

thewilsonator deleted the cl-images-2 branch August 1, 2022 10:46

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[dcompute] add support for OpenCL image I/O #3835

[dcompute] add support for OpenCL image I/O #3835

thewilsonator commented Sep 20, 2021

JohanEngelen Sep 20, 2021

thewilsonator Sep 20, 2021

JohanEngelen Sep 20, 2021

thewilsonator Sep 20, 2021

JohanEngelen Sep 21, 2021

thewilsonator Sep 21, 2021

kinke Sep 26, 2021

thewilsonator Sep 26, 2021

thewilsonator Sep 27, 2021

kinke Sep 30, 2021 •

edited

Loading

thewilsonator commented Sep 26, 2021

kinke Sep 26, 2021

thewilsonator Sep 26, 2021

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 27, 2021 •

edited

Loading

kinke commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

kinke commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 28, 2021

kinke commented Sep 28, 2021

kinke commented Sep 28, 2021

thewilsonator commented Sep 28, 2021

kinke commented Sep 28, 2021

thewilsonator commented Sep 28, 2021

thewilsonator commented Sep 29, 2021

kinke commented Sep 30, 2021

thewilsonator commented Oct 10, 2021

thewilsonator commented Oct 10, 2021

thewilsonator commented Oct 12, 2021

	IrTypeStruct IrTypeStruct::get(StructDeclaration sd) {
	IF_LOG Logger::println("Building struct type %s @ %s", sd->toPrettyChars(),
	sd->loc.toChars());
	LOG_SCOPE;

	auto t = new IrTypeStruct(sd);
	getIrType(sd->type) = t;

[dcompute] add support for OpenCL image I/O #3835

[dcompute] add support for OpenCL image I/O #3835

Conversation

thewilsonator commented Sep 20, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kinke Sep 30, 2021 • edited Loading

Choose a reason for hiding this comment

thewilsonator commented Sep 26, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 27, 2021 • edited Loading

kinke commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

kinke commented Sep 27, 2021

thewilsonator commented Sep 27, 2021

thewilsonator commented Sep 28, 2021

kinke commented Sep 28, 2021

kinke commented Sep 28, 2021

thewilsonator commented Sep 28, 2021

kinke commented Sep 28, 2021

thewilsonator commented Sep 28, 2021

thewilsonator commented Sep 29, 2021

kinke commented Sep 30, 2021

thewilsonator commented Oct 10, 2021

thewilsonator commented Oct 10, 2021

thewilsonator commented Oct 12, 2021

kinke Sep 30, 2021 •

edited

Loading

thewilsonator commented Sep 27, 2021 •

edited

Loading