Decide on standard wording for APIs that are available only when compiler is C++20 or later

[gmlueck]

We decided in a previous F2F meeting that the SYCL-Next specification will allow the host compiler to be C++17 or later. We have also discussed adding some APIs to the specification that will be available only if the compiler has a certain version. For example, we might add an API that uses std::span and document that the API is available only when the compiler is C++20 or later. We need to decide what the specification wording will be in cases like this. The choice of wording also affects the way a vendor must implement these APIs.

To illustrate the options, let's consider a mythical API:

void frob(std::span data);

Option 1 is to document the API like this:

void frob(std::span data);

Required C++ version: C++20 or later.

Option 2 is to document the API in terms of the C++ feature-test macros like:

void frob(std::span data);

Required C++ features: The implementation must define the __cpp_lib_span feature-test macro (which is defined in C++20 and later).

@Pennycook proposed option 2, so I think he prefers this option. I agree that this option allows an application to make use of these APIs even if the compiler is not fully conformant to C++20. (In the case above, it is only necessary that the compiler supports std::span in order for the application to use the frob API.)

Despite that, I think I have a preference for option 1. I like that the wording is simpler. Imagine a case where an API uses two or more C++ features that have different feature-test macros, and the wording could get even more complex than the example above. I think that most application developers will just want to know which version of C++ supports each SYCL API, rather than the finer-level granularity of the feature-test macros.

I also think there's a consistency concern. Would we add Required C++ features clauses even to APIs that use std::byte, for example? This is a C++17 feature, but it also has a feature-tests macro (__cpp_lib_byte). In the existing SYCL spec, we seem to have taken the position that the underlying compiler must support all C++17 features. If SYCL-Next allows C++20, it seems like it would be most consistent to adopt the same all-or-nothing wording in the spec.

This does not mean that an implementation also needs to adopt an all-or-nothing approach. An implementation could still choose to make these APIs available based on the C++ feature-test macros if it wanted. This would still be conformant with the specification even if we went with option 1. Thus, making these APIs available with a partially-conformant C++ compiler could be a quality-of-implementation thing rather than a mandated-by-specification thing.

Finally, we should consider how this wording will seem years from now when C++20 is more established and fully supported by all compilers. I think the option 2 wording will seem overly complicated at that time.

[kevin-harms]

If the spec says Option 1, then to me that implies that the implementation is checking just for C++20 and if Option 2, then the implementation is checking the specific feature macro. Ideally we don't select Option 1 and then have implementation actually implement Option 2. I don't know if we could have a case where frob would be reused in the spec somewhere as a dependency in another API/definition which then needs to decide if it should also state Option 1 or 2. Sticking with Option 1 seems to be cleaner in that regard.

[Pennycook]

Thanks for writing this up and opening the issue. There's just two things I want to add.

First, I'm not tied to the specific wording I proposed, and there are potentially ways to simplify it if we don't like the complexity, e.g.,

Required C++ Features: __cpp_lib_span >= /* some specific version of span */
Required C++ Features: <span>

Second, requiring only a specific header also constrains implementations (i.e., using more C++20 features than the ones in the list would be invalid). We're not happy with our current way of pre-adopting features, and so I'm trying to explore this as a potential replacement. This would effectively allow SYCL to start introducing features as soon as they're broadly adopted and stable, similar to what we were trying to achieve with things like sycl::span.

[keryell]

Second, requiring only a specific header also constrains implementations (i.e., using more C++20 features than the ones in the list would be invalid).

For example I could not use concept in a SYCL code because it is mentioned nowhere?

We're not happy with our current way of pre-adopting features, and so I'm trying to explore this as a potential replacement.

Do you have a concrete example?

[Pennycook]

For example I could not use concept in a SYCL code because it is mentioned nowhere?

Right, that's the sort of thing I'm imagining.

If the SYCL specification (or a specific SYCL implementation) says that it requires C++20, then the implementation should be able to use any C++20 feature it wants because it's the user's responsibility to ensure they are using a compatible compiler. But if we just want to make a feature like <span> available, we might be able to do that in a way that doesn't pull in everything else.

Do you have a concrete example?

I think @gmlueck brought this up recently, but I don't remember where. But if I understood correctly, the issue is with the way that we describe pre-adoption of future ISO C++ features in 3.9.2. Alignment with future versions of C++:

The following features are pre-adopted by SYCL 2020 and made available in the sycl:: namespace: std::span, std::dynamic_extent, std::bit_cast. The implementations of pre-adopted features are compliant with the next C++ specification, and are expected to forward directly to standard C++ features in a future version of SYCL.

The intent of this wording was that in a future version of SYCL, we would be able to say something to the effect of "When a SYCL program is compiled in C++20 mode, sycl::span is defined as an alias to std::span." This seemed like a reasonable way to provide the functionality of span to C++17 programs while providing a simple transition to using the std:: implementation when it became available. The issue is that swapping sycl::span for std::span will be an ABI break for any implementation that didn't already define sycl::span as an alias of std::span.

The alternative I'm proposing here is that instead of the SYCL specification pre-adopting features, we would effectively be permitting SYCL implementations to expose features early. A specific implementation could choose to make std::span -- and any SYCL APIs reliant on std::span -- available when compiling SYCL-Next in C++17 mode. It would be up to each implementation to decide which features were technically feasible to expose early and whether to provide ABI guarantees between different SYCL and C++ versions.

[gmlueck]

The alternative I'm proposing here is that instead of the SYCL specification pre-adopting features, we would effectively be permitting SYCL implementations to expose features early. A specific implementation could choose to make std::span -- and any SYCL APIs reliant on std::span -- available when compiling SYCL-Next in C++17 mode.

This is not how the C++ feature-test macros work today in either clang of gcc. For example, __cpp_lib_span is not defined when compiling in C++17 mode even though both compilers support that feature in C++20. You have to compile in C++20 mode in order for the compiler to define that macro.

I think these feature-test macros exist so that a compiler can claim to be C++20 even before it implements all of the C++20 features. The macros do not provide a way for a C++17 compiler to expose features that are added in a newer version of the language.

This reinforces my opinion that we should use the simpler wording in the spec:

Required C++ version: C++20 or later.

This is a true statement. The compiler does need to be C++20 in order for it to provide std::span.

[Pennycook]

This is not how the C++ feature-test macros work today in either clang of gcc. For example, __cpp_lib_span is not defined when compiling in C++17 mode even though both compilers support that feature in C++20. You have to compile in C++20 mode in order for the compiler to define that macro.

I agree that this is not how most of the feature-test macros work, but I think this is an implementation decision. clang does provide an option to enable char8_t prior to C++20, and enabling that option defines the C++20 feature-test macro even in C++17 mode (see https://godbolt.org/z/Mnr1qx17o). Given that precedent, I can't see why an implementation wouldn't be allowed to provide something like an -fmdspan option.

We are already in a position with clang today where we could say "Required C++ Features: __cpp_lib_char8_t" and developers could choose whether to compile with -std=c++17 -fchar8_t or -std=c++20.

I think these feature-test macros exist so that a compiler can claim to be C++20 even before it implements all of the C++20 features. The macros do not provide a way for a C++17 compiler to expose features that are added in a newer version of the language.

The rationale document for the feature-test macros (see here) argues the opposite, suggesting that implementations don't typically claim support for C++20 until they've implemented everything:

"As a result, testing for a specific revision of the standard (e.g. by examining the value of the __cplusplus macro) often gives the wrong answer. Implementers generally don’t want to appear to be claiming full conformance to a standard revision until all of its features are implemented. That leaves programmers with no portable way to determine which features are actually available to them."

This is a true statement. The compiler does need to be C++20 in order for it to provide std::span.

I'm sorry for arguing semantics here, but I don't think that's true. I don't think the conditions for a "compiler to be C++20" are well-defined. I think it's true that a compiler which claims to conform to the C++20 standard will provide std::span. I think it's also true that many compilers provide a C++20 mode that enables std::span. But a compiler doesn't need to implement all of C++20 to provide std::span (and I think you agree with this).

What criteria should we use to determine if a compiler is C++20?

[gmlueck]

Given that precedent, I can't see why an implementation wouldn't be allowed to provide something like an -fmdspan option.

Are there any other precedents of this other than -fchar8_t? Putting on my DPC++ hat, I would not be in favor of adding options like this unless the clang community wants to adopt them into upstream clang.

I think this is the crux of the issue. If compilers did generally add options like this to enable newer C++ features when compiling in older C++ modes, I would be in favor of the wording you propose. I'm not aware that this is the case, though. Is -fchar8_t the only case of this so far? The char8_t type was added in C++20. If there are no other examples since then, it makes me think that the compiler community has decided not to adopt this practice.

[Pennycook]

Are there any other precedents of this other than -fchar8_t? Putting on my DPC++ hat, I would not be in favor of adding options like this unless the clang community wants to adopt them into upstream clang.

I haven't done an exhaustive search, but there appear to be other examples. It looks like it's much more common in gcc, which provides -fconcepts and -fcoroutines in addition to -fchar8_t. MSVC has /await:strict for enabling C++20 coroutines with earlier C++ versions. MSVC also has /std:c++latest to enable whatever C++ features are currently implemented in the compiler; interestingly, there's also a note here explaining that MSVC's std options don't actually set the __cplusplus macro, and you have to request it separately.

I think this is the crux of the issue. If compilers did generally add options like this to enable newer C++ features when compiling in older C++ modes, I would be in favor of the wording you propose. I'm not aware that this is the case, though. Is -fchar8_t the only case of this so far? The char8_t type was added in C++20. If there are no other examples since then, it makes me think that the compiler community has decided not to adopt this practice.

I'm not a member of the compiler community, so I can't speak for them. My opinions here are based on what I've seen being done with feature-test macros in large C++ projects (e.g., Kokkos) and what I've seen in compiler documentation.

@tahonermann, do you have an opinion here? I noticed you worked on the char8_t paper (P0482R6).

[tahonermann]

Use of the -fchar8_t option in C++17 mode or -fno-char8_t option in C++20 mode selects a non-conforming C++ compilation mode. Likewise, use of -fno-concepts or -fno-coroutines when compiling for C++20 selects a non-conforming C++ compilation mode.

There is only one specification of C++. While implementors might have to be concerned with use of non-conforming compilation modes, I don't think the SYCL specification needs to be or should be. My recommendation is that the SYCL specification specify minimum C++ version requirements as needed for interfaces it specifies. Implementors can then make those interfaces available in earlier C++ compilation modes based on feature test macros if so desired. If it is desirable to encourage implementors to do so, then a non-normative note included with such interfaces could be added that details which feature test macros would indicate the minimum feature support required.

Stated otherwise, the following from @Pennycook's previous comment reflects the direction I recommend:

The alternative I'm proposing here is that instead of the SYCL specification pre-adopting features, we would effectively be permitting SYCL implementations to expose features early. A specific implementation could choose to make std::span -- and any SYCL APIs reliant on std::span -- available when compiling SYCL-Next in C++17 mode. It would be up to each implementation to decide which features were technically feasible to expose early and whether to provide ABI guarantees between different SYCL and C++ versions.

[Pennycook]

There is only one specification of C++. While implementors might have to be concerned with use of non-conforming compilation modes, I don't think the SYCL specification needs to be or should be. My recommendation is that the SYCL specification specify minimum C++ version requirements as needed for interfaces it specifies. Implementors can then make those interfaces available in earlier C++ compilation modes based on feature test macros if so desired. If it is desirable to encourage implementors to do so, then a non-normative note included with such interfaces could be added that details which feature test macros would indicate the minimum feature support required.

I hadn't considered the conformance aspect of this. Given SYCL has a conformance test, what I proposed could lead to some really nasty corner cases, where SYCL says that a program should be valid, a conforming C++ implementation says it isn't, but a non-conforming C++ implementation says it is. Oof.

You and @gmlueck have convinced me that stating things in terms of C++ version is a better idea. In terms of specific wording, I'll propose a slight tweak to Greg's wording:

Minimum C++ Version: C++20

...because this saves us writing "or later" every time this paragraph appears.

For code in a synopsis block, I propose:

void foo() noexcept; // C++20

...because guarding with the __cplusplus macro would likely break things unexpectedly (e.g., for MSVC), but it would still be good to highlight in the synopsis which functions require more than C++17.

Then we could modify Section 3.9.1 to say something like:

Implementations may support newer C++ versions than the minimum required by SYCL, and some SYCL features may only be available when using a newer C++ version than the minimum required by SYCL. The descriptions of all such features include a Minimum C++ Version element that references one of the C++ versions defined in Section 3.3.

[Note: Some implementations may choose to support SYCL and/or C++ features when using earlier C++ versions than required by this specification. Recommended best practice is to use __has_include and appropriate C++ feature-test macros to improve the portability of SYCL applications across implementations.]

...including a non-normative note as Tom suggested.

[gmlueck]

I used this as an opportunity to finish a PR I had started which broadens the base C++ version in the SYCL spec. I tried to incorporate your proposal: #680.