Run examples in CI #336

wprzytula · 2025-07-06T09:08:56Z

What's done

Stubs were added for functions that are called by examples. This is to enable building all examples.
libuv is now linked to all examples. This is because some examples need libuv. Note that it might make more sense to only link libuv against the particular examples that require it, but it seemed to me to be too much fuss.
Makefile has now targets to build all examples and run a specified list of examples.
- One ScyllaDB node is run in Docker for the purpose of running examples.
All examples were checked.
- successfully running examples were enabled;
- failing examples were annotated with a comment why they fail.
A CI step that builds and runs examples is added to the build, lint and test workflow.

Fixes: #305

Pre-review checklist

I have split my patch into logically separate commits.
All commit messages clearly explain what they change and why.
PR description sums up the changes and reasons why they should be introduced.
~~[ ] I have implemented Rust unit tests for the features/changes introduced.~~
~~[ ] I have enabled appropriate tests in Makefile in {SCYLLA,CASSANDRA}_(NO_VALGRIND_)TEST_FILTER.~~
I added appropriate Fixes: annotations to PR description.

examples/tuple/tuple.c

Lorak-mmk · 2025-07-07T12:04:18Z

examples/drop_examples_keyspace/drop_examples_keyspace.c

Commit: examples: add drop_examples_keyspace example

The other alternative is to make examples also work with existing keyspaces (`CREATE KEYSPACE / TABLE IF NOT EXISTS). Imo it would be cleaner, not requiring this additional weird target.

They would also have to drop the keyspace beforehand, because they often write the same data to the same keyspace/table. I'm still not convinced it's worth modyfing the examples.

Lorak-mmk · 2025-07-07T12:06:34Z

tests/examples_cluster/docker-compose.yml

I don't really understand why use docker here. You have already existing workflows in CI that use CCM. You could do the same instead of introducing another method.

Which workflows in CI use CCM? Well, I agree that integration tests use CCM. However, they are calling it through the bridge layer, from C++. Here I'd have to use it straight from the CLI. Why is it better than a ready docker-compose taken from Rust Driver?

CMakeLists.txt

wprzytula · 2025-07-07T14:05:28Z

Rebased on master.

wprzytula · 2025-07-15T08:37:19Z

Rebased on master, dropping the commit that adjusts the examples from VARCHAR to TEXT.

The example will be used by the example runner in Makefile to drop the `examples` keyspace between runs of consecutive examples. This is because the examples often create the `example` keyspace, don't drop it, and at the same time expect that the keyspace does not exist when they run. To avoid this, we add a new example that drops the `examples` keyspace, and the example runner will run it before running the next example. Alternatively, we could use cqlsh, but this would require adding it as a new dependency, which is not desirable.

The simplest way to run examples in CI is to use docker-compose. The docker-compose file, adapted from the Rust Driver, is configured to run a single Scylla node.

The driver does not support DSE, so let's remove the examples.

Some examples, e.g. `callbacks`, require libuv, so we enable it when building examples.

`cpp_integration_testing` is now required to build examples, because some of them use functions that are only available when `cpp_integration_testing` config option is enabled (because they are only implemented as stubs for testing purposes).

wprzytula · 2025-07-17T14:50:27Z

Rebased on master.
CMake is reverted to its original form, in which it only supports building all examples.

Copilot

Pull Request Overview

This PR adds CI support for building and running examples against a ScyllaDB Docker container. The implementation includes comprehensive stub additions for unimplemented functions required by examples, infrastructure for running examples in CI, and cleanup of DSE-specific examples that are not applicable to ScyllaDB.

Adds function stubs for authenticator, host listener, and version types to enable example compilation
Introduces Docker-based ScyllaDB test infrastructure with automated example execution
Removes DSE-specific examples (geotypes, date_range, gssapi, plaintext, proxy_execution) that don't apply to ScyllaDB

Reviewed Changes

Copilot reviewed 25 out of 25 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/examples_cluster/docker-compose.yml	Adds ScyllaDB Docker container configuration for example testing
scylla-rust-wrapper/src/lib.rs	Imports new type modules for authenticator, host listener, and version types
scylla-rust-wrapper/src/integration_testing.rs	Expands stub functions and exports for examples compilation
scylla-rust-wrapper/build.rs	Adds bindgen preparation for new type modules
scylla-rust-wrapper/CMakeLists.txt	Enables integration testing flags for examples builds
Makefile	Adds targets for building and running examples with ScyllaDB
CMakeLists.txt	Links libuv for examples that require it
.github/workflows/build-lint-and-test.yml	Integrates example execution into CI pipeline
examples/drop_examples_keyspace/*	Refactors example to use standard Cassandra API instead of DSE
examples/dse/*	Removes DSE-specific examples not applicable to ScyllaDB

scylla-rust-wrapper/src/integration_testing.rs

Makefile have been updated to support building examples.

Makefile now includes a variable `SCYLLA_EXAMPLES_TO_RUN`, which lists the examples that are expected to run successfully. In a next commit, this variable will be used to run the examples in another Make target.

The `run-examples-scylla` target is added to the Makefile to run examples. The examples are built and run against a ScyllaDB instance running in a Docker container. The `SCYLLA_EXAMPLES_TO_RUN` variable is used to specify which examples to build and run. As some examples create the `examples` keyspace and some assume it is not present, a special example `drop_examples_keyspace` is run to drop the `examples` keyspace before running any example. This lets us avoid adding a dependency on `cqlsh` to the system.

This makes examples run in the CI workflow, ensuring that they are tested and functional.

Lorak-mmk · 2025-07-17T15:27:26Z

scylla-rust-wrapper/CMakeLists.txt


-if(CASS_BUILD_INTEGRATION_TESTS)
+if(CASS_BUILD_INTEGRATION_TESTS OR CASS_BUILD_EXAMPLES)
  set(CMAKE_Rust_FLAGS "${CMAKE_Rust_FLAGS} --cfg cpp_integration_testing")
 endif()



We need to make sure this flag is not set in published binaries - we don't want stubs there.

I've read the packaging logic carefully and my conclusion is that the used build command is correct wrt not including stubs in the generated package.

For redhat derivates, this line is important (dist/redhat/scylla-cpp-rust-driver.spec):

%cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RustFLAGS="-Cforce-frame-pointers=yes --cap-lints=warn"

For debian, it's similar (dist/debian/debian/rules):

dh_auto_configure -- -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_VERBOSE_MAKEFILE=ON -DCMAKE_RustFLAGS="--cap-lints warn -C linker=$(DEB_HOST_GNU_TYPE)-gcc -C link-arg=-Wl,-Bsymbolic-functions -C link-arg=-Wl,-z,relro"

I'm suspicious about the --cap-lints warn part. Why would anyone want to include this? Essentially, this converts all errors to warnings, effectively making the compilation process pass even if it would fail otherwise.
I've talked to ChatGPT on this topic.

wprzytula self-assigned this Jul 6, 2025

wprzytula added the CI Related to continuous integration label Jul 6, 2025

wprzytula added this to the 0.6 milestone Jul 6, 2025

wprzytula force-pushed the run-examples-in-ci branch from 8e5c8ca to 291be5c Compare July 6, 2025 09:10

wprzytula marked this pull request as draft July 6, 2025 10:07

wprzytula force-pushed the run-examples-in-ci branch 2 times, most recently from 3df182f to ea17b61 Compare July 6, 2025 10:35

wprzytula marked this pull request as ready for review July 6, 2025 11:36

wprzytula requested review from Copilot, Lorak-mmk and dkropachev July 6, 2025 11:36

This comment was marked as outdated.

Sign in to view

Lorak-mmk requested changes Jul 7, 2025

View reviewed changes

wprzytula force-pushed the run-examples-in-ci branch from ea17b61 to aac3ca5 Compare July 7, 2025 14:05

wprzytula force-pushed the run-examples-in-ci branch from aac3ca5 to 3378226 Compare July 15, 2025 08:36

dkropachev modified the milestones: 0.6, 1.0.0 Jul 16, 2025

wprzytula added 5 commits July 17, 2025 15:34

add single-node docker-compose for running examples

ec54aca

The simplest way to run examples in CI is to use docker-compose. The docker-compose file, adapted from the Rust Driver, is configured to run a single Scylla node.

examples: delete DSE examples

c05137c

The driver does not support DSE, so let's remove the examples.

cmake: link libuv for examples

83f6912

Some examples, e.g. `callbacks`, require libuv, so we enable it when building examples.

wprzytula force-pushed the run-examples-in-ci branch from 3378226 to 4d5a7d9 Compare July 17, 2025 14:45

wprzytula requested review from Lorak-mmk and Copilot July 17, 2025 14:50

Copilot AI reviewed Jul 17, 2025

View reviewed changes

scylla-rust-wrapper/src/integration_testing.rs Show resolved Hide resolved

scylla-rust-wrapper/src/integration_testing.rs Outdated Show resolved Hide resolved

IT: add stubs required for examples to compile

904c26e

wprzytula added 4 commits July 17, 2025 16:55

makefile: add target to build examples

925958d

Makefile have been updated to support building examples.

Makefile: list supported examples

90d635b

Makefile now includes a variable `SCYLLA_EXAMPLES_TO_RUN`, which lists the examples that are expected to run successfully. In a next commit, this variable will be used to run the examples in another Make target.

CI: add run-examples step to CI workflow

3453eb4

This makes examples run in the CI workflow, ensuring that they are tested and functional.

wprzytula force-pushed the run-examples-in-ci branch from 4d5a7d9 to 3453eb4 Compare July 17, 2025 14:55

Lorak-mmk approved these changes Jul 17, 2025

View reviewed changes

wprzytula merged commit ee7064f into scylladb:master Jul 21, 2025
9 checks passed

wprzytula deleted the run-examples-in-ci branch July 21, 2025 12:40

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Run examples in CI #336

Run examples in CI #336

Uh oh!

wprzytula commented Jul 6, 2025 •

edited

Loading

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Lorak-mmk Jul 7, 2025

Uh oh!

wprzytula Jul 7, 2025

Uh oh!

Lorak-mmk Jul 7, 2025

Uh oh!

wprzytula Jul 10, 2025

Uh oh!

Uh oh!

Uh oh!

wprzytula commented Jul 7, 2025

Uh oh!

wprzytula commented Jul 15, 2025

Uh oh!

wprzytula commented Jul 17, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Lorak-mmk Jul 17, 2025

Uh oh!

wprzytula Jul 21, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Run examples in CI #336

Run examples in CI #336

Uh oh!

Conversation

wprzytula commented Jul 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What's done

Pre-review checklist

Uh oh!

This comment was marked as outdated.

Uh oh!

Uh oh!

Lorak-mmk Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

wprzytula Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

Lorak-mmk Jul 7, 2025

Choose a reason for hiding this comment

Uh oh!

wprzytula Jul 10, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

wprzytula commented Jul 7, 2025

Uh oh!

wprzytula commented Jul 15, 2025

Uh oh!

wprzytula commented Jul 17, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Lorak-mmk Jul 17, 2025

Choose a reason for hiding this comment

Uh oh!

wprzytula Jul 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

wprzytula commented Jul 6, 2025 •

edited

Loading

wprzytula Jul 21, 2025 •

edited

Loading