Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] PHP SDK integration #485

Open
wants to merge 73 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
73 commits
Select commit Hold shift + click to select a range
79d08a6
feat: add PHP initialization
roxblnfk May 14, 2024
9094700
chore(PHP): implement RoadRunner run command generator
roxblnfk May 16, 2024
3a77372
feat(PHP): add ClassLocator; implement Workflow and Activity classes …
roxblnfk May 16, 2024
816da41
chore(PHP): RoadRunner now is run from PHP script that will start als…
roxblnfk May 21, 2024
134d79b
chore(PHP): implemented checks starting; refactoring; implemented que…
roxblnfk May 21, 2024
604af1a
chore(PHP): better notification about failures
roxblnfk May 29, 2024
579f976
fix(PHP): fixed task queue binding and workflow run with correct task…
roxblnfk May 29, 2024
a6c16dc
chore(PHP): add workflow injector that runs workflow with correct que…
roxblnfk May 29, 2024
5288948
Merge branch 'main' into php
roxblnfk Jun 3, 2024
9a27208
feat(PHP): add the RoadRunner runner service to be able to stop and r…
roxblnfk Jun 3, 2024
fe89bf7
chore(PHP): add feature `query/unexpected_arguments`
roxblnfk Jun 3, 2024
5836b51
chore(PHP): finish all the `query` features
roxblnfk Jun 3, 2024
048cd59
chore(PHP): add activity features: `basic_no_workflow_timeout`, `canc…
roxblnfk Jun 4, 2024
32397df
chore(PHP): add Child Workflow feature `signal`
roxblnfk Jun 6, 2024
85f37c1
chore(PHP): add Child Workflow features: `result` and `throws_on_exec…
roxblnfk Jun 6, 2024
a1abdd0
chore(PHP): add feature `continue_as_new/continue_as_same`
roxblnfk Jun 6, 2024
53f5a87
chore(PHP): add feature `eager_workflow/successful_start"`
roxblnfk Jun 6, 2024
8a720f0
feat(PHP): add ability to inject client Interceptor provider via feat…
roxblnfk Jun 6, 2024
33ff0e6
chore(PHP): add feature `data_converter/json`
roxblnfk Jun 6, 2024
282108a
chore(PHP): add feature `data_converter/json_protobuf`
roxblnfk Jun 6, 2024
511e009
chore(PHP): add feature `data_converter/empty`
roxblnfk Jun 12, 2024
29aed00
chore(PHP): add feature `data_converter/failure`
roxblnfk Jun 12, 2024
65e75a7
feat(PHP): support custom Payload Converters in client and server sides
roxblnfk Jun 12, 2024
249dd7f
chore(PHP): add feature `data_converter/codec`
roxblnfk Jun 12, 2024
34d1f53
fix(PHP): fix constants conflict in feature files; place binary proto…
roxblnfk Jun 12, 2024
baeae7e
chore(PHP): add feature `data_converter/binary_protobuf`
roxblnfk Jun 12, 2024
a963c9e
chore(PHP): add feature `data_converter/binary`
roxblnfk Jun 13, 2024
fd20fb7
chore(PHP): add feature `eager_activity/non_remote_activities_worker`
roxblnfk Jun 13, 2024
b2d0e68
chore(PHP): add feature `schedule\backfill`
roxblnfk Jun 13, 2024
d9be0c0
chore(PHP): add feature `schedule\basic`
roxblnfk Jun 13, 2024
6273ef6
chore(PHP): add feature `schedule\pause`
roxblnfk Jun 14, 2024
26afce6
chore(PHP): add feature `schedule\trigger`
roxblnfk Jun 14, 2024
c26c534
chore(PHP): add feature `signal\activities`
roxblnfk Jun 14, 2024
d313870
chore(PHP): add feature `signal\basic`
roxblnfk Jun 14, 2024
0db858d
chore(PHP): add feature `signal\child_workflow`
roxblnfk Jun 14, 2024
1856f02
chore(PHP): add feature `signal\external`
roxblnfk Jun 14, 2024
3d4cd29
chore(PHP): add feature `signal\prevent_close`
roxblnfk Jun 14, 2024
9352318
chore(PHP): add feature `signal\signal_with_start`
roxblnfk Jun 14, 2024
aad7f62
chore(PHP): add case into the `signal\signal_with_start` feature
roxblnfk Jun 14, 2024
97019fb
chore(PHP): add case `update\activities`
roxblnfk Jun 17, 2024
ea685ae
chore(PHP): add case `update\async_accepted`
roxblnfk Jun 17, 2024
abc1b88
chore(PHP): add case `update/basic`
roxblnfk Jun 18, 2024
2d7fcdc
chore(PHP): add case `update/basic_async`
roxblnfk Jun 18, 2024
687f57d
chore(PHP): add case `update/client_interceptor`
roxblnfk Jun 18, 2024
b419976
chore(PHP): add case `update/deduplication`
roxblnfk Jun 18, 2024
4e24644
chore(PHP): add case `update/non_durable_reject`
roxblnfk Jun 18, 2024
e28e329
chore(PHP): add case `update/self`
roxblnfk Jun 18, 2024
1403c20
chore(PHP): add case `update/task_failure`
roxblnfk Jun 18, 2024
f60b896
chore(PHP): add case `update/validation_replay`
roxblnfk Jun 19, 2024
7b7181f
feat(PHP): configure KV module; add case `update/worker_restart`
roxblnfk Jun 19, 2024
77daa38
chore(PHP): support testing in a separated dir
roxblnfk Jun 19, 2024
b0fdd42
chore(PHP): add PHP dockerfile; cleanup
roxblnfk Jun 19, 2024
7da3f93
chore(PHP): fix PHP dockerfile; add github workflow
roxblnfk Jun 19, 2024
f8bf366
Merge branch 'main' into php
roxblnfk Jun 19, 2024
99369b4
chore(PHP): polish update/* tests
roxblnfk Jul 5, 2024
28c6a1c
Merge remote-tracking branch 'refs/remotes/temporalio/main' into php
roxblnfk Jul 17, 2024
6e4d991
chore(PHP): fix todos
roxblnfk Sep 23, 2024
20a9fb6
Merge remote-tracking branch 'temporalio/main' into php
roxblnfk Sep 23, 2024
8c8d620
chore: Sync with PHP SDK 2.11
roxblnfk Oct 8, 2024
765b3ba
Merge remote-tracking branch 'refs/remotes/temporalio/main' into php
roxblnfk Oct 8, 2024
4d5f84d
ci: Fix PHP version detection before Docker image building
roxblnfk Oct 8, 2024
6c5d91d
ci: add commands to build PHP image
roxblnfk Oct 8, 2024
67008d5
ci: add prefix `v` for php-ver input
roxblnfk Oct 8, 2024
a3836ae
ci: fix typo
roxblnfk Oct 8, 2024
18e8804
Ignore platform req on composer install
roxblnfk Oct 8, 2024
ecb9300
Merge branch 'refs/heads/ci-fix-php-version-detection' into php
roxblnfk Oct 8, 2024
fe0bf03
Skip data_converter/failure last check
roxblnfk Oct 8, 2024
caffa59
Skip one of update/task_failure tests
roxblnfk Oct 8, 2024
b99c37e
Fix schedule/basic test: add 10 sec timeout to find schedule
roxblnfk Oct 8, 2024
d1b3c8b
Fix installing dependencies on PHP image building
roxblnfk Oct 8, 2024
210ff83
Optimize PHP dockerfile
roxblnfk Oct 8, 2024
630e9f3
Mark eager_activity tests skipped
roxblnfk Oct 8, 2024
075143b
Add GITHUB_TOKEN to download RoadRunner without a limit
roxblnfk Oct 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/all-docker-images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ on:
description: Python SDK ver to build. Skipped if not specified. Must start with v.
type: string
php-ver:
description: PHP SDK ver to build. Skipped if not specified.
description: PHP SDK ver to build. Skipped if not specified. Must start with v.
type: string
ts-ver:
description: TypeScript SDK ver to build. Skipped if not specified. Must start with v.
Expand Down
39 changes: 38 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,9 @@ on: # rebuild any PRs and main branch changes
java_sdk_version:
default: ''
type: string
php_sdk_version:
default: ''
type: string
python_sdk_version:
default: ''
type: string
Expand All @@ -39,6 +42,7 @@ jobs:
go_latest: ${{ steps.latest_version.outputs.go_latest }}
typescript_latest: ${{ steps.latest_version.outputs.typescript_latest }}
java_latest: ${{ steps.latest_version.outputs.java_latest }}
php_latest: ${{ steps.latest_version.outputs.php_latest }}
python_latest: ${{ steps.latest_version.outputs.python_latest }}
csharp_latest: ${{ steps.latest_version.outputs.csharp_latest }}
steps:
Expand Down Expand Up @@ -75,6 +79,13 @@ jobs:
fi
echo "java_latest=$java_latest" >> $GITHUB_OUTPUT

php_latest="${{ github.event.inputs.php_sdk_version }}"
if [ -z "$php_latest" ]; then
php_latest=$(./temporal-features latest-sdk-version --lang php)
echo "Derived latest PHP SDK release version: $php_latest"
fi
echo "php_latest=$php_latest" >> $GITHUB_OUTPUT

python_latest="${{ github.event.inputs.python_sdk_version }}"
if [ -z "$python_latest" ]; then
python_latest=$(./temporal-features latest-sdk-version --lang py)
Expand Down Expand Up @@ -122,6 +133,23 @@ jobs:
- run: poetry install --no-root
- run: poe lint

build-php:
strategy:
fail-fast: true
matrix:
os: [ubuntu-latest]
runs-on: ${{ matrix.os }}
steps:
- name: Print build information
run: 'echo head_ref: "$GITHUB_HEAD_REF", ref: "$GITHUB_REF", os: ${{ matrix.os }}'
- uses: actions/checkout@v4
- name: Setup PHP 8.2
uses: shivammathur/setup-php@v2
with:
php-version: 8.2
tools: composer:v2
extensions: dom, sockets, grpc, curl, protobuf

build-java:
strategy:
fail-fast: true
Expand Down Expand Up @@ -182,6 +210,15 @@ jobs:
features-repo-ref: ${{ github.head_ref }}
features-repo-path: ${{ github.event.pull_request.head.repo.full_name }}

feature-tests-php:
needs: build-go
uses: ./.github/workflows/php.yaml
with:
version: ${{ needs.build-go.outputs.php_latest }}
version-is-repo-ref: false
features-repo-ref: ${{ github.head_ref }}
features-repo-path: ${{ github.event.pull_request.head.repo.full_name }}

feature-tests-java:
needs: build-go
uses: ./.github/workflows/java.yaml
Expand Down Expand Up @@ -209,6 +246,6 @@ jobs:
go-ver: 'v${{ needs.build-go.outputs.go_latest }}'
ts-ver: 'v${{ needs.build-go.outputs.typescript_latest }}'
java-ver: 'v${{ needs.build-go.outputs.java_latest }}'
php-ver: '${{ needs.build-go.outputs.php_latest }}'
php-ver: 'v${{ needs.build-go.outputs.php_latest }}'
py-ver: 'v${{ needs.build-go.outputs.python_latest }}'
cs-ver: 'v${{ needs.build-go.outputs.csharp_latest }}'
2 changes: 2 additions & 0 deletions .github/workflows/docker-images.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,8 @@ jobs:

# This step will set the FEATURES_BUILT_IMAGE_TAG env key
- name: Build docker image
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
go run . build-image --lang ${{ inputs.lang }} \
${{ inputs.sdk-repo-ref && format('--repo-ref {0}', inputs.sdk-repo-ref) || '' }} \
Expand Down
101 changes: 101 additions & 0 deletions .github/workflows/php.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,101 @@
name: PHP Features Testing
on:
workflow_call:
inputs:
php-repo-path:
type: string
default: 'temporal/sdk'
version:
required: true
type: string
# When true, the default version will be used (actually it's the latest tag)
version-is-repo-ref:
required: true
type: boolean
features-repo-path:
type: string
default: 'temporalio/features'
features-repo-ref:
type: string
default: 'main'
# If set, download the docker image for server from the provided artifact name
docker-image-artifact-name:
type: string
required: false

jobs:
test:
runs-on: ubuntu-latest
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
defaults:
run:
working-directory: ./features
steps:
- name: Print git info
run: 'echo head_ref: "$GITHUB_HEAD_REF", ref: "$GITHUB_REF", PHP sdk version: ${{ inputs.version }}'
working-directory: '.'

- name: Download docker artifacts
if: ${{ inputs.docker-image-artifact-name }}
uses: actions/download-artifact@v3
with:
name: ${{ inputs.docker-image-artifact-name }}
path: /tmp/server-docker

- name: Load server Docker image
if: ${{ inputs.docker-image-artifact-name }}
run: docker load --input /tmp/server-docker/temporal-autosetup.tar
working-directory: '.'

- name: Override IMAGE_TAG environment variable
if: ${{ inputs.docker-image-artifact-name }}
run: |
image_tag=latest
# image_tag won't exist on older builds (like 1.22.0), so default to latest
if [ -f /tmp/server-docker/image_tag ]; then
image_tag=$(cat /tmp/server-docker/image_tag)
fi
echo "IMAGE_TAG=${image_tag}" >> $GITHUB_ENV
working-directory: '.'

- name: Checkout SDK features repo
uses: actions/checkout@v4
with:
path: features
repository: ${{ inputs.features-repo-path }}
ref: ${{ inputs.features-repo-ref }}

- uses: actions/setup-go@v2
with:
go-version: '^1.22'
- name: Setup PHP 8.2
uses: shivammathur/setup-php@v2
with:
php-version: 8.2
tools: composer:v2
extensions: dom, sockets, grpc, curl, protobuf
- name: Start containerized server and dependencies
if: inputs.docker-image-artifact-name
run: |
docker compose \
-f ./dockerfiles/docker-compose.for-server-image.yaml \
-f /tmp/server-docker/docker-compose.yml \
up -d temporal-server cassandra elasticsearch

- name: Run SDK-features tests directly
if: inputs.docker-image-artifact-name == ''
run: go run . run --lang php ${{ inputs.docker-image-artifact-name && '--server localhost:7233 --namespace default' || ''}} --version "${{ inputs.version-is-repo-ref && '' || inputs.version }}"

# Running the tests in their own step keeps the logs readable
- name: Run containerized SDK-features tests
if: inputs.docker-image-artifact-name
run: |
docker compose \
-f ./dockerfiles/docker-compose.for-server-image.yaml \
-f /tmp/server-docker/docker-compose.yml \
up --no-log-prefix --exit-code-from features-tests-php features-tests-php

- name: Tear down docker compose
if: inputs.docker-image-artifact-name && (success() || failure())
run: docker compose -f ./dockerfiles/docker-compose.for-server-image.yaml -f /tmp/server-docker/docker-compose.yml down -v
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ These features serve several purposes:
With latest [Go](https://golang.org/) installed, run:

```
go build -o temporal-features # or temporal-features.exec on Windows
go build -o temporal-features # or temporal-features.exe on Windows
```

## Running
Expand All @@ -31,14 +31,16 @@ Prerequisites:
- [Poetry](https://python-poetry.org/): `poetry install`
- `setuptools`: `python -m pip install -U setuptools`
- [.NET](https://dotnet.microsoft.com) 7+
- [PHP](https://www.php.net/) 8.1+
- [Composer](https://getcomposer.org/)

Command:

temporal-features run --lang LANG [--version VERSION] [PATTERN...]

Note, `go run .` can be used in place of `go build` + `temporal-features` to save on the build step.

`LANG` can be `go`, `java`, `ts`, `py`, or `cs`. `VERSION` is per SDK and if left off, uses the latest version set for
`LANG` can be `go`, `java`, `ts`, `php`, `py`, or `cs`. `VERSION` is per SDK and if left off, uses the latest version set for
the language in this repository.

`PATTERN` must match either the features relative directory _or_ the relative directory + `/feature.<ext>` via
Expand Down
2 changes: 2 additions & 0 deletions cmd/prepare.go
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,8 @@ func (p *Preparer) Prepare(ctx context.Context) error {
_, err = p.BuildJavaProgram(ctx, true)
case "ts":
_, err = p.BuildTypeScriptProgram(ctx)
case "php":
_, err = p.BuildPhpProgram(ctx)
case "py":
_, err = p.BuildPythonProgram(ctx)
case "cs":
Expand Down
14 changes: 12 additions & 2 deletions cmd/run.go
Original file line number Diff line number Diff line change
Expand Up @@ -295,6 +295,16 @@ func (r *Runner) Run(ctx context.Context, patterns []string) error {
if err == nil {
err = r.RunTypeScriptExternal(ctx, run)
}
case "php":
if r.config.DirName != "" {
r.program, err = sdkbuild.PhpProgramFromDir(
filepath.Join(r.rootDir, r.config.DirName),
r.rootDir,
)
}
if err == nil {
err = r.RunPhpExternal(ctx, run)
}
case "py":
if r.config.DirName != "" {
r.program, err = sdkbuild.PythonProgramFromDir(filepath.Join(r.rootDir, r.config.DirName))
Expand Down Expand Up @@ -545,7 +555,7 @@ func (r *Runner) destroyTempDir() {
func normalizeLangName(lang string) (string, error) {
// Normalize to file extension
switch lang {
case "go", "java", "ts", "py", "cs":
case "go", "java", "ts", "php", "py", "cs":
case "typescript":
lang = "ts"
case "python":
Expand All @@ -561,7 +571,7 @@ func normalizeLangName(lang string) (string, error) {
func expandLangName(lang string) (string, error) {
// Expand to lang name
switch lang {
case "go", "java", "typescript", "python":
case "go", "java", "typescript", "php", "python":
case "ts":
lang = "typescript"
case "py":
Expand Down
77 changes: 77 additions & 0 deletions cmd/run_php.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
package cmd

import (
"context"
"fmt"
"path/filepath"
"github.com/temporalio/features/harness/go/cmd"
"github.com/temporalio/features/sdkbuild"
)

// PreparePhpExternal prepares a PHP run without running it. The preparer
// config directory if present is expected to be a subdirectory name just
// beneath the root directory.
func (p *Preparer) BuildPhpProgram(ctx context.Context) (sdkbuild.Program, error) {
p.log.Info("Building PHP project", "DirName", p.config.DirName)

prog, err := sdkbuild.BuildPhpProgram(ctx, sdkbuild.BuildPhpProgramOptions{
DirName: p.config.DirName,
Version: p.config.Version,
RootDir: p.rootDir,
})
if err != nil {
p.log.Error("failed preparing: %w", err)
return nil, fmt.Errorf("failed preparing: %w", err)
}
return prog, nil
}

// RunPhpExternal runs the PHP run in an external process. This expects
// the server to already be started.
func (r *Runner) RunPhpExternal(ctx context.Context, run *cmd.Run) error {
// If program not built, build it
if r.program == nil {
var err error
if r.program, err = NewPreparer(r.config.PrepareConfig).BuildPhpProgram(ctx); err != nil {
return err
}
}

// Compose RoadRunner command options
args := append(
[]string{
// Namespace
"namespace=" + r.config.Namespace,
// Server address
"address=" + r.config.Server,
},
// Features
run.ToArgs()...,
)
// TLS
if r.config.ClientCertPath != "" {
clientCertPath, err := filepath.Abs(r.config.ClientCertPath)
if err != nil {
return err
}
args = append(args, "tls.cert="+clientCertPath)
}
if r.config.ClientKeyPath != "" {
clientKeyPath, err := filepath.Abs(r.config.ClientKeyPath)
if err != nil {
return err
}
args = append(args, "tls.key="+clientKeyPath)
}

// Run
cmd, err := r.program.NewCommand(ctx, args...)
if err == nil {
// r.log.Debug("Running PHP separately", "Args", cmd.Args)
err = cmd.Run()
}
if err != nil {
return fmt.Errorf("failed running: %w", err)
}
return nil
}
10 changes: 10 additions & 0 deletions dockerfiles/docker-compose.for-server-image.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,3 +69,13 @@ services:
- temporal-server
networks:
- temporal-dev-network

features-tests-php:
image: temporaliotest/features:php
environment:
- WAIT_EXTRA_FOR_NAMESPACE
command: ['--server', 'temporal-server:7233', '--namespace', 'default']
depends_on:
- temporal-server
networks:
- temporal-dev-network
12 changes: 4 additions & 8 deletions dockerfiles/php.Dockerfile
Original file line number Diff line number Diff line change
@@ -1,20 +1,16 @@
# Build in a full featured container
FROM php:8.2 as build
FROM php:8.2-cli as build

# Install protobuf compiler
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive \
apt-get install --no-install-recommends --assume-yes \
protobuf-compiler=3.* libprotobuf-dev=3.* wget=*
protobuf-compiler=3.* libprotobuf-dev=3.* wget=* git=*

# Get go compiler
ARG PLATFORM=amd64
RUN wget -q https://go.dev/dl/go1.22.4.linux-${PLATFORM}.tar.gz \
&& tar -C /usr/local -xzf go1.22.4.linux-${PLATFORM}.tar.gz
# Install Rust for compiling the core bridge - only required for installation from a repo but is cheap enough to install
# in the "build" container (-y is for non-interactive install)
# hadolint ignore=DL4006
RUN wget -q -O - https://sh.rustup.rs | sh -s -- -y
RUN wget -q https://go.dev/dl/go1.22.5.linux-${PLATFORM}.tar.gz \
&& tar -C /usr/local -xzf go1.22.5.linux-${PLATFORM}.tar.gz

# Install composer
COPY --from=composer:2.3 /usr/bin/composer /usr/bin/composer
Expand Down
Loading
Loading