Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for Mac M1 #192

Closed
thettler opened this issue Mar 1, 2021 · 21 comments
Closed

Support for Mac M1 #192

thettler opened this issue Mar 1, 2021 · 21 comments
Labels
improvement Improvement or new feature solved

Comments

@thettler
Copy link

thettler commented Mar 1, 2021

Which release, branch or hash of Docker SDK are you using?

1.30.0

Which operating system (platform/version/architecture) are you using?

Mac Os Big Sur, Mac Book Pro M1

What is the issue that you're experiencing?

On docker/sdk bootstrap deploy.dev.yml the warning WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested appears.
And if i try to start the containers with docker/sdk up this Error appears:

[+] Building 6.7s (21/21) FINISHED                                                                                                                                               
 => [internal] load build definition from Dockerfile                                                                                                                        0.3s
 => => transferring dockerfile: 37B                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                           0.5s
 => => transferring context: 2B                                                                                                                                             0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                       2.3s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                  0.0s
 => [internal] load metadata for docker.io/spryker/php:7.3-alpine3.12                                                                                                       0.9s
 => [internal] load build context                                                                                                                                           0.2s
 => => transferring context: 376B                                                                                                                                           0.0s
 => [ 1/14] FROM docker.io/spryker/php:7.3-alpine3.12@sha256:84ccd6d38962b649b3ff349831ca3b31b090154399ec06ae1f9f35c4ca2fe30f                                               0.0s
 => CACHED [ 2/14] WORKDIR /data                                                                                                                                            0.0s
 => CACHED [ 3/14] RUN mkdir -p /var/log/spryker && chown spryker:spryker /var/log/spryker                                                                                  0.0s
 => CACHED [ 4/14] RUN mkdir -p /home/spryker/.ssh && chmod 0700 /home/spryker/.ssh                                                                                         0.0s
 => CACHED [ 5/14] RUN bash -c '[ ! -z "${KNOWN_HOSTS}" ] && ssh-keyscan -t rsa ${KNOWN_HOSTS} >> /home/spryker/.ssh/known_hosts || true'                                   0.0s
 => CACHED [ 6/14] RUN chown spryker:spryker -R /home/spryker/.ssh                                                                                                          0.0s
 => CACHED [ 7/14] COPY php/php-fpm.d/worker.conf /usr/local/etc/php-fpm.d/worker.conf                                                                                      0.0s
 => CACHED [ 8/14] COPY php/php.ini /usr/local/etc/php/                                                                                                                     0.0s
 => CACHED [ 9/14] COPY php/conf.d/90-opcache.ini /usr/local/etc/php/conf.d                                                                                                 0.0s
 => CACHED [10/14] RUN rm -f /usr/local/etc/php/conf.d/opcache.ini                                                                                                          0.0s
 => CACHED [11/14] RUN mv /usr/local/etc/php/disabled/blackfire.ini /usr/local/etc/php/conf.d/90-blackfire.ini                                                              0.0s
 => CACHED [12/14] COPY php/conf.d/99-from-deploy-yaml-php.ini /usr/local/etc/php/conf.d/                                                                                   0.0s
 => CACHED [13/14] COPY --chown=spryker:spryker jenkins/jenkins.docker.xml.twig /home/spryker/jenkins.docker.xml.twig                                                       0.0s
 => CACHED [14/14] COPY --chown=spryker:spryker php/build.php /home/spryker/build.php                                                                                       0.0s
 => exporting to image                                                                                                                                                      1.6s
 => => exporting layers                                                                                                                                                     0.0s
 => => writing image sha256:ac0090d6674e0bbcc7bec2cc67a4448ba1dbb6fed9a684a3c494009925289d48                                                                                0.0s
 => => naming to docker.io/library/spryker_b2c_dev_base_app:dev                                                                                                             0.0s
[+] Building 3.5s (5/5) FINISHED                                                                                                                                                 
 => [internal] load build definition from Dockerfile                                                                                                                        0.3s
 => => transferring dockerfile: 37B                                                                                                                                         0.0s
 => [internal] load .dockerignore                                                                                                                                           0.5s
 => => transferring context: 35B                                                                                                                                            0.0s
 => resolve image config for docker.io/docker/dockerfile:experimental                                                                                                       0.7s
 => CACHED docker-image://docker.io/docker/dockerfile:experimental@sha256:600e5c62eedff338b3f7a0850beb7c05866e0ef27b2d2e8c02aa468e78496ff5                                  0.0s
 => ERROR [internal] load metadata for docker.io/library/spryker_b2c_dev_base_app:dev                                                                                       1.3s
------
 > [internal] load metadata for docker.io/library/spryker_b2c_dev_base_app:dev:
------
failed to solve with frontend dockerfile.v0: failed to solve with frontend gateway.v0: rpc error: code = Unknown desc = pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed

What are the steps to reproduce the issue?

Simply try to install Spryker on an Mac M1 machine.

What is the expected result of these steps in the absence of the issue?

The Containers would be getting build und started.

What is the actual result?

The error appears and stops the Process

What possible solutions and/or workarounds for the issue do you see?

Make the Image ARM compatible. Or let it run through Rosetta 2.

Is there any other information that might be helpful?

@sprymiker sprymiker added the improvement Improvement or new feature label Mar 3, 2021
@thettler
Copy link
Author

thettler commented Mar 5, 2021

I tinkered araound a little bit more and have some new insights and got it nearly running.

You can remove the warning form the bootstrap command if you add --platform linux/amd64 to bin/command/install/bootstrap.sh:111. This will pull the correct Image.

in/command/install/bootstrap.sh:111

    docker run -i --rm "${userToRun[@]}" \
        -e SPRYKER_DOCKER_SDK_PLATFORM="${_PLATFORM}" \
        -e SPRYKER_DOCKER_SDK_DEPLOYMENT_DIR="${DESTINATION_DIR}" \
        -e VERBOSE="${VERBOSE}" \
        --platform linux/amd64 \
        -v "${tmpDeploymentDir}":/data/deployment:rw \
        spryker_docker_sdk

To Fix theupcommand you have to modify bin/sdk/images/common.sh and also add some --platform linux/amd64

# ...
function Images::_buildApp() {

    local -a sshArgument=()
    local folder=${1}
    local baseAppImage="${SPRYKER_DOCKER_PREFIX}_base_app:${SPRYKER_DOCKER_TAG}"
    local appImage="${SPRYKER_DOCKER_PREFIX}_app:${SPRYKER_DOCKER_TAG}"
    local localAppImage="${SPRYKER_DOCKER_PREFIX}_local_app:${SPRYKER_DOCKER_TAG}"
    local runtimeImage="${SPRYKER_DOCKER_PREFIX}_run_app:${SPRYKER_DOCKER_TAG}"
    local baseCliImage="${SPRYKER_DOCKER_PREFIX}_base_cli:${SPRYKER_DOCKER_TAG}"
    local cliImage="${SPRYKER_DOCKER_PREFIX}_cli:${SPRYKER_DOCKER_TAG}"
    local runtimeCliImage="${SPRYKER_DOCKER_PREFIX}_run_cli:${SPRYKER_DOCKER_TAG}"

    if [ -n "${SSH_AUTH_SOCK_IN_CLI}" ]; then
        sshArgument=('--ssh' 'default')
    fi

    Images::_prepareSecrets
    Registry::Trap::addExitHook 'removeBuildSecrets' "rm -f ${SECRETS_FILE_PATH}"

    Console::verbose "${INFO}Building Application images${NC}"

    docker build \
        -t "${baseAppImage}" \
        -f "${DEPLOYMENT_PATH}/images/common/application/Dockerfile" \
        --progress="${PROGRESS_TYPE}" \
        --build-arg "SPRYKER_PLATFORM_IMAGE=${SPRYKER_PLATFORM_IMAGE}" \
        --build-arg "SPRYKER_LOG_DIRECTORY=${SPRYKER_LOG_DIRECTORY}" \
        --build-arg "SPRYKER_PIPELINE=${SPRYKER_PIPELINE}" \
        --build-arg "APPLICATION_ENV=${APPLICATION_ENV}" \
        --build-arg "SPRYKER_DB_ENGINE=${SPRYKER_DB_ENGINE}" \
        --build-arg "KNOWN_HOSTS=${KNOWN_HOSTS}" \
        --build-arg "SPRYKER_BUILD_HASH=${SPRYKER_BUILD_HASH:-"current"}" \
        --build-arg "SPRYKER_BUILD_STAMP=${SPRYKER_BUILD_STAMP:-""}" \
        "${DEPLOYMENT_PATH}/context" 1>&2

    docker build \
        -t "${appImage}" \
        -f "${DEPLOYMENT_PATH}/images/${folder}/application/Dockerfile" \
        "${sshArgument[@]}" \
        --secret "id=secrets-env,src=$SECRETS_FILE_PATH" \
        --progress="${PROGRESS_TYPE}" \
        --build-arg "SPRYKER_PARENT_IMAGE=${baseAppImage}" \
        --build-arg "SPRYKER_DOCKER_PREFIX=${SPRYKER_DOCKER_PREFIX}" \
        --build-arg "SPRYKER_DOCKER_TAG=${SPRYKER_DOCKER_TAG}" \
        --build-arg "USER_UID=${USER_FULL_ID%%:*}" \
        --build-arg "DEPLOYMENT_PATH=${DEPLOYMENT_PATH}" \
        --build-arg "SPRYKER_PIPELINE=${SPRYKER_PIPELINE}" \
        --build-arg "APPLICATION_ENV=${APPLICATION_ENV}" \
        --build-arg "SPRYKER_DB_ENGINE=${SPRYKER_DB_ENGINE}" \
        --build-arg "SPRYKER_COMPOSER_MODE=${SPRYKER_COMPOSER_MODE}" \
        --build-arg "SPRYKER_COMPOSER_AUTOLOAD=${SPRYKER_COMPOSER_AUTOLOAD}" \
        --build-arg "SPRYKER_BUILD_HASH=${SPRYKER_BUILD_HASH:-"current"}" \
        --build-arg "SPRYKER_BUILD_STAMP=${SPRYKER_BUILD_STAMP:-""}" \
        --platform linux/amd64 \
        . 1>&2

    docker build \
        -t "${localAppImage}" \
        -t "${runtimeImage}" \
        -f "${DEPLOYMENT_PATH}/images/common/application-local/Dockerfile" \
        --progress="${PROGRESS_TYPE}" \
        --build-arg "SPRYKER_PARENT_IMAGE=${appImage}" \
        --platform linux/amd64 \
        "${DEPLOYMENT_PATH}/context" 1>&2

    if [ -n "${SPRYKER_XDEBUG_MODE_ENABLE}" ]; then
        docker build \
            -t "${runtimeImage}" \
            -f "${DEPLOYMENT_PATH}/images/debug/application/Dockerfile" \
            --progress="${PROGRESS_TYPE}" \
            --build-arg "SPRYKER_PARENT_IMAGE=${localAppImage}" \
            --platform linux/amd64 \
            "${DEPLOYMENT_PATH}/context" 1>&2
    fi

    Console::verbose "${INFO}Building CLI images${NC}"

    docker build \
        -t "${baseCliImage}" \
        -f "${DEPLOYMENT_PATH}/images/common/cli/Dockerfile" \
        --progress="${PROGRESS_TYPE}" \
        --build-arg "SPRYKER_PARENT_IMAGE=${localAppImage}" \
        --platform linux/amd64 \
        "${DEPLOYMENT_PATH}/context" 1>&2

    docker build \
        -t "${cliImage}" \
        -t "${runtimeCliImage}" \
        -f "${DEPLOYMENT_PATH}/images/${folder}/cli/Dockerfile" \
        "${sshArgument[@]}" \
        --secret "id=secrets-env,src=$SECRETS_FILE_PATH" \
        --progress="${PROGRESS_TYPE}" \
        --build-arg "SPRYKER_PARENT_IMAGE=${baseCliImage}" \
        --build-arg "DEPLOYMENT_PATH=${DEPLOYMENT_PATH}" \
        --build-arg "SPRYKER_PIPELINE=${SPRYKER_PIPELINE}" \
        --build-arg "SPRYKER_BUILD_HASH=${SPRYKER_BUILD_HASH:-"current"}" \
        --build-arg "SPRYKER_BUILD_STAMP=${SPRYKER_BUILD_STAMP:-""}" \
        --platform linux/amd64 \
        .  1>&2

    if [ -n "${SPRYKER_XDEBUG_MODE_ENABLE}" ]; then
        docker build \
            -t "${runtimeCliImage}" \
            -f "${DEPLOYMENT_PATH}/images/debug/cli/Dockerfile" \
            --progress="${PROGRESS_TYPE}" \
            --build-arg "SPRYKER_PARENT_IMAGE=${cliImage}" \
            --platform linux/amd64 \
            "${DEPLOYMENT_PATH}/context" 1>&2
    fi

    Registry::Trap::releaseExitHook 'removeBuildSecrets'
}
# ...

Unfortunately there is a qemu bug in the elasticsearch image witch prevents spryker from fully working.
qemu: uncaught target signal 11 (Segmentation fault) - core dumped
docker/for-mac#5123

@camilou
Copy link

camilou commented May 3, 2021

Hi @thettler,

Thanks for the last update, it's getting closer on my M1 mac, but after applying the changes you mentioned, and after waiting for a while, i get the following error message:

Service broker is not running. Please check the name.

Do you know how to fix this issue?

Thanks in advance,

Camilo

@dpmR
Copy link

dpmR commented May 20, 2021

Hi all,

I can confirm the same error as @camilou when using the workaround from @thettler with my MacBook Pro (13-inch, M1, 2020).


Recreating spryker_item_b2b_dev_gateway_1 ... done
Restarting spryker_item_b2b_dev_gateway_1  ... done
Restarting spryker_item_b2b_dev_frontend_1 ... done
Service `broker` is not running. Please check the name. 

Any ideas how to solve this?

Also are there plans to provide images for linux/arm64 ?

@fwartner
Copy link

fwartner commented Jun 1, 2021

Anyone fixed this yet? Or got around that issue?

@sebwegscheider
Copy link

Same issue on fresh M1 MacBook Air with Docker for Silicone, as well. Would be great if someone could pick this up.

@DmitriyDem
Copy link

Hello. Any news? Could you please answer when this problem can be resolved for MAC with M1 ?

@alexanderM91
Copy link
Collaborator

Hello. Any news? Could you please answer when this problem can be resolved for MAC with M1 ?

Hello, @DmitriyDem and sorry for the log response time.

The support of the MAC with M1 is planned until the end of the Q4.

If this is a blocker for you, you can take a look at the experimental branch.

@DmitriyDem
Copy link

Thanks for response!✌️

@janwidmer
Copy link

Hi there..
Any news if the planned support for M1 will arrive these days or if there are significant delays?

@sebastianwindau
Copy link

Hi,
any news on this ?

@dpmR
Copy link

dpmR commented Jan 7, 2022

I got a response in the slack channel that it was postponed to Q1 2022

@jordan-jarolim
Copy link

Q1 is almost over, any updates on this? Shall we wait 2 more days or it got delayed to Q2? Thanks.

@ynnoig
Copy link

ynnoig commented Mar 30, 2022

To me personally a few weeks ago it was confirmed that it was not postponed, but at this point a day before the end of Q1 I actually have doubts that by tomorrow support for M1 can be officially released.

@jordan-jarolim
Copy link

To me personally a few weeks ago it was confirmed that it was not postponed, but at this point a day before the end of Q1 I actually have doubts that by tomorrow support for M1 can be officially released.

In the end it does not matter if it's one or X weeks later. It's more about setting right expectations from spy team towards partners. As it was promised for 21/Q4 then 22/Q1, now its Q2, then maybe Q3 and so on... This is a major issue blocking me and my colleagues from switching to M1 macbooks.

I am just trying to make this issue somehow more visible and maybe give it higher priority in your pipeline. Thanks

@ynnoig
Copy link

ynnoig commented Mar 30, 2022

To me personally a few weeks ago it was confirmed that it was not postponed, but at this point a day before the end of Q1 I actually have doubts that by tomorrow support for M1 can be officially released.

In the end it does not matter if it's one or X weeks later. It's more about setting right expectations from spy team towards partners. As it was promised for 21/Q4 then 22/Q1, now its Q2, then maybe Q3 and so on... This is a major issue blocking me and my colleagues from switching to M1 macbooks.

I am just trying to make this issue somehow more visible and maybe give it higher priority in your pipeline. Thanks

I'm not an internal member of the Spryker, so I can only tell you what was communicated to me.

In any case you are completely correct. Although I've been using Spryker Docker SDK with M1 for over a month now and have absolutely no issues whatsoever.

So one thing is the official support and we are all right in complaining, secondly this is not a block at the moment because as I said you can work with it without any problem.

p.s.: of course there is a guide (slack threads...) and some aspects to take into account for the project to be compatible etc...

@ynnoig
Copy link

ynnoig commented Apr 21, 2022

Ping (🍎 M1)

soon comes the end of Q2

any news about the official support for M1?

@alexanderM91
Copy link
Collaborator

alexanderM91 commented Apr 21, 2022

Ping (🍎 M1)

soon comes the end of Q2

any news about the official support for M1?

the current status - In QA.
Yes, you are absolutely right, it was planned for the end of Q1 but unfortunately it has been delayed due to several critical issues.

@ynnoig
Copy link

ynnoig commented May 4, 2022

I just Seen that in Release 1.47.0 has been released the support for ARM Architecture.
…but I see also that experimental branch ‘apple-m1-adjustement’ is still alive and not merged.
I asked now me self: is release 1.47.0 also for M1 ARM Provessor or not?
I am a little bit confused 🫤 @alexanderM91

@alexanderM91
Copy link
Collaborator

alexanderM91 commented May 5, 2022

Hi,
good news, we have officially released support of ARM architecture (M1)

Small guidance how to make your project up and running with ARM architecture (M1) - https://docs.spryker.com/docs/scos/dev/technical-enhancement-integration-guides/switch-to-arm-architecture-m1-chip.html.

Hi @ynnoig,
apple-m1-adjustement it was a temporary solution and it will not be merged into the master branch.
Please use the latest master and attached guidance.

@namikk
Copy link
Contributor

namikk commented May 5, 2022

Hi, @alexanderM91
it seems like this official support for M1 requires elasticsearch version >7.6. That's likely to be a major problem for us that are still on version 5.6.

Why was this requirement introduced how can we continue using 5.6? AFAIK the experimental M1 branch worked just fine with the older version...

Also, I hope you're not planning to remove the experimental branch anytime soon - doing that might prevent some people from working with Spryker on their M1 completely.

@alexanderM91
Copy link
Collaborator

alexanderM91 commented May 6, 2022

Hi @namikk,
The current images weren't provided due to the fact of their EOL - https://www.elastic.co/support/eol BUT to make your migration smooth, they are already prepared - ES (5.6, 6.8) and KIbana (5.6, 6.8) and will be tested soon.
As a suggestion please consider migration.

No, I am not planning to remove a temporary branch in the nearest future due to the migration process (approx. one month) but it will not be supported anymore.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
improvement Improvement or new feature solved
Development

No branches or pull requests