How to handle pushes in multimodule project with immutable tags

[chtpl]

I have a multimodule Maven project with multiple Spring Boot microservices, one per maven module
Then I have an ECR with 2 repositories per microservice.
On repository receives tags "dev" and "stage" which are created by jib. Everytime deploying to dev or stage, the images will be overwritten by jib.

However, for production, I use proper version number tags and these must be immutable. Jib receives the version of the tag from ${project.version}
So if I would release microservice A 1.0.0 now and B 1.0.0 and then I discover a bug in B, fix it, bump the version of B to 1.0.1 and keep the version of A at 1.0.0, I guess the next build (CI builds all modules everytime for main branch) then the push for A 1.0.0 would fail because A 1.0.0 already exists on the ECR and is immutable.
Thus, the whole build would fail, right? So B 1.0.1 would not be pushed.

What can I do in this case?

[elefeint]

If your releases are immutable, they are probably done by your CI, not from a local machine, right? Would the CI then release both A and B to version 1.0.1 in tandem? Or is it expected that each microservice will be released on its own schedule?

[chtpl]

they will be released together by the CI

[elefeint]

Could you try -DskipExistingImages option, as in this answer by @chanseokoh ?

[chtpl]

@elefeint it does not work :(

This is the profile I'm using

        <profile>
            <id>jib-prod</id>
            <activation>
                <property>
                    <name>jib-prod</name>
                    <value>true</value>
                </property>
            </activation>
            <dependencies>
                <!-- https://mvnrepository.com/artifact/com.google.cloud.tools/jib-maven-plugin -->
                <dependency>
                    <groupId>com.google.cloud.tools</groupId>
                    <artifactId>jib-maven-plugin</artifactId>
                    <version>3.0.0</version>
                </dependency>
            </dependencies>
            <build>
                <defaultGoal>compile test jib:build</defaultGoal>
                <plugins>
                    <plugin>
                        <groupId>com.google.cloud.tools</groupId>
                        <artifactId>jib-maven-plugin</artifactId>
                        <version>3.0.0</version>
                        <configuration>
                            <container>
                                <entrypoint>
                                    <arg>/bin/sh</arg>
                                    <arg>-c</arg>
                                    <arg>echo "PROFILE: $PROFILE"; exec java \
                                        -Dspring.profiles.active=$PROFILE \
                                        $JAVA_OPTS -cp /app/resources:/app/classes:/app/libs/* \
                                        com.company.MyApplication
                                    </arg>
                                </entrypoint>
                            </container>
                            <from>
                                <image>
                                    <ACCOUNTID>.dkr.ecr.eu-central-1.amazonaws.com/baseimages:openjdk-8
                                </image>
                            </from>
                            <to>
                                <image>
                                    <ACCOUNTID>.dkr.ecr.eu-central-1.amazonaws.com/myapp:${project.version}
                                </image>
                            </to>
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>

I use this command to run the build mvn -B -Djib-prod=true -Djib.skipExistingImages

@karlmuscat could it be that this is because the image SHA is different on my subsequent builds (you mentioned the SHA here #2370 (comment))? I include a git-commit info in the application's resources which changes (FYI: I use a monorepo with multiple Spring Boot services - so when changing service A, all services are built and they all get the updated commit info).

[chtpl]

@elharo could you please reopen the issue? Thx :)

[karlmuscat]

If you have a different SHA every time you build, then you have non-reproducible builds. This means that a different image is being built every time you commit (even though no code changes have happened in the actual module) and hence JIB will try to upload it. The option skipExistingImages only works if an image with the same SHA already exists on ECR.

[chtpl]

@karlmuscat thanks for the clarification.

An additional option like "-D.jib.skipExistingImagesMode" would be nice which would allow to use an option of either "SHA" or "TAG". When using "TAG", it should not check for the SHA but the TAG name being already on the registry

[chanseokoh]

@chtpl so we gave a thought on the skip-push-if-tag-exists idea, which is not difficult to implement. I can understand your situation, but the feature feels a bit weird to us and I don't think we will implement that. It's like "I built a new, different image (in terms of SHA) but I will discard it only because there exists an outdated image in a target registry."

There may be some feasible workarounds in your situation. For example, instead of a static tag (e.g., 1.0.0), you could create a unique tag for each new image (e.g., suffixing a git commit like 1.0.0-<commit>). It's also possible to selectively run Jib in a multi-module Maven project, if the main issue is that a failure in one sub-module blocks other sub-modules, like ./mvnw --also-make --projects module-a,module-b package jib:build.

[chtpl]

I fixed this by using com.vackosar.gitflowincrementalbuilder plugin which only builds the services when there are changes so existing images will not be pushed again