chore: adopt buildx bake when building container images #1145

MartinForReal · 2022-02-15T04:48:19Z

What type of PR is this?

/kind cleanup

What this PR does / why we need it:

adopt buildx bake to boost image building process

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

May introduce regression to e2e tests.

Does this PR introduce a user-facing change?

NONE

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot · 2022-02-15T04:48:36Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: MartinForReal

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [MartinForReal]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

coveralls · 2022-02-15T04:52:46Z

Coverage remained the same at 80.182% when pulling 81a75d0 on MartinForReal:chores/adopt_buildx_bake into 10a510e on kubernetes-sigs:master.

MartinForReal · 2022-02-15T04:57:58Z

/assign @nilo19

MartinForReal · 2022-02-15T05:00:08Z

/cc @lodrem

k8s-ci-robot · 2022-02-15T05:00:09Z

@MartinForReal: GitHub didn't allow me to request PR reviews from the following users: lodrem.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @lodrem

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

zarvd · 2022-02-15T05:12:07Z

Makefile

 buildx-setup:
 	docker buildx inspect img-builder > /dev/null || docker buildx create --name img-builder --use
 	# enable qemu for arm64 build
 	# https://github.com/docker/buildx/issues/464#issuecomment-741507760
 	docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
 	docker run --rm --privileged tonistiigi/binfmt --install all

-.PHONY: build-ccm-image
-build-ccm-image: buildx-setup docker-pull-prerequisites ## Build controller-manager image.


Could we keep this or refine it to build for the specified arch/platform? The internal pipeline may depend on this (build and export to tar).

build-ccm-image-% target is added. e.g make build-ccm-image-amd64

same case for cnm, and we already use the following syntax to build cnm to mcr.
OUTPUT_TYPE="docker,dest=$mydir/azure-cloud-node-manager.tar" ARCH=$(arch) make build-node-image-linux

zarvd · 2022-02-15T05:19:59Z

buildx-bake.hcl

+}
+
+group "default" {
+    targets = ["ccm-amd64","ccm-arm64","ccm-arm","cnm-amd64","cnm-arm64","cnm-arm"]


Could this also support the Windows targets?

MartinForReal · 2022-02-15T08:05:28Z

/cc @AbelHu

k8s-ci-robot · 2022-02-15T08:05:33Z

@MartinForReal: GitHub didn't allow me to request PR reviews from the following users: AbelHu.

Note that only kubernetes-sigs members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

/cc @AbelHu

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

MartinForReal · 2022-02-16T07:49:18Z

/assign @feiskyer

MartinForReal · 2022-02-16T11:05:35Z

/cc @mainred

mainred

Thanks for introducing buildx baker, left several comments.

mainred · 2022-02-17T02:37:53Z

Makefile

 buildx-setup:
 	docker buildx inspect img-builder > /dev/null || docker buildx create --name img-builder --use
 	# enable qemu for arm64 build
 	# https://github.com/docker/buildx/issues/464#issuecomment-741507760
 	docker run --privileged --rm tonistiigi/binfmt --uninstall qemu-aarch64
 	docker run --rm --privileged tonistiigi/binfmt --install all

-.PHONY: build-ccm-image
-build-ccm-image: buildx-setup docker-pull-prerequisites ## Build controller-manager image.


same case for cnm, and we already use the following syntax to build cnm to mcr.
OUTPUT_TYPE="docker,dest=$mydir/azure-cloud-node-manager.tar" ARCH=$(arch) make build-node-image-linux

Makefile

mainred · 2022-02-17T02:41:49Z

Makefile


+build-ccm-image-%: 
+	ENABLE_GIT_COMMAND="$(ENABLE_GIT_COMMAND)" VERSION="$(VERSION)" OUTPUT=registry docker buildx bake --pull  --progress plain -f buildx-bake.hcl ccm-$*


ditto and same for the build-cnm-image-% you are going to add.

Makefile

MartinForReal · 2022-03-14T11:33:57Z

/retest

MartinForReal · 2022-03-14T14:16:55Z

/retest

Makefile

mainred · 2022-03-15T01:13:21Z

Makefile


+.PHONY: build-node-image-linux
+build-node-image-linux: buildx-setup 


OUTPUT_TYPE="docker,dest=$(image.release.dir)/linux/$(arch)/azure-cloud-node-manager.tar" ARCH=$(arch) make build-node-image-linux is used to build cnm with specific arch, can we support this?

https://dev.azure.com/AzureContainerUpstream/Kubernetes/_git/ado-build-definitions?path=/pipelines/cloud-provider.azure.build.yml&version=GBmaster&line=53&lineEnd=54&lineStartColumn=1&lineEndColumn=1&lineStyle=plain&_a=contents

IMAGE_REGISTRY=martindlut OUTPUT=registry make build-node-image-linux can be used to build image for all platform.

Is it possible to build arm64 cnm image alone? Through the buildx baker target you provided "cnm-linux-amd64","cnm-linux-arm64","cnm-linux-arm",

try build-node-image-linux

MartinForReal · 2022-03-15T02:40:20Z

/retest

Signed-off-by: MartinForReal <[email protected]>

mainred · 2022-03-15T03:30:14Z

Makefile


+.PHONY: build-node-image-linux
+build-node-image-linux: buildx-setup 
+	ENABLE_GIT_COMMAND="$(ENABLE_GIT_COMMAND)" VERSION="$(VERSION)" OUTPUT="$(OUTPUT_TYPE)" IMAGE_TAG="$(IMAGE_TAG)" docker buildx bake --pull --progress plain -f buildx-bake.hcl cnm-linux-amd64


docker buildx bake --pull --progress plain -f buildx-bake.hcl cnm-linux-amd64, -amd64?

Yes, It is amd64 only

lzhecheng · 2022-03-15T03:59:39Z

buildx-bake.hcl

+    tags = [
+        "${IMAGE_REGISTRY}/azure-cloud-node-manager:${IMAGE_TAG}-windows-ltsc2022-amd64"
+    ]
+}


nit: an empty line here.

MartinForReal · 2022-03-15T05:14:36Z

/test pull-cloud-provider-azure-e2e-ccm-capz

MartinForReal · 2022-03-15T07:24:09Z

/test pull-cloud-provider-azure-e2e-ccm-capz

🐛 Handle pointer to alias in DeepCopy

k8s-ci-robot requested review from feiskyer and nilo19 February 15, 2022 04:48

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 15, 2022

MartinForReal changed the title ~~chore: adopt buildx bake when build container images~~ chore: adopt buildx bake when building container images Feb 15, 2022

k8s-ci-robot assigned nilo19 Feb 15, 2022

zarvd reviewed Feb 15, 2022

View reviewed changes

k8s-ci-robot assigned feiskyer Feb 16, 2022

mainred reviewed Feb 17, 2022

View reviewed changes

mainred reviewed Mar 15, 2022

View reviewed changes

chore: adopt buildx bake when build container images

81a75d0

Signed-off-by: MartinForReal <[email protected]>

mainred reviewed Mar 15, 2022

View reviewed changes

lzhecheng reviewed Mar 15, 2022

View reviewed changes

MartinForReal closed this Mar 15, 2022

JoelSpeed pushed a commit to JoelSpeed/cloud-provider-azure that referenced this pull request Feb 18, 2025

Merge pull request kubernetes-sigs#1145 from cbandy/alias-deepcopy

dac769d

🐛 Handle pointer to alias in DeepCopy

MartinForReal mentioned this pull request Feb 24, 2025

normalize the image building targets #8400

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: adopt buildx bake when building container images #1145

chore: adopt buildx bake when building container images #1145

MartinForReal commented Feb 15, 2022

k8s-ci-robot commented Feb 15, 2022

coveralls commented Feb 15, 2022 •

edited

Loading

MartinForReal commented Feb 15, 2022

MartinForReal commented Feb 15, 2022

k8s-ci-robot commented Feb 15, 2022

zarvd Feb 15, 2022

MartinForReal Feb 15, 2022

mainred Feb 17, 2022

zarvd Feb 15, 2022

MartinForReal Feb 15, 2022

MartinForReal commented Feb 15, 2022

k8s-ci-robot commented Feb 15, 2022

MartinForReal commented Feb 16, 2022

MartinForReal commented Feb 16, 2022 •

edited

Loading

mainred left a comment

mainred Feb 17, 2022

mainred Feb 17, 2022

MartinForReal Mar 14, 2022

MartinForReal commented Mar 14, 2022

MartinForReal commented Mar 14, 2022

mainred Mar 15, 2022

MartinForReal Mar 15, 2022

mainred Mar 15, 2022

MartinForReal Mar 15, 2022

MartinForReal commented Mar 15, 2022

mainred Mar 15, 2022

MartinForReal Mar 15, 2022

lzhecheng Mar 15, 2022

MartinForReal commented Mar 15, 2022

MartinForReal commented Mar 15, 2022


		build-ccm-image-%:
		ENABLE_GIT_COMMAND="$(ENABLE_GIT_COMMAND)" VERSION="$(VERSION)" OUTPUT=registry docker buildx bake --pull --progress plain -f buildx-bake.hcl ccm-$*


		.PHONY: build-node-image-linux
		build-node-image-linux: buildx-setup

chore: adopt buildx bake when building container images #1145

chore: adopt buildx bake when building container images #1145

Conversation

MartinForReal commented Feb 15, 2022

What type of PR is this?

What this PR does / why we need it:

Which issue(s) this PR fixes:

Special notes for your reviewer:

Does this PR introduce a user-facing change?

Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

k8s-ci-robot commented Feb 15, 2022

coveralls commented Feb 15, 2022 • edited Loading

MartinForReal commented Feb 15, 2022

MartinForReal commented Feb 15, 2022

k8s-ci-robot commented Feb 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MartinForReal commented Feb 15, 2022

k8s-ci-robot commented Feb 15, 2022

MartinForReal commented Feb 16, 2022

MartinForReal commented Feb 16, 2022 • edited Loading

mainred left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MartinForReal commented Mar 14, 2022

MartinForReal commented Mar 14, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MartinForReal commented Mar 15, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

MartinForReal commented Mar 15, 2022

MartinForReal commented Mar 15, 2022

coveralls commented Feb 15, 2022 •

edited

Loading

MartinForReal commented Feb 16, 2022 •

edited

Loading