[buildkit] cache 기록

forums.docker.com/t/how-to-share-data-between-container-same-service/22571/3

forums.docker.com/t/swarm-manager-copying-files-to-nodes/31010

docs.docker.com/engine/reference/commandline/buildx_build/#cache-to 

docs.docker.com/develop/develop-images/build_enhancements/

docs.docker.com/search/?q=cache%20export

testdriven.io/blog/faster-ci-builds-with-docker-cache/

medium.com/titansoft-engineering/docker-build-cache-sharing-on-multi-hosts-with-buildkit-and-buildx-eb8f7005918e

 

$ docker buildx create --use

$ docker buildx build --cache-to=type=local,dest=./cache .

 

2021. 04. 15 목

디렉토리, Dockerfile, nodejs 패키지 파일

- 디렉토리 구조

$ tree .
.
├── Dockerfile
├── index.js
├── package.json
└── package-lock.json

 

- Dockerfile

FROM node:12
WORKDIR /home/node
COPY package*.json ./
RUN npm install

COPY . .
EXPOSE 3000
CMD ["node", "index.js"]

 

- index.js

const express = require('express')
const app = express()
const port = 3000

app.get('/', (req, res) => {
  res.send('Hello World!')
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})

 

- package.json

{
  "name": "simpleweb",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1"
  }
}

 

- package-lock.json : 생략

 

buildx 사용

- buildx 활성화

$ export DOCKER_BUILDKIT=1

 

- builder 생성

$ docker buildx create --name builder --use

 

- builder 생성 확인

$ docker buildx ls
NAME/NODE  DRIVER/ENDPOINT             STATUS  PLATFORMS
builder *  docker-container                    
  builder0 unix:///var/run/docker.sock running linux/amd64, linux/386
default    docker                              
  default  default                     running linux/amd64, linux/386

 

- build cache 로컬에 저장(export)

$ docker buildx build --load -t node:latest --cache-to=type=local,dest=/home/cent/node_cache .

 

- 로컬 build cache 로부터 Dockerfild build

$ docker buildx build --load -t node:latest --cache-from=type=local,src=/home/cent/node_cache .

 

빌드 시간 비교

- normal build

$ docker build -t node:normal .
[+] Building 31.3s (10/10) FINISHED                                           
 => [internal] load build definition from Dockerfile                     0.1s
 => => transferring dockerfile: 227B                                     0.0s
 => [internal] load .dockerignore                                        0.1s
 => => transferring context: 126B                                        0.0s
 => [internal] load metadata for docker.io/library/node:12               3.5s
 => [1/5] FROM docker.io/library/node:12@sha256:609103746810535f5a3a98  25.1s
 => => resolve docker.io/library/node:12@sha256:609103746810535f5a3a987  0.0s
 => => sha256:3ff9a21f91c215efa3b65ae958df9bd5011e55301 2.21kB / 2.21kB  0.0s
 => => sha256:29be39bd6917eb2ecdd1145bcb6d471c73a5e9b38 7.83kB / 7.83kB  0.0s
 => => sha256:2e2bafe8a0f40509cc10249087268e66a662e43 11.29MB / 11.29MB  1.4s
 => => sha256:b53ce1fd2746e8d2037f1b0b91ddea0cc7411eb3e 4.34MB / 4.34MB  1.3s
 => => sha256:609103746810535f5a3a987a26ba4ce95d96225d28e9d 776B / 776B  0.0s
 => => sha256:76b8ef87096fa726adbe8f073ef69bb5664bac1 45.38MB / 45.38MB  2.8s
 => => sha256:84a8c1bd5887cc4a89e1f286fed9ee31ce12dba 49.79MB / 49.79MB  4.6s
 => => sha256:7a803dc0b40fcd10faee3fb3ebb2d7aaa885 214.35MB / 214.35MB  15.6s
 => => sha256:b800e94e7303e276b8fb4911a40bfe28f46180d99 4.19kB / 4.19kB  3.1s
 => => sha256:a5e0c18be249394ae7365dce1364dfbd9c888de 23.75MB / 23.75MB  4.9s
 => => extracting sha256:76b8ef87096fa726adbe8f073ef69bb5664bac19474c5c  2.7s
 => => sha256:e0ba08bf689ebb4935e5ebf5e7ede61c151c55b73 2.38MB / 2.38MB  5.1s
 => => sha256:20f8d3e06d470f5659ac84232f4d5b309ad360a0b87a0 294B / 294B  5.2s
 => => extracting sha256:2e2bafe8a0f40509cc10249087268e66a662e437f10e95  0.7s
 => => extracting sha256:b53ce1fd2746e8d2037f1b0b91ddea0cc7411eb3e5949f  0.2s
 => => extracting sha256:84a8c1bd5887cc4a89e1f286fed9ee31ce12dba9f6813c  2.9s
 => => extracting sha256:7a803dc0b40fcd10faee3fb3ebb2d7aaa88500520e6295  6.9s
 => => extracting sha256:b800e94e7303e276b8fb4911a40bfe28f46180d997022c  0.0s
 => => extracting sha256:a5e0c18be249394ae7365dce1364dfbd9c888dec4bf948  1.2s
 => => extracting sha256:e0ba08bf689ebb4935e5ebf5e7ede61c151c55b73d2fe8  0.1s
 => => extracting sha256:20f8d3e06d470f5659ac84232f4d5b309ad360a0b87a01  0.0s
 => [internal] load build context                                        0.1s
 => => transferring context: 15.08kB                                     0.0s
 => [2/5] WORKDIR /home/node                                             0.1s
 => [3/5] COPY package*.json ./                                          0.1s
 => [4/5] RUN npm install                                                2.1s
 => [5/5] COPY index.js ./                                               0.1s
 => exporting to image                                                   0.2s
 => => exporting layers                                                  0.2s
 => => writing image sha256:e78b490861e0d95e8aa6d71c01bbae7cdd7f3878b68  0.0s
 => => naming to docker.io/library/node:normal                           0.0s

 

- 로컬 캐시를 이용한 빌드

$ docker buildx build --load -t node:latest --cache-from=type=local,src=/home/cent/node_cache .
[+] Building 23.2s (11/11) FINISHED                                             
 => [internal] load build definition from Dockerfile                       0.1s
 => => transferring dockerfile: 213B                                       0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 126B                                          0.0s
 => [internal] load metadata for docker.io/library/node:12                 3.3s
 => importing cache manifest from local:7718261985460007794                0.0s
 => [1/5] FROM docker.io/library/node:12@sha256:609103746810535f5a3a987a2  0.0s
 => [internal] load build context                                          0.0s
 => => transferring context: 15.42kB                                       0.0s
 => CACHED [2/5] WORKDIR /home/node                                        0.0s
 => CACHED [3/5] COPY package*.json ./                                     0.0s
 => CACHED [4/5] RUN npm install                                           0.0s
 => CACHED [5/5] COPY . .                                                 19.6s
 => => pulling sha256:76b8ef87096fa726adbe8f073ef69bb5664bac19474c5cce4dd  1.3s
 => => pulling sha256:2e2bafe8a0f40509cc10249087268e66a662e437f10e9598a09  0.4s
 => => pulling sha256:b53ce1fd2746e8d2037f1b0b91ddea0cc7411eb3e5949fe10c0  0.3s
 => => pulling sha256:84a8c1bd5887cc4a89e1f286fed9ee31ce12dba9f6813cf1408  1.8s
 => => pulling sha256:7a803dc0b40fcd10faee3fb3ebb2d7aaa88500520e6295295f5  5.1s
 => => pulling sha256:b800e94e7303e276b8fb4911a40bfe28f46180d997022c69bf1  0.5s
 => => pulling sha256:a5e0c18be249394ae7365dce1364dfbd9c888dec4bf9489b3d9  1.1s
 => => pulling sha256:e0ba08bf689ebb4935e5ebf5e7ede61c151c55b73d2fe8f6ce7  0.3s
 => => pulling sha256:20f8d3e06d470f5659ac84232f4d5b309ad360a0b87a01f8042  0.1s
 => => pulling sha256:550a9716dde82f7fe3bcae909b438e926cb976b19e7eeb2950d  0.2s
 => => pulling sha256:505cb93ab7a4280c5e70dd612d29c65e3a84a32aac97d1bd8ca  0.3s
 => => pulling sha256:22c1b48e191688ff29942e1d76dd3f4ecf815b9ac88b28dc36b  0.1s
 => exporting to image                                                     0.1s
 => => exporting layers                                                    0.0s
 => => writing image sha256:a221243cab622137e78358197621df3e6ca10232bf6a1  0.0s
 => => naming to docker.io/library/node:latest                             0.0s

 

TensorFlow 컨테이너

- normal

$ docker build -t tf:normal .
[+] Building 296.3s (10/10) FINISHED                                           
 => [internal] load build definition from Dockerfile                      0.1s
 => => transferring dockerfile: 262B                                      0.1s
 => [internal] load .dockerignore                                         0.0s
 => => transferring context: 2B                                           0.0s
 => [internal] load metadata for docker.io/library/python:3.7             1.1s
 => [1/6] FROM docker.io/library/python:3.7@sha256:c8ab54d05f9795da69954  0.0s
 => CACHED [2/6] RUN mkdir /home/ubuntu                                   0.0s
 => CACHED [3/6] WORKDIR /home/ubuntu                                     0.0s
 => CACHED [4/6] RUN apt-get update && apt-get install -y libgl1-mesa-de  0.0s
 => [5/6] RUN pip install Tensorflow==2.4.1                             233.4s
 => [6/6] RUN pip install opencv-python==4.5.1.48                        31.3s 
 => exporting to image                                                   30.2s 
 => => exporting layers                                                  30.1s 
 => => writing image sha256:bb086e8949d43bc71b46ed65186c519bcd7b063299b8  0.0s 
 => => naming to docker.io/library/tf:normal                              0.0s 
ubuntu@ubuntu-VirtualBox:~/TensorFlow$ docker ps                               
CONTAINER ID   IMAGE     COMMAND   CREATED   STATUS    PORTS     NAMES         
ubuntu@ubuntu-VirtualBox:~/TensorFlow$ docker images
REPOSITORY   TAG       IMAGE ID       CREATED          SIZE
tf           normal    bb086e8949d4   52 seconds ago   2.69GB