csapi 프로젝트 (1) - 코드 동작하게 만들기

https://not-to-be-reset.tistory.com/343 글에 작성한 대로, 모델 다운로드 링크가 만료되고 코드가 동작하지 않아서 다시 포스팅 합니다.
단일 인스턴스 부터 쿠버네티스 클러스터까지 확장해 나가려고 합니다.

개발환경

2023. 3. 22 기준 m1 맥북에서 진행

github 주소

https://github.com/kyeongjun-dev/csapi

 

(구) 코드 동작하게 만들기

먼저 코드가 동작하게 만들어 봅시다. 아래와 같이 필요한 파일들을 준비 및 작성합니다.

tree .
.
├── Dockerfile
├── model.h5
├── requirements.txt
├── run.py
└── test_input.png

 

model.h5 파일은 아래 명령어로 다운로드 가능합니다 (언제 막힐지모름...)

 wget --load-cookies ~/cookies.txt "https://docs.google.com/uc?export=download&confirm=$(wget --quiet --save-cookies ~/cookies.txt --keep-session-cookies --no-check-certificate 'https://docs.google.com/uc?export=download&id=1cJiPA1XoCn-htXFEq7UYHb_Row3J3agf' -O- | sed -rn 's/.*confirm=([0-9A-Za-z_]+).*/\1\n/p')&id=1cJiPA1XoCn-htXFEq7UYHb_Row3J3agf" -O ./model.h5

 

Dockerfile

FROM python:3.7.16-slim
ENV PYTHONUNBUFFERED 1
WORKDIR /csapi
RUN apt-get update && apt-get install -y libgl1-mesa-dev libglib2.0-0
COPY requirements.txt requirements.txt
RUN pip install -r requirements.txt --no-cache-dir
COPY . .

 

run.py

import cv2
import numpy as np
from tensorflow.keras.models import load_model
import tensorflow as tf
import sys
from tensorflow.compat.v1 import ConfigProto
from tensorflow.compat.v1 import InteractiveSession
config = ConfigProto()
config.gpu_options.allow_growth = True
session = InteractiveSession(config=config)


f = sys.argv[1]

saved = load_model("./model.h5")

class fashion_tools(object):
    def __init__(self,imageid,model,version=1.1):
        self.imageid = imageid
        self.model   = model
        self.version = version

    def get_dress(self,stack=False):
        """limited to top wear and full body dresses (wild and studio working)"""
        """takes input rgb----> return PNG"""
        name =  self.imageid
        file = cv2.imread(name)
        file = tf.image.resize_with_pad(file,target_height=512,target_width=512)
        rgb  = file.numpy()
        file = np.expand_dims(file,axis=0)/ 255.
        seq = self.model.predict(file)
        seq = seq[3][0,:,:,0]
        seq = np.expand_dims(seq,axis=-1)
        c1x = rgb*seq
        c2x = rgb*(1-seq)
        cfx = c1x+c2x
        dummy = np.ones((rgb.shape[0],rgb.shape[1],1))
        rgbx = np.concatenate((rgb,dummy*255),axis=-1)
        rgbs = np.concatenate((cfx,seq*255.),axis=-1)
        if stack:
            stacked = np.hstack((rgbx,rgbs))
            return stacked
        else:
            return rgbs


    def get_patch(self):
        return None

# running code
api = fashion_tools(f, saved)
image_ = api.get_dress(False)
cv2.imwrite("out.png", image_)

 

requirements.txt

opencv-python==4.5.1.48
tensorflow==2.11.0
tensorflow-io==0.24.0

 

도커 이미지를 빌드하고

docker build -t tf:test .

 

컨테이너 생성 및 접속 및 test_input.png 이미지로 run.py 실행

docker run -it --rm tf:test bash
root@cfc3b110b894:/csapi# ls
Dockerfile  model.h5  requirements.txt  run.py  test_input.png
root@cfc3b110b894:/csapi# python run.py test_input.png 
2023-07-22 05:06:22.911250: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-07-22 05:06:23.045160: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2023-07-22 05:06:23.045212: I tensorflow/compiler/xla/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2023-07-22 05:06:24.100135: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2023-07-22 05:06:24.100212: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2023-07-22 05:06:24.100223: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2023-07-22 05:06:25.200047: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2023-07-22 05:06:25.201218: W tensorflow/compiler/xla/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcuda.so.1'; dlerror: libcuda.so.1: cannot open shared object file: No such file or directory
2023-07-22 05:06:25.201268: W tensorflow/compiler/xla/stream_executor/cuda/cuda_driver.cc:265] failed call to cuInit: UNKNOWN ERROR (303)
2023-07-22 05:06:25.201301: I tensorflow/compiler/xla/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (cfc3b110b894): /proc/driver/nvidia/version does not exist
1/1 [==============================] - 6s 6s/step

 

뭔가 에러가 나긴하지만 이미지 처리(segmentation)는 동작한다.

# 다른 터미널창을 열어서 현재 실행중인 tf 컨테이너에서 결과물인 out.png 복사
docker ps
CONTAINER ID   IMAGE     COMMAND   CREATED              STATUS              PORTS     NAMES
cfc3b110b894   tf:test   "bash"    About a minute ago   Up About a minute             quizzical_merkle

docker cp cfc3b110b894:/csapi/out.png ./out.png
Successfully copied 263kB to /home/ubuntu/tmp/out.png

 

pip install 팁

pip install 할때, --no-cache-dir 옵션을 추가했을 때와 안했을 때 차이가 꽤 크다.

docker images | grep tf
tf   cache    84be0a8d615d   29 seconds ago   3.27GB
tf   test     340864f44e01   2 minutes ago    2.86GB