nvidia-dockerはdockerコンテナ内でnvidia gpuを使うためのOSSのこと
これを使って、dockerコンテナ内でgpu資源を使えるようにする
nvidia, docker, docker-composeはインストール済のところから始める
環境
- ubuntu: 17.04
- docker: 17.12.0-ce
- docker-compose: 1.19.0
- NVIDIA Docker: 2.0.2
手順
おおむね以下の手順通りに行った https://github.com/NVIDIA/nvidia-docker/wiki/Installation-(version-2.0)
事前準備
NVIDIA/nvidia-dockerより、各種バージョンの確認。gpuとdockerについて新しいものを用意しておけば事前準備は問題ないと思われる
-
GNU/Linux x86_64 with kernel version > 3.10
uname -a
Linux 4.10.0-42-generic #46-Ubuntu SMP Mon Dec 4 14:38:01 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
バージョン4.10.0でx86_64なので大丈夫
2. Docker >= 1.12
```bash
docker version
Client:
Version: 17.12.0-ce
API version: 1.35
...バージョンは17.12.0-ceなので大丈夫
-
NVIDIA GPU with Architecture > Fermi (2.1)
lspci | grep NVIDIA 01:00.0 3D controller: NVIDIA Corporation GP107M [GeForce GTX 1050 Mobile] (rev a1)
FermiというのはNVIDIA GPUの製品の1種らしい。指定されているのは結構古いタイプのようなので、新しいGPUを使っていれば問題ないと思われる
4. NVIDIA drivers ~= 361.93 (untested on older versions)
```bash
nvidia-smi
Thu Feb 22 01:37:08 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.111 Driver Version: 384.111 |
...バージョンは384.111なので大丈夫
インストール
Repository configurationより、GPGキーの登録、リポジトリの追加、nvidia-docker2のインストール、docker daemonの再起動を行う
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
sudo apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/ubuntu16.04/amd64/nvidia-docker.list | \
sudo tee /etc/apt/sources.list.d/nvidia-docker.list
sudo apt-get update
sudo apt-get install nvidia-docker2
# reload docker daemon process
sudo pkill -SIGHUP dockerd動作確認
dockerコマンドにて確認
docker run --runtime=nvidia --rm nvidia/cuda nvidia-smi
...
Status: Downloaded newer image for nvidia/cuda:latest
Wed Feb 21 17:25:45 2018
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 384.111 Driver Version: 384.111 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 1050 Off | 00000000:01:00.0 Off | N/A |
| N/A 59C P0 N/A / N/A | 2159MiB / 4041MiB | 32% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
+-----------------------------------------------------------------------------+docker-composeで使う
https://github.com/docker/compose/issues/5360 によると1.19.0でdocker-compose対応するらしい
releasesを見ると https://github.com/docker/compose/releases
Compose file version 2.3
Added support for the runtime option in service > definitions
version 2.3にruntimeオプションが追加されている様子
動作確認
version: '2.3'
services:
nvidia:
image: nvidia/cuda
runtime: nvidia
command: nvidia-smiとして試してみると
docker-compose up
ERROR: The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.nvidia: 'runtime'runtimeオプションがないといわれた
docker-compose version
docker-compose version 1.15.0, build e12f3b9docker-composeのバージョン古かった。更新して
sudo curl -L https://github.com/docker/compose/releases/download/1.19.0/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose
sudo rm /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose
docker-compose version
docker-compose version 1.19.0, build 9e633ef試すと、実行できた
docker-compose up
Recreating nvidia_1 ... done
Attaching to nvidia_1
nvidia_1 | Wed Feb 21 18:05:29 2018
nvidia_1 | +-----------------------------------------------------------------------------+
nvidia_1 | | NVIDIA-SMI 384.111 Driver Version: 384.111 |
nvidia_1 | |-------------------------------+----------------------+----------------------+
nvidia_1 | | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
nvidia_1 | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
nvidia_1 | |===============================+======================+======================|
nvidia_1 | | 0 GeForce GTX 1050 Off | 00000000:01:00.0 Off | N/A |
nvidia_1 | | N/A 55C P0 N/A / N/A | 2133MiB / 4041MiB | 30% Default |
nvidia_1 | +-------------------------------+----------------------+----------------------+
nvidia_1 |
nvidia_1 | +-----------------------------------------------------------------------------+
nvidia_1 | | Processes: GPU Memory |
nvidia_1 | | GPU PID Type Process name Usage |
nvidia_1 | |=============================================================================|
nvidia_1 | +-----------------------------------------------------------------------------+
nvidia_1 exited with code 0kerasを実行してみる
gpuコンテナでkerasを使って、gpuを認識するか試す python3を実行できるコンテナを用意する dockerイメージはnvidiaのものをベースにする
Pillow
numpy
tensorflow-gpu==1.4
kerasFROM nvidia/cuda:8.0-cudnn6-devel
RUN apt-get update -y && \
apt-get install -y --no-install-recommends \
python3-dev \
python3-pip \
python3-setuptools \
python3-wheel
RUN mkdir -p /usr/src/app
WORKDIR /usr/src/app
COPY requirements.txt ./
RUN pip3 install --no-cache-dir -r requirements.txt
COPY . /usr/src/appversion: '2.3'
services:
app:
build: .
volumes:
- .:/usr/src/app
runtime: nvidiaコンテナをビルドして、コンテナ内に入る
docker-compose build
...
docker-compose run --rm app /bin/bash
# コンテナ内でpythonを実行
# python3 train.py
Using TensorFlow backend.
...
2018-02-22 20:14:13.309056: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1030] Found device 0 with properties:
name: GeForce GTX 1050 major: 6 minor: 1 memoryClockRate(GHz): 1.493
pciBusID: 0000:01:00.0
totalMemory: 3.95GiB freeMemory: 2.03GiB
2018-02-22 20:14:13.309092: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1120] Creating TensorFlow device (/device:GPU:0) -> (device: 0, name: GeForce GTX 1050, pci bus id: 0000:01:00.0, compute capability: 6.1)
...gpuで実行された