업그레이드 MCCS Enterprise (MDRM from 4.4.2 to 4.5.2) 고려사항

 1. Environment   

  • MDRM 4.5.2
  • CENTOS 7
  • Docker version 1.12.6
  • Docker Root Dir: /var/lib/docker
  • Base Device Size: 10.11 GB

 2. Symptoms

  • gam 4.4.2 -> 4.5.2 로 업그레이드가 실패한다.
  • | 14:55:52.624 | mv: error writing ‘/var/lib/postgresql/9.3/main/pg_xlog/000000010000000A00000025’: No space left on device
    | 14:55:52.626 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/pg_xlog/000000010000000A00000025’: No space left on device
    | 14:55:52.637 | mv: error writing ‘/var/lib/postgresql/9.3/main/PG_VERSION’: No space left on device
    | 14:55:52.639 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/PG_VERSION’: No space left on device
    | 14:55:52.657 | mv: error writing ‘/var/lib/postgresql/9.3/main/pg_ctl.conf’: No space left on device
    | 14:55:52.658 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/pg_ctl.conf’: No space left on device
    | 14:55:52.658 | mv: error writing ‘/var/lib/postgresql/9.3/main/pg_hba.conf’: No space left on device
    | 14:55:52.661 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/pg_hba.conf’: No space left on device
    | 14:55:52.662 | mv: error writing ‘/var/lib/postgresql/9.3/main/pg_ident.conf’: No space left on device
    | 14:55:52.663 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/pg_ident.conf’: No space left on device
    | 14:55:52.664 | mv: error writing ‘/var/lib/postgresql/9.3/main/postgresql.conf’: No space left on device
    | 14:55:52.666 | mv: failed to extend ‘/var/lib/postgresql/9.3/main/postgresql.conf’: No space left on device
    | 14:55:52.667 | mv: cannot create regular file ‘/var/lib/postgresql/9.3/main/postmaster.opts’: No space left on device
    | 14:55:52.669 | mv: cannot create regular file ‘/var/lib/postgresql/9.3/main/start.conf’: No space left on device
    | 14:55:52.670 | mv: cannot create regular file ‘/var/lib/postgresql/9.3/main/postmaster.pid’: No space left on device

 3. Cause

  • 컨테이너 내부의 용량
    root@mccs-ent1:postgresql > df -hT
    Filesystem Type Size Used Avail Use% Mounted on
    /dev/mapper/docker-253:0-67401708-9fd6f94caa73c804bd21460751f3a13ba2673ce90b10a3aeb95f63f0e0d989bc xfs 10G 5.8G 4.3G 58% /
    tmpfs tmpfs 16G 0 16G 0% /dev
    tmpfs tmpfs 16G 0 16G 0% /sys/fs/cgroup
    /dev/sdc1 xfs 30G 11G 20G 35% /gampkgs/data
    /dev/mapper/centos-root xfs 28G 21G 7.1G 75% /etc/hosts
    shm tmpfs 64M 12K 64M 1% /dev/shm
  • 실제 postgresql DB용량
    [root@cj-mag container1]# du -sh db/
    4.3G db/

* 실제 DB 용량이 커서 마이그레이션 작업 도중 컨테니어 내부의 “/” 디렉토리의 사용량이 100%가 되어 DB 생성이 제대로 되지 않음

 4. Solution

컨테이너의 Base Device Size를 DB 용량이 맞게 증설

수정 할 파일 : /etc/docker/daemon.json
{
“graph”: “/var/lib/docker”,
“storage-driver”: “devicemapper”,
“storage-opts”: [
“dm.basesize=15G”
]
}

이 후 
1. docker 데몬 중지 : systemctl stop docker
2. Docker Root Dir 디렉토리 삭제 : rm -rf /var/lib/docker
3. docker 데몬 시작 : systemctl start docker
GAM 이미지 로드

 5. Workaround

참조 URL : https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-storage-driver