공유 스토리지 구성을 위해 고정 공간이 할당된 가상 디스크 10개를 추가로 생성합니다.
| 파일 이름 | 용량 | 타입 | 용도 | 비고 |
|---|---|---|---|---|
CRS1.vdi | 10GB | Fixed / Shareable | CRS 저장 영역 | 공유(shareable) 디스크로 사용하기 위해서 Fixed Size로 생성합니다. |
| CRS2.vdi | 10GB | Fixed / Shareable | ||
| CRS3.vdi | 10GB | Fixed / Shareable | ||
DATA1.vdi | 10GB | Fixed / Shareable | Data 저장 영역 | |
| DATA2.vdi | 10GB | Fixed / Shareable | ||
| DATA3.vdi | 10GB | Fixed / Shareable | ||
| DATA4.vdi | 10GB | Fixed / Shareable | ||
| FRA1.vdi | 10GB | Fixed / Shareable | Fast Recovery Area | |
| FRA2.vdi | 10GB | Fixed / Shareable |
1. 디스크 생성 (호스트에서 수행)
A) 물리 Virtual Disk 생성
호스트 머신에서 실행
vboxmanage createmedium disk --filename CRS1.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename CRS2.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename CRS3.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename DATA1.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename DATA2.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename DATA3.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename DATA4.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename FRA1.vdi --size 10240 --format VDI --variant Fixed vboxmanage createmedium disk --filename FRA2.vdi --size 10240 --format VDI --variant Fixed
Host 컴퓨터에서 VM에 연결시킬 CRS, DATA와 Fast Recovery Area 디스크를 각각 10GB 사이즈로 생성합니다.
디스크는 2개 이상의 노드에서 공유해야 하므로 반드시 Fixed 모드로 초기에 디스크 공간을 할당하여 생성해야 합니다.
B) Virtual Disk 속성 변경
호스트 머신에서 실행
vboxmanage modifymedium disk CRS1.vdi --type shareable vboxmanage modifymedium disk CRS2.vdi --type shareable vboxmanage modifymedium disk CRS3.vdi --type shareable vboxmanage modifymedium disk DATA1.vdi --type shareable vboxmanage modifymedium disk DATA2.vdi --type shareable vboxmanage modifymedium disk DATA3.vdi --type shareable vboxmanage modifymedium disk DATA4.vdi --type shareable vboxmanage modifymedium disk FRA1.vdi --type shareable vboxmanage modifymedium disk FRA2.vdi --type shareable
생성된 디스크들은 모두 shareable 디스크로 속성 변경이 필요합니다.
VirtualBox의 GUI 툴을 사용하여 생성해도 됩니다.
2. 디스크 설정 (게스트에서 수행)
A) 디스크 포맷
root 유저로 수행
ls -ls /dev/sd* fdisk -c /dev/sdb ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdc ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdd ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sde ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdf ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdg ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdh ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdi ### "n", "p", "1", "엔터", "엔터", "w" 입력 fdisk -c /dev/sdj ### "n", "p", "1", "엔터", "엔터", "w" 입력 ls -ls /dev/sd*
[root@rac1 ~]# ls -ls /dev/sd*
0 brw-rw----. 1 root disk 8, 0 2017-04-15 21:34 /dev/sda
0 brw-rw----. 1 root disk 8, 1 2017-04-15 21:34 /dev/sda1
0 brw-rw----. 1 root disk 8, 2 2017-04-15 21:34 /dev/sda2
0 brw-rw----. 1 root disk 8, 16 2017-04-15 21:34 /dev/sdb
0 brw-rw----. 1 root disk 8, 32 2017-04-15 21:34 /dev/sdc
0 brw-rw----. 1 root disk 8, 48 2017-04-15 21:34 /dev/sdd
0 brw-rw----. 1 root disk 8, 64 2017-04-15 21:34 /dev/sde
0 brw-rw----. 1 root disk 8, 80 2017-04-15 21:34 /dev/sdf
0 brw-rw----. 1 root disk 8, 96 2017-04-15 21:34 /dev/sdg
0 brw-rw----. 1 root disk 8, 112 2017-04-15 21:34 /dev/sdh
0 brw-rw----. 1 root disk 8, 128 2017-04-15 21:34 /dev/sdi
0 brw-rw----. 1 root disk 8, 144 2017-04-15 21:34 /dev/sdj
[root@rac1 ~]# fdisk -c /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x658d8a2c.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x44883ea3.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xfcfe29e6.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x30dd34fe.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xef83f667.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdg
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x7a57b10b.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdh
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x47c3a6b5.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdi
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x89bcdc27.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdj
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xf33b9838.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)
WARNING: cylinders as display units are deprecated. Use command 'u' to
change units to sectors.
Command (m for help): n
Command action
e extended
p primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1):
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305):
Using default value 1305
Command (m for help): w
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# ls -ls /dev/sd*
0 brw-rw----. 1 root disk 8, 0 2017-04-15 21:34 /dev/sda
0 brw-rw----. 1 root disk 8, 1 2017-04-15 21:34 /dev/sda1
0 brw-rw----. 1 root disk 8, 2 2017-04-15 21:34 /dev/sda2
0 brw-rw----. 1 root disk 8, 16 2017-04-15 21:40 /dev/sdb
0 brw-rw----. 1 root disk 8, 17 2017-04-15 21:40 /dev/sdb1
0 brw-rw----. 1 root disk 8, 32 2017-04-15 21:40 /dev/sdc
0 brw-rw----. 1 root disk 8, 33 2017-04-15 21:40 /dev/sdc1
0 brw-rw----. 1 root disk 8, 48 2017-04-15 21:40 /dev/sdd
0 brw-rw----. 1 root disk 8, 49 2017-04-15 21:40 /dev/sdd1
0 brw-rw----. 1 root disk 8, 64 2017-04-15 21:41 /dev/sde
0 brw-rw----. 1 root disk 8, 65 2017-04-15 21:41 /dev/sde1
0 brw-rw----. 1 root disk 8, 80 2017-04-15 21:41 /dev/sdf
0 brw-rw----. 1 root disk 8, 81 2017-04-15 21:41 /dev/sdf1
0 brw-rw----. 1 root disk 8, 96 2017-04-15 21:41 /dev/sdg
0 brw-rw----. 1 root disk 8, 97 2017-04-15 21:41 /dev/sdg1
0 brw-rw----. 1 root disk 8, 112 2017-04-15 21:41 /dev/sdh
0 brw-rw----. 1 root disk 8, 113 2017-04-15 21:41 /dev/sdh1
0 brw-rw----. 1 root disk 8, 128 2017-04-15 21:41 /dev/sdi
0 brw-rw----. 1 root disk 8, 129 2017-04-15 21:41 /dev/sdi1
0 brw-rw----. 1 root disk 8, 144 2017-04-15 21:41 /dev/sdj
0 brw-rw----. 1 root disk 8, 145 2017-04-15 21:41 /dev/sdj1
VM에 디스크를 모두 연결시키고, 디스크를 모두 포맷합니다.
디스크 전체를 마운트 시킬 것 이므로 모두 단일 파티션으로 포맷합니다
B) ASM 및 GI 패키지 설치
root 유저로 수행
cd <GI 설치 파일 압축 해제 경로>/grid/rpm rpm -ivh cvuqdisk-* cd <ASM Library 파일 저장 경로> rpm -ivh oracleasmlib-*
[root@rac1 ~]# cd /media/sf_oracle/grid/rpm [root@rac1 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm 준비 중... ########################################### [100%] Using default group oinstall to install package 1:cvuqdisk ########################################### [100%] [root@rac1 rpm]# cd /media/sf_oracle/ [root@rac1 sf_oracle]# rpm -ivh oracleasmlib-2.0.12-1.el6.x86_64.rpm 경고: oracleasmlib-2.0.12-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY 준비 중... ########################################### [100%] 1:oracleasmlib ########################################### [100%]
Grid Infrastructure (이하 GI)를 설치하기 위해 패키지를 추가로 설치해야 합니다.
cvuqdisk 패키지는 GI 설치 파일에서 구할 있으며, ASM library 패키지는 아래 경로에서 다운 받을 수 있습니다
http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol6-1709075.html#oracleasm_ol6_amd64
C) ASM 환경 설정
root 유저로 수행
oracleasm configure oracleasm configure -i oracleasm configure
[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=false
ORACLEASM_UID=
ORACLEASM_GID=
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
[root@rac1 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"
ASM library에 대한 추가적인 환경 설정(사용자, 그룹, 부팅시 활성화, 부팅시 스캔 여부)을 해줍니다.
D) ASM 환경 적용
root 유저로 수행
oracleasm status oracleasm init oracleasm status #reboot
[root@rac1 ~]# oracleasm status Checking if ASM is loaded: no Checking if /dev/oracleasm is mounted: no [root@rac1 ~]# oracleasm init Creating /dev/oracleasm mount point: /dev/oracleasm Loading module "oracleasm": oracleasm Configuring "oracleasm" to use device physical block size Mounting ASMlib driver filesystem: /dev/oracleasm [root@rac1 ~]# oracleasm status Checking if ASM is loaded: yes Checking if /dev/oracleasm is mounted: yes
설정한 내용을 적용하기 위해 oracleasm 환경을 초기화 합니다.
E) ASM DIsk 생성
root 유저로 수행
oracleasm createdisk CRS1 /dev/sdb1 oracleasm createdisk CRS2 /dev/sdc1 oracleasm createdisk CRS3 /dev/sdd1 oracleasm createdisk DATA1 /dev/sde1 oracleasm createdisk DATA2 /dev/sdf1 oracleasm createdisk DATA3 /dev/sdg1 oracleasm createdisk DATA4 /dev/sdh1 oracleasm createdisk FRA1 /dev/sdi1 oracleasm createdisk FRA2 /dev/sdj1 oracleasm scandisks oracleasm listdisks
[root@rac1 ~]# oracleasm createdisk CRS1 /dev/sdb1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk CRS2 /dev/sdc1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk CRS3 /dev/sdd1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk DATA1 /dev/sde1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk DATA2 /dev/sdf1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk DATA3 /dev/sdg1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk DATA4 /dev/sdh1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk FRA1 /dev/sdi1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm createdisk FRA2 /dev/sdj1 Writing disk header: done Instantiating disk: done [root@rac1 ~]# oracleasm scandisks Reloading disk partitions: done Cleaning any stale ASM disks... Scanning system for ASM disks... [root@rac1 ~]# oracleasm listdisks CRS1 CRS2 CRS3 DATA1 DATA2 DATA3 DATA4 FRA1 FRA2
VM에 마운트된 디스크를 각각 ASM Disk로 할당해줍니다.
순서대로 마운트가 되었다면, 왼쪽의 명령으로 디스크를 생성하면 됩니다.
