공유 스토리지 구성을 위해 고정 공간이 할당된 가상 디스크 10개를 추가로 생성합니다.

파일 이름용량타입용도비고

CRS1.vdi

10GBFixed / ShareableCRS 저장 영역공유(shareable) 디스크로 사용하기 위해서 Fixed Size로 생성합니다.
CRS2.vdi10GBFixed / Shareable
CRS3.vdi10GBFixed / Shareable

DATA1.vdi

10GBFixed / ShareableData 저장 영역
DATA2.vdi10GBFixed / Shareable
DATA3.vdi10GBFixed / Shareable
DATA4.vdi10GBFixed / Shareable
FRA1.vdi10GBFixed / ShareableFast Recovery Area
FRA2.vdi10GBFixed / Shareable

디스크 생성 (호스트에서 수행)

물리 Virtual Disk 생성

vboxmanage createmedium disk --filename CRS1.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename CRS2.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename CRS3.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename DATA1.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename DATA2.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename DATA3.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename DATA4.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename FRA1.vdi --size 10240 --format VDI --variant Fixed
vboxmanage createmedium disk --filename FRA2.vdi --size 10240 --format VDI --variant Fixed

Host 컴퓨터에서 VM에 연결시킬 CRS, DATA와 Fast Recovery Area 디스크를 각각 10GB 사이즈로 생성합니다.

디스크는 2개 이상의 노드에서 공유해야 하므로 반드시 Fixed 모드로 초기에 디스크 공간을 할당하여 생성해야 합니다.

Virtual Disk 속성 변경

vboxmanage modifymedium disk CRS1.vdi --type shareable
vboxmanage modifymedium disk CRS2.vdi --type shareable
vboxmanage modifymedium disk CRS3.vdi --type shareable
vboxmanage modifymedium disk DATA1.vdi --type shareable
vboxmanage modifymedium disk DATA2.vdi --type shareable
vboxmanage modifymedium disk DATA3.vdi --type shareable
vboxmanage modifymedium disk DATA4.vdi --type shareable
vboxmanage modifymedium disk FRA1.vdi --type shareable
vboxmanage modifymedium disk FRA2.vdi --type shareable

생성된 디스크들은 모두 shareable 디스크로 속성 변경이 필요합니다.

VirtualBox의 GUI 툴을 사용하여 생성해도 됩니다.

 

 

 

 

 

디스크 설정 (게스트에서 수행)

디스크 포맷

ls -ls /dev/sd*
fdisk -c /dev/sdb  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdc  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdd  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sde  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdf  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdg  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdh  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdi  ### "n", "p", "1", "엔터", "엔터", "w" 입력
fdisk -c /dev/sdj  ### "n", "p", "1", "엔터", "엔터", "w" 입력
ls -ls /dev/sd*
[root@rac1 ~]# ls -ls /dev/sd*
0 brw-rw----. 1 root disk 8,   0 2017-04-15 21:34 /dev/sda
0 brw-rw----. 1 root disk 8,   1 2017-04-15 21:34 /dev/sda1
0 brw-rw----. 1 root disk 8,   2 2017-04-15 21:34 /dev/sda2
0 brw-rw----. 1 root disk 8,  16 2017-04-15 21:34 /dev/sdb
0 brw-rw----. 1 root disk 8,  32 2017-04-15 21:34 /dev/sdc
0 brw-rw----. 1 root disk 8,  48 2017-04-15 21:34 /dev/sdd
0 brw-rw----. 1 root disk 8,  64 2017-04-15 21:34 /dev/sde
0 brw-rw----. 1 root disk 8,  80 2017-04-15 21:34 /dev/sdf
0 brw-rw----. 1 root disk 8,  96 2017-04-15 21:34 /dev/sdg
0 brw-rw----. 1 root disk 8, 112 2017-04-15 21:34 /dev/sdh
0 brw-rw----. 1 root disk 8, 128 2017-04-15 21:34 /dev/sdi
0 brw-rw----. 1 root disk 8, 144 2017-04-15 21:34 /dev/sdj

[root@rac1 ~]# fdisk -c /dev/sdb
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x658d8a2c.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdc
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x44883ea3.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdd
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xfcfe29e6.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sde
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x30dd34fe.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdf
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xef83f667.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdg
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x7a57b10b.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdh
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x47c3a6b5.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# fdisk -c /dev/sdi
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0x89bcdc27.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@rac1 ~]# fdisk -c /dev/sdj
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xf33b9838.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

WARNING: cylinders as display units are deprecated. Use command 'u' to
         change units to sectors.

Command (m for help): n
Command action
   e   extended
   p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-1305, default 1): 
Using default value 1
Last cylinder, +cylinders or +size{K,M,G} (1-1305, default 1305): 
Using default value 1305

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.

[root@rac1 ~]# ls -ls /dev/sd*
0 brw-rw----. 1 root disk 8,   0 2017-04-15 21:34 /dev/sda
0 brw-rw----. 1 root disk 8,   1 2017-04-15 21:34 /dev/sda1
0 brw-rw----. 1 root disk 8,   2 2017-04-15 21:34 /dev/sda2
0 brw-rw----. 1 root disk 8,  16 2017-04-15 21:40 /dev/sdb
0 brw-rw----. 1 root disk 8,  17 2017-04-15 21:40 /dev/sdb1
0 brw-rw----. 1 root disk 8,  32 2017-04-15 21:40 /dev/sdc
0 brw-rw----. 1 root disk 8,  33 2017-04-15 21:40 /dev/sdc1
0 brw-rw----. 1 root disk 8,  48 2017-04-15 21:40 /dev/sdd
0 brw-rw----. 1 root disk 8,  49 2017-04-15 21:40 /dev/sdd1
0 brw-rw----. 1 root disk 8,  64 2017-04-15 21:41 /dev/sde
0 brw-rw----. 1 root disk 8,  65 2017-04-15 21:41 /dev/sde1
0 brw-rw----. 1 root disk 8,  80 2017-04-15 21:41 /dev/sdf
0 brw-rw----. 1 root disk 8,  81 2017-04-15 21:41 /dev/sdf1
0 brw-rw----. 1 root disk 8,  96 2017-04-15 21:41 /dev/sdg
0 brw-rw----. 1 root disk 8,  97 2017-04-15 21:41 /dev/sdg1
0 brw-rw----. 1 root disk 8, 112 2017-04-15 21:41 /dev/sdh
0 brw-rw----. 1 root disk 8, 113 2017-04-15 21:41 /dev/sdh1
0 brw-rw----. 1 root disk 8, 128 2017-04-15 21:41 /dev/sdi
0 brw-rw----. 1 root disk 8, 129 2017-04-15 21:41 /dev/sdi1
0 brw-rw----. 1 root disk 8, 144 2017-04-15 21:41 /dev/sdj
0 brw-rw----. 1 root disk 8, 145 2017-04-15 21:41 /dev/sdj1

VM에 디스크를 모두 연결시키고, 디스크를 모두 포맷합니다.

디스크 전체를 마운트 시킬 것 이므로 모두 단일 파티션으로 포맷합니다

ASM 및 GI 패키지 설치

cd <GI 설치 파일 압축 해제 경로>/grid/rpm
rpm -ivh cvuqdisk-*
cd <ASM Library 파일 저장 경로>
rpm -ivh oracleasmlib-*
[root@rac1 ~]# cd /media/sf_oracle/grid/rpm

[root@rac1 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm 
준비 중...               ########################################### [100%]
Using default group oinstall to install package
   1:cvuqdisk               ########################################### [100%]

[root@rac1 rpm]# cd /media/sf_oracle/


[root@rac1 sf_oracle]# rpm -ivh oracleasmlib-2.0.12-1.el6.x86_64.rpm 
경고: oracleasmlib-2.0.12-1.el6.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID ec551f03: NOKEY
준비 중...               ########################################### [100%]
   1:oracleasmlib           ########################################### [100%]

Grid Infrastructure (이하 GI)를 설치하기 위해 패키지를 추가로 설치해야 합니다.

cvuqdisk 패키지는 GI 설치 파일에서 구할 있으며, ASM library 패키지는 아래 경로에서 다운 받을 수 있습니다

http://www.oracle.com/technetwork/server-storage/linux/asmlib/ol6-1709075.html#oracleasm_ol6_amd64

ASM 환경 설정

oracleasm configure
oracleasm configure -i
oracleasm configure
[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=false
ORACLEASM_UID=
ORACLEASM_GID=
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

[root@rac1 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver.  The following questions will determine whether the driver is
loaded on boot and what permissions it will have.  The current values
will be shown in brackets ('[]').  Hitting <ENTER> without typing an
answer will keep that current value.  Ctrl-C will abort.

Default user to own the driver interface []: grid
Default group to own the driver interface []: asmadmin
Start Oracle ASM library driver on boot (y/n) [n]: y
Scan for Oracle ASM disks on boot (y/n) [y]: y
Writing Oracle ASM library driver configuration: done
 
[root@rac1 ~]# oracleasm configure
ORACLEASM_ENABLED=true
ORACLEASM_UID=grid
ORACLEASM_GID=asmadmin
ORACLEASM_SCANBOOT=true
ORACLEASM_SCANORDER=""
ORACLEASM_SCANEXCLUDE=""
ORACLEASM_USE_LOGICAL_BLOCK_SIZE="false"

ASM library에 대한 추가적인 환경 설정(사용자, 그룹, 부팅시 활성화, 부팅시 스캔 여부)을 해줍니다.

ASM 환경 적용

oracleasm status
oracleasm init
oracleasm status
#reboot
[root@rac1 ~]# oracleasm status
Checking if ASM is loaded: no
Checking if /dev/oracleasm is mounted: no


[root@rac1 ~]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

[root@rac1 ~]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes

설정한 내용을 적용하기 위해 oracleasm 환경을 초기화 합니다.

ASM DIsk 생성

oracleasm createdisk CRS1 /dev/sdb1
oracleasm createdisk CRS2 /dev/sdc1
oracleasm createdisk CRS3 /dev/sdd1
oracleasm createdisk DATA1 /dev/sde1
oracleasm createdisk DATA2 /dev/sdf1
oracleasm createdisk DATA3 /dev/sdg1
oracleasm createdisk DATA4 /dev/sdh1
oracleasm createdisk FRA1 /dev/sdi1
oracleasm createdisk FRA2 /dev/sdj1
oracleasm scandisks
oracleasm listdisks
[root@rac1 ~]# oracleasm createdisk CRS1 /dev/sdb1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk CRS2 /dev/sdc1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk CRS3 /dev/sdd1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk DATA1 /dev/sde1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk DATA2 /dev/sdf1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk DATA3 /dev/sdg1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk DATA4 /dev/sdh1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk FRA1 /dev/sdi1
Writing disk header: done
Instantiating disk: done

[root@rac1 ~]# oracleasm createdisk FRA2 /dev/sdj1
Writing disk header: done
Instantiating disk: done
 
[root@rac1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@rac1 ~]# oracleasm listdisks
CRS1
CRS2
CRS3
DATA1
DATA2
DATA3
DATA4
FRA1
FRA2

VM에 마운트된 디스크를 각각 ASM Disk로 할당해줍니다.

순서대로 마운트가 되었다면, 왼쪽의 명령으로 디스크를 생성하면 됩니다.