Virtualization has been a need of time over several past years as we have machines now even with 16 cores and memory in TBs. A single machine is now capable of accommodating even more than 100 VMs at a time. Oracle VM for SPARC formerly known as LDOMs has played a key role in oracles virtualization strategies and is improving with every version. Before start configuring our first oracle VM for SPARC let us understand types of ldoms, ldom services and virtual devices.
Types of logical domains
|Guest||No direct access to underlying hardware and does not provide virtual device or services to other ldoms. Uses virtual device.|
|I/O||has direct access to underlying hardware in the server. It can be used in cases like oracle DB which wants direct/raw access to the storage devices.|
|Service||provides virtualized devices and services to guest domains.|
|Control||Service domain that also runs the ldoms manager software to control the configuration of hypervisor. This ldom manager is responsible for mapping between physical and virtual devices.|
Virtual Services and Devices
|VLDC||virtual logical domain channel||communication channel between logical domain and hypervisor|
|VCC||Virtual console concentrator||Acts as a virtual console for each logical domain|
|VSW||Virtual switch service||provides network access for guest ldoms to the physical network ports|
|VDS||virtual disk service||provides virtual storage service for guest ldoms|
|VCPU||virtual CPU||Each thread of a T series CPU acts as a virtual CPU|
|MAU||Mathematical arithmetic unit||Each core of T series CPU will have a MAU for accelerated RAS/DSA encryption|
|Memory||Physical memory is mapped into virtual memory and assigned to ldoms|
|VCONS||Virtual console||a port in guest ldom that connects to the VCC service in control domain|
|VNET||Virtual network||network port in guest ldom which is connected to the VSW service in the control domain|
|VSDEV||Virtual disk service device||physical storage device that is virtualized by VDS service in control domain|
|VDISK||Virtual disk||VDISK in guest domain is connected to the VDS service in control domain/service domain|
Installing the OVM software
To install the LDOM software simply unzip the software zip and run the install-ldm script with -s option in case you don’t want to use the configuration assistant to configure the primary and guest ldoms.
primary # unzip OVM_Server_SPARC_latest.zip primary # ./install-ldm -s
Creating the default services
Create the essential services like vsw, vcc and vds required to serve the guest LDOMs.
primary# ldm add-vcc port-range=5000-5100 primary-vcc0 primary primary# ldm add-vds primary-vds0 primary primary# ldm add-vsw net-dev=nxge0 primary-vsw0 primary primary# ldm list-services primary VDS NAME VOLUME OPTIONS DEVICE primary-vds0 VCC NAME PORT-RANGE primary-vcc0 5000-5100 VSW NAME MAC NET-DEV DEVICE MODE primary-vsw0 02:04:4f:fb:9f:0d nxge0 [email protected] prog,promisc
Initial configuration of the control domain
By default all the VCPUs, Memory and MAUs are assigned to the primary domain which is the default domain created after installing the OVM for SPARC software. Primary or control domain is used to configure all the guest ldoms and provide necessary virtual services to them like vcc, vsw and vds. Logical domain manager is responsible to create, delete, modify and control ldoms, thus make sure the ldmd service is running before configuring the primary and guest domains. Use delayed reconfiguration in order to configure the primary ldom without rebooting for previous changes to take effect.
primary# svcs -a | grep ldmd online 14:23:34 svc:/ldoms/ldmd:default primary# ldm set-mau 1 primary primary# ldm set-vcpu 8 primary primary# ldm start-reconf primary (delayed reconfiguration) primary# ldm set-memory 4G primary primary# ldm add-config new_config primary# ldm list-config factory-default new_config [current]
Reboot the primary domain for configuration settings to take effect
primary# shutdown -y -g0 -i6
Enable networking between primary and guest domains
By default communication between control domain and all the guest domains is disabled. To enable it, virtual switch has to be configured as the network device instead of nxge0.
primary# ifconfig nxge0 down unplumb primary# ifconfig vsw0 plumb primary# ifconfig vsw0 192.168.1.2 netmask + broadcast + up primary# mv /etc/hostname.nxge0 /etc/hostname.vsw0
Enable virtual network terminal server daemon
The vntsd daemon is responsible to provide the virtual network terminal services to the guest ldoms. If this service is not running enable it with svcadm command.
primary# svcadm enable vntsd primary# svcs vntsd STATE STIME FMRI online Oct_12 svc:/ldoms/vntsd:default
Setting up the Guest Domain
We would assign 8 VCPUs, 2 GB of memory and 1 MAU to our first guest ldom. Also a virtual network vnet1 will be created and associated with the virtual switch vsw0.
primary# ldm add-domain ldom01 primary# ldm add-vcpu 8 ldom01 primary# ldm add-memory 2G ldom01 primary-domain# ldm set-mau 1 ldom01 primary# ldm add-vnet vnet1 primary-vsw0 ldom01
Adding storage to the guest domain
Here we first need to specify the physical device that needs to be exported by vdsdev to the guest domain and then we actually add the virtual disk thus created to the guest domain. Now use any one of the 3 methods mentioned below.
1. Adding physical disks
primary# ldm add-vdsdev /dev/dsk/c2t1d0s2 [email protected] primary# ldm add-vdisk vdisk1 [email protected] ldom01
2. Adding file
primary# mkfile 10g /ldoms/ldom01_boot primary# ldm add-vdsdev /ldoms/ldom01_boot [email protected] primary# ldm add-vdisk vdisk1 [email protected] ldom01
3. Adding a volume
primary# zfs create -V 5gb pool/vol01 primary# ldm add-vdsdev /dev/zvol/dsk/pool/vol01 [email protected] primary# ldm add-vdisk vdisk1 [email protected] ldom01
Setup the boot environment variable for the guest ldom.
primary# ldm set-var auto-boot?=true ldom01 primary# ldm set-var boot-device=vdisk1 ldom01
Setting up the solaris ISO image for installing guest ldom
Now we can also do a jumpstart installation of the guest domain. But one of the easiest and most widely used method is add iso image as virtual disk to the guest ldom and install it from it. Here you can access the vdisk sol10_iso in the ok prompt and boot from it.
primary# ldm add-vdsdev options=ro /data/sol_10.iso [email protected] primary# ldm add-vdisk sol10_iso [email protected] ldom01
Bind and start installing the ldom
primary# ldm bind ldom01 primary# ldm start ldom01 LDom ldom01 started ok> devalias ok> boot sol10_iso
Connect the guest domain
Now check the port which is bound with the guest domain and connect the virtual console of the guest domain.
primary:~ # ldm list NAME STATE FLAGS CONS VCPU MEMORY UTIL UPTIME primary active -n-cv SP 8 4G 0.3% 8h 46m ldom01 active -n--- 5000 8 2G 48% 1h 52m
primary# telnet localhost 5000 Trying 127.0.0.1... Connected to localhost. Escape character is ’^]’. Connecting to console "ldom01" in group "ldom01" .... Press ~? for control options ..
Now you can see various flags in the “ldm list” command output. The falgs represent the current state of the ldom.
|column 1||column 2||column 3|
|s starting or stopping– placeholder||n normalt transition||d delayed reconfiguration– placeholder|
|column 4||column 5||column 6|
|c control domain– placeholder||v virtual I/O service domain– placeholder||s source domain in migrationt target domain in migratione error occurred in migration– placeholder|
Other useful Commands
View current version of Oracle VM server for SPARC software
primary# ldm -V
Long listing of domains
primary# ldm list -l
List the resource for all LDOMs and per LDOM
# ldm list -o cpu primary # ldm list -o network,memory ldom01
List the boot variables
List the bindings of all the LDOMs
# ldm list-bindings ldom
List all server resources, bound and unbound.
# ldm list-devices -a # ldm list-devices mem
Every file in a Linux/Unix operating system has an inode associated with it with an exception of Solaris ZFS, which does not have inodes. Inodes basically work very similar to an appendix of a book. Every Inode will have below information about the file.
4. time of last access
5. creation time
6. group id
7. Pointers to data blocks associated with the file content
Note: Inode does not provide filename however.
There are basically 5 types of files in any unix operating system.
3. Symbolic links (hard link & soft link)
4. Device files (character special and block special device)
5. Named pipes
The Character in the first column of ls -l command identifies the type of a file.
# cd / # ls -l bin lrwxrwxrwx 1 root root 9 Sep 19 15:41 bin -> ./usr/bin
- Regular files d Directories l Symbolic links b Block-special device files c Character-special device files p Named pipes
Files and Directories
The regular files can store different types of data and can be easily created using touch command or vi editor. Directories hold the association between files and/or directories and inode numbers.
As shown in the diagram soft links or symbolic links simply points to another file. It only contains the pathname of the file to which it is pointing
1. Creation method
# touch file1 # ln -s file1 link1 # ls -l -rw-r--r-- 1 root root 0 Sep 19 14:41 link1 lrwxrwxrwx 1 root root 5 Sep 19 15:41 link1 -> file1
2. The size of the soft link created in the example above is the no of characters in the pathname (file1), which is 5 (it can be absolute or relative).
3. If you delete the original file (file1) the soft link render as useless.
4. Soft links can reside on different file systems.
Every file uses atleast one hard link. So when you create a new file a new directory entry is created which is called link count. So when you create a new hard link to this file the link count increaments by 1.
1. creation method
# touch file1 # ls -l -rw-r--r-- 1 root root 0 Sep 23 13:19 file1 # ln file1 file2 # ls -l -rw-r--r-- 2 root root 0 Sep 23 13:19 file1 -rw-r--r-- 2 root root 0 Sep 23 13:19 file2 # ls -li 1282 -rw-r--r-- 2 root 0 root 0 Sep 23 13:19 file1 1282 -rw-r--r-- 2 root 0 root 0 Sep 23 13:19 file2 # find . -inum 1282 ./file1 ./file2
2. The link count increases by one, everytime you create a new hard link to the file as shown above.
3. Even if you delete any one of the file, it has no effect on the other file. Only the link count decrements
4. Hard links can not cross the file system.
In UNIX operating system any physical device has a file associated with it called as device file. It’s an interface that interacts with device drivers. Unlike other file types they do not hold any data in data blocks instead they use the inodes to store the major and minor no for any device file.
# cd /dev/ # ls -l crw-r----- 1 root tty 4, 0 Sep 23 12:51 tty0 brw-rw----- 1 root disk 8, 1 Sep 23 12:51 sda0
Major device no – Specific device driver required to access a device
minor device no – specific unit of the type that the device driver controls.
For example if you have 10 HP printers, the major no will be the HP printer devices driver and minor no would be the instance of printer (1,2 .. upto 10).
Device files are of 2 types
1. Character special
2. Block special
Character special Device files
1. Character “c” in the filrst column of ls -l command output identifies a character special device file
2. Data is accessed as data stream (character by character , 1 byte at a time)
3. Example : tty, serial, virtual terminals
# ls -l crw-r----- 1 root tty 4, 0 Sep 23 12:51 tty0
Block special Device file
1. Character “b” in the first column of ls -l command output identifies a character special device file
2. Data is accessed as defined block size for that device
3. Example : Hard Disk, CD/DVD
# ls -l brw-rw----- 1 root disk 8, 1 Sep 23 12:51 sda0
– Named pipes are special files which are used for interprocessor communications. Unlike normal pipes you can read from and write to the named pipes. For this reason they are also called as FIFO (file in file out).
– mknod() or mkfifio() are common examples which make use of named pipes in order to access the pipe by name.
– As shown in the example below 2 processes (gzip and cat) can simultaneously access the Named pipe to write to and read data from it.
# mkfifo test_pipe # gzip -9 -c > test_pipe < out.gz # cat file1 > test_pipe # ls -l test_pipe prw-rw----- 1 root root 0 Sep 23 12:51 test_pipe