This chapter provides conceptual information about disk sets. For information about performing related tasks, see Chapter 19, Disk Sets (Tasks).This chapter includes the following information:What's New in Disk Sets Introduction to Disk Sets Solaris Volume Manager Disk Set Administration Guidelines for Working With Disk Sets Asynchronous Shared Storage in Disk Sets Scenario—Disk Sets This section describes new disk set features in this Solaris release.For a complete listing of new Solaris features and a description of Solaris releases, see What’s New in Solaris Express.Solaris Express 6/06: The metaimport command has been enhanced to allow you to import partial disk sets. A partial disk set is a disk set that has at least one disk that is not available. With this enhancement, a partial disk set can still be imported provided one disk with a metadb is present in the set.This new functionality is associated with enhancements in the metaimport r command. The command's output now includes the following information:Time stampThe time the disk set is created is displayed. Unavailable disksIf a disk has become unresponsive or might have been removed from the system, the disk is labeled unavailable. Disk conflictsIf a disk is listed as belonging to more than one disk set, the disk is marked as in conflict. Import recommendationIf disk conflicts exist, metaimport recommends which disk set to import with the disk in conflict. The information generated by metaimport r helps reduce the risk of data corruption when importing disk sets. In cases of disk conflicts, metaimport imports only the most recent disk set that is created with the disk in conflict. If a user imports a disk set with time conflicts contrary to the recommendation, the attempt is blocked.The metaimport rv command now contains more information in the output. This information allows the user to see the basic structure of the volumes that are available for import.Finally, you can import the recommended partial disk set by using the following syntax:metaimport -s setname -f disknameFor examples of the output of the metaimport r and metaimport rv commands, see How to Print a Report on Disk Sets Available for Import. A disk set is a set of physical storage volumes that contain logical volumes and hot spares. Volumes and hot spare pools must be built on drives from within that disk set. Once you have created a volume within the disk set, you can use the volume just as you would use a physical slice. You can use the volume to create and to mount a file system and to store data.Disk sets are supported on both SPARC and x86 based platforms. This section discusses the different types of disk sets available in Solaris Volume Manager.Each host has a local disk set. The local disk set consists of all of the disks on a host that are not in a named disk set. A local disk set belongs exclusively to a specific host. The local disk set contains the state database for that specific host's configuration. Volumes and hot spare pools in the local disk set consist only of drives from within the local disk set. In addition to local disk sets, hosts can participate in named disk sets. A named disk set is any disk set that is not in the local disk set. You can implement the following types of named disk sets to manage volumes, depending on the configuration of your system.A shared disk set can be shared by multiple hosts. Although a shared disk set is visible from all the participating hosts, only the owner of the disk set can access it. Each host can control a shared disk set, but only one host can control it at a time. Additionally, shared disk sets provide a distinct namespace within which the volume is managed.A shared disk set supports data redundancy and data availability. If one host fails, another host can take over the failed host's disk set (this type of configuration is known as a failover configuration).Shared disk sets are intended, in part, for use with Sun Cluster, Solstice HA (High Availability), or another supported third-party HA framework. Solaris Volume Manager by itself does not provide all the functionality necessary to implement a failover configuration. Although each host can control the set of disks, only one host can control it at a time. Before the autotake feature became available in the Solaris 9 4/04 release, Solaris Volume Manager did not support the automatic mounting of file systems on disk sets through the /etc/vfstab file. Solaris Volume Manager required the system administrator to manually issue a disk set take command by using the metaset s setname t command before the file systems on the disk set could be accessed.With the autotake feature, you can set a disk set to be automatically accessed at boot time by using the metaset s setname A enable command. The autotake feature makes it possible for you to define at boot the mount options for a file system in the /etc/vfstab file. This feature allows you to define the mount options in the /etc/vfstab file for file systems on volumes in the enabled disk set.Only single-host disk sets support the autotake feature. The autotake feature requires that the disk set is not shared with any other systems. A disk set that is shared cannot be set to use the autotake feature, and the metaset A command will fail. However, after other hosts are removed from the disk set, it may then be set to autotake. Similarly, an autotake disk set cannot have other hosts added to it. If the autotake feature is disabled, additional hosts can then be added to the disk set. In a Sun Cluster environment, the autotake feature is disabled. Sun Cluster handles the take and release of a disk set. For more information on the autotake feature see the A option of the metaset1M command. Named disk sets created in a Sun Cluster environment are called multi-owner disk sets. Multi-owner disk sets allow multiple nodes to share the ownership of the disk sets and to simultaneously access the shared disks. All disks and volumes in a multi-owner disk set can be directly accessed by all the nodes in a cluster. Each multi-owner disk set contains a list of hosts that have been added to the disk set. Consequently, each multi-owner disk set within a cluster configuration can have a different (and sometimes overlapping) set of hosts.Each multi-owner disk set has a master node. The function of the master node is to manage and update the state database replica changes. Since there is a master node per disk set, multiple masters can exist simultaneously. There are two ways that the master is chosen. The first way is that a node becomes the master if it is the first node to add a disk to the disk set. The second way is when a master node panics and fails. The node with the lowest node id becomes the master node.Multi-owner disk set functionality is enabled only in a Sun Cluster environment to manage multi-owner disk set storage. The Solaris Volume Manager for Sun Cluster feature works with releases of Sun Cluster beginning with the Sun Cluster 10/04 software collection and with applications like Oracle Real Applications Clusters. For more information on Solaris Volume Manager for Sun Cluster, see Chapter 4, Solaris Volume Manager for Sun Cluster (Overview).Before you can configure multi-owner disk sets, the following software must be installed in addition to the Solaris OS:Sun Cluster initial cluster framework Sun Cluster Support for Oracle Real Application Clusters software Oracle Real Application Clusters software For information on setting up Sun Cluster and Oracle Real Application Clusters software, see Sun Cluster Software Installation Guide for Solaris OS and Sun Cluster Data Service for Oracle Real Application Clusters Guide for Solaris OS. Unlike local disk set administration, you do not need to manually create or delete disk set state databases. Solaris Volume Manager places one state database replica (on slice 7) on each disk across all disks in the disk set, up to a maximum of 50 total replicas in the disk set.When you add disks to a disk set, Solaris Volume Manager automatically creates the state database replicas on the disk set. When a disk is accepted into a disk set, Solaris Volume Manager might repartition the disk so that the state database replica for the disk set can be placed on the disk (see Automatic Disk Partitioning).A file system that resides on a volume in a disk set normally is not mounted automatically at boot time with the /etc/vfstab file. The necessary Solaris Volume Manager RPC daemons (rpc.metad and rpc.metamhd) do not start early enough in the boot process to permit this. Additionally, the ownership of a disk set is lost during a reboot. Do not disable the Solaris Volume Manager RPC daemons in the /etc/inetd.conf file. They are configured to start by default. These daemons must remain enabled to allow Solaris Volume Manager to use its full functionality.When the autotake feature is enabled using the A option of the metaset command, the disk set is automatically taken at boot time. Under these circumstances, a file system that resides on a volume in a disk set can be automatically mounted with the /etc/vfstab file. To enable an automatic take during the boot process, the disk set must be associated with only a single host, and must have the autotake feature enabled. A disk set can be enabled either during or after disk set creation. For more information on the autotake feature, see Autotake Disk Sets.Although disk sets are supported in single-host configurations, they are often not appropriate for “local” (not dual-connected) use. Two common exceptions are the use of disk sets to provide a more manageable namespace for logical volumes, and to more easily manage storage on a Storage Area Network (SAN) fabric (see Scenario—Disk Sets. Disk sets can be created and configured by using the Solaris Volume Manager command-line interface (the metaset command) or the Enhanced Storage tool within the Solaris Management Console. After disks are added to a disk set, the disk set can be reserved (or taken) and released by hosts in the disk set. When a disk set is reserved by a host, the other host in the disk set cannot access the data on the disks in the disk set. To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. A host takes implicit ownership of the disk set by putting the first disk into the set.Disk sets, including disk sets created on a different system, can be imported into existing Solaris Volume Manager configurations using the metaimport command.Before a host can use the disks in a disk set, the host must reserve the disk set. There are two methods of reserving a disk set: Safely - When you safely reserve a disk set, Solaris Volume Manager attempts to take the disk set, and the other host attempts to release the disk set. The release (and therefore the reservation) might fail. Forcibly - When you forcibly reserve a disk set, Solaris Volume Manager reserves the disk set whether or not another host currently has the set reserved. This method is generally used when a host in the disk set is down or not communicating. All disks within the disk set are taken over. The state database is read in on the host performing the reservation and the shared volumes configured in the disk set become accessible. If the other host had the disk set reserved at this point, it would panic due to reservation loss.Normally, two hosts in a disk set cooperate with each other to ensure that the disks in a disk set are reserved by only one host at a time. A normal situation is defined as both hosts being up and communicating with each other. If a disk has been determined unexpectedly not to be reserved (perhaps because another host using the disk set forcibly took the disk), the host will panic. This behavior helps to minimize data loss which would occur if two hosts were to simultaneously access the same disk. For more information about taking or reserving a disk set, see How to Take a Disk Set. Releasing a disk set can be useful when you perform maintenance on the physical disks in the disk set. When a disk set is released, it cannot be accessed by the host. If both hosts in a disk set release the set, neither host in the disk set can access the disks in the disk set.For more information about releasing a disk set, see How to Release a Disk Set. Beginning with the Solaris 9 9/04 release, the metaimport command enables you to import disk sets, including replicated disk sets, into existing Solaris Volume Manager configurations that have device ID support in the disk set. You can also use the metaimport command to report on disk sets that are available for import.Replicated disk sets are created using remote replication software. In order for a replicated disk set to be imported with the metaimport command, the slice containing the state database replica for each disk in the disk set must also be replicated onto the same slice of the replicated disk set. This corresponds to slice 7 for non-EFI disks and slice 6 for EFI disks. Before replicating a disk set, make sure that the disk configurations of the data to be replicated and of the remote site match. This step ensures that both the state database replica and the data are replicated accurately.The metaimport command also does not import a disk in a disk set if the disk does not contain a volume or a state database replica. This scenario occurs if a volume or a state database replica have not been added to a disk or have been deleted from a disk. In this case, when you import the disk set to another system, you would find that the disk is missing from the disk set. For example, maximum of 50 state database replicas are allowed per Solaris Volume Manager disk set. If you have 60 disks in a disk set, the 10 disks that do not contain a state database replica must contain a volume in order to be imported with the disk set.For tasks associated with importing a disk set, see Importing Disk Sets.In a Sun Cluster environment, the metaimport command is not available. When you add a new disk to a disk set, Solaris Volume Manager checks the disk format. If necessary, Solaris Volume Manager repartitions the disk to ensure that the disk has an appropriately configured slice 7 with adequate space for a state database replica. The precise size of slice 7 depends on the disk geometry. However, the size will be no less than 4 Mbytes, and probably closer to 6 Mbytes (depending on where the cylinder boundaries lie). By default, Solaris Volume Manager places one state database replica on slice 7. You can increase the default size of slice 7 or decrease the size of the state database replica in order to fit more than one state database replica onto the slice.The minimal size for slice 7 will likely change in the future, based on a variety of factors, including the size of the state database replica and information to be stored in the state database replica. The default size for a state database replica in a multi-owner disk set is 16 Mbytes. For use in disk sets, disks must have a slice 7 that meets these criteria:Starts at sector 0 Includes enough space for a disk label and state database replicas Cannot be mounted Does not overlap with any other slices, including slice 2 If the existing partition table does not meet these criteria, Solaris Volume Manager repartitions the disk. A small portion of each drive is reserved in slice 7 for use by Solaris Volume Manager. The remainder of the space on each drive is placed into slice 0. Any existing data on the disks is lost by repartitioning. After you add a drive to a disk set, you may repartition it as necessary, with the exception that slice 7 is not altered in any way. The following output from the prtvtoc command shows a disk before it is added to a disk set.[root@lexicon:apps]$ prtvtoc /dev/rdsk/c1t6d0s0 * /dev/rdsk/c1t6d0s0 partition map * * Dimensions: * 512 bytes/sector * 133 sectors/track * 27 tracks/cylinder * 3591 sectors/cylinder * 4926 cylinders * 4924 accessible cylinders * * Flags: * 1: unmountable * 10: read-only * * First Sector Last * Partition Tag Flags Sector Count Sector Mount Directory 0 2 00 0 4111695 4111694 1 3 01 4111695 1235304 5346998 2 5 01 0 17682084 17682083 3 0 00 5346999 4197879 9544877 4 0 00 9544878 4197879 13742756 5 0 00 13742757 3939327 17682083The above output shows that the disk does not contain a slice 7. Therefore, when the disk is added to a disk set, Solaris Volume Manager repartitions the disk. The following output from the prtvtoc command shows the disk after it is added to a disk set.[root@lexicon:apps]$ prtvtoc /dev/rdsk/c1t6d0s0 * /dev/rdsk/c1t6d0s0 partition map * * Dimensions: * 512 bytes/sector * 133 sectors/track * 27 tracks/cylinder * 3591 sectors/cylinder * 4926 cylinders * 4924 accessible cylinders * * Flags: * 1: unmountable * 10: read-only * * First Sector Last * Partition Tag Flags Sector Count Sector Mount Directory 0 0 00 10773 17671311 17682083 7 0 01 0 10773 10772 [root@lexicon:apps]$ The output shows that the disk has been repartitioned to include a slice 7 that starts at cylinder 0 and that has sufficient space for the state database replica. If disks you add to a disk set each have an acceptable slice 7, they are not reformatted.If you have disk sets that you upgraded from Solstice DiskSuite software, the default state database replica size on those sets is 1034 blocks, not the 8192 block size from Solaris Volume Manager. Also, slice 7 on the disks that were added under Solstice DiskSuite software are correspondingly smaller than slice 7 on disks that were added under Solaris Volume Manager. Disk set volume names are similar to other Solaris Volume Manager component names. However, the disk set name is included as part of the name. For example, volume path names include the disk set name after /dev/md/ and before the actual volume name in the path.The following table shows some example disk set volume names./dev/md/blue/dsk/d0 Block volume d0 in disk set blue /dev/md/blue/dsk/d1 Block volume d1 in disk set blue /dev/md/blue/rdsk/d126 Raw volume d126 in disk set blue /dev/md/blue/rdsk/d127 Raw volume d127 in disk set blue Similarly, hot spare pools have the disk set name as part of the hot spare name. Figure 18–1 shows an example configuration that uses two disk sets. In this configuration, Host A and Host B share the disk sets red and blue. They each have their own local disk set, which is not shared. If Host A fails, Host B can take over control of Host A's shared disk set, the disk set red. Likewise, if Host B fails, Host A can take control of Host B's shared disk set, the disk set blue.

Diagram shows how two hosts can share some disks through shared disk sets and retain exclusive use of other disks in local disk sets.

When working with disk sets, consider the following guidelines:Solaris Volume Manager must be configured on each host that will be connected to the disk set. Each host must have its local state database set up before you can create disk sets. The sequence of steps for creating a disk set and creating the volumes for that disk set is to, first, create the disk set. Then, add the disks to the disk set. Finally, create the volumes in the disk set. To create and work with a disk set in a clustering environment, root must be a member of Group 14. Alternatively, the /.rhosts file on each host must contain an entry for the other host names associated with the disk set.This step is not necessary in a SunCluster 3.x environment. To perform maintenance on a disk set, a host must be the owner of the disk set or have reserved the disk set. A host takes implicit ownership of the disk set by putting the first drives into the disk set. You cannot add a drive to a disk set that is in use for a file system, database or any other application. Before you add a drive, make sure that it is not currently being used. Do not add to a disk set a drive containing existing data that you want to preserve. The process of adding the disk to the disk set repartitions the disk and destroys existing data. Unlike local volume administration, it is not necessary to manually create or delete state database replicas on the disk set. Solaris Volume Manager tries to balance a reasonable number of state database replicas across all drives in a disk set. When drives are added to a disk set, Solaris Volume Manager rebalances the state database replicas across the remaining drives. Later, if necessary, you can change the replica layout with the metadb command. In previous versions of Solaris Volume Manager, all of the disks that you planned to share between hosts in the disk set had to be connected to each host. They also had to have the exact same path, driver, and name on each host. Specifically, a shared disk drive had to be seen by both hosts in the same location (/dev/rdsk/c#t#d#). In addition, the shared disks had to use the same driver name (ssd).In the current Solaris OS release, systems that have different views of commonly accessible storage can nonconcurrently share access to a disk set. With the introduction of device ID support for disk sets, Solaris Volume Manager automatically tracks disk movement within named disk sets.Device ID support for disk sets is not supported in a Sun Cluster environment. When you upgrade to the latest Solaris OS, you need to take the disk set once in order to enable disk tracking. For more information on taking a disk set, see How to Take a Disk Set. If the autotake feature is not enabled, you have to take each disk set manually. If this feature is enabled, this step is done automatically when the system is rebooted. For more information on the autotake feature, see Autotake Disk Sets.This expanded device ID support also enables you to import disk sets, even disk sets that were created on different systems. For more information on importing disk sets, see Importing a Disk Set. The following example, drawing on the sample system shown in Chapter 5, Configuring and Using Solaris Volume Manager (Scenario), describes how disk sets should be used to manage storage that resides on a SAN (Storage Area Network) fabric. Assume that the sample system has an additional controller that connects to a fiber switch and SAN storage. Storage on the SAN fabric is unavailable to the system as early in the boot process as other devices, such as SCSI and IDE disks. In addition, Solaris Volume Manager would report logical volumes on the fabric as unavailable at boot. However, by adding the storage to a disk set, and then using the disk set tools to manage the storage, this problem with boot time availability is avoided. Also, the fabric-attached storage can be easily managed within a separate, disk set-controlled, namespace from the local storage.