This chapter provides tasks for administering interface groups with IP network multipathing (IPMP). The following major topics are discussed:Configuring IPMP (Task Maps) Configuring IPMP Groups Maintaining IPMP Groups Replacing a Failed Physical Interface on Systems That Support Dynamic Reconfiguration Recovering a Physical Interface That Was Not Present at System Boot Modifying IPMP Configurations For an overview of IPMP concepts, refer to Chapter 30, Introducing IPMP (Overview). This section contains links to the tasks that are described in this chapter.Task Description For Instructions Plan for an IPMP group. Lists all ancillary information and required tasks before you can configure an IPMP group. How to Plan for an IPMP Group Configure an IPMP interface group with multiple interfaces. Configures multiple interfaces as members of an IPMP group. How to Configure an IPMP Group With Multiple Interfaces Configure an IPMP group where one of the interfaces is a standby interface. Configures one of the interfaces in a multiple interface IPMP group as a standby interface. How to Configure a Standby Interface for an IPMP Group Configure an IPMP group that consists of a single interface. Creates a single interface IPMP group. How to Configure a Single Interface IPMP Group Display the IPMP group to which a physical interface belongs. Explains how to obtain the name of an interface's IPMP group from the output of the ifconfig command. How to Display the IPMP Group Membership of an Interface Add an interface to an IPMP group. Configures a new interface as a member of an existing IPMP group. How to Add an Interface to an IPMP Group Remove an interface from an IPMP group. Explains how to remove an interface from an IPMP group. How to Remove an Interface From an IPMP Group Move an interface from an existing IPMP group to a different group. Moves interfaces among IPMP groups. How to Move an Interface From One IPMP Group to Another Group Change three default settings for the in.mpathd daemon. Customizes failure detection time and other parameters of the in.mpathd daemon. How to Configure the /etc/default/mpathd File Task Description For Instructions Remove an interface that has failed. Removes a failed interface on a system. How to Remove a Physical Interface That Has Failed (DR-Detach) Replace an interface that has failed. Replaces a failed interface. How to Replace a Physical Interface That Has Failed (DR-Attach) Recover an interface that was not configured at boot time. Recovers a failed interface. How to Recover a Physical Interface That Was Not Present at System Boot This section provides procedures for configuring IPMP groups. It also describes how to configure an interface as a standby.Before you configure interfaces on a system as part of an IPMP group, you need to do some preconfiguration planning. The following procedure includes the planning tasks and information to be gathered prior to configuring the IPMP group. The tasks do not have to be performed in sequence. Decide which interfaces on the system are to be part of the IPMP group.An IPMP group usually consists of at least two physical interfaces that are connected to the same IP link. However, you can configure a single interface IPMP group, if required. For an introduction to IPMP groups, refer to IPMP Interface Configurations. For example, you can configure the same Ethernet switch or the same IP subnet under the same IPMP group. You can configure any number of interfaces into the same IPMP group.You cannot use the group parameter of the ifconfig command with logical interfaces. For example, you can use the group parameter with hme0, but not with hme0:1. Verify that each interface in the group has a unique MAC address.For instructions, refer to How to Ensure That the MAC Address of an Interface Is Unique, in Solaris 10 3/05 ONLY or How to Ensure That the MAC Address of an Interface Is Unique. Choose a name for the IPMP group.Any non-null name is appropriate for the group. You might want to use a name that identifies the IP link to which the interfaces are attached. Ensure that the same set of STREAMS modules is pushed and configured on all interfaces in the IPMP group.All interfaces in the same group must have the same STREAMS modules configured in the same order.Check the order of STREAMS modules on all interfaces in the prospective IPMP group.You can print out a list of STREAMS modules by using the ifconfig interface modlist command. For example, here is the ifconfig output for an hme0 interface:# ifconfig hme0 modlist 0 arp 1 ip 2 hmeInterfaces normally exist as network drivers directly below the IP module, as shown in the output from ifconfig hme0 modlist. They should not require additional configuration.However, certain technologies, such as NCA or IP Filter, insert themselves as STREAMS modules between the IP module and the network driver. Problems can result in the way interfaces of the same IPMP group behave.If a STREAMS module is stateful, then unexpected behavior can occur on failover, even if you push the same module onto all of the interfaces in a group. However, you can use stateless STREAMS modules, provided that you push them in the same order on all interfaces in the IPMP group. Push the modules of an interface in the standard order for the IPMP group.ifconfig interface modinsert module-nameifconfig hme0 modinsert ip Use the same IP addressing format on all interfaces of the IPMP group.If one interface is configured for IPv4, then all interfaces of the group must be configured for IPv4. Suppose you have an IPMP group that is composed of interfaces from several NICs. If you add IPv6 addressing to the interfaces of one NIC, then all interfaces in the IPMP group must be configured for IPv6 support. Check that all interfaces in the IPMP group are connected to the same IP link. Verify that the IPMP group does not contain interfaces with different network media types.The interfaces that are grouped together should be of the same interface type, as defined in /usr/include/net/if_types.h. For example, you cannot combine Ethernet and Token ring interfaces in an IPMP group. As another example, you cannot combine a Token bus interface with asynchronous transfer mode (ATM) interfaces in the same IPMP group. For IPMP with ATM interfaces, configure the ATM interfaces in LAN emulation mode. IPMP is not supported for interfaces using Classical IP over ATM. Before you configure an IPMP group, you must verify that every interface in the prospective group has a unique MAC address. Almost all interfaces come configured with a factory-set unique MAC address. However, every SPARC-based system has a system-wide MAC address, which by default is used by all interfaces. In an IPMP group, each interface must have a unique MAC address. Therefore, you must ensure that the EEPROM parameter local-mac-address? is set to true so that the interfaces use their factory-set MAC addresses. You can use the eeprom command to check the current value of local-mac-address? and change it, if necessary. On the system with the interfaces to be configured, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Determine whether all interfaces on the system currently use the system-wide MAC address.# eeprom local-mac-address? local-mac-address?=falseIn the example, the value of local-mac-address?=false indicates that all interfaces do use the system-wide MAC address. The value of local-mac-address?=false must be changed to true before the interfaces can become members of an IPMP group. If necessary, change the value of local-mac-address? as follows:# eeprom local-mac-address?=trueWhen you reboot the system, the interfaces with factory-set MAC addresses instead use these factory settings. Interfaces without factory-set MAC addresses continue to use the system-wide MAC address. Check the MAC addresses of the interfaces on the system.ifconfig -a lo0: flags=1000849 <UP,LOOPBACK,RUNNING,MULTICAST,IPv4> mtu 8232 index 1 inet 127.0.0.1 netmask ff000000 hme0: flags=1004843 <UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.0.0.112 netmask ffffff80 broadcast 10.0.0.127 ether 8:0:20:0:0:1 hme1: flags=1004843 <UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.0.0.114 netmask ffffff80 broadcast 10.0.0.127 ether 8:0:20:0:0:1 ge0: flags=1004843 <UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 10.0.0.118 netmask ffffff80 broadcast 10.0.0.127 ether 8:0:20:1:1:1Look for cases where multiple interfaces have the same MAC address. In the previous example, hme0 and hme1 both have the same MAC address.Continue to the next step only if more than one network interface still has the same MAC address. If necessary, manually configure the remaining interfaces so that all interfaces have unique MAC addresses.Place a unique MAC address in the /etc/hostname.interface for the particular interface. To prevent any risk of manually configured MAC addresses conflicting with other MAC addresses on your network, you must always configure locally administered MAC addresses, as defined by the IEEE 802.3 standard. In the previous example, you must configure either hme0 or hme1 with a locally-administered MAC address. For example, to reconfigure hme1 with the locally-administered MAC address 06:05:04:03:02, you would add the following line to /etc/hostname.hme1: ether 06:05:04:03:02 You also can use the ifconfig ether command to configure an interface's MAC address for the current session. However, any changes made directly with ifconfig are not preserved across reboots. Refer to the ifconfig1M man page for details. Reboot the system. This section contains configuration tasks for a typical IPMP group with at least two physical interfaces.For an introduction to multiple interface IPMP groups, refer to IPMP Group. For planning tasks, refer to Planning for an IPMP Group. To configure an IPMP group with only one physical interface, refer to Configuring IPMP Groups With a Single Physical Interface. You need to have already configured the IPv4 addresses, and, if appropriate, the IPv6 addresses of all interfaces in the prospective IPMP group. On the system with the interfaces to be configured, assume the Primary Administrator role, or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Place each physical interface into an IPMP group.# ifconfig interface group group-nameFor example, to place hme0 and hme1 under group testgroup1, you would type the following commands:# ifconfig hme0 group testgroup1 # ifconfig hme1 group testgroup1Avoid using spaces in group names. The ifconfig status display does not show spaces. Consequently, do not create two similar group names where the only difference is that one name also contains a space. If one of the group names contains a space, these group names look the same in the status display.In a dual-stack environment, placing the IPv4 instance of an interface under a particular group automatically places the IPv6 instance under the same group. (Optional) Configure an IPv4 test address on one or more physical interfaces.You need to configure a test address only if you want to use probe-based failure detection on a particular interface. Test addresses are configured as logical interfaces of the physical interface that you specify to the ifconfig command.If one interface in the group is to become the standby interface, do not configure a test address for that interface at this time. You configure a test address for the standby interface as part of the task How to Configure a Standby Interface for an IPMP Group.Use the following syntax of the ifconfig command for configuring a test address:# ifconfig interface addif ip-address <parameters> -failover deprecated upFor example, you would create the following test address for the primary network interface hme0:# ifconfig hme0 addif 192.168.85.21 netmask + broadcast + -failover deprecated upThis command sets the following parameters for the primary network interface hme0:Address set to 192.168.85.21 Netmask and broadcast address set to the default value failover and deprecated options setYou must mark an IPv4 test address as deprecated to prevent applications from using the test address. Check the IPv4 configuration for a specific interface.You can always view the current status of an interface by typing ifconfig interface. For more information on viewing an interface's status, refer to How to Get Information About a Specific Interface.You can get information about test address configuration for a physical interface by specifying the logical interface that is assigned to the test address.# ifconfig hme0:1 hme0:1: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 2 inet 192.168.85.21 netmask ffffff00 broadcast 192.168.85.255 (Optional) If applicable, configure an IPv6 test address.# ifconfig interface inet6 -failoverPhysical interfaces with IPv6 addresses are placed into the same IPMP group as the interfaces' IPv4 addresses. This happens when you configure the physical interface with IPv4 addresses into an IPMP group. If you first place physical interfaces with IPv6 addresses into an IPMP group, physical interfaces with IPv4 addresses are also implicitly placed in the same IPMP group.For example, to configure hme0 with an IPv6 test address, you would type the following:# ifconfig hme0 inet6 -failoverYou do not need to mark an IPv6 test address as deprecated to prevent applications from using the test address. Check the IPv6 configuration.# ifconfig hme0 inet6 hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6,NOFAILOVER> mtu 1500 index 2 inet6 fe80::a00:20ff:feb9:17fa/10 groupname testThe IPv6 test address is the link-local address of the interface. (Optional) Preserve the IPMP group configuration across reboots.For IPv4, add the following line to the /etc/hostname.interface file:interface-address <parameters> group group-name up \ addif logical-interface -failover deprecated <parameters> upIn this instance, the test IPv4 address is configured only on the next reboot. If you want the configuration to be invoked in the current session, do steps 1, 2, and, optionally 3. For IPv6, add the following line to the /etc/hostname6.interface file:-failover group group-name upThis test IPv6 address is configured only on the next reboot. If you want the configuration to be invoked in the current session, do steps 1, 2, and, optionally, 5. (Optional) Add more interfaces to the IPMP group by repeating steps 1 through 6. You can add new interfaces to an existing group on a live system. However, changes are lost across reboots. Suppose you want to do the following:Have the netmask and broadcast address set to the default value. Configure the interface with a test address 192.168.85.21. You would type the following command:# ifconfig hme0 addif 192.168.85.21 netmask + broadcast + -failover deprecated upYou must mark an IPv4 test address as deprecated to prevent applications from using the test address. See How to Configure an IPMP Group With Multiple Interfaces. To turn on the failover attribute of the address, you would use the failover option without the dash All test IP addresses in an IPMP group must use the same network prefix. The test IP addresses must belong to a single IP subnet. Suppose you want to create an IPMP group called testgroup1 with the following configuration:Physical interface hme0 with address 192.168.85.19 A logical interface address of 192.168.85.21 deprecated and failover options set Netmask and broadcast address set to the default value You would add the following line to the /etc/hostname.hme0 file:192.168.85.19 netmask + broadcast + group testgroup1 up \ addif 192.168.85.21 deprecated -failover netmask + broadcast + upSimilarly, to place the second interface hme1 under the same group testgroup1 and to configure a test address, you would add the following line:192.168.85.20 netmask + broadcast + group testgroup1 up \ addif 192.168.85.22 deprecated -failover netmask + broadcast + up To create a test group for interface hme0 with an IPv6 address, you would add the following line to the /etc/hostname6.hme0 file:-failover group testgroup1 upSimilarly, to place the second interface hme1 in group testgroup1 and to configure a test address, you would add the following line to the /etc/hostname6.hme1 file:-failover group testgroup1 up During IPMP group configuration, in.mpathd outputs a number of messages to the system console or to the syslog file. These messages are informational in nature and indicate that the IPMP configuration functions correctly.This message indicates that interface hme0 was added to IPMP group testgroup1. However, hme0 does not have a test address configured. To enable probe-based failure detection, you need to assign a test address to the interface.May 24 14:09:57 host1 in.mpathd[101180]: No test address configured on interface hme0; disabling probe-based failure detection on it. testgroup1 This message appears for all interfaces with only IPv4 addresses that are added to an IPMP group. May 24 14:10:42 host4 in.mpathd[101180]: NIC qfe0 of group testgroup1 is not plumbed for IPv6 and may affect failover capability This message should appear when you have configured a test address for an interface.Created new logical interface hme0:1 May 24 14:16:53 host1 in.mpathd[101180]: Test address now configured on interface hme0; enabling probe-based failure detection on it If you want the IPMP group to have an active-standby configuration, go on to How to Configure a Standby Interface for an IPMP Group. Probe-based failure detection involves the use of target systems, as explained in Probe-Based Failure Detection. For some IPMP groups, the default targets used by in.mpathd is sufficient. However, for some IPMP groups, you might want to configure specific targets for probe-based failure detection. You accomplish probe-based failure detection by setting up host routes in the routing table as probe targets. Any host routes that are configured in the routing table are listed before the default router. Therefore, IPMP uses the explicitly defined host routes for target selection. You can use either of two methods for directly specifying targets: manually setting host routes or creating a shell script that can become a startup script. Consider the following criteria when evaluating which hosts on your network might make good targets. Make sure that the prospective targets are available and running. Make a list of their IP addresses. Ensure that the target interfaces are on the same network as the IPMP group that you are configuring. The netmask and broadcast address of the target systems must be the same as the addresses in the IPMP group. The target host must be able to answer ICMP requests from the interface that is using probe-based failure detection. Log in with your user account to the system where you are configuring probe-based failure detection. Add a route to a particular host to be used as a target in probe-based failure detection.$ route add -host destination-IP gateway-IP -staticReplace the values of destination-IP and gateway-IP with the IPv4 address of the host to be used as a target. For example, you would type the following to specify the target system 192.168.85.137, which is on the same subnet as the interfaces in IPMP group testgroup1.$ route add -host 192.168.85.137 192.168.85.137 -static Add routes to additional hosts on the network to be used as target systems. On the system where you have configured an IPMP group, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Create a shell script that sets up static routes to your proposed targets. For example, you could create a shell script called ipmp.targets with the following contents:TARGETS="192.168.85.117 192.168.85.127 192.168.85.137" case "$1" in 'start') /usr/bin/echo "Adding static routes for use as IPMP targets" for target in $TARGETS; do /usr/sbin/route add -host $target $target done ;; 'stop') /usr/bin/echo "Removing static routes for use as IPMP targets" for target in $TARGETS; do /usr/sbin/route delete -host $target $target done ;; esac Copy the shell script to the startup script directory. # cp ipmp.targets /etc/init.d Change the permissions on the new startup script. # chmod 744 /etc/init.d/ipmp.targets Change ownership of the new startup script. # chown root:sys /etc/init.d/ipmp.targets Create a link for the startup script in the /etc/init.d directory.# ln /etc/init.d/ipmp.targets /etc/rc2.d/S70ipmp.targetsThe S70 prefix in the file name S70ipmp.targets orders the new script properly with respect to other startup scripts. Use this procedure if you want the IPMP group to have an active-standby configuration. For more information on this type of configuration, refer to IPMP Interface Configurations. You must have configured all interfaces as members of the IPMP group. You should not have configured a test address on the interface to become the standby interface. For information on configuring an IPMP group and assigning test addresses, refer to How to Configure an IPMP Group With Multiple Interfaces. On the system with the standby interfaces to be configured, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Configure an interface as a standby and assign the test address.# ifconfig interface plumb ip-address <other-parameters> deprecated -failover standby upA standby interface can have only one IP address, the test address. You must set the failover option before you set the standby up option. For <other-parameters>, use the parameters that are required by your configuration, as described in the ifconfig1M man page. For example, to create an IPv4 test address, you would type the following command:# ifconfig hme1 plumb 192.168.85.22 netmask + broadcast + deprecated -failover standby uphme1Defines hme1 as the physical interface to be configured as the standby interface. 192.168.85.22Assigns this test address to the standby interface. deprecatedIndicates that the test address is not used for outbound packets. failoverIndicates that the test address does not fail over if the interface fails. standbyMarks the interface as a standby interface. For example, to create an IPv6 test address, you would type the following command:# ifconfig hme1 plumb -failover standby up Check the results of the standby interface configuration.# ifconfig hme1 hme1: flags=69040843<UP,BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER, STANDBY,INACTIVE mtu 1500 index 4 inet 192.168.85.22 netmask ffffff00 broadcast 19.16.85.255 groupname testThe INACTIVE flag indicates that this interface is not used for any outbound packets. When a failover occurs on this standby interface, the INACTIVE flag is cleared.You can always view the current status of an interface by typing the ifconfig interface command. For more information on viewing interface status, refer to How to Get Information About a Specific Interface. (Optional) Preserve the IPv4 standby interface across reboots.Assign the standby interface to the same IPMP group, and configure a test address for the standby interface.For example, to configure hme1 as the standby interface, you would add the following line to the /etc/hostname.hme1 file:192.168.85.22 netmask + broadcast + deprecated group test -failover standby up (Optional) Preserve the IPv6 standby interface across reboots.Assign the standby interface to the same IPMP group, and configure a test address for the standby interface.For example, to configure hme1 as the standby interface, add the following line to the /etc/hostname6.hme1 file:-failover group test standby up Suppose you want to create a test address with the following configuration:Physical interface hme2 as a standby interface Test address of 192.168.85.22 deprecated and failover options set Netmask and broadcast address set to the default value You would type the following:# ifconfig hme2 plumb 192.168.85.22 netmask + broadcast + deprecated -failover standby upThe interface is marked as a standby interface only after the address is marked as a NOFAILOVER address.You would remove the standby status of an interface by typing the following:# ifconfig interface -standby When you have only one interface in an IPMP group, failover is not possible. However, you can enable failure detection on that interface by assigning the interface to an IPMP group. You do not have to configure a dedicated test IP address to establish failure detection for a single interface IPMP group. You can use a single IP address for sending data and detecting failure. On the system with the prospective single interface IPMP group, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. For IPv4, create the single interface IPMP group.You can use either of the following methods:Use the following syntax to assign the single interface to an IPMP group.# ifconfig interface -failover group group-nameThe following example assigns the interface hme0 into the IPMP group v4test:# ifconfig hme0 -failover group v4testUnlike the multiple physical interface configuration, you would not mark a single physical interface as deprecated. This example includes the use of the failover option of the ifconfig command to create an IFF_NOFAILOVER flag for the interface. Consider using failover if you might later add more interfaces to the group. The in.mpathd daemon sends probe packets by using that address. Later, when you add more interfaces, the configuration should work properly. Alternatively, you can use the following syntax to add a single physical interface to an IPMP group:# ifconfig interface group group-nameWhen you use this configuration, in.mpathd chooses a data address to send probe packets. For IPv6, create the single interface IPMP group.Use either of the following two methods:Use the following syntax to assign the single interface to an IPMP group:# ifconfig interface inet6 -failover group group-nameFor example, you would type the following to add the single interface hme0 into the IPMP group v6test:# ifconfig hme0 inet6 -failover group v6test Use the following syntax if you do not want to set the NOFAILOVER flag:# ifconfig interface inet6 group group-nameWhen the in.mpathd daemon detects failures, the interface is marked and logged appropriately on the console. In a single physical interface configuration, you cannot verify whether the target system that is being probed has failed or whether the interface has failed. The target system can be probed through only one physical interface. If only one default router is on the subnet, turn off IPMP if a single physical interface is in the group. If a separate IPv4 and IPv6 default router exists, or multiple default routers exist, more than one target system needs to be probed. Hence, you can safely turn on IPMP. This section contains tasks for maintaining existing IPMP groups and the interfaces that compose those groups. The tasks presume that you have already configured an IPMP group, as explained in Configuring IPMP Groups.On the system with the IPMP group configuration, become superuser or assume an equivalent role.Roles contain authorizations and privileged commands. For more information about roles, see Configuring RBAC (Task Map) in System Administration Guide: Security Services. Display information about the interface, including the group to which the interface belongs.# ifconfig interface If applicable, display IPv6 information for the interface.# ifconfig interface inet6 To display the group name for hme0, you would type the following:# ifconfig hme0 hme0: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 192.168.85.19 netmask ffffff00 broadcast 192.168.85.255 groupname testgroup1To display the group name for only the IPv6 information, you would type the following:# ifconfig hme0 inet6 hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2 inet6 fe80::a00:20ff:feb9:19fa/10 groupname testgroup1 On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Add the interface to the IPMP group.# ifconfig interface group group-nameThe interface specified in interface becomes a member of IPMP group group-name. To add hme0 to the IPMP group testgroup2, you would type the following command:# ifconfig hme0 group testgroup2 hme0: flags=9000843<UP ,BROADCAST,RUNNING,MULTICAST,IPv4,NOFAILOVER> mtu 1500 index 2 inet 192.168.85.19 netmask ff000000 broadcast 10.255.255.255 groupname testgroup2 ether 8:0:20:c1:8b:c3 When you execute the ifconfig command's group parameter with a null string, the interface is removed from its current IPMP group. Be careful when removing interfaces from a group. If some other interface in the IPMP group has failed, a failover could have happened earlier. For example, if hme0 failed previously, all addresses are failed over to hme1, if hme1 is part of the same group. The removal of hme1 from the group causes the in.mpathd daemon to return all the failover addresses to some other interface in the group. If no other interfaces are functioning in the group, failover might not restore all the network accesses.Similarly, when an interface in a group needs to be unplumbed, you should first remove the interface from the group. Then, ensure that the interface has all the original IP addresses configured. The in.mpathd daemon tries to restore the original configuration of an interface that is removed from the group. You need to ensure that the configuration is restored before unplumbing the interface. Refer to What Happens During Interface Failover to see how interfaces look before and after a failover. On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Remove the interface from the IPMP group.# ifconfig interface group ""The quotation marks indicate a null string. To remove hme0 from the IPMP group test, you would type the following command:# ifconfig hme0 group "" # ifconfig hme0 hme0: flags=9000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2 inet 192.168.85.19 netmask ffffff00 broadcast 192.168.85.255 # ifconfig hme0 inet6 hme0: flags=a000841<UP,RUNNING,MULTICAST,IPv6> mtu 1500 index 2 inet6 fe80::a00:20ff:feb9:19fa/10 You can place an interface in a new IPMP group when the interface belongs to an existing IPMP group. You do not need to remove the interface from the current IPMP group. When you place the interface in a new group, the interface is automatically removed from any existing IPMP group. On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Move the interface to a new IPMP group.# ifconfig interface group group-namePlacing the interface in a new group automatically removes the interface from any existing group. To change the IPMP group of interface hme0, you would type the following: # ifconfig hme0 group cs-linkThis command removes the hme0 interface from IPMP group test and then puts the interface in the group cs-link. This section contains procedures that relate to administering systems that support dynamic reconfiguration (DR).The tasks pertain only to IP layers that are configured by using the ifconfig command. Layers before or after the IP layer, such as ATM or other services, require specific manual steps if the layers are not automated. The steps in the next procedures are used to unconfigure interfaces during predetachment and configure interface after postattachment. This procedure shows how to remove a physical interface on a system that supports DR. The procedure assumes that the following conditions already exist:Physical interfaces hme0 and hme1 are the example interfaces. Both interfaces are in the same IPMP group. hme0 has failed. Logical interface hme0:1 has the test address. You are replacing the failed interface with the same physical interface name, for example, hme0 with hme0. You can skip Step 2 if the test address is plumbed by using the /etc/hostname.hme0 file. On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Display the test address configuration.# ifconfig hme0:1 hme0:1: flags=9040842<BROADCAST,RUNNING,MULTICAST,DEPRECATED,IPv4,NOFAILOVER> mtu 1500 index 3 inet 192.168.233.250 netmask ffffff00 broadcast 192.168.233.255You need this information to replumb the test address when replacing the physical interface. Remove the physical interface.Refer to the following sources for a complete description of how to remove the physical interface:cfgadm1M man page Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide Sun Enterprise 10000 DR Configuration Guide This procedure shows how to replace a physical interface on a system that supports DR. On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Replace the physical interface.Refer to the instructions in the following sources:cfgadm1M man page Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide Sun Enterprise 10000 DR Configuration Guide, or Sun Fire 880 Dynamic Reconfiguration User's Guide Plumb and bring up the test address.# ifconfig hme0 test-address-configurationUse the same test address configuration that was configured in the /etc/hostname.hme0 file. The test configuration is the same configuration that is displayed in Step 2 in the procedure How to Remove a Physical Interface That Has Failed (DR-Detach).This configuration triggers the in.mpathd daemon to resume probing. As a result of this probing, in.mpathd detects the repair. Consequently, in.mpathd causes the original IP address to fail back from hme1.See Test Addresses for more details about test addresses.The failback of IP addresses during the recovery of a failed physical interface takes up to three minutes. This time might vary, depending on network traffic. The time also depends on the stability of the incoming interface to fail back the failed-over interfaces by the in.mpathd daemon. The following procedure pertains only to IP layers that are configured by using the ifconfig command. Layers before or after the IP layer, such as ATM or other services, require specific manual steps if the layers are not automated. The specific steps in the next procedure are used to unconfigure interfaces during predetachment and to configure interfaces after postattachment. Recovery after dynamic reconfiguration is automatic for an interface that is part of the I/O board on a Sun Fire™ platform. If the NIC is a Sun Crypto Accelerator I - cPCI board, the recovery is also automatic. Consequently, the following steps are not required for an interface that is coming back as part of a DR operation. For more information on the Sun Fire x800 and Sun Fire 15000 systems, see the cfgadm_sbd1M man page. The physical interface fails back to the configuration that is specified in the /etc/hostname.interface file. See Configuring IPMP Groups for details on how to configure interfaces to preserve the configuration across reboots.On Sun Fire legacy (Exx00) systems, DR detachments are still subject to manual procedures. However, DR attachments are automated. You must complete the following procedure before you recover a physical interface that was not present at system boot. The example in this procedure has the following configuration:Physical interfaces hme0 and hme1 are the interfaces. Both interfaces are in the same IPMP group. hme0 was not installed at system boot. The failback of IP addresses during the recovery of a failed physical interface takes up to three minutes. This time might vary, depending on network traffic. The time also depends on the stability of the incoming interface to fail back the failed-over interfaces by the in.mpathd daemon. On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Retrieve the failed network information from the failure error message of the console log.See the syslog3Cman page. The error message might be similar to the following:moving addresses from failed IPv4 interfaces: hme1 (moved to hme0)This message indicates that the IPv4 addresses on the failed interface hme1 have failed over to the hme0 interface.Alternatively, you might receive the following similar message:moving addresses from failed IPv4 interfaces: hme1 (couldn't move, no alternative interface)This message indicates that no active interface could be found in the same group as failed interface hme1. Therefore, the IPv4 addresses on hme1 could not fail over. Attach the physical interface to the system. Refer to the following for instructions on how to replace the physical interface:cfgadm1M man page Sun Enterprise 10000 DR Configuration Guide Sun Enterprise 6x00, 5x00, 4x00, and 3x00 Systems Dynamic Reconfiguration User's Guide Refer to the message content from Step 2. If the addresses could not be moved, go to Step 6. If the addresses were moved, continue to Step 5. Unplumb the logical interfaces that were configured as part of the failover process.Review the contents of the /etc/hostname.moved-from-interface file to determine what logical interfaces were configured as part of the failover process. Unplumb each failover IP address.# ifconfig moved-to-interface removeif moved-ip-addressFailover addresses are marked with the failover parameter, or are not marked with the -failover parameter. You do not need to unplumb IP addresses that are marked -failover. For example, assume that the contents of the /etc/hostname.hme0 file contains the following lines:inet 10.0.0.4 -failover up group one addif 10.0.0.5 failover up addif 10.0.0.6 failover upTo unplumb each failover IP address, you would type the following commands:# ifconfig hme0 removeif 10.0.0.5 # ifconfig hme0 removeif 10.0.0.6 Reconfigure the IPv4 information for the replaced physical interface by typing the following command for each interface that was removed:# ifconfig removed-from-NIC <parameters>For example, you would type the following commands:# ifconfig hme1 inet plumb # ifconfig hme1 inet 10.0.0.4 -failover up group one # ifconfig hme1 addif 10.0.0.5 failover up # ifconfig hme1 addif 10.0.0.6 failover up Use the IPMP configuration file /etc/default/mpathd to configure the following system-wide parameters for IPMP groups.FAILURE_DETECTION_TIME FAILBACK TRACK_INTERFACES_ONLY_WITH_GROUPS These parameters apply to all IPMP groups that are created on an individual system.On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Edit the /etc/default/mpathd file. (Optional) Type the new value for the FAILURE_DETECTION_TIME parameter.FAILURE_DETECTION_TIME=nwhere n is the amount of time in seconds for ICMP probes to detect whether an interface failure has occurred. The default is 10 seconds. (Optional) Type the new value for the TRACK_INTERFACES_ONLY_WITH_GROUPS parameter.TRACK_INTERFACES_ONLY_WITH_GROUPS=[yes | no]yes- The yes value is the default behavior of IPMP. This parameter causes IPMP to ignore network interfaces that are not configured into an IPMP group. no - The no value sets failure and repair detection for all network interfaces, regardless of whether they are configured into an IPMP group. However, when a failure or repair is detected on an interface that is not configured into an IPMP group, no failover or failback occurs. Therefore, theno value is only useful for reporting failures and does not directly improve network availability. (Optional) Type the new value for the FAILBACK parameter.FAILBACK=[yes | no]yes- The yes value is the default failback behavior of IPMP. When the repair of a failed interface is detected, network access fails back to the repaired interface, as described in IPMP Failure Detection and Recovery Features. no - The no indicates that data traffic does not move back to a repaired interface. When a failed interfaces is detected as repaired, the INACTIVE flag of ifconfig is set. This flag indicates that the interface is currently not to be used for data traffic. The interface can still be used for probe traffic.For example, suppose an IPMP group consists of two interfaces, ce0 and ce1. Then assume that the value FAILBACK=no is set in the mpathd file. If ce0 fails, its traffic fails over to ce1, as is the expected behavior of IPMP. However, when IPMP detects that ce0 is repaired, traffic does not fail back from ce1, due to the FAILBACK=no parameter in the mpathd file. ce0 retains its INACTIVE status and is not used for traffic unless the ce1 interface fails. Restart the in.mpathd daemon.# pkill -HUP in.mpathd The IPMP configuration file /etc/default/mpathd contains three parameters that you can modify for your configuration requirements:FAILURE_DETECTION_TIME TRACK_INTERFACES_ONLY_WITH_GROUPS FAILBACK On the system with the IPMP group configuration, assume the Primary Administrator role or become superuser.The Primary Administrator role includes the Primary Administrator profile. To create the role and assign the role to a user, see Chapter 2, Working With the Solaris Management Console (Tasks), in System Administration Guide: Basic Administration. Edit the /etc/default/mpathd file. Change the default value of one or more of the three parameters.Type the new value for the FAILURE_DETECTION_TIME parameter.FAILURE_DETECTION_TIME=nwhere n is the amount of time in seconds for ICMP probes to detect whether an interface failure has occurred. The default is 10 seconds. Type the new value for the FAILBACK parameter.FAILBACK=[yes | no] Type the new value for the TRACK_INTERFACES_ONLY_WITH_GROUPS parameter.TRACK_INTERFACES_ONLY_WITH_GROUPS=[yes | no] Restart the in.mpathd daemon.# pkill -HUP in.mpathd