The IP Security Architecture (IPsec) provides cryptographic protection for IP datagrams in IPv4 and IPv6 network packets.This chapter contains the following information:What's New in IPsec? Introduction to IPsec IPsec Packet Flow IPsec Security Associations IPsec Protection Mechanisms IPsec Protection Policies Transport and Tunnel Modes in IPsec Virtual Private Networks and IPsec IPsec and NAT Traversal IPsec and SCTP IPsec and Solaris Zones IPsec Utilities and Files Changes to IPsec for the Solaris 10 Release To implement IPsec on your network, see Chapter 20, Configuring IPsec (Tasks). For reference information, see Chapter 21, IP Security Architecture (Reference). Solaris Express, Developer Edition 2/07: In this release, IPsec fully implements tunnels in tunnel mode, and modifies the utilities that support tunnels.IPsec implements tunnels in tunnel mode for Virtual Private Networks (VPNs). In tunnel mode, IPsec supports multiple clients behind a single NAT. In tunnel mode, IPsec is interoperable with implementations of IP-in-IP tunnels by other vendors. IPsec continues to support tunnels in transport mode, so is compatible with earlier Solaris releases. The syntax to create a tunnel is simplified. To manage IPsec policy, the ipsecconf command has been expanded. The ifconfig command is deprecated for managing IPsec policy. In this release, the /etc/ipnodes file is removed. Use the /etc/hosts file to configure network IPv6 addresses.Solaris 10 1/06: In this release, IKE is fully compliant with NAT-Traversal support as described in RFC 3947 and RFC 3948. IKE operations use the PKCS #11 library from the cryptographic framework, which improves performance.The cryptographic framework provides a softtoken keystore for applications that use the metaslot. When IKE uses the metaslot, you have the option of storing the keys on disk, on an attached board, or in the softtoken keystore.To use the softtoken keystore, see the cryptoadm1M man page. For a complete listing of new Solaris features and a description of Solaris releases, see Solaris Express, Developer Edition What’s New. IPsec protects IP packets by authenticating the packets, by encrypting the packets, or by doing both. IPsec is performed inside the IP module, well below the application layer. Therefore, an Internet application can take advantage of IPsec while not having to configure itself to use IPsec. When used properly, IPsec is an effective tool in securing network traffic.IPsec protection involves five main components:Security protocols – The IP datagram protection mechanisms. The authentication header (AH) signs IP packets and ensures integrity. The content of the datagram is not encrypted, but the receiver is assured that the packet contents have not been altered. The receiver is also assured that the packets were sent by the sender. The encapsulating security payload (ESP) encrypts IP data, thus obscuring the content during packet transmission. ESP also can ensure data integrity through an authentication algorithm option. Security associations database (SADB) – The database that associates a security protocol with an IP destination address and an indexing number. The indexing number is called the security parameter index (SPI). These three elements (the security protocol, the destination address, and the SPI) uniquely identify a legitimate IPsec packet. The database ensures that a protected packet that arrives to the packet destination is recognized by the receiver. The receiver also uses information from the database to decrypt the communication, verify that the packets are unchanged, reassemble the packets, and deliver the packets to their ultimate destination. Key management – The generation and distribution of keys for the cryptographic algorithms and for the SPI. Security mechanisms – The authentication and encryption algorithms that protect the data in the IP datagrams. Security policy database (SPD) – The database that specifies the level of protection to apply to a packet. The SPD filters IP traffic to determine how the packets should be processed. A packet can be discarded. A packet can be passed in the clear. Or, a packet can be protected with IPsec. For outbound packets, the SPD and the SADB determine what level of protection to apply. For inbound packets, the SPD helps to determine if the level of protection on the packet is acceptable. If the packet is protected by IPsec, the SPD is consulted after the packet has been decrypted and has been verified. When you invoke IPsec, IPsec applies the security mechanisms to IP datagrams that travel to the IP destination address. The receiver uses information in its SADB to verify that the arriving packets are legitimate, and to decrypt them. Applications can invoke IPsec to apply security mechanisms to IP datagrams on a per-socket level as well.Note that sockets behave differently from ports:Per-socket SAs override their corresponding port entry in the SPD. Also, if a socket on a port is connected, and IPsec policy is later applied to that port, then traffic that uses that socket is not protected by IPsec.Of course, a socket that is opened on a port after IPsec policy is applied to the port is protected by IPsec policy. The Internet Engineering Task Force (IETF) has published a number of Requests for Comment (RFCs) that describe the security architecture for the IP layer. All RFCs are copyrighted by the Internet Society. For a link to the RFCs, see http://ietf.org/. The following list of RFCs covers the more general IP security references:RFC 2411, “IP Security Document Roadmap,” November 1998 RFC 2401, “Security Architecture for the Internet Protocol,” November 1998 RFC 2402, “IP Authentication Header,” November 1998 RFC 2406, “IP Encapsulating Security Payload (ESP),” November 1998 RFC 2408, “Internet Security Association and Key Management Protocol (ISAKMP),” November 1998 RFC 2407, “The Internet IP Security Domain of Interpretation for ISAKMP,” November 1998 RFC 2409, “The Internet Key Exchange (IKE),” November 1998 RFC 3554, “On the Use of Stream Control Transmission Protocol (SCTP) with IPsec,” July 2003 [ not implemented in the Solaris 10 release ] The IPsec RFCs define a number of terms that are useful to recognize when implementing IPsec on your systems. The following table lists IPsec terms, provides their commonly used acronyms, and defines each term. For a list of terminology used in key negotiation, see Table 22–1.IPsec Term Acronym Definition Security association SA A unique connection between two nodes on a network. The connection is defined by a triplet: a security protocol, a security parameter index, and an IP destination. The IP destination can be an IP address or a socket. Security associations database SADB Database that contains all active security associations. Security parameter index SPI The indexing value for a security association. An SPI is a 32-bit value that distinguishes among SAs that have the same IP destination and security protocol. Security policy database SPD Database that determines if outbound packets and inbound packets have the specified level of protection. Key exchange The process of generating keys for asymmetric cryptographic algorithms. The two main methods are RSA protocols and the Diffie-Hellman protocol. Diffie-Hellman protocol DH A key exchange protocol that involves key generation and key authentication. Often called authenticated key exchange. RSA protocol RSA A key exchange protocol that involves key generation and key distribution. The protocol is named for its three creators, Rivest, Shamir, and Adleman. Internet Security Association and Key Management Protocol ISAKMP The common framework for establishing the format of SA attributes, and for negotiating, modifying, and deleting SAs. ISAKMP is the IETF standard for handling IPsec SAs. Figure 19–1 shows how an IP addressed packet, as part of an IP datagram, proceeds when IPsec has been invoked on an outbound packet. The flow diagram illustrates where authentication header (AH) and encapsulating security payload (ESP) entities can be applied to the packet. How to apply these entities, as well as how to choose the algorithms, are described in subsequent sections.Figure 19–2 shows the IPsec inbound process.

Flow diagram shows that the outbound packet is first protected by ESP, and then by AH. The packet then goes to a tunnel or a physical interface.

Flow diagram shows that IPsec first processes the AH header, then the ESP header on inbound packets. A packet that is not protected enough is dropped.

An IPsec security association (SA) specifies security properties that are recognized by communicating hosts. A single SA protects data in one direction. The protection is either to a single host or to a group (multicast) address. Because most communication is either peer-to-peer or client-server, two SAs must be present to secure traffic in both directions.The following three elements uniquely identify an IPsec SA:The security protocol (AH or ESP) The destination IP address The security parameter index (SPI) The SPI, an arbitrary 32-bit value, is transmitted with an AH or ESP packet. The ipsecah7P and ipsecesp7P man pages explain the extent of protection that is provided by AH and ESP. An integrity checksum value is used to authenticate a packet. If the authentication fails, the packet is dropped.Security associations are stored in a security associations database (SADB). A socket-based administration engine, the pf_key interface, enables privileged applications to manage the database.For a more complete description of the IPsec SADB, see Security Associations Database for IPsec. For more information about how to manage the SADB, see the pf_key7P man page. Security associations (SAs) require material to create the keys for authentication and for encryption. The managing of this keying material is called key management. The Internet Key Exchange (IKE) protocol handles key management automatically. You can also manage keys manually with the ipseckey command. SAs on IPv4 and IPv6 packets can use either method of key management. Unless you have an overriding reason to use manual key management, automatic key management is preferred. For example, to interoperate with systems other than Solaris systems might require manual key management.The in.iked daemon provides automatic key management. For a description of IKE, see Chapter 22, Internet Key Exchange (Overview). For more information on the in.iked daemon, see the in.iked1M man page. The ipseckey command provides manual key management. For a description of the command, see Utilities for Key Generation in IPsec. For a detailed description of the ipseckey command options, see the ipseckey1M man page. IPsec provides two security protocols for protecting data:Authentication Header (AH) Encapsulating Security Payload (ESP) An AH protects data with an authentication algorithm. An ESP protects data with an encryption algorithm. Optionally, an ESP protects data with an authentication algorithm. Each implementation of an algorithm is called a mechanism.The authentication header provides data authentication, strong integrity, and replay protection to IP datagrams. AH protects the greater part of the IP datagram. As the following illustration shows, AH is inserted between the IP header and the transport header. Diagram shows the AH header between the IP header and the TCP header. The transport header can be TCP, UDP, SCTP, or ICMP. If a tunnel is being used, the transport header can be another IP header. The encapsulating security payload (ESP) module provides confidentiality over what the ESP encapsulates. ESP also provides the services that AH provides. However, ESP only provides its protections over the part of the datagram that ESP encapsulates. The authentication services of ESP are optional. These services enable you to use ESP and AH together on the same datagram without redundancy. Because ESP uses encryption-enabling technology, ESP must conform to U.S. export control laws.ESP encapsulates its data, so ESP only protects the data that follows its beginning in the datagram, as shown in the following illustration. Diagram shows the ESP header between the IP header and the TCP header. The TCP header is encrypted by the ESP header. In a TCP packet, ESP encapsulates only the TCP header and its data. If the packet is an IP-in-IP datagram, ESP protects the inner IP datagram. Per-socket policy allows self-encapsulation, so ESP can encapsulate IP options when ESP needs to.If self-encapsulation is set, a copy of the IP header is made to construct an IP-in-IP datagram. For example, when self-encapsulation is not set on a TCP socket, the datagram is sent in the following format:[ IP(a -> b) options + TCP + data ]When self-encapsulation is set on that TCP socket, the datagram is sent in the following format:[ IP(a -> b) + ESP [ IP(a -> b) options + TCP + data ] ]For further discussion, see Transport and Tunnel Modes in IPsec.The following table compares the protections that are provided by AH and ESP.Protocol Packet Coverage Protection Against Attacks AH Protects packet from the IP header to the transport header Provides strong integrity, data authentication:Ensures that the receiver receives exactly what the sender sent Is susceptible to replay attacks when an AH does not enable replay protection Replay, cut-and-paste ESP Protects packet following the beginning of ESP in the datagram. With encryption option, encrypts the IP datagram. Ensures confidentiality Eavesdropping With authentication option, provides the same protection as AH Replay, cut-and-paste With both options, provides strong integrity, data authentication, and confidentiality Replay, cut-and-paste, eavesdropping IPsec security protocols use two types of algorithms, authentication and encryption. The AH module uses authentication algorithms. The ESP module can use encryption as well as authentication algorithms. You can obtain a list of the algorithms on your system and their properties by using the ipsecalgs command. For more information, see the ipsecalgs1M man page. You can also use the functions that are described in the getipsecalgbyname3NSL man page to retrieve the properties of algorithms.IPsec on a Solaris system uses the Solaris cryptographic framework to access the algorithms. The framework provides a central repository for algorithms, in addition to other services. The framework enables IPsec to take advantage of high performance cryptographic hardware accelerators. The framework also provides resource control features. For example, the framework enables you to limit the amount of CPU time spent in cryptographic operations in the kernel. For more information, see the following:Chapter 12, Solaris Cryptographic Framework (Overview), in System Administration Guide: Security Services Chapter 8, Introduction to the Solaris Cryptographic Framework, in Solaris Security for Developers Guide Authentication algorithms produce an integrity checksum value or digest that is based on the data and a key. The AH module uses authentication algorithms. The ESP module can use authentication algorithms as well. Encryption algorithms encrypt data with a key. The ESP module in IPsec uses encryption algorithms. The algorithms operate on data in units of a block size. By default, the DES-CBC, 3DES-CBC, AES-CBC, and Blowfish-CBC algorithms are installed. The key sizes that are supported by the AES-CBC and Blowfish-CBC algorithms are limited to 128 bits.AES-CBC and Blowfish-CBC algorithms that support key sizes that are greater than 128 bits are available to IPsec when you install the Solaris Encryption Kit. However, not all encryption algorithms are available outside of the United States. The kit is available on a separate CD that is not part of the Solaris 10 installation box. The Solaris 10 Encryption Kit Installation Guide describes how to install the kit. For more information, see the Sun Downloads web site. To download the kit, click the Downloads A-Z tab, then click the letter S. The Solaris 10 Encryption Kit is among the first twenty entries. IPsec protection policies can use any of the security mechanisms. IPsec policies can be applied at the following levels:On a system-wide level On a per-socket level IPsec applies the system-wide policy to outbound datagrams and inbound datagrams. Outbound datagrams are either sent with protection or without protection. If protection is applied, the algorithms are either specific or non-specific. You can apply some additional rules to outbound datagrams, because of the additional data that is known by the system. Inbound datagrams can be either accepted or dropped. The decision to drop or accept an inbound datagram is based on several criteria, which sometimes overlap or conflict. Conflicts are resolved by determining which rule is parsed first. The traffic is automatically accepted, except when a policy entry states that traffic should bypass all other policies.The policy that normally protects a datagram can be bypassed. You can either specify an exception in the system-wide policy, or you can request a bypass in the per-socket policy. For traffic within a system, policies are enforced, but actual security mechanisms are not applied. Instead, the outbound policy on an intra-system packet translates into an inbound packet that has had those mechanisms applied.You use the ipsecinit.conf file and the ipsecconf command to configure IPsec policies. For details and examples, see the ipsecconf1M man page. The IPsec standards define two distinct modes of IPsec operation, transport mode and tunnel mode. The modes do not affect the encoding of packets. The packets are protected by AH, ESP, or both in each mode. The modes differ in policy application when the inner packet is an IP packet.In transport mode, the outer header determines the IPsec policy that protects the inner IP packet. In tunnel mode, the inner IP packet determines the IPsec policy that protects its contents. In transport mode, the outer header, the next header, and any ports that the next header supports, can be used to determine IPsec policy. In effect, IPsec can enforce different transport mode policies between two IP addresses to the granularity of a single port. For example, if the next header is TCP, which supports ports, IPsec policy can be set for a TCP port of the outer IP address. Similarly, if the next header is an IP header, the outer header and the inner IP header can be used to determine IPsec policy.Tunnel mode works only on IP-in-IP datagrams. Tunneling in tunnel mode can be useful when computer workers at home are connecting to a central computer location. In tunnel mode, IPsec policy is enforced on the contents of the inner IP datagram. Different IPsec policies can be enforced for different inner IP addresses. That is, the inner IP header, its next header, and the ports that the next header supports, can enforce a policy. Unlike transport mode, in tunnel mode the outer IP header does not dictate the policy of its inner IP datagram.Therefore, in tunnel mode, IPsec policy can be specified for subnets of a LAN behind a router and can be specified for ports on those subnets. IPsec policy can also be specified for particular IP addresses, that is, hosts, on those subnets. The ports of those hosts can also have a specific IPsec policy. However, if a dynamic routing protocol is run over a tunnel, neither subnet selection nor address selection should be used, because the view of the network topology on the peer network could change. Changes would invalidate the static IPsec policy. For examples of tunneling procedures that include configuring static routes, see Protecting a VPN With IPsec.In Solaris, tunnel mode can be enforced only on an IP tunneling network interface. The ipsecconf command provides a tunnel keyword to select an IP tunneling network interface. When the tunnel keyword is present in a rule, all selectors that are specified in that rule apply to the inner packet.In transport mode, ESP, AH, or both ESP and AH, can protect the datagram.Figure 19–3 shows an IP header with an unprotected TCP packet.

Diagram shows the IP header followed by the TCP header. The TCP header is not protected.

In transport mode, ESP protects the data as shown in Figure 19–4.

Diagram shows the ESP header between the IP header and the TCP header. The TCP header is encrypted by the ESP header.

In transport mode, AH protects the data as shown in Figure 19–5.

Diagram shows the AH header between the IP header and the TCP header.

AH actually covers the data before the data appears in the datagram. Consequently, the protection that is provided by AH, even in transport mode, covers some of the IP header.In tunnel mode, the entire datagram is inside the protection of an IPsec header. The datagram in Figure 19–3 is protected in tunnel mode by an outer IPsec header, and in this case ESP, as is shown in Figure 19–6.

Diagram shows the ESP header after the IP header and before an IP header and a TCP header. The last 2 headers are protected by encryption.

The ipsecconf command includes keywords to set tunnels in tunnel mode or in transport mode.For details on per-socket policy, see the ipsec7P man page. For an example of per-socket policy, see How to Secure a Web Server With IPsec. For more information about tunnels, see the ipsecconf1M man page. For an example of tunnel configuration, see How to Protect a VPN With an IPsec Tunnel in Tunnel Mode Over IPv4. A configured tunnel is a point-to-point interface. The tunnel enables one IP packet to be encapsulated within another IP packet. A correctly configured tunnel requires both a tunnel source and a tunnel destination. For more information, see the tun7M man page and Configuring Tunnels for IPv6 Support. A tunnel creates an apparent physical interface to IP. The physical link's integrity depends on the underlying security protocols. If you set up the security associations (SAs) securely, then you can trust the tunnel. Packets that exit the tunnel must have originated from the peer that was specified in the tunnel destination. If this trust exists, you can use per-interface IP forwarding to create a virtual private network (VPN).You can use IPsec to construct a VPN. IPsec secures the connection. For example, an organization that uses VPN technology to connect offices with separate networks can deploy IPsec to secure traffic between the two offices.The following figure illustrates how two offices use the Internet to form their VPN with IPsec deployed on their network systems.

Diagram shows that Offices 1 and 2 use the hme0 interface to communicate with each other. Each office uses hme1 for internal communication.

For a detailed example of the setup procedure, see How to Protect a VPN With an IPsec Tunnel in Tunnel Mode Over IPv4. For IPv6 networks, see How to Protect a VPN With an IPsec Tunnel in Tunnel Mode Over IPv6. IKE can negotiate IPsec SAs across a NAT box. This ability enables systems to securely connect from a remote network, even when the systems are behind a NAT device. For example, employees who work from home, or who log on from a conference site can protect their traffic with IPsec.NAT stands for network address translation. A NAT box is used to translate a private internal address into a unique Internet address. NATs are very common at public access points to the Internet, such as hotels. For a fuller discussion, see Using Solaris IP Filter's NAT Feature.The ability to use IKE when a NAT box is between communicating systems is called NAT traversal, or NAT-T. In the Solaris 10 release, NAT-T has the following limitations:NAT-T works on IPv4 networks only. NAT-T cannot take advantage of the IPsec ESP acceleration provided by the Sun Crypto Accelerator 4000 board. However, IKE acceleration with the Sun Crypto Accelerator 4000 board works. The AH protocol depends on an unchanging IP header, therefore AH cannot work with NAT-T. The ESP protocol is used with NAT-T. The NAT box does not use special processing rules. A NAT box with special IPsec processing rules might interfere with the implementation of NAT-T. NAT-T works only when the IKE initiator is the system behind the NAT box. An IKE responder cannot be behind a NAT box unless the box has been programmed to forward IKE packets to the appropriate individual system behind the box. The following RFCs describe NAT functionality and the limits of NAT-T. Copies of the RFCs can be retrieved from http://www.rfc-editor.org.RFC 3022, “Traditional IP Network Address Translator (Traditional NAT),” January 2001 RFC 3715, “IPsec-Network Address Translation (NAT) Compatibility Requirements,” March 2004 RFC 3947, “Negotiation of NAT-Traversal in the IKE,” January 2005 RFC 3948, “UDP Encapsulation of IPsec Packets,” January 2005 To use IPsec across a NAT, see Configuring IKE for Mobile Systems (Task Map). The Solaris 10 release supports the Streams Control Transmission Protocol (SCTP). The use of the SCTP protocol and SCTP port number to specify IPsec policy is supported, but is not robust. The IPsec extensions for SCTP as specified in RFC 3554 are not yet implemented. These limitations can create complications in creating IPsec policy for SCTP.SCTP can make use of multiple source and destination addresses in the context of a single SCTP association. When IPsec policy is applied to a single source or a single destination address, communication can fail when SCTP switches the source or the destination address of that association. IPsec policy only recognizes the original address. For information about SCTP, read the RFCs and SCTP Protocol. IPsec is configured from the global zone. The IPsec policy configuration file, ipsecinit.conf, exists in the global zone only. The file can have entries that apply to non-global zones, as well as entries that apply to the global zone. For information on how to use IPsec with zones, see Protecting Traffic With IPsec. For information about zones, see Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Solaris Containers-Resource Management and Solaris Zones. Table 19–3 describes the files and commands that are used to configure and manage IPsec. For completeness, the table includes key management files and commands. For instructions on implementing IPsec on your network, see Protecting Traffic With IPsec (Task Map). For more details about IPsec utilities and files, see Chapter 21, IP Security Architecture (Reference). IPsec Utility or File Description Man Page /etc/inet/ipsecinit.conf file IPsec policy file. If this file exists, IPsec is activated at boot time. ipsecconf1M ipsecconf command IPsec policy command. The boot scripts use ipsecconf to read the /etc/inet/ipsecinit.conf file and activate IPsec. Useful for viewing and modifying the current IPsec policy, and for testing. ipsecconf1M PF_KEY socket interface Interface for security associations database (SADB). Handles manual key management and automatic key management. pf_key7P ipseckey command IPsec security associations (SAs) keying command. ipseckey is a command-line front end to the PF_KEY interface. ipseckey can create, destroy, or modify SAs. ipseckey1M /etc/inet/secret/ipseckeys file Keys for IPsec SAs. If the ipsecinit.conf file exists, the ipseckeys file is automatically read at boot time. ipsecalgs command IPsec algorithms command. Useful for viewing and modifying the list of IPsec algorithms and their properties. ipsecalgs1M /etc/inet/ipsecalgs file Contains the configured IPsec protocols and algorithm definitions. This file is managed by the ipsecalgs utility and must never be edited manually. /etc/inet/ike/config file IKE configuration and policy file. If this file exists, the IKE daemon, in.iked, provides automatic key management. The management is based on rules and global parameters in the /etc/inet/ike/config file. See IKE Utilities and Files. ike.config4 For a complete listing of new Solaris features and a description of Solaris releases, see Solaris Express, Developer Edition What’s New. Since the Solaris 9 release, IPsec includes the following functionality:When a Sun Crypto Accelerator 4000 board is attached, the board automatically caches IPsec SAs for packets that use the board's Ethernet interface. The board also accelerates the processing of the IPsec SAs. IPsec can take advantage of automatic key management with IKE over IPv6 networks. For more information, see Chapter 22, Internet Key Exchange (Overview).For new IKE features, see Changes to IKE for the Solaris 10 Release. The parser for theipseckey command provides clearer help. The ipseckey monitor command timestamps each event. For details, see the ipseckey1M man page. IPsec algorithms now come from a central storage location, the Solaris cryptographic framework. The ipsecalgs1M man page describes the characteristics of the algorithms that are available. The algorithms are optimized for the architecture that they run on. For a description of the framework, see Chapter 12, Solaris Cryptographic Framework (Overview), in System Administration Guide: Security Services. IPsec works in the global zone. IPsec policy is managed in the global zone for a non-global zone. Keying material is created and is managed manually in the global zone for a non-global zone. IKE cannot be used to generate keys for a non-global zone. For more information on zones, see Chapter 16, Introduction to Solaris Zones, in System Administration Guide: Solaris Containers-Resource Management and Solaris Zones. IPsec policy can work with the Streams Control Transmission Protocol (SCTP) and SCTP port number. However, the implementation is not complete. The IPsec extensions for SCTP that are specified in RFC 3554 are not yet implemented. These limitations can cause complications when creating IPsec policy for SCTP. For details, consult the RFCs. Also, read IPsec and SCTP and SCTP Protocol. IPsec and IKE can protect traffic that originates behind a NAT box. For details and limitations, see IPsec and NAT Traversal. For procedures, see Configuring IKE for Mobile Systems (Task Map).