What is IPsec? How it works and how to connect, explained

As remote work spreads, more and more organizations use VPNs (virtual private networks). A VPN is a virtual dedicated line that only specific people and devices can access, so you can build a secure network. This article explains IPsec, one of the protocols used to build internet VPNs.

What is IPsec?

IPsec stands for “Security Architecture for Internet Protocol” or “Internet Protocol Security.” It is mainly used to build internet VPNs. It is a security architecture for communication over the Internet Protocol, and it uses various protocols to prevent information leaks and tampering. Because the traffic is encrypted, no one can read it even if it is intercepted.

Internet VPNs are divided into several types depending on which protocol they use and which layer they protect. IPsec is one of these types.

Protocols used for internet VPNs

• IPsec-VPN
• SSL-VPN
• L2TP/IPsec
• PPTP

The protocols other than IPsec used to build internet VPNs are explained in the following article.

Protocols used for internet VPNs (Japanese)

Examples of IPsec use

One example is connecting a company’s sites with IPsec to build an internal network that spans the sites. IPsec is also used when employees connect to the company network from outside the office to use internal file servers. In this case, the employee’s client device needs to authenticate with dedicated software to connect from outside.

How IPsec works

IPsec encrypts at the network layer (Layer 3) of the OSI reference model. Even if the upper layers, such as the transport layer and application layer, do not support encryption, the lower network layer does, so communication is more secure. On the other hand, note that IPsec has the drawback of not working well with firewalls.

OSI reference model

	Name
Layer 7	Application layer
Layer 6	Presentation layer
Layer 5	Session layer
Layer 4	Transport layer
Layer 3	Network layer
Layer 2	Data link layer
Layer 1	Physical layer

VPN communication over IPsec works roughly as follows.

First, a communication path called an SA (Security Association) is established between the sender and the receiver (tunneling). One SA cannot carry traffic in both directions, so two SAs, one for sending and one for receiving, are needed for two-way communication.

IKE (Internet Key Exchange Protocol) is needed to set up the SA. Here we explain the common procedure that uses a pre-shared key. A pre-shared key (PSK) is configured in advance on the routers that use IPsec, and the routers authenticate each other with the PSK. The connected routers then exchange keys for encrypting data, and the keys are generated automatically. If a key leaks, the traffic can be intercepted, so it is important that keys do not leak. Keys therefore have an expiration time. When a key expires, a new key is created to replace it, which keeps key management secure.

Once the SA is established, the AH or ESP protocol carries the actual data and prevents information leaks and tampering. ESP (Encapsulated Security Payload) encrypts the data and checks its integrity (authentication), which prevents both leaks and tampering. AH (Authentication Header) only checks integrity (authentication). AH does not encrypt, so it is less secure, and ESP is used in almost all cases today.*

*Some jurisdictions prohibit encrypted data communication, so ESP may be subject to import and export controls. AH is provided so that IPsec authentication can still be used in such cases.

IPsec connection modes

IPsec has two modes of operation: transport mode and tunnel mode.

1. Transport mode

The first mode is transport mode. Data on the internet is sent and received in units called packets. Each packet has a header and a data part (payload).

Transport mode encrypts only the data part (payload) and does not encrypt the header.

2. Tunnel mode

Tunnel mode, on the other hand, encrypts both the header and the data part. More precisely, it encrypts the whole packet, including the original header, and adds a new header (this is called encapsulation).
Transport mode can be used when both the sender and the receiver are endpoints, such as in host-to-host communication. For this reason, VPNs between sites almost always use tunnel mode, which connects routers to each other or a router to a server, for example to link a head office and branch offices.
However, in tunnel mode the header is also encrypted, so when combined with a firewall, traffic may not pass through the firewall flexibly. This is why IPsec is generally said to work less well with firewalls than other protocols.

IPsec types compared

As described above, IPsec uses AH or ESP to prevent information leaks and tampering, and it has two modes of operation: transport mode and tunnel mode. The most commonly used combination is ESP with transport.

	Tunnel mode	Transport mode
ESP	Most commonly used	Limited use cases
AH	No encryption, less secure	No encryption, less secure Limited use cases

IPsec comes in many types and uses various protocols. This processing takes place in the router, so to build a VPN with IPsec, you need a router that supports IPsec.

IPv6, which is gradually spreading, includes IPsec as standard. However, most networks still use IPv4, where IPsec is optional rather than standard, so you still need to check IPsec support.

Technologies used for IPsec over mobile networks

We have explained how IPsec works and how to connect. IPsec over a wireless mobile network, rather than a wired connection such as a LAN cable, needs some different technologies. They are explained below.

About Aggressive Mode

There are two negotiation modes in IKE Phase 1: Main Mode and Aggressive Mode. Main Mode assumes that the peer is identified by its IP address during authentication. Aggressive Mode identifies the peer by its own ID and the peer’s ID, not by IP address. When a company connects multiple sites with a VPN, each site can have a static IP address, so Main Mode is usually used. On the other hand, when a portable PC accesses a private network through a public hotspot or mobile network, it is hard to fix the IP address, so Aggressive Mode is used.

About NAT traversal

In a typical IPsec connection, data is sent and received with the ESP protocol after IKE establishes the SA. In the OSI reference model, ESP is a signal on the same layer as TCP and UDP, the transport layer (Layer 4), directly above the network layer (Layer 3). It is not a protocol that runs on top of TCP or UDP (like SSL or HTTP), and unlike TCP and UDP, it has no concept of port numbers. However, typical corporate and carrier NAT uses TCP and UDP port numbers for processing, so ESP traffic often cannot pass through NAT.

To allow IPsec tunnels in NAT environments, data can be sent and received over UDP instead of ESP. This is called NAT traversal (NAT-T). UDP port 4500 is normally used in place of ESP.

OSI reference model and protocols

	Name	Example protocols
Layer 7	Application layer	HTTP FTP SSL
Layer 6	Presentation layer
Layer 5	Session layer
Layer 4	Transport layer	TCP UDP ESP
Layer 3	Network layer	IP ICMP IPsec
Layer 2	Data link layer	ARP RARP PPP
Layer 1	Physical layer	Hardware

IPsec and SSL-VPN compared

SSL-VPN is a VPN method often compared with IPsec. SSL-VPN uses SSL technology for encryption. SSL stands for “Secure Sockets Layer,” a protocol widely used on websites and elsewhere.

SSL-VPN uses SSL encrypted communication, an encryption and authentication mechanism at the session layer (Layer 5) of TCP/IP and the OSI reference model. Because it encrypts at a higher layer than IPsec, it is somewhat less secure. However, it is good for remote access and low in cost, so it is widely used for access from outside the office and for remote work.

IPsec also requires VPN software on the client PC, while SSL needs no special software and works with just a web browser. This convenience is one reason people choose SSL-VPN. In addition, unlike IPsec (transport mode), the header is not encrypted, so firewalls can be applied flexibly.

On the other hand, some applications are more likely not to work with SSL-VPN than with IPsec. This is because IPsec runs at the lower network layer, while SSL runs at the higher session layer.

The table below compares IPsec and SSL-VPN.

	IPsec	SSL-VPN
Layer	Network layer (Layer 3)	Session layer (Layer 5)
Cost	High	Low
Convenience	Low	High
Security	High	Low

L2TP, used together with IPsec

L2TP is another VPN method that often comes up, but it is almost always used together with IPsec. L2TP alone has no encryption, so IPsec is used to make it secure.

L2TP stands for Layer 2 Tunneling Protocol. It is a protocol for tunneling, which builds a virtual tunnel between the sender and the receiver at the data link layer (Layer 2). It makes VPN connections by establishing PPP (Point-to-Point Protocol) connections. Specifically, the sender encapsulates PPP frames and the receiver decapsulates them. L2TP has no encryption of its own, so it is often used with IPsec for security and is often written as “L2TP/IPsec.”

One advantage of L2TP is that a single VPN tunnel can carry multiple user sessions. Another is that it supports many protocols. L2TP can use protocols such as UDP/IP, ATM, and Frame Relay (FR).

As this shows, IPsec is sometimes combined with other methods to build a VPN.

Summary

IPsec is a technology widely used in VPN connections to keep communication secure. With the growth of remote work, the security and confidentiality of internet communication will become even more important, and IPsec will continue to be used to keep the internet secure. Understand it correctly and use it to build a secure network.

amnimo LTE devices with IPsec support

All amnimo LTE devices, such as our gateways and routers (except the portable cameras), implement IPsec. amnimo devices have been connected over IPsec to Cisco and Yamaha routers. These LTE devices also offer dual firmware, multiple SIM cards, and protection against momentary power interruptions, so they are well suited to building VPNs that need high stability. If you are planning a VPN with IPsec, consider amnimo LTE devices.

You can download the general catalog, which covers all amnimo LTE devices, below.