Disclaimer: This is an example of a student written essay.
Click here for sample essays written by our professional writers.

Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of UKEssays.com.

Ipsec implementation

Paper Type: Free Essay Subject: Information Technology
Wordcount: 5409 words Published: 18th Jun 2015

Reference this

There are many methods to IP Security. In the Microsoft’s Windows 2003 the server version, there are many such techniques and tweaks which are useful to help provide a secure base to the system. The operating system is very good in protecting the system from attacks which may be active attacks or in some cases passive attacks. The procedures for IPSec are placed properly into the system which helps it tackle such attacks. This is possible by providing a secure packet filter for packet transport and also using cryptography. This technology is very in high use for communication types like host to host, router to router, gateway to gateway, site to site and also in virtual private networks. Some other places for successful implementation are secure servers. The IPSec comes as a basic Group Policy controlled by Dos Prompt commands and it also has user interface with custom programs installed.

The implementation of IPSec involves these steps:

  • Overview of IPSec Deployment
  • Determining Our Internet Protocol Security targets
  • Preparing a Internet Protocol Security Policy
  • Implementation of the designed policies

Overview of IPSec Deployment:

There are many techniques such as usage of cryptography and authentication software to secure the communication in a network. The connection may be between two parties or between a group of users. Security is to make sure that the communication is not broken, the communication is not intervened and the data is not altered. The Internet Protocol Security has some features which help running a secure transmission. IPSec has set procedures which help achieve these ends.

Get Help With Your Essay

If you need assistance with writing your essay, our professional essay writing service is here to help!

Essay Writing Service

Today there are many companies and it is getting harder day by day to ensure the security of such humungous networks with a heavy workload on the networks. It is also very hard to track any attacks on the network in millions and millions of request. Using firewalls to protect a network did work for some days but it has proved to be impractical as it has no proper rules to detect intrusions or attacks. The advent of IPSec is a leap in computer and network security. This protocol has a wide spectrum of features which are strong enough to contain such high volume of network requests and also handle intrusions.

The Internet Protocol for Security is not a versatile detection protocol. It is capable of managing most of the policies to grant or reject, block and negotiate the traffic in a network. This can also be permitted to a specific set of addresses or in some cases protocol and also a different policy to each port. It is recommended that we use ICF (Internet Connection Firewall) when we are in need of a firewall which is capable of providing a network interface for very big networks. This is because the internet protocol for Security has a very strict and a very stern strategy which is based on static filtering based on IP addresses. But this is it is totally different in the case of the Internet Connection Firewall. The ICF has policies which has a set of filter for all the addresses which are capable of being accessed. The Internet Protocol for Security can be used when the restriction is only to a particular set of addresses or the communication between a group of computers.

There are many ways to use IPSec in a network but the best way is bye using a directory with all the domains and also a GP when needed.

Some areas of interest when implementing IPSecurity:

The decision of where we must secure computers and how on our network which can be done by running a group of systems in a directory which is also called as the Active Directory Organisational Units or OUs. The next step is to determine the strength of the policies we assign.

Determining Our Internet Protocol Security targets:

The first step in deploying IPSec on our server or the network can be done properly by deciding which set of systems are in dire need of security. There are surely some places on the network which are in a need of higher security than the other sections. It is for sure that IPSec is capable of providing optimal security but the problems starts when the network slows down due to the excessive data to be processed and also a large number of systems for IPSec to follow and maintain. In some cases, there are systems which are not upgraded to be able to support the IPSecurity.

Determining Your IPsec Needs

In beginning of the protocol design procedure, proper planning should be done to make sure that our current network environment available for use. It is always helpful to have a set of the network topology with all of its hardware and software components. This procedure is of high importance mainly in the designing procedure. IPSecurity is highly vulnerable to a network topology. There are many network topologies in which IPSec is not well suited.

Preparing a Internet Protocol Security Policy:

As said earlier, there are many network topologies which are not suitable to the default structure of the IPSec policies. There is a need to develop a custom IPSec set of policies well suited for the current network. Some organisations can run their network with a small set of policies. But in companies with a very huge network, there are many policies which are to be implemented properly and also a stringent structure is to be maintained.

Steps below shows how IP Sec policies work.

Designing IPsec Policies

There might be some cases where the company or the organisation is interested in implementing a policy which sets a secure communication between two specified computers. This can be done by restricting all traffic and adding exceptions which relates to these two systems. This method can be done vice versa. A network can be setup with policies to allow all requests and block specific ports or computers. To implement such exceptions, a thorough analysis of the network is clearly needed.

Computer roles:

Security for Data Transmission: Security needs are different from each and every data packet transmitted. The security policies are also very different. There are many levels in this instance. When considering encryption, there are many types such as AES, DE5, RSA and many more. RSA is the best encryption software available presently. These programs can be used to secure files on transmission, on a network or even in the system.

Operating System – Computers: IPSecurity is a very unique technique used to implement security in a network. There are many operating systems which are not so advanced to implement IPSec. There is no support for IPSec. But there are many operating systems which are capable of running IPSec in transport mode. Some other cases, the policies are stored locally which makes it easier to decide as the implementation doesn’t take long enough. In some cases, IPSec policies are implemented through the Group Policy.

General IPSec Policy Settings

General IPSec policy settings must be specified whether we want the policy to provide packet filtering or end-to-end networks.

IPSec Rules

IPSec rules determine which traffic is affected by an IPSec policy and which actions take place when that type of traffic is encountered. Table6.5 describes the contents of IPSec rules that two computers use to establish a secure, authenticated channel.

Specifies a named list of filters. Each filter in the filter list specifies the types of traffic to which the filter action is applied. Filters can be defined to match specific IP protocols, source and destination TCP and UDP ports, and source and destination IP addresses.

The filter list name might include the version number, the last update time, and the administrative owner. Each computer discards the filter list name during policy processing.

Filter action

Specifies whether a packet is permitted, blocked, or secured. If packets are to be secured, specifies how they are secured. A list of security methods specifies the security protocol, cryptographic algorithm, and session key regeneration frequency.

Request Security

Authentication methods

One or more authentication methods, which are specified in order of preference. Available options are KerberosV5, certificate, or preshared key.

Specifies whether to use tunnel mode and, if so, the tunnel’s endpoint.

Specifies whether the rule applies to LAN connections, remote access connections, or both.

Assigning IPSec Policies:

As a domain administrator, we can configure IPSec policies to meet the security requirements of a user, group, application, domain, site, or global enterprise from a domain controller. IPSec policy can also be implemented in a non-Windows2000-based domain environment by using local IPSec policies.

Deploying Our IPSec Solution:

After scoping our needs, building IPSec policies, and determining our strategy for assigning the policies to specific OUs, test the IPSec policies in a lab environment and conduct a pilot project before rolling them out for production use.

To ensure that IPSec policy functions as expected and provides the appropriate level of security, test specific IPSec policy configurations on clients and servers in a lab environment, and then conduct pilot or beta tests in a limited operational environment before conducting a full-scale deployment.

A Cryptographic Evaluation of IPsec:

Even with all the serious critisisms that we have on IPsec, it is probably

the best IP security protocol available at the moment. We have looked at other,

functionally similar, protocols in the past (including PPTP [SM98, SM99]) in

much the same manner as we have looked at IPsec. None of these protocols

come anywhere near their target, but the others manage to miss the mark by a

wider margin than IPsec. This difference is less significant from a security point

of view; there are no points for getting security nearly right. From a marketing

point of view, this is important. IPsec is the current best practice,” no matter

how badly that reects on our ability to create a good security standard.

Our main criticism of IPsec is its complexity. IPsec contains too many options

and too much exibility; there are often several ways of doing the same or similar

things. This is a typical committee efiect. Committees are notorious for adding

features, options, and additional exibility to satisfy various factions within the

committee. As we all know, this additional complexity and bloat is seriously

detrimental to a normal (functional) standard. However, it has a devastating

efiect on a security standard.

It is instructive to compare this to the approach taken by NIST for the development

of AES [NIST97a, NIST97b]. Instead of a committee, NIST organized a

contest. Several small groups each created their own proposal, and the process is limited to picking one of them. At the time of writing there has been one stage

of elimination, and any one of the five remaining candidates will make a much better standard than any committee could ever have made.

The Complexity Trap: Security’s worst enemy is complexity.

Complexity of IPsec In our opinion, IPsec is too complex to be secure. The

design obviously tries to support many difierent situations with difierent options.

We feel very strongly that the resulting system is well beyond the level of

complexity that can be analysed or properly implemented with current methodologies.

Thus, no IPsec system will achieve the goal of providing a high level of


IPsec has two modes of operation: transport mode and tunnel mode. There

are two protocols: AH and ESP. AH provides authentication, ESP provides authentication,

encryption, or both. This creates a lot of extra complexity: two

machines that wish to authenticate a packet can use a total of four difierent

modes: transport/AH, tunnel/AH, transport/ESP with NULL encryption, and

tunnel/ESP with NULL encryption. The difierences between these options, both

in functionality and performance, are minor. The documentation also makes it

clear that under some circumstances it is envisioned to use two protocols: AH

for the authentication and ESP for the encryption.

Modes As far as we can determine, the functionality of tunnel mode is a superset

of the functionality of transport mode. (From a network point of view, one

can view tunnel mode as a special case of transport mode, but from a security

point of view this is not the case.) The only advantage that we can see to transport

mode is that it results in a somewhat smaller bandwidth overhead. However,

the tunnel mode could be extended in a straightforward way with a specialized

header-compression scheme that we will explain shortly. This would achieve virtually

the same performance as transport mode without introducing an entirely

new mode. We therefore recommend that transport mode be eliminated.

Recommendation 1 Eliminate transport mode.

Without any documented rationale, we do not know why IPsec has two

modes. In our opinion it would require a very compelling argument to introduce

a second major mode of operation. The extra cost of a second mode (in

terms of added complexity and resulting loss of security) is huge, and it certainly

should not be introduced without clearly documented reasons.

Eliminating transport mode also eliminates the need to separate the machines

on the network into the two categories of hosts and security gateways. The main

distinction seems to be that security gateways may not use transport mode;

without transport mode the distinction is no longer necessary.

Protocols The functionality provided by the two protocols overlaps somewhat.

AH provides authentication of the payload and the packet header, while ESP

provides authentication and confidentiality of the payload.

In transport mode, AH provides a stronger authentication than ESP can provide,

as it also authenticates the IP header fields. One of the standard modes of

operation would seem to be to use both AH and ESP in transport mode. In tunnel

mode, ESP provides the same level of authentication (as the payload includes

the original IP header), and AH is typically not combined with ESP [KA98c,

section 4.5]. (Implementations are not required to support nested tunnels that

would allow ESP and AH to both be used in tunnel mode.)

One can question why the IP header fields are being authenticated at all. The

authentication of the payload proves that it came from someone who knows the

proper authentication key. That by itself should provide adequate information.

The IP header fields are only used to get the data to the recipient, and should

not afiect the interpretation of the packet. There might be a very good reason

why the IP header fields need to be authenticated, but until somebody provides

that reason the rationale remains unclear to us.

The AH protocol [KA98a] authenticates the IP headers of the loour layers.

This is a clear violation of the modularization of the protocol stack. It creates

all kind of problems, as some header fields change in transit. As a result, the

AH protocol needs to be aware of all data formats used at loour layers so that

these mutable fields can be avoided. This is a very ugly construction, and one

that will create more problems when future extensions to the IP protocol are

made that create new fields that the AH protocol is not aware of. Also, as some

header fields are not authenticated, the receiving application still cannot rely on

the entire packet. To fully understand the authentication provided by AH, an

application needs to take into account the same complex IP header parsing rules

that AH uses. The complex definition of the functionality that AH provides can

easily lead to security-relevant errors.

The tunnel/ESP authentication avoids this problem, but uses more bandwidth.

The extra bandwidth requirement can be reduced by a simple specialized

compression scheme: for some suitably chosen set of IP header fields X, a single

bit in the ESP header indicates whether the X fields in the inner IP header are

identical to the corresponding fields in the outer header.2 The fields in question

are then removed to reduce the payload size. This compression should be

applied after computing the authentication but before any encryption. The authentication

is thus still computed on the entire original packet. The receiver

reconstitutes the original packet using the outer header fields, and verifies the

authentication. A suitable choice of the set of header fields X allows tunnel/ESP

to achieve virtually the same low message expansion as transport/AH.

We conclude that eliminating transport mode allows the elimination of the

AH protocol as well, without loss of functionality. We therefore recommend that

the AH protocol be eliminated.

IPSEC methodology using different operating systems:

IPSEC is a framework for security that operates at the Network Layer by extending the IP packet header. This gives it the ability to encrypt any higher layer protocol, including TCP and UDP sessions, so it offers the greatest flexibility of all the existing TCP/IP cryptosystems. While conceptually simple, setting up IPSEC is much more complex that installing SSH, for example.

IPSEC also has the disadvantage of requiring operating system support, since most O/S kernels don’t allow direct manipulation of IP headers. Linux IPSEC support (the FreeS/WAN project), for example, isn’t included in the standard kernel distribution for this reason, and has to be applied as an add-on. Furthermore, putting the cryptography in the kernel isolates it from the application, making it more difficult to code crypto-aware software. Using SSL, for example, simply requires linking a library into the application and allows the application to easily query what certificates have been used to authenticate a client.

IPSEC defines a “Security Association” (SA) as its primitive means of protecting IP packets. An SA is defined by the packet’s destination IP address and a 32-bit Security Parameter Index (SPI), that functions somewhat like a TCP or UDP port number. SAs can operate in transport mode, where the IPSEC data field begins with upper level packet headers (usually TCP, UDP, or ICMP), or in tunnel mode, where the IPSEC data field begins with an entirely new IP packet header, ala RFC 2003. Furthermore, SAs can be encapsulated within SAs, forming SA bundles, allowing layered IPSEC protection.

For example, one SA might protect all traffic through a gateway, while another SA would protect all traffic to a particular host. The packets finally routed across the network would be encapsulated in an SA bundle consisting of both SAs.

A common use of IPSEC is the construction of a Virtual Private Network (VPN), where multiple segments of a private network are linked over a public network using encrypted tunnels. This allows applications on the private network to communicate securely without any local cryptographic support, since the VPN routers perform the encryption and decryption. IPSEC is well suited for this environment, more so than tunneling PPP over SSL or SSH, since it operates directly on the IP packets and preserves a one-to-one correspondence between packets inside and outside the network. In the case of tunneling PPP over an encrypted TCP connection, any packet loss in the public network would trigger a TCP retransmission, stalling the link until the packet was delivered. In particular, running Voice Over IP (VoIP) traffic through a TCP/PPP tunnel would largely defeat the RTP protocol used for VoIP; IPSEC is better suited in this case.

IPsec Development for Linux:

In the Linux IPv4 IPsec world, a lot of people use

FreeS/WAN project’s implementation. It consists of an inkernel

IPsec processing part, Key Exchange daemon ‘Pluto’

and some utility commands/scripts.

To run Pluto with small changes on our IPsec kernel

implementation and reduce impact for user who use

FreeS/WAN implementation, we have decided to keep compatibility

with FreeS/WAN’s IPsec programming interface

between kernel and userland. For this, we use the same

PF KEY interface which FreeS/WAN project extended.

In kernel IPsec packet processing part, we developed

AH, ESP, SAD and SPD from scratch. PF KEY interface

PF KEY(v2), which is described in RFC2367, is key

management API mainly for IPsec. PF KEY is used for

handling the IPsec Security Association Database. Additionally

we have to handle the IPsec Security Policy

Database, but there is no standard for the IPsec Security

Policy management API. In FreeS/WAN implementation,

PF KEY interface is extended to manage the IPsec Security

Policy Database. Our kernel 2.4 IPsec implementation

also uses the same PF KEY interface as FreeS/WAN’s one.

It is important to be able to run the FreeS/WAN’s userland

application (e.g., Pluto) with small changes.

Encryption and Authentication algorithm

We provide HMAC-SHA1 and HMAC-MD5 for authentication,

NULL, DES-CBC, 3DES-CBS and AES for encryption.

We thought encryption and authentication algorithm

is not only used by IPsec and there are many algorithms

so that we consider encryption and authentication algorithm

and those interface should have good modularity.

We adopted cipher modules which provided by CryptoAPI


Security Association and Security Policy

SA and SP themselves don’t depend substantially on the

IP version. FreeS/WAN project architecture depends on

their special virtual network interface for IPsec because it

might focus on IPv4 tunnel mode (Their implementation

also provides IPv4 transport mode). Their SA, SP, SAD and

SPD also depend on their special virtual network interface.

We considered and decided it was not suit to IPv6 because the IPv6 stack needed the neighbor discovery and the auto address configuration in its basic specification. If we had

implemented IPv6 IPsec stack with their architecture, we

had to implement those basic specification in their special

virtual network interface. Therefore we implemented our

own SAD and SPD in order to handle both IPv4 and IPv6.

To improve the system performance, Each database will

be locked by smallest granularity. And in many cases we

use the ‘read lock’. SA and SP are managed by the reference

counter to prevent used SA from removing by accident.

IPsec Packet Processing


There are various packet output paths from the IP(v4/6)

layer to the network driver layer in Linux kernel networking

stack (TCP, UDP/ICMP, and NDP[10] for IPv6).

The packets which may be applied IPsec will go

through these paths. We had to add IPsec functionality

for these output paths, e.g, in IPv6 ip6 xmit()

for TCP, ip6 build xmit() for UDP/ICMP and

ndisc send ns()/ndisc send rs() for neighbor

discovery packets.

Output process is as follows):

  1. check IPsec SP
  2. lookup the IPsec SA by the IPsec SP
  3. apply IPsec processing to the packet
  4. output the packet to the network driver layer

To reduce SA searhing time, we link the SP and the

found SA after lookup from the first time.


At input, there is only path for IP packets. We added IPsec

processing part in ip6 input finish.

Input process is as follows:

  1. receive the packet
  2. lookup the IPsec SA by SPI(which resides in AH/ESP header)
  3. check integrity and decrypt
  4. check IPsec Policy.

IPsec Tunnel mode

We are using IPv6-over-IPv6(and IPv4-over-IPv4) virtual

tunnel device to implement IPsec tunnel mode. This

implementation can avoid to duplication code of encapsulation/

decapsulation outer IP header compairing with having

these code in the IPsec processing part itself. The virtual

tunnel device is not different from the normal IP-over-IP

virtual tunnel device in Linux.

4 IPsec implementation for kernel 2.6

The most important difference between ours and them is SAD/SPD part. They thought the whole SPD/SAD mechanic should be flow cache based lookup system shared by

IPv4 and IPv6. One month later, they introduced the new

network architecture called ‘XFRM’ to Linux kernel 2.5.

At first their developing code lacked IPv6 IPsec only for

IPv4 IPsec. In order to suport IPv6 IPsec, we have implemented

IPv6 IPsec code based on XFRM (and discarded our

original code).

PF KEY interface

The PF KEY interface of Linux kernel 2.6(and 2.5) is

compatible with KAME[3] PF KEY interface. We can use

‘setkey’ command for configuring SA and SP and ‘Racoon’

for IKE. Additionally we can add IPsec Policy each socket

via Netlink3. They have suported only IPv4 in their first

code, we have added IPv6 support.

Security Association and Security Policy

On the XFRM architecture, IPsec SP, which is represented

as xfrm policy structure, will be bound to

the routing flow cache (and IPsec policy will point

IPsec SA bundle) and IPsec SA, which is represented as

xfrm state structure, is included in destination cache,

dst entry structure. The chaining destination cache

means IPsec SA bundle.

IPsec Packet Processing


The output part of the XFRM architecture is placed between

the IP layer and the network driver layer. In general, non

IPsec packet will be passed to the network driver layer by a

single destination output function, which is resolved routing

lookup. But IPsec packet will be need to apply some IPsec

processing (e.g., encryption, hash). XFRM functions make

a chain of destination output functions (We call Stackable

Destination, as shown in Figure3). Each function match

each IPsec processing (AH, ESP and IPcomp[11]).

To be more specific, in order to pass a packet to the network

driver layer we have to do as follows.

  1. lookup routing table to decide output function by

    ip6 route output

  2. lookup IPsec Security Policy
  3. lookup IPsec Security Association(s) suitable for IPsec

    Security Policy and create destination chain

  4. to apply IPsec, pass a packet to the destination chain


The input part of the XFRM architecture is simpler than output.

The XFRM input function is handled as same as upper

layer protocols like TCP, UDP, etc. In IPv6, IPsec headers

are defined as IPv6 extension header but IPsec input functions

are handled as an upper layer protocol handler. As the

result of introducing IPv6 IPsec input processing in Linux.

kernel, inconsistencies existed between IPsec headers and other IPv6 extension headers. In order to resolve this, we

moved to the other IPv6 extension header handler functions

to upper layer protocol handler. In detail, we registered

IPsec header (both AH and ESP) handler functions with upper

layer protocol handler array inet6 protos.

Incoming IPsec packet processing flow is as follows:

  1. process IP packet from IP header in sequence
  2. process IPsec part (check integrity and decrypt) if


  3. check IPsec Security Policy
  4. pass IP packet next handler

IPsec Tunnel mode

Linux kernel 2.6 IPsec tunnel mode doesn’t use the virtual

tunnel device to create tunnel. The IPsec stack builds

the outer IP header during IPsec processing by itself.

IPSec Best practices

Best practices

IPSEC in transport mode has some serious advantages over other solutions. Compared to other technologies, IPSEC is built into to the Linux kernel. In other words there is nodaemonrunning in the background. Better yet, IPSEC does not require port-forwarding; some people elect to useSSH, stunnel, and other technologies that rely onport forwarding. With IPSEC, you simply have to run a program and it’s configuration file. After running it,encryptionbetween hosts is mandatory. Connections will be refused if the other connection does not have the appropriate keys. Groups of computers can share the same key, and it can even be done on a per-port setting (for example securing VNC, etc).


IPSEC in transport mode does have a couple draw backs. In transport mode you cannot have any dynamic setups where the IP addresses change from time to time. In other words, IPSEC is usually insufficient for workstation environments or dynamically assigned networks. Also, if you want to do a per-port setup the configuration becomes harder.

Security Implications

A very astute user can use IPSEC to bypass firewalls and other security measures. Since IPSEC uses cryptography, information is passed between machines in encrypted format. If the keys are not known, there is no practical way to decrypt the information (it is virtual impossible due to the sheer amount of time it would take).

Machine-to-Machine IPSEC installations should be considered as Virtual Private Networks (VPNs) for security considerations. Please check with yoursystem administrator, business policies, and laws and regulations of your locality in order to establish whether or not to institute IPSEC.


  • ipsec-tools package
  • static IP addresses for each machine

Configuration file

The configuration file, /etc/setkey.conf, contains the information about the IPSECpolicy. Below is a sampleconfiguration policy(i.e. don’t implement this policy because it is insecure).

These lines are the actual keys and the encryption that will be used. The first block has the keys that will be used for authentication. In this case, it is the “hmac-md5″algorithm. The second block contains the keys that will be used for privacy, and the method of encryption. In the example, AES-CBC will be used, which is probably stronger than should be required; the key that we will be using is 194bits, meaning that it is good enough for US Government Secret and below classifications.

The final block includes the actual policy. This is where you can put port numbers and even define whether it will be TCP orUDP.

Generating the keys

The more random the key, the better. Obviously, the example above is insufficient to secure a network. The following command will generate a random key. While running this command, you’ll need to wiggle the mouse to make it run faster. Or, if you are using a terminal use/dev/urandom instead.

dd if=/dev/random count=16 bs=1| xxd -ps

Depending on the size of the key that you want, adjust the count (16 will produce a 128 bit key, 24 will produce a 196 bit key, and 32 will produce a 512 bit key)

The size of the key is important. If you really paranoid or just haveCPUcycles to


Cite This Work

To export a reference to this article please select a referencing stye below:

Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.

Related Services

View all

DMCA / Removal Request

If you are the original writer of this essay and no longer wish to have your work published on UKEssays.com then please: