Firewalls and Virtual Private Networks

Frederick Avolio
Trusted Information Systems, Inc.

Building VPNs with Internet Firewalls

The Necessity of VPNs

VPNs are necessary because communications between sites using a public network (like the Internet) are vulnerable to an eavesdropping (or snooping) attack. The risk of this happening depends on the importance the transmitted information holds for someone who has the ability to intercept it.

VPNs allow a corporation, at key gateways or communication points, to ensure that all network traffic is private. Network communication, over the Internet for example, is vulnerable to “snooping” — electronic eavesdropping. Armed with a PC, a network interface card for the PC, and access to the communications flow, a hacker or corporate spy can copy all information flowing between one site and another: e-mail, terminal sessions, anything. Setting up a VPN between two points guarantees private communication between those points.

If a VPN is set up between site A and site B, all traffic between those sites will be encrypted. All traffic between either of these sites and other sites on the Internet, for example, with which no VPN relationship exists will be sent “in the clear.”

VPNs also can represent a terrific cost saving over private networks. The March 1996 issue of US Computer reported that using encrypted “tunnels” over the Internet to connect LANs and WANs can reduce costs 23-50%.

Data Communications had a sidebar in the May 21, 1997 with this information:

Hypothetical network with 3 fully meshed sites in LA, Houston, and Boston, plus a transatlantic link to London, at 64 kbits/s. Charges based on AT&T and include local access circuits of 5 km to the nearest POP. Leased-line and frame relay figures come from Lynx Technologies, Inc. (Fairfield, NJ). Internet access based on average monthly ISP charges in the US.  
Leased Lines  
Annual charges:
$ 2,700
Frame Relay VPN  
Annual charges:
$ 89,998
$ 5,760
Internet VPN  
Annual Charges:
$ 38,400
Four VPN encrypting devices
$ 54,400

Firewalls and VPNs

Many firewalls have some kind of VPNs — encrypted firewall-to-firewall tunnels. All traffic between one firewall and another is encrypted, stuck inside of another IP packet, and sent over the Internet. At the remote site, the firewall pulls the encrypted payload out of the IP packet and decrypts it to get the original IP packet, which is forwarded to the final destination.

Firewall-to-Firewall With Controlled Access

As VPNs become more widespread on the Internet, and VPN establishment more automatic, most VPNs will be for privacy between sites, without a complete trust relationship between those sites. By “complete trust relationship,” we mean the relationship that exists between business partners, suppliers and consumers, providers and customers. Privacy for the communications may be desirable or needed, but an Internet firewall can be used to control or prohibit access to the internal, private network.

As an example let us say XYZ company has 100 resellers. XYZ could set up VPNs between XYZ’s Internet gateway and each of those 100 resellers. All e-mail, file transfers, everything, would go over an encrypted channel. Any TCP/IP traffic to other systems on the Internet would be sent, as usual, in the clear. Further, we could allow limited access through the corporate firewall to selected servers inside our network security perimeter for our resellers (to our sales support web pages for example).

Firewall-to-Firewall With Open Access

A common configuration for VPNs between Internet Firewalls currently is in a trust relationship between offices of the same company. For example let’s look at the XYZ corporate network.

XYZ has two offices in Maryland, one in Virginia, two in California, one in London, and one in Munich. The Internet is used as the XYZ corporate backbone. Without a VPN, all traffic between XYZ offices would be vulnerable to disclosure as it flowed over the Internet. Further, using less secure TCP/IP services (such as file and directory sharing) would leave the XYZ corporate network vulnerable to attack using those services.

With VPN connectivity using VPN-enabled Internet Firewalls between the sites, all traffic is encrypted, and so private. Additionally, when we add the trust derived from all sites being administered by the same organization, all having the same security policy implemented, and all being under the same management organization, we can, over this VPN, allow all network services. In this way we are virtually going around the firewall, though actually the communications flow is still under the protection of the firewalls. In this way we extend the network security perimeter to include those other offices. All sites are now virtually on the same LAN, with a virtual network perimeter.

Firewall-to-Remote System

The same sort of VPN technology can be used between a firewall and a single site. Often this is used to allow private access to a corporate from mobile users, working at a customer site, or connected in from home or a hotel. As with firewall-to-firewall VPNs, these can be set up with controlled or open access. Controlled access is useful for clients, customers, and partners needing access to particular systems on the inside of the security perimeter for particular services at particular hours of the day. Open access is useful for employees on the road who need to get access to shared files, printers, etc. on the inside of the network security perimeter. With a VPN they can do this securely.


Various technologies are used to implement VPNs. The basics of cryptography and a discussion of general encryption standards and methods are beyond the scope of this paper. For the remainder of this paper we will concentrate on established and emerging standards being used by vendors to implement VPNs.

The Need for Standardization

Many commercial firewalls today support VPN functionality. Two VPN-enabled firewalls can be used to establish a VPN. When this technology was first developed, there were no standards. Consequently, vendors established their own ways of implementing IP encryption. Some vendors went with a mechanism called swIPe (for Software IP Encryption). This was freely available in source code form with no licensing restrictions, so it was attractive because 1) it was already written (although it needed to be ported to platforms other than SunOS) and 2) it was the only mechanism approaching a standard, albeit “de facto,” that existed. Still, one vendor’s VPN using swIPe didn’t work with others and it was not possible to establish an inter-vendor VPN.

In order for VPNs to become widely and routinely used, sites using differing encryption products need to be able to communicate.

In 1995, RSA formed an interoperability group called S/WAN (Secure Wide Area Network). Its purpose was to support the emerging IPSEC standard being developed by the Internet Engineering Task Force (IETF), and to provide a framework for inter-vendor interoperability testing. Many vendors now interoperate using the emerging IPSEC standards. See the web page for up-to-date information about interoperability testing.


The IETF’s IP Security (IPSEC) Working Group is developing standards for IP-layer security mechanisms for both IPv4 (the version use on the Internet at the time of this writing) and IPv6 (the next generation of TCP/IP). The IPSEC architecture includes authentication (how to know if the site communicating to your site really is who it claims to be) and encryption. These mechanisms can be used together or independently.

Authentication Header

As described in RFC 1826, The Authentication Header is a mechanism for providing strong integrity and authentication for IP datagrams. It might also provide non-repudiation, depending on which cryptographic algorithm is used and how keying is performed. For example, use of an asymmetric digital signature algorithm, such as RSA, could provide non-repudiation.

The IP Authentication Header seeks to provide security by adding authentication information to an IP datagram. This authentication information is calculated using all of the fields in the IP datagram (including not only the IP Header but also other headers and the user data) which do not change in transit. Fields or options which need to change in transit (e.g., “hop count”, “time to live”, “ident”, “fragment offset”, or “routing pointer”) are considered to be zero for the calculation of the authentication data. This provides significantly more security than is currently present in IPv4 and might be sufficient for the needs of many users.

Encapsulating Security Payload (ESP)

As described in RFC 1847, ESP is a mechanism for providing integrity and confidentiality to IP datagrams. In some circumstances it can also provide authentication to IP datagrams. The mechanism works with both IPv4 and IPv6.

ESP is a mechanism for providing integrity and confidentiality to IP datagrams. It may also provide authentication, depending on which algorithm and algorithm mode are used. … The IP Authentication Header may be used in conjunction with ESP to provide authentication. Users desiring integrity and authentication without confidentiality should use the IP Authentication Header (AH) instead of ESP.

ESP in IPSEC can be used to encrypt the entire IP datagram or just information at the transport layer (TCP, UDP, etc.).

Again quoting from the RFC,

In Tunnel-mode ESP, the original IP datagram is placed in the encrypted portion of the Encapsulating Security Payload and that entire ESP frame is placed within a datagram having unencrypted IP headers. The information in the unencrypted IP headers is used to route the secure datagram from origin to destination. An unencrypted IP Routing Header might be included between the IP Header and the Encapsulating Security Payload.

In Transport-mode ESP, the ESP header is inserted into the IP datagram immediately prior to the transport-layer protocol header (e.g., TCP, UDP, or ICMP). In this mode bandwidth is conserved because there are no encrypted IP headers or IP options.

Key Management: The Challenge and the Emerging Standards

Key exchange mechanisms — how to decide on and communicate what cryptographic keys to use to secure the cryptographic communication — and keeping that exchange secure (private) is critical to private communications and the establishment of VPNs. Most implementations of VPNs up until the date of this writing used a manual key exchange mechanism with out of band communication. This often meant, software run manually to come up with a random key for communication, communicated over encrypted e-mail, the telephone, or via floppy disk. It should be clear that this mechanism does not scale.

There are frameworks and mechanisms being developed to do this work. The IPSEC effort is focusing on ISAKMP and Oakley.


The Internet Security Association & Key Management Protocol (ISAKMP) provides a framework for Internet key management. It also defines the protocol for the negotiation of security attributes (algorithm to use, length of key).


The Oakley Key Determination Protocol uses a hybrid Diffie-Hellman (a public key) technique to establish session keys.

ISAKMP and Oakley Combined

As stated above, both private key exchange and a mechanism for negotiating keys, algorithms, etc. are required for automatic establishment of VPNs. Combining ISAKMP and Oakley provides this.

As described in the IETF draft entitled The resolution of ISAKMP with Oakley,

Oakley defines a method to establish an authenticated key exchange. This includes how payloads are constructed, the information they carry, the order in which they are processed and how they are used.

While Oakley defines “modes”, ISAKMP defines “phases”. The relationship between the two is very straightforward. ISAKMP’s phase 1 is where the two ISAKMP peers establish a secure, authenticated channel with which to communicate. This is called the ISAKMP Security Association (SA). …

Phase 2 is where Security Associations are negotiated on behalf of services such as IPSEC or any other service which needs key material and/or parameter negotiation.


This was developed originally as an extension to PPP (Point-to-Point Protocol) used for TCP/IP of serial lines (dial-up modem connections), by Microsoft in collaboration with others. PPTP “tunnels” or encapsulates IP, IPX, or NetBEUI protocols within IP packets. Supported by Microsoft and NT servers, this could become the standard protocol for setting up a VPN from remote system to a corporate private network.

Conceptually, there is no difference between using PPTP for client to firewall communications, and using an IPSEC enabled IP stack on the client to talk to the firewall and establish a VPN. PPTP does support tunneling of non-IP protocols. But, as of the time of this writing, PPTP was only supported on NT servers. In order to support this, a site must be running an NT-based firewall or must provide a hole through the firewall to allow PPTP traffic through. The latter mean forgoing any firewall control over what is permitted over that PPTP session (i.e., it means VPNs with complete access and no VPNs with controlled access). PPTP has not yet been widely adopted.

Final Questions

Because of the growth of business use of the Internet, and the rise in reported Internet attacks and break-ins, security is no longer a forgotten requirement as in the past. Internet firewalls are now on everyone’s checklist when considering Internet connectivity. Virtual Private Networks are also starting to be considered “must haves.” VPNs are not yet in wide use, but with the move to standards such as IPSEC with ISAKMP and Oakley, and the automation in VPN establishment they should provide, we expect to see a dramatic growth in VPN use in the next year. Some issues, beyond the firming up of the above-mentioned standards, still need to be considered.

Global Virtual Private Networks

The Internet is a worldwide network. Companies cross international boundaries, partnerships cross international boundaries, and privacy in business transactions is critical to success. As we have described, network communications privacy means employing encryption. This encryption must be useable and available globally, and so we have coined the term Global Virtual Private Network, or GVPN. By this we mean a VPN that is useable globally. Additionally, it must use strong cryptography (meaning at least 56 bit DES, maybe 3DES, 128 bit RC2, etc.) and be platform independent (Macintosh, Microsoft Windows, UNIX, etc.).

Do Firewalls Go Away?

Firewalls do not go away, even when (if) IPSEC is found on all computers in an organization. Internet firewalls enforce an enterprise network security policy and are part of a perimeter defense. IPSEC on every desktop provides for privacy and authentication, but does not ensure that the corporate network security policy is enforced (what services are allowed, when to force virus scanning, etc.). Firewalls will be able to enforce a policy that requires private links between networks (sites) even if desktop users cannot or do not use an encrypted connection.

Closing Thoughts

Establishing VPNs require agreement in four different areas: the mechanism for encryption, the algorithm for encryption (and key size to use), the mechanism for key exchange negotiation, and the algorithm to use for key exchange. Standards of one sort and another exist in all of these area:
Mechanism for encryption swIPe, IPSEC, PPTP
Algorithm for encryption RC2, RC4, DES, 3DES, IDEA, CAST
Mechanism to negotiate encryption and key exchange ISAKMP, SKIP
Algorithm for encrypting key exchange information RSA, Diffie-Hellman
All of these need to be coordinated and communicated. The technologies that exist today, and are being implemented in firewall-based VPNs, provide these mechanisms and will allow a growth in deployment and use of VPNs in the near future.