Monday, September 24, 2012

How Does WAN Optimization Work?




WAN optimization can work wonders for spread out enterprise networks, reducing latency, relieving congestion and speeding up bandwidth-hungry Web apps. Learn how this technology works.


WAN optimization controllers (WOCs) can breathe new life into slow wide area network links, relieving congestion, speeding up file transfers and making applications more responsive. But how exactly do vendors like Riverbed Technology, Juniper Networks, Blue Coat Systems and Expand Networks get their devices to work their magic?


  • They have limited capacity, so they can become congested
  • They suffer from high latency because they are long (relative to  LAN links)
  • Caching
  • Compression
  • Data reduction
  • Latency reduction
  • Quality of Service (QoS) tagging
  • Packet coalescing
CachingCompressionData ReductionLatency ReductionQuality of Service (Qos)Packet Coalescing



To answer this question, consider the two fundamental problems that WAN links present:
The best strategy for overcoming these two problems is simply to avoid using the WAN links whenever possible, and to minimize their use when it's not possible to avoid them. It's this strategy that underpins all the techniques that WOCs employ to optimize WAN traffic.
These most commonly used techniques include:
This is one of the most obvious ways of improving WAN performance. When a file is transferred over a WAN, say from a head office to a branch office, a copy of it is cached by the branch office's WOC. When other users in the branch office request the same file, the request is intercepted by the WOC before it goes over the WAN link, and the file is served locally from the device's cache.  Changes made to files in any location are communicated across the network to ensure that files are always kept in sync.
Using caching, the first access of any file is still slow because it still has to pass over the WAN before it can be cached it's only subsequent accesses that are much faster. To speed up the first access, the cache can be pre-populated overnight with commonly used files so that they are immediately available in the cache the following day.
This is another obvious step that can be taken to boost WAN performance. It tackles the problem of limited bandwidth by reducing the amount of data that has to be sent over the WAN using a variety of data compression techniques.
Some WOCs also include header compression, which can reduce the size of packet headers dramatically. This is particularly effective when the size of the header is large compared to the size of the rest of the packet.
Data reduction works like a combination of compression and caching. Driven by the principle that the best way of overcoming the problems presented by a WAN is not to use it if possible, a WOC using data reduction examines data as it travels over the WAN, and stores data it receives. If it detects a piece of data that it has already transmitted in a file that it is sending, that byte sequence is removed, and is replaced with a reference. When the WOC at the remote office receives the reference it can then retrieve that piece of data from its own cache and substitute it back in. This avoids transmitting over the WAN any data that has already been sent – even as part of a completely different file. In some circumstances the amount of data traveling over a WAN can be reduced by 75 percent or more using data reduction techniques.
Latency, as mentioned earlier, can be a problem with WANs. This is particularly true when dealing with "chatty" protocols like Common Internet File System (CIFS). CIFS (and other CIFS implementations like Samba on Linux) are frequently  used when remote disks are browsed and files are transferred across a WAN, but the protocol was never really intended for use over high latency links. The term "chatty" refers to the fact that in order to send data (in chunks of no more than 61kb) a large number of background communications has to travel back and forth over the WAN link. For example, the next chunk of data will only start to be sent over the network once a response has been received for the previous one. Hundreds or thousands of communications have to be sent across the WAN during the process of sending a single file, and due to the high latency of the WAN this means that accessing a file which would be more or less instantaneous on a LAN can take several minutes on a WAN.
The way that WOCs deal with this problem is by understanding that a file transfer is taking place, and pre-sending some or all of the file to the remote WOC as quickly as it can. Protocol communications at the remote end destined to for server at head office are then intercepted by the remote WOC which generates the appropriate response, so that much of the protocol "chat" never actually crosses the WAN—it is dealt with by the WOC, which  already has the file that the protocol is trying to transfer.
As long as the WOC "understands" a particular protocol, it can be used to accelerate transmissions—whether they are at the down at the TCP level, or up at the application level.
QoS is complex, although the underlying idea is simple. Traffic is identified, usually by its application, source, or destination, and given a priority for transmission over the WAN. This may include how long it has to wait before being sent over the WAN, or the amount of available bandwidth reserved for a given application. The result is to ensure that time sensitive packets, such as VoIP packets, are sent as quickly as possible - at the expense of less time sensitive packets during busy periods.
Packet coalescing is useful in circumstances where the packet header is large relative to the size of the data it is transporting. In essence it saves bandwidth by consolidating multiple packets into a single (coalesced) packet, with a single header. This can make save considerable amounts of bandwidth, especially in applications like VoIP.
All of the WOCs sold into this multi-billion dollar market  offer some combination of the techniques mentioned above. The results speak for themselves: applications running up to 50 times faster, file transfers reduced from minutes to seconds, and WAN bandwidth requirements as much as halved. It's no surprise that over the last few years the market for WAN optimization controllers has been very strong indeed.

What is Riverbed and how does it work?




There are many types of WAN accelerators on the market right now, Riverbed has managed to stay ahead in the Gartner magic quadrant reports, and has led the field in this arena.
In summary, Riverbed devices are deployed on either end of a connection, typically a WAN or high latency connection, the devices will peer with each other and then build up tables in what Riverbed calls SDR (scaleable data referencing).
In simple terms imagine an IP conversation across a network is comprised of packets of data, if you think of these packets as words in a conversation it makes Riverbed easy to understand.
So using this example each packet equals a "word" in a conversation so if we had a short statement which said "THE BROWN DOG" when the conversation flowed through each Steelhead it will add these 'words' to its database and give each word a unique reference.
Next time the edge device sees one of these words. instead of sending the whole word(packet) it sends the reference only across the WAN link, then the remote site Steelhead rebuilds this packet and sends it out LAN side.
What this means is common and repetitive data which runs across WAN and high latency connections can be drastically reduced, if you think of emails as an example they often contain header information and signatures/logos etc, all of this equates to many packets of data which are sent repeatedly across the WAN.
The Steelheads will do run SDR on all types of TCP traffic by default but as well as this they also have specific optimisation streams for common 'chatty' types of traffic as well which optimise these even further.
These include: CIFS (including encrypted SMB1 and SMB2), HTTP(S), Oracle forms, MAPI (Exchange), SQL, NFS, Lotus Notes, Citrix (ICA and Session reliabilty), FCIP and SRDF.
In the deployments we manage at SAS we have seen drastic reductions in areas such as print traffic (again something which sends a lot of repeat information) and especially CIFS (such as Windows mapped drives) and MAPI (Outlook to Exchange conversations) The end result gives the user a much more LAN-like experience when working either remotely or in a remote office.
For mobile workers Riverbed has a mobile client which runs on laptops and acts as a built-in accelerator. This enables remote/home working to be much faster and more productive by indirectly reducing the latency of working on Internet connections. This mobile client works in the same way as an appliance with SDR but uses the local hard drive to store its database and reference