Load Balancing for Network Servers - Revision history

Sadlerjw at 02:31, 13 April 2009

2009-04-13T02:31:11Z

← Older revision		Revision as of 02:31, 13 April 2009
Line 69:		Line 69:
	# [http://www.flowgram.com/f/p.html#2qi3k8eicrfgkv Jason Sobel: ''Needle in a Haystack: Efficient Storage of Billions of Photos'']		# [http://www.flowgram.com/f/p.html#2qi3k8eicrfgkv Jason Sobel: ''Needle in a Haystack: Efficient Storage of Billions of Photos'']
	# [http://www.infoq.com/presentations/Facebook-Software-Stack Aditya Agarwal: ''Facebook: Science and the Social Graph'']		# [http://www.infoq.com/presentations/Facebook-Software-Stack Aditya Agarwal: ''Facebook: Science and the Social Graph'']
		+
		+
		+	--[[User:Sadlerjw\|Sadlerjw]] 22:31, 12 April 2009 (EDT)

Sadlerjw: /* Optimizing the Return Trip */

2009-04-13T02:29:41Z

Optimizing the Return Trip

← Older revision		Revision as of 02:29, 13 April 2009
Line 45:		Line 45:

	====Optimizing the Return Trip====		====Optimizing the Return Trip====
-	The advantage of the algorithm described above is that it is simple and can be done on a relatively high level. Since only IP addresses are rewritten, it can be dealt with entirely on Level 3 (Networking layer) of the '''[http://en.wikipedia.org/wiki/OSI_model OSI model]'''. However it can be optimized by eliminating step three—this would reduce traffic through the load balancing server by ~~half~~. This optimization is entitled '''Direct Server Return''' and operates on Level 2 (Data Link layer) of the OSI model, using '''MAC Address Translation''' (or '''MAT''') [1].	+	The advantage of the algorithm described above is that it is simple and can be done on a relatively high level. Since only IP addresses are rewritten, it can be dealt with entirely on Level 3 (Networking layer) of the '''[http://en.wikipedia.org/wiki/OSI_model OSI model]'''. However it can be optimized by eliminating step three—this would reduce traffic through the load balancing server. (Since most Internet usage has a ratio 8:1 incoming traffic to outgoing traffic, this optimization can reduce the amount of traffic going through the load balancing server by 8 times.) This optimization is entitled '''Direct Server Return''' and operates on Level 2 (Data Link layer) of the OSI model, using '''MAC Address Translation''' (or '''MAT''') [1].
	*The real server (host C in the above scenario) is assigned its own IP Address as well as the load balancer's IP. Since it is impossible to have duplicate IP addresses on the same network, so this address is bound to host C's loopback interface. (This is done through aliases, so that the loopback interface also maintains its traditional 127.0.0.1 address.)		*The real server (host C in the above scenario) is assigned its own IP Address as well as the load balancer's IP. Since it is impossible to have duplicate IP addresses on the same network, so this address is bound to host C's loopback interface. (This is done through aliases, so that the loopback interface also maintains its traditional 127.0.0.1 address.)
	*The server software (such as a web server like Apache) is bound to this new address on the loopback interface.		*The server software (such as a web server like Apache) is bound to this new address on the loopback interface.
-	The packet goes through the following steps:	+	The packet goes through the following steps [1]:
-	#	+	#The packet is sent from the user's computer (host A) to the load balancer (host B). The source address is A and the destination address is B.
		+	#The load balancer (B) chooses a server (host C) which will service the user's request. Instead of rewriting the packet header to point at C, it keeps its own address (B) as the destination. (Remember, host C has this address configured on its loopback interface) Instead of changing the destination IP address, it rewrites the destination MAC address. This means that the packet will be sent to C, but whereas the packet would normally be dropped (since C would not normally also be configured on B's IP address), C accepts the packet.
		+	#Host C finishes serving the request and sends a response back with source B and destination A. The packet is delivered directly to the user's computer, which accepts the packet.

	===Caching===		===Caching===

Sadlerjw: /* Optimizing the Return Trip */

2009-04-13T02:22:42Z

Optimizing the Return Trip

← Older revision		Revision as of 02:22, 13 April 2009
Line 45:		Line 45:

	====Optimizing the Return Trip====		====Optimizing the Return Trip====
		+	The advantage of the algorithm described above is that it is simple and can be done on a relatively high level. Since only IP addresses are rewritten, it can be dealt with entirely on Level 3 (Networking layer) of the '''[http://en.wikipedia.org/wiki/OSI_model OSI model]'''. However it can be optimized by eliminating step three—this would reduce traffic through the load balancing server by half. This optimization is entitled '''Direct Server Return''' and operates on Level 2 (Data Link layer) of the OSI model, using '''MAC Address Translation''' (or '''MAT''') [1].
		+	*The real server (host C in the above scenario) is assigned its own IP Address as well as the load balancer's IP. Since it is impossible to have duplicate IP addresses on the same network, so this address is bound to host C's loopback interface. (This is done through aliases, so that the loopback interface also maintains its traditional 127.0.0.1 address.)
		+	*The server software (such as a web server like Apache) is bound to this new address on the loopback interface.
		+	The packet goes through the following steps:
		+	#

	===Caching===		===Caching===

Sadlerjw: /* Traditional Load Balancing */

2009-04-13T02:09:38Z

Traditional Load Balancing

← Older revision		Revision as of 02:09, 13 April 2009
Line 32:		Line 32:

	====Traditional Load Balancing====		====Traditional Load Balancing====
-	Load balancing must occur transparently to the user. It is therefore necessary to use '''Virtual IPs''' or '''VIPs'''. A Virtual IP is "the load-balancing instance where there world points its browsers to get to a site. A VIP has an IP address which must be publicly available to be useable." [1]	+	Load balancing must occur transparently to the user. It is therefore necessary to use '''Virtual IPs''' or '''VIPs'''. A Virtual IP is "the load-balancing instance where there world points its browsers to get to a site. A VIP has an IP address which must be publicly available to be useable" [1]. There can be one or more machines which answer to this IP—these are the load balancing machines. All traffic travels through these computers, which route traffic to hosts based on a variety of algorithms. Where there is more than one load balancing machine, failover protection is used so that if one machine fails the other load balancer can take over its traffic. This can be done in a variety of ways, such as with the '''[http://en.wikipedia.org/wiki/VRRP Virtual Router Redundancy Protocol]''' and its proprietary cousins, and fail-over cable, among others.
		+
		+	Many load balancing servers address the issue of persistence or '''stickiness''', which stipulates that whichever server is used at the beginning of a user's session must be used for the rest of the duration of that session. This requires a higher awareness of the TCP communications on the part of the load balancer, but is essential for persistent web applications such as stores.
		+
		+	Load balancing servers are responsible for manipulating packet headers through a process termed '''Network Address Translation''' ('''NAT''') to properly route the packets. This nicely circumvents the TCP requirement that the source and destination addresses in incoming and outgoing packets on a single connection to match. (This would be a problem since the server answering the user's request does not have the same IP address as the server to which the connection was made—the load balancer.)
		+
		+	A packet in a typical load balancing system usually goes through the following progression [1]:
		+	#The packet is sent from the user's computer (host A) to the load balancer (host B). Naturally, the source address is A and the destination address is B.
		+	#The load balancer chooses a server (host C) which will service the user's request. It rewrites the packet header so that the destination address reads as C. The source address remains as A.
		+	#Host C finishes serving the request and sends a response back with source C and destination A. However, as discussed above, A will discard the packet because its source address does not match the connection (it expects a packet with source B). However the default route outgoing from C is B, so the load balancing server is able to intercept the packet on its way back to the user.
		+	#Host B rewrites the packet header again, setting the source field to B. The packet now has source B and destination A, and is delivered properly to the user's computer.

	====Optimizing the Return Trip====		====Optimizing the Return Trip====

Sadlerjw: /* Multiple Redundant Servers */

2009-04-13T01:38:05Z

Multiple Redundant Servers

← Older revision		Revision as of 01:38, 13 April 2009
Line 29:		Line 29:

	===Multiple Redundant Servers===		===Multiple Redundant Servers===
		+	In this scenario there are multiple servers which all serve the same function. For example, when a user connects to a large website like Google, they may be connecting to any one of several hundred or thousand servers, but regardless of which server he is connected to, the user still gets the same Google homepage and can make searches just the same way. This is what is usually meant by the phrase "load balancing."

	====Traditional Load Balancing====		====Traditional Load Balancing====
		+	Load balancing must occur transparently to the user. It is therefore necessary to use '''Virtual IPs''' or '''VIPs'''. A Virtual IP is "the load-balancing instance where there world points its browsers to get to a site. A VIP has an IP address which must be publicly available to be useable." [1]

	====Optimizing the Return Trip====		====Optimizing the Return Trip====

Sadlerjw: /* Caching */

2009-04-13T00:43:23Z

Caching

← Older revision		Revision as of 00:43, 13 April 2009
Line 38:		Line 38:

	In LAMP stacks one of the most popular caching tools is '''[http://www.danga.com/memcached/ Memcache]'''. Memcache implements what is essentially a very large hash map which allows very quick access to cached data. It works with PHP, Java, Ruby, Python, and several other languages.		In LAMP stacks one of the most popular caching tools is '''[http://www.danga.com/memcached/ Memcache]'''. Memcache implements what is essentially a very large hash map which allows very quick access to cached data. It works with PHP, Java, Ruby, Python, and several other languages.
-
-

	==See Also==		==See Also==

Sadlerjw: /* Case Study: Facebook's Photo System */

2009-04-13T00:42:55Z

Case Study: Facebook's Photo System

← Older revision		Revision as of 00:42, 13 April 2009
Line 39:		Line 39:
	In LAMP stacks one of the most popular caching tools is '''[http://www.danga.com/memcached/ Memcache]'''. Memcache implements what is essentially a very large hash map which allows very quick access to cached data. It works with PHP, Java, Ruby, Python, and several other languages.		In LAMP stacks one of the most popular caching tools is '''[http://www.danga.com/memcached/ Memcache]'''. Memcache implements what is essentially a very large hash map which allows very quick access to cached data. It works with PHP, Java, Ruby, Python, and several other languages.

-	~~==Case Study: Facebook's Photo System==~~	+

	==See Also==		==See Also==

Sadlerjw: /* See Also */

2009-04-12T20:47:29Z

Sadlerjw: /* See Also */

2009-04-12T20:47:03Z

Sadlerjw: /* See Also */

2009-04-12T20:45:03Z

← Older revision		Revision as of 20:47, 12 April 2009
Line 42:		Line 42:

	==See Also==		==See Also==
-	[http://en.wikipedia.org/wiki/Content_delivery_network Content Delivery Network]	+	*[http://en.wikipedia.org/wiki/Content_delivery_network Content Delivery Network]
-		+	*[http://en.wikipedia.org/wiki/LAMP_(software_bundle) LAMP]
-	[http://en.wikipedia.org/wiki/LAMP_(software_bundle) LAMP]	+	*[http://en.wikipedia.org/wiki/Memcache Memcache]
-		+	*[http://en.wikipedia.org/wiki/Multitier_architecture Multitier Architecture]
-	[http://en.wikipedia.org/wiki/Memcache Memcache]	+
-		+
-	[http://en.wikipedia.org/wiki/Multitier_architecture Multitier Architecture]	+

	==References==		==References==

← Older revision		Revision as of 20:47, 12 April 2009
Line 42:		Line 42:

	==See Also==		==See Also==
-	[http://en.wikipedia.org/wiki/Content_delivery_network Content ~~delivery network~~]	+	[http://en.wikipedia.org/wiki/Content_delivery_network Content Delivery Network]
		+
		+	[http://en.wikipedia.org/wiki/LAMP_(software_bundle) LAMP]
		+
	[http://en.wikipedia.org/wiki/Memcache Memcache]		[http://en.wikipedia.org/wiki/Memcache Memcache]
		+
		+	[http://en.wikipedia.org/wiki/Multitier_architecture Multitier Architecture]

	==References==		==References==

← Older revision		Revision as of 20:45, 12 April 2009
Line 43:		Line 43:
	==See Also==		==See Also==
	[http://en.wikipedia.org/wiki/Content_delivery_network Content delivery network]		[http://en.wikipedia.org/wiki/Content_delivery_network Content delivery network]
		+	[http://en.wikipedia.org/wiki/Memcache Memcache]

	==References==		==References==