Packet distributing system and method for distributing access packets to a plurality of server apparatuses
Claim Score by NHIP
Abstract
A system includes at least two servers for executing the same application protocol among a plurality of servers, load balancing apparatus respectively connected to the servers, and a router connected to these load balancing apparatuses and to a network. Receiving an access packet from the network, the router selects and routes an arbitrary load balancing apparatus. Receiving the access packet, each load balancing apparatus selects an arbitrary server and transmits the access packet.

Term
Term ended
Projected expiry passed 21 March 2023, 3.5 years ago.
- Priority
- Filed
- Published
- Projected expiry
- Today
18 claims: 3 independent, 15 dependent
- 1Broadest claimClaim Score 55, average(NHIP)A load balancing system for distributing access packets to a plurality of servers prepared for at least one kind of application protocol, comprising:a plurality of load balancing apparatuses respectively connected to at least two servers for executing the same application protocol among said plurality of servers, each of said load balancing apparatus selecting an arbitrary server when receiving said access packets to said at least two severs, and transmitting said access packet so received to said selected server;and a router connected to said plurality of load balancing apparatuses and to a network, said router selecting an arbitrary load balancing apparatus from said at least one load balancing apparatus to which at least two servers for executing an arbitrary kind of application protocol are connected, when receiving said access packet to said server for executing said arbitrary kind of application protocol from said network, and routing said access packet so received to said selected load balancing apparatus.
- 7A load balancing system for distributing access packets to a plurality of servers prepared for at least one kind of application protocol, comprising:a plurality of load balancing apparatuses respectively connected to at least two servers for executing the same application protocol among said plurality of servers, each of said load balancing apparatuses including a balancing unit for selecting an arbitrary server when receiving said access packet to said at least two servers, and transmitting said access packet so received to said selected server;and a router connected to said plurality of load balancing apparatuses and to a network, said router including: a plurality of interfaces connected to said network or to each of said load balancing apparatuses;a storage unit for storing in advance a destination address, application protocol information, a hash value and routing destination information in association with one another;and a routing unit for judging whether or not a destination address and application protocol information contained in a header of a packet received are stored in said storage unit when receiving said access packet from said network, calculating a hash value from arbitrary information contained in said header when the destination address and the application protocol information are judged as being stored in said storage unit, and extracting said routing destination information associated with said hash value, said destination address and said application protocol information.
- 12A load balancing method in a load balancing system including a plurality of load balancing apparatuses connected to at least two Web servers for executing the same application protocol among a plurality of Web servers prepared for each of at least one kind of application protocol and distributing access packets to said plurality of Web servers, and a router connected to said plurality of load balancing apparatuses and routing the access packet received through a network to each of said load balancing apparatuses, said router executing at least the following steps:storing look-up key information set in advance;receiving said access packet from said network;comparing information contained in a header of said access packet with said look-up key information;calculating a hash value by use of arbitrary information contained in said header when information contained in said header is coincident with said look-up key information;deciding one load balancing apparatus from said hash value;and transmitting said access packet to said load balancing apparatus so decided.
Independent claims3
125 paragraphs in 5 sections, as filed
CROSS-REFERENCE TO RELATED APPLICATION
P-0001[0001] This application relates to U.S. patent application Ser. No. 10/189,468 filed on Jul. 8, 2002, the content of the application is incorporated herein by reference.
BACKGROUND OF THE INVENTION
P-0002[0002] This invention relates to a load balancing system. More particularly, the invention relates to a system capable of distributing traffics to a plurality of Web servers provided to an information service site connected to the Internet.
P-0003[0003] A wide variety of Internet information services such as EC (Electronic Commerce) have been rapidly increasing in recent years. The phenomena in which traffics to Web servers installed at a specific site providing the information service abruptly increase have often occurred. On the other hand, services having a higher speed and higher reliability have been required for business concerns that provide data centers and ASP (Application Service Providers).
P-0004[0004] These sites generally use a router for connection with the Internet. A Web system at the site includes load balancing apparatuses connected to the router and a plurality of Web servers connected to the load balancing apparatuses. Large quantities of traffics (access packets) from outside are once collected to the router in such a Web system. To secure reliability and stability of the system, the router must be able to distribute the traffics to a plurality of Web servers.
P-0005[0005] A technology called “policy routing function” in the router is another known technology. The router generally selects an optimum path of a network on the basis of a destination address contained in a header of an IP packet and establishes a route. When the policy routing function is employed, on the other hand, the router selects the routing path on the basis of other header information such as a destination port number contained in a TCP header, for example, in place of the destination address. The destination port number and a source port number inside the TCP header specify the kind of application protocols. In consequence, the router can select a routing path for each application protocol.
P-0006[0006] The technology disclosed in U.S. Pat. No. 6,175,874 (JP-A-11-27320) is one of the known technologies relating to the policy routing function in the router.
P-0007[0007] Still another known technology connects the load balancing apparatuses in multiple stages to the router. Traffics are distributed and routed from load balancing apparatuses on the upstream side to load balancing apparatuses on the downstream side on the basis of a predetermined rule (such as the kind of application protocols).
SUMMARY OF THE INVENTION
P-0008[0008] In the technology described in US2003/0009559A1, a plurality of lines are interposed between one router and one apparatus connected to the router such as a load balancing apparatus. The router determines a hash value in accordance with a certain hash function for the traffics routed to the load balancing apparatus. The router selects a line for practical routing on the basis of the hash value so determined. Therefore, this technology can distribute the traffics between the router and the load balancing apparatus. In other words, this technology is the one that can be applied only to 1:1 connection but cannot be applied between one router and a plurality of load balancing apparatuses, that is, 1:N connection.
P-0009[0009] The policy routing function makes it possible to distribute the traffics for each application protocol. However, the traffics for the same application protocol are routed only to the same line. For example, it is possible to distribute HTTP (Hyper Text Transfer Protocol) traffics used for gaining access to ordinary Web servers and HTTPS traffics created by encrypting ordinary HTTP by use of an SSL (Secure Sockets Layer) protocol. According to the policy routing function, however, the router cannot further segment and distribute only the HTTP traffics, for example.
P-0010[0010] On the other hand, according to the technology described in U.S. Pat. No. 6,176,874, the router inspects whether or not a pattern including a protocol type, a source address, a source port number and a destination port number that are contained in a header of a reception packet matches with a predetermined pattern. When the pattern of the reception packet matches with a specific pattern, the router calculates a hash value by use of header information and decides a processing node (server) that transmits the packet in accordance with the calculation result. This technology can distribute and route a packet containing a specific destination port number to N dedicated servers prepared for each application protocol. In this technology, however, a plurality of processing nodes (servers) and one router together constitute a cluster. The router receives a packet containing a virtual address (representative address) allocated to this cluster as the destination address and distributes it to a plurality of servers inside the cluster. In other words, one router allocates all the packets sent from user terminals to a plurality of servers. Therefore, performance of the router may result in the bottleneck of the capacity of providing the services through the cluster.
P-0011[0011] When the load balancing apparatuses are connected in multiple stages, each load balancing apparatus distributes the traffics by using conjointly an address translation function. A system using the load balancing apparatuses generally uses properly a virtual address as the representative of all the Web servers and individual addresses of the Web servers. The virtual address is set to the load balancing apparatuses. A client transmits a packet containing the virtual address as a destination address. Receiving the packet containing the virtual address as the destination, the load balancing apparatus selects the Web servers in accordance with a designated rule. The load balancing apparatus translates the destination address contained in the packet to the actual address of the selected Web server and -routes it. Similarly, the load balancing apparatus on the upstream side executes address translation when routing is made from the load balancing apparatuses on the upstream side to the load balancing apparatuses on the downstream side. Since the address translation function has a high load on the process of its own, a higher processing rate cannot be expected. Ordinary load balancing apparatuses distribute the packets on the basis of higher order layer information of layers 4 to 7. Therefore, the load balancing apparatuses are in many cases the apparatuses based on software. In other words, performance of the load balancing apparatus itself becomes a weak point and performance of the overall system cannot be easily improved.
P-0012[0012] The invention provides an apparatus, a system and a method each capable of distributing large quantities of traffics from the Internet to a plurality of load balancing apparatuses inside a site at the site providing information services through the Internet, and further distributing the traffics from each load balancing apparatus to a plurality of Web servers. The invention provides an apparatus, a system and a method each capable of distributing traffics to a plurality of load balancing apparatuses without conducting address translation that has been necessary in load balancing apparatuses according to the prior art.
P-0013[0013] The invention provides a network system including a plurality of Web servers, a plurality of load balancing apparatuses connected to at least one of these Web servers and distributing access packets to each Web server, and a router connected to the load balancing apparatuses and routing the access packets received through the network to each load balancing apparatus. The router decides the load balancing apparatus to be routed by use of at least application protocol information contained in header information of each packet and a hash value calculated in accordance with header information.
P-0014[0014] More concretely, when receiving the packet from the network, the router compares header information of the reception packet with a predetermined retrieval condition. When this header information is coincident with the retrieval condition, the router calculates the hash value on the basis of the header information. The router decides the load balancing apparatus to which the reception packet is to be routed in accordance with the hash value, and routes the packet.
P-0015[0015] The router can route at least two packets having the same application protocol information to an arbitrary load balancing apparatus decided in accordance with the hash value.
BRIEF DESCRIPTION OF THE DRAWINGS
P-0016[0016]FIG. 1 shows a construction of a Web system <b>10</b> according to an embodiment of the invention;
P-0017[0017]FIG. 2 shows a construction of a policy routing table;
P-0018[0018]FIG. 3 shows a construction of a hash table “#1”;
P-0019[0019]FIG. 4 shows a construction of a hash table “#2”;
P-0020[0020]FIG. 5 shows a construction of a TCP/IP packet;
P-0021[0021]FIG. 6 shows a construction of a Web system <b>100</b>;
P-0022[0022]FIG. 7 shows an example of a content of the policy routing table;
P-0023[0023]FIG. 8 shows a construction of an encrypted TCP/IP packet;
P-0024[0024]FIG. 9 shows a construction of a load balancing apparatus;
P-0025[0025]FIG. 10 shows a construction of a pattern table;
P-0026[0026]FIG. 11 shows a construction of a hash table;
P-0027[0027]FIG. 12 shows a construction of an output destination look-up table under an initial state; and
P-0028[0028]FIG. 13 shows a construction of the forwarding cache after entries are registered.
DETAILED DESCRIPTION OF THE EMBODIMENTS
P-0029[0029] Embodiments of the invention will be hereinafter explained in detail with reference to the drawings.
P-0030[0030]FIG. 1 shows a construction of a Web system that constitutes an information service site according to an embodiment of the invention.
P-0031[0031] A plurality of client apparatuses <b>11</b><i>a </i>to <b>11</b><i>d </i>using PC (Personal Computers), etc, for receiving information services and a Web system <b>10</b> of a site providing the information services are connected to one another through a network such as the Internet <b>12</b>. The Web system <b>10</b> includes a router <b>13</b> connected to the Internet <b>12</b> and load balancing apparatuses A<b>14</b>, B<b>15</b>, C<b>16</b> and D<b>17</b> for distributing traffics (access packets) allocated by the router <b>13</b> to a plurality of Web servers. The load balancing apparatus A<b>14</b> and B<b>15</b> execute ordinary HTTP and are connected to the Web servers <b>18</b><i>a </i>to <b>18</b><i>d </i>responding to HTTP packets from the client apparatuses. The load balancing apparatuses C<b>16</b> and D<b>17</b> execute HTTPS and are connected to the Web servers <b>19</b><i>a </i>to <b>19</b><i>d </i>responding to HTTPS packets. Generally, the Web servers <b>18</b><i>a </i>to <b>18</b><i>d </i>and <b>19</b><i>a </i>to <b>19</b><i>d </i>are so constituted as to provide the same information service. A virtual IP address “10.0.0.1” is allocated as the address representing all the Web servers <b>18</b><i>a </i>to <b>18</b><i>d </i>and the Web servers <b>19</b><i>a </i>to <b>19</b><i>d</i>. This virtual IP address “10.0.0.1” is set to all load balancing apparatuses A<b>14</b>, B<b>15</b>, C<b>16</b> and D<b>17</b>. Each load balancing apparatus receives access packets containing the virtual IP address “10.0.0.1” as a destination address from the router <b>13</b> and distributes them to each Web server. The load balancing apparatus A<b>14</b>, B<b>15</b>, C<b>16</b> and D<b>17</b> respectively have their individual IP addresses, that is, “50.0.0.1”, “60.0.0.1”, “70.0. 0.1” and “80.0.0.1”.
P-0032[0032] Referring to FIG. 1, the router <b>13</b> is connected to four load balancing apparatuses and each load balancing apparatus is connected to two Web servers. However, the numbers of the load balancing apparatuses connected to the router and the Web servers connected to each load balancing apparatus are not limited to the example, but they may be greater numbers, respectively.
P-0033[0033] The router <b>13</b> includes a reception unit <b>31</b> for executing reception control of the packets, a transmission unit <b>33</b> for executing transmission control, a buffer <b>32</b> for temporarily storing the packets and a route search unit <b>30</b> for deciding a routing destination of the packet received, as shown in FIG. 1. The route search unit <b>30</b> includes a policy routing retrieval unit <b>34</b> for executing a policy routing function, a policy routing table <b>37</b>, a hash table retrieval unit <b>35</b> for retrieving a hash table, the hash table <b>38</b> and a transmission control unit <b>36</b> for indicating the routing destination of the packet to the transmission unit <b>33</b>. A memory (not shown) provided to the router <b>13</b> stores the policy routing table <b>37</b> and the hash table <b>38</b>.
P-0034[0034]FIG. 2 shows a construction of the policy routing table <b>37</b>.
P-0035[0035] As shown in the drawing, the policy routing table <b>37</b> includes a plurality of entries. Each entry includes a plurality of fields respectively containing information of a policy number <b>50</b> representing an item number, a look-up key <b>51</b>, a next hop address <b>57</b> obtained as a result of retrieval and a transmission interface <b>58</b>. The memory (not shown) provided to the router <b>13</b> stores the information contained in the policy routing table <b>37</b>.
P-0036[0036] Relying on the routing function and the policy routing function of the router, the router generally decides an optimum route to be next forwarded with the address of an adjacent apparatus of a routing destination on the basis of a destination IP address and application protocol information. The next hop address <b>57</b> represents the address of the adjacent apparatus as the routing destination. In this embodiment, the address of the adjacent apparatus of the routing destination or hash table number information is set as the next hop address <b>57</b> to the policy routing table <b>37</b>.
P-0037[0037] A destination IP address <b>52</b>, a source IP address <b>53</b> and a protocol type <b>54</b> that are contained in a IP header and a destination port number <b>55</b> and a source port number <b>56</b> contained in the TCP header are used as the look-up key <b>51</b>. Therefore, the field in the policy routing table <b>37</b> inclusive of the look-up key <b>51</b> further contains five fields corresponding to the look-up keys. Though this embodiment uses these five kinds of header information as the look-up key <b>51</b>, a greater number of, or different, header information may be used, too.
P-0038[0038] Incidentally, symbol “*” in FIG. 2 means “Don't Care”. In other words, the header information corresponding to the field to which “*” is set is excluded from the object of the look-up key. Further, the header information corresponding to the field having “*” may have an arbitrary value.
P-0039[0039]FIG. 3 shows a construction of the hash table “#1” <b>38</b><i>a</i>. FIG. 4 shows a construction of the hash table “#2” <b>38</b><i>b</i>. As shown in the drawings, each of the hash tables <b>38</b><i>a </i>and <b>38</b><i>b </i>contains a plurality of entries. Each entry includes a plurality of fields containing a hash value <b>60</b>, a next hop address <b>61</b> as routing destination information and information of a transmission interface <b>62</b>. The memory (not shown) provided to the router <b>13</b> stores these kinds of information contained in the hash table <b>38</b>.
P-0040[0040] A range of hash values is set as a hash value <b>60</b> to the hash table so that the transmission can be decided in accordance with the range of the hash value. When this range of the hash value is flexibly changed, setting with weight can be set to the transmission. For example, two uniform ranges of “0 to 127” and “128 to 255” are set as the hash value <b>60</b> to the hash table <b>38</b><i>a</i>. On the other hand, two ranges of “0 to 191” and “192 to 255” are set as the hash value <b>60</b> to the hash table <b>38</b><i>b</i>. In this case, the range of the hash value of the former is broader than the range of the latter. Therefore, setting is made in this case so that the weight of the transmission corresponding to the former range is greater. Incidentally, an example of a calculation method of the hash value will be explained elsewhere.
P-0041[0041] An example of a concrete operation of the Web system <b>10</b> shown in FIG. 1 will be now explained.
P-0042[0042] First, when the client apparatus <b>11</b><i>a </i>gains access to the Web server that executes HTTP, the client apparatus <b>11</b><i>a </i>forwards the TCP/IP packet (HTTP packet) carrying HTTP information to the Web system <b>10</b> of the site.
P-0043[0043]FIG. 5 shows a construction of the TCP/IP packet. As shown in the drawing, the TCP/IP packet includes an IP header <b>70</b>, a TCP header <b>71</b> and data <b>72</b>. The IP header <b>70</b> contains a destination IP address <b>73</b>, a source IP address <b>74</b> and a protocol type <b>75</b>. The TCP header <b>71</b> contains a destination port number <b>76</b> and a source port number <b>77</b>. The destination port number <b>76</b> is a kind of information (application protocol information) that represents the kind of the application protocol to be executed by the Web server. In this example, the client <b>11</b><i>a </i>sets the virtual address “10.0.0.1” as the destination IP address <b>73</b> of the TCP/IP packet, the address “192.10.0.100” of the client <b>11</b><i>a </i>itself as the source IP address <b>74</b>, “TCP” as the protocol type <b>75</b> and “HTTP” as the destination port number <b>76</b>, and forwards this TCP/IP packet.
P-0044[0044] In practice, however, the number allocated in advance to each application protocol is stored in the destination port number field <b>76</b> inside the TCP/IP packet. Since “80” is allocated to HTTP, for example, “80” is stored in the destination port number field <b>76</b>. Here, the destination port number <b>76</b> is expressed as “HTTP” for ease of explanation.
P-0045[0045] The virtual address is the address as the representative of all Web servers as described above. In a system that uses load balancing apparatuses, this virtual address and the individual address of the Web server itself are used separately and properly. The virtual address is set to each load balancing apparatus. The client forwards the packet to the virtual address. Receiving the packet containing the virtual address as the destination address, each load balancing apparatus selects the Web server in accordance with an algorithm such as a round robin system. Each load balancing system translates the destination address contained in the packet to the actual address of the selected Web server and routes the packet. In this embodiment, “10.0.1.1” is set in advance as the virtual address.
P-0046[0046] The router <b>13</b> inside the Web system <b>10</b> receives the TCP/IP packet by the reception unit <b>31</b> and stores it in the buffer <b>32</b>. The reception unit <b>31</b> extracts the header information of the reception packet and transfers it to the policy routing retrieval unit <b>34</b> inside the route search unit <b>30</b>. The policy routing retrieval unit <b>34</b> extracts the destination IP address <b>73</b>, the source IP address <b>74</b>, the protocol type <b>75</b>, the destination port number <b>76</b> and the source port number <b>77</b> as the look-up key in the header information and retrieves the policy routing table <b>37</b>. The reception packet contains “10.0.0.1” as the destination IP address <b>73</b>, “TCP” as the protocol type <b>75</b> and “HTTP” as the destination port number <b>76</b>. The information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “1” in the policy routing table <b>37</b>. As a result, the policy routing retrieval unit <b>34</b> looks up the next hop address <b>57</b> of the entry having the policy number <b>50</b> “1”. In this case, the value of the next hop address <b>57</b> is the hash table number information and its value is “#1”. Therefore, the policy routing retrieval unit <b>34</b> further transfers the header information and the hash table number information to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> calculates the hash value by use of a predetermined hash function and the header information. The hash table retrieval unit <b>35</b> retrieves the hash table <b>38</b><i>a </i>designated by the hash table number information “#1”. Various hash functions are available but this embodiment simply uses a modulo value of 256 for the lowest order byte of the source IP address. The header information that the hash table retrieval unit <b>35</b> receives contains “192.10.0.100” as the source IP address <b>74</b>, and its lowest order byte is “100”. Therefore, the hash value the hash table retrieval unit <b>35</b> calculates is “100”. This hash value “100” falls within the range of “0 to 127” in the hash table <b>38</b><i>a. </i>
P-0047[0047] The next hop address <b>61</b> corresponding to the range of the hash value is “50.0.0.1” and the transmission interface <b>62</b> corresponding to this range is “If1”. The next hop address <b>61</b> “50.0.0.1” is the IP address of the load balancing apparatus A<b>14</b>. Therefore, the hash retrieval unit <b>35</b> reads out the routing information from the hash table <b>38</b><i>a </i>and transfers them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> forwards the routing destination information to the transmission unit <b>33</b> and gives an output instruction. Receiving this instruction, the transmission unit <b>33</b> reads the reception packet from the buffer <b>32</b> and transmits the packet from “If1” to the line. The router <b>13</b> routes in this way the reception packet to the load balancing apparatus A<b>14</b>. Incidentally, the router <b>13</b> merely routes the packet on the basis of the next hop address information acquired from the policy routing table <b>37</b> or the hash table <b>38</b>. Therefore, the router <b>13</b> does not execute address translation of the destination IP address <b>73</b> that is executed when each load balancing apparatus distributes the access packet to the Web server.
P-0048[0048] Receiving the packet, the load balancing apparatus A<b>14</b> allocates the packet to the Web server <b>18</b><i>a </i>or <b>18</b><i>b </i>in accordance with a designated rule. This rule includes algorithms such as the round robin method and the hash method using the hash function. FIG. 1 represents the case where the load balancing apparatus A<b>14</b> allocates the packet to the Web server <b>18</b><i>a. </i>
P-0049[0049] All the load balancing apparatuses A<b>14</b>, B<b>15</b>, C<b>16</b> and D<b>17</b> have the same construction. Therefore, the construction of the load balancing apparatus A<b>14</b> and its operation will be explained concretely by way of example.
P-0050[0050]FIG. 9 shows the construction of the load balancing apparatus A<b>14</b>.
P-0051[0051] The load balancing apparatus A<b>14</b> includes a reception unit <b>91</b> for receiving packets, a transmission unit <b>93</b> for transmitting the packets, a buffer <b>92</b> for temporarily storing the reception packets and a load balancing process unit <b>90</b> for deciding the transmission of the reception packets, as shown in FIG. 9. The load balancing process unit <b>90</b> includes a forwarding unit <b>94</b> for retrieving the transmission of the reception packets, a pattern table <b>97</b>, a hash table <b>98</b>, a forwarding cache <b>99</b> and a header translation unit <b>96</b> for translating the header information of the reception packet and indicating the transmission to the transmission unit <b>93</b>. A memory (not shown) provided to the load balancing apparatus A<b>14</b> stores the pattern table <b>97</b>, the hash table <b>98</b> and the forwarding cache <b>99</b>. The router <b>13</b> is connected to If0 of the load balancing apparatus A<b>14</b> through a communication line, and the Web server <b>18</b><i>a </i>and the Web server <b>18</b><i>b </i>are connected respectively to If1 and If2 through the communication line. As shown in FIG. 9, the load balancing process unit <b>90</b> of the load balancing apparatus A<b>14</b> has the hash table <b>98</b> and distributes the packets to the two Web servers by use of the hash method. However, the load balancing process unit <b>90</b> can distribute the packets to the two Web servers in accordance with other algorithms such as the round robin method without having the hash table.
P-0052[0052]FIG. 10 shows a construction of the pattern table <b>97</b>.
P-0053[0053] As shown in this drawing, the pattern table <b>97</b> contains a plurality of entries. Each entry has a plurality of fields including an entry number <b>111</b> and a look-up key <b>112</b>. The memory (not shown) provided to the load balancing apparatus A<b>14</b> stores the information contained in the pattern table <b>97</b>.
P-0054[0054] A destination IP address <b>113</b> and a source IP address <b>114</b> contained in the IP header of the packet and a destination port number <b>115</b> and a source port number <b>116</b> contained in the TCP header are used as the look-up key <b>112</b>. Therefore, the field containing the look-up key <b>112</b> of the pattern table <b>98</b> contains four fields corresponding to these look-up keys, on the contrary. Though this embodiment uses these four header information as the look-up key <b>112</b>, a greater number or, or different, header information may be used, too.
P-0055[0055] In FIG. 10, too, symbol “*” means “Don't Care”.
P-0056[0056]FIG. 11 shows the construction of the hash table <b>98</b>.
P-0057[0057] As shown in the drawing, the hash table <b>98</b> contains a plurality of entries. Each entry has a plurality of fields including a hash value <b>117</b>, a destination IP address <b>118</b> as transmission information and a transmission interface <b>119</b>. The memory (not shown) provided to the load balancing apparatus A<b>14</b> stores the information contained in the pattern table <b>98</b>.
P-0058[0058] A range of the hash value is set as the hash value <b>117</b> to the hash table so that the transmission can be decided in accordance with the range of the hash value. Two ranges of “0 to 127” and “128 to 255” are set as the hash value to the hash table <b>98</b>, for example.
P-0059[0059]FIG. 12 shows a construction of the forwarding cache <b>99</b>.
P-0060[0060] As shown in the drawing, the forwarding cache <b>99</b> has a plurality of entries. Each entry has a plurality of fields including information of an entry number <b>121</b>, a look-up key <b>122</b>, translation information <b>127</b> and a transmission interface <b>130</b>. The memory (not shown) provided to the load balancing apparatus A<b>14</b> stores the information contained in the forwarding cache <b>99</b>.
P-0061[0061] A destination IP address <b>123</b> and a source IP address <b>124</b> contained in the IP header of the packet and a destination port number <b>125</b> and a source port number <b>126</b> contained in the TCP header are used as the look-up key <b>122</b>. Therefore, the field containing the look-up key <b>122</b> of the forwarding cache <b>97</b> further contains four fields corresponding to these look-up keys. Though this embodiment uses these four head information as the look-up key <b>122</b>, a greater number or, or different, header information may be used, too.
P-0062[0062] When the load balancing apparatus is under the initial condition such as at the start of operation, the forwarding cache <b>99</b> does not have the entry having a concrete value registered to each field. FIG. 12 shows the forwarding cache <b>99</b> before each entry is registered. When the load balancing apparatus starts distributing the packets to each Web server, the entry containing the information used for deciding the transmission of the packet is registered to the forwarding cache <b>99</b>.
P-0063[0063]FIG. 13 shows the state after the entry is registered to the forwarding cache <b>99</b>.
P-0064[0064] When the packet received from the router <b>13</b> is distributed to any Web server, two entries are registered to the forwarding cache <b>99</b>. One of the entries contains the information used for outputting the reception packet to any Web server. The other entry contains the information used for outputting the packet transmitted from the Web server in response to the reception packet to the router <b>13</b>. As these two entries are registered in this way, the load balancing apparatus A<b>14</b> can execute two-way routing.
P-0065[0065] A concrete operation of the load balancing apparatus A<b>14</b> shown in FIG. 9 will be hereinafter explained.
P-0066[0066] First, the reception unit <b>91</b> of the load balancing apparatus A<b>14</b> receives the packed routed by the router <b>13</b> and stores the reception packet in the buffer <b>92</b>. In the example described above, the reception packet is the TCP/IP packet that the client apparatus <b>11</b><i>a </i>transmits for the purpose of the HTTP packet. Therefore, the IP header of the reception packet contains the virtual address “10.0.0.1” as the destination IP address <b>73</b>, “192.10.0.100”, that is, the address of the client <b>11</b><i>a </i>of its own, as the source IP address <b>74</b>, and “TCP” as the protocol type <b>75</b>. The TCP header contains “HTTP” as the destination port number <b>76</b> and “65000” as the source port number <b>77</b>. The reception unit <b>91</b> extracts the header information of the reception packet and transfers it to the forwarding unit <b>94</b> of the load balancing process unit <b>90</b>. The forwarding unit <b>94</b> extracts the destination IP address <b>73</b>, the source IP address <b>74</b>, the destination port number <b>76</b> and the source port number <b>77</b> as the look-up key from the header information and retrieves the forwarding cache <b>99</b>. In this example, the forwarding cache <b>99</b> is under the state shown in FIG. <b>12</b>. Therefore, the forwarding unit <b>94</b> does not judge that the entry having the information coincident with the extracted look-up key registered thereto exists in the forwarding cache <b>99</b>. Subsequently, the forwarding unit <b>94</b> retrieves the pattern table <b>97</b> with the extracted look-up key. The extracted look-up key is coincident with the information contained in the entry having the entry number <b>11</b> “1” in the pattern table <b>97</b>. Therefore, the forwarding unit <b>94</b> calculates the hash value by use of a predetermined hash function and the header information so received.
P-0067[0067] The hash function uses the modulo value of 256 for the lowest order byte of the source IP address in the same way as the hash table retrieval unit <b>35</b> of the router as described above. The header information the forwarding unit <b>94</b> receives contains “192.10.0.100” as the source IP address <b>74</b> and its lowest order byte is “100”. Therefore, the hash value the forwarding unit <b>94</b> calculates is “100”. Next, the forwarding unit <b>94</b> retrieves the hash table <b>98</b>. The hash value “100” is contained in the range of “0 to 127” in the hash table <b>98</b>. The destination IP address <b>118</b> and the transmission interface <b>119</b> corresponding to the range of this hash value are “10.0.0.10” and “If1”, respectively. The IP address “10.0.0.10” is the address of the Web server <b>18</b><i>a</i>. The forwarding unit <b>94</b> reads the transmission information from the hash table <b>98</b> and transfers them with the header information to the header translation unit <b>96</b>.
P-0068[0068] The forwarding unit <b>94</b> registers the information of the look-up key extracted from the header information so received and the transmission information read out from the hash table <b>98</b> to the forwarding cache <b>99</b>. The forwarding unit <b>94</b> first registers the destination IP address <b>123</b> “10.0.0.1”, the source IP address <b>124</b> “192.10.0.100”, the destination port number <b>125</b> “HTTP” and the source port number <b>126</b> “65000” to the look-up key field <b>122</b> of the entry having the entry number <b>121</b> “1”. The output retrieval unit <b>194</b> registers the destination IP address <b>128</b> “10.0.0.10” to the translation information field <b>127</b> of the entry and “If1” to the transmission interface field <b>130</b>. The forwarding unit <b>94</b> further registers the destination IP address <b>123</b> “192.10.0.100”, the source IP address <b>124</b> “10.0.0.1”, the destination port number <b>125</b> “65000” and the source port number <b>126</b> “HTTP” to the look-up key field <b>122</b> of the entry having the entry number <b>121</b> “2”. The forwarding unit <b>94</b> registers the source IP address <b>128</b> “10.0.0.1” to the translation information field <b>127</b> of the entry and “If0” to the transmission interface field <b>130</b>. The information registered in this way to the forwarding cache <b>99</b> is shown in FIG. 13. The information of the entry having the entry number <b>121</b> “1” is used for retrieving the transmission of the packet that the forwarding unit <b>94</b> receives from the router <b>13</b>. The information of the entry having the entry number <b>121</b> “2” is used for retrieving the transmission of the packet that the forwarding unit <b>94</b> receives from the Web server <b>18</b><i>a. </i>
P-0069[0069] Incidentally, when the entry is registered to the forwarding cache <b>99</b>, the forwarding unit <b>94</b> need not simultaneously register two entries but may register each entry with a time interval. As to the entry having the entry number <b>121</b> “2”, the entry may be registered when the packet is received from the Web server <b>18</b><i>a. </i>
P-0070[0070] The header translation unit <b>96</b> receives the header information and the transmission information from the forwarding unit <b>94</b>, and translates the destination IP address “10.0.0.1” contained in the header information to the destination IP address “10.0.0.10” in transmission information. The header translation unit <b>96</b> transmits the header information after address translation and the information of the transmission interface to the transmission unit <b>93</b>.
P-0071[0071] Receiving the information of the transmission interface from the header translation unit <b>96</b>, the transmission unit <b>93</b> reads the reception packet from the buffer <b>92</b>. The transmission unit <b>93</b> changes the header information of the reception packet to the header information received from the header translation unit <b>96</b> and transmits the packet from “If1” designated.
P-0072[0072] Incidentally, when the packet is stored in the buffer <b>92</b>, the reception unit <b>91</b> may store only the data contained in the packet in the buffer <b>92</b>. In this case, the transmission unit <b>93</b> reads the data from the buffer <b>92</b>, adds the header information received from the header translation unit <b>96</b> to the data and creates the packet.
P-0073[0073] The Web server <b>18</b><i>a </i>that receives the HTTP packet from the load balancing unit A<b>14</b> executes a process required from the client apparatus <b>11</b><i>a </i>as the source transmitting party and returns the response packet containing the process result (required information). The response packet contains the destination IP address “192.10.0.100”, the source IP address “10.0.0.10”, the destination port number “65000” and the source port number “HTTP”. These kinds of information are the information all contained in the HTTP packet the Web server <b>18</b><i>a </i>receives.
P-0074[0074] The reception unit <b>91</b> of the load balancing apparatus A<b>14</b> receives the response packet and stores it in the buffer <b>92</b>. The reception unit <b>91</b> extracts the header information of the response packet and forwards it to the forwarding unit <b>94</b> of the load balancing process unit. The forwarding unit <b>94</b> extracts the destination IP address, the source IP address, the destination port number and the source port number as the look-up key from the header information and retrieves the forwarding cache <b>99</b>. Two entries are registered to the forwarding cache <b>99</b> as shown in FIG. 13. The forwarding unit <b>94</b> judges the information contained in the look-up key <b>122</b> having the entry number <b>121</b> “2” as being coincident with the information extracted from the header information. Therefore, the forwarding unit <b>94</b> reads out the source IP address “10.0.0.1” from the translation information field <b>127</b> of that entry and “If0” from the transmission interface field <b>130</b>, and sends the information and the header information to the header translation unit <b>96</b>.
P-0075[0075] The header translation unit <b>96</b> receives the information sent and translates the source IP address “10.0.0.10” contained in the header information to the source IP address “10.0.0.1” received. The header translation unit <b>96</b> forwards the header information after address translation and the information of the transmission interface to the transmission unit <b>93</b>.
P-0076[0076] Receiving the header information and the information of the transmission interface from the header translation unit <b>96</b>, the transmission unit <b>93</b> reads the response packet from the buffer <b>92</b>, changes the header information and transmits the response packet from designated “If0”.
P-0077[0077] Receiving the response packet from the load balancing apparatus A<b>14</b>, the router <b>13</b> decides the routing destination from the destination IP address “192.10.0.100” of the response packet, and routes the response packet to the client apparatus <b>11</b><i>a</i>. The policy routing retrieval unit <b>34</b> or the hash table retrieval unit <b>35</b> may decide the routing destination of the response packet. The route search unit <b>30</b> of the router <b>13</b> may further have a routing unit not shown in FIG. 1. In this case, the routing unit decides the routing destination of the response packet.
P-0078[0078] Next, it will be assumed that the client apparatus <b>11</b><i>b </i>similarly makes the HTTP packet. The destination IP address <b>73</b> of the TCP/IP packet transmitted by the client apparatus <b>11</b><i>b </i>is “10.0.0.1”. The source IP address <b>74</b> is “192.10.0.200” that is the own address of the source IP address <b>74</b>. The protocol type <b>75</b> is “TCP”. The destination port number <b>76</b> is “HTTP”. The router <b>13</b> receives and allocates this packet in the same way as the HTTP packet by the client apparatus <b>11</b><i>a </i>described above. More concretely, the content contained in the header information of this packet is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “1” in the policy routing table <b>37</b>. Therefore, the policy routing retrieval unit <b>34</b> reads “hash table #1” from the next hop address field <b>57</b> of the entry. The policy routing retrieval unit <b>34</b> forwards the header information and the hash table number information “#1” to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> calculates the hash value from the header information. Since the source IP address is “192.10.0.200”, the hash value is “200”. The hash table retrieval unit <b>35</b> retrieves the hash table <b>38</b><i>a </i>designated by the hash table number information “#1”, reads the next hop address <b>61</b> “60.0.0.1” (IP address of load balancing apparatus B<b>15</b>) corresponding to the hash value “200” and the transmission interface <b>62</b> “If2”, and forwards them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> delivers the routing destination information to the transmission unit <b>33</b> and gives an input instruction. Receiving the instruction, the transmission unit <b>33</b> transmits the reception packet from “If2” to the line and routes it to the load balancing apparatus B<b>15</b>.
P-0079[0079] The load balancing apparatus B<b>15</b> operates in the same way as the load balancing apparatus A<b>14</b> and allocates the packet to the Web server <b>18</b><i>c </i>or <b>18</b><i>d</i>. FIG. 1 shows the case where the load balancing apparatus B<b>15</b> allocates the packet to the Web server <b>18</b><i>d. </i>
P-0080[0080] Next, explanation will be given on the case where the client apparatus <b>11</b><i>c </i>gains access to the Web server that executes HTTPS. When the HTTPS packet is made, too, the client apparatus <b>11</b><i>c </i>transmits the TCP/IP packet in the same way as it makes the HTTP packet. The destination IP address <b>73</b> of the TCP/IP packet transmitted by the client apparatus <b>11</b><i>c </i>is “10.0.0.1”. The source IP address <b>74</b> is “192.10.0.50” that is the address of the source IP address itself. The protocol type <b>75</b> is “TCP”. The destination port number <b>76</b> is “HTTPS”. The router <b>13</b> receives this packet in the reception unit <b>31</b> and stores it in the buffer <b>32</b>. The reception unit <b>31</b> extracts the header information of the reception packet and forwards it to the policy routing retrieval unit <b>34</b>. The content contained in this header information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “2”. Therefore, the policy routing retrieval unit <b>34</b> reads “hash table #2” contained in the next hop address field <b>57</b> of the entry. The policy routing retrieval unit <b>34</b> transfers the header information and the hash table number information “#2” to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> calculates the hash value from the header information. Since the source IP address <b>74</b> is “192.10.0.50”, the hash value is “50”. The hash table retrieval unit <b>35</b> retrieves the hash table <b>38</b><i>b </i>designated by the hash table number information “#2”, reads the next hop address <b>61</b> “70.0.0.1” (IP address of load balancing apparatus C<b>16</b>) corresponding to the hash value “50” and the transmission interface <b>62</b> “If3” and forwards them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> delivers the routing destination information to the transmission unit <b>33</b> and gives an output instruction. Receiving the instruction, the transmission unit <b>33</b> reads the reception packet from the buffer <b>32</b>, transmits it from “If3” to the line and routes the packet to the load balancing apparatus C<b>16</b>. The load balancing apparatus C<b>16</b> allocates the packet to the Web server <b>19</b><i>a </i>or <b>19</b><i>b </i>in the same way as the load balancing apparatus A<b>14</b> described above. FIG. 1 shows the case where the load balancing apparatus C<b>16</b> allocates the packet to the Web server <b>19</b><i>a. </i>
P-0081[0081] When the client apparatus <b>11</b><i>d </i>makes the HTTPS packet, too, the router <b>13</b> routes the packet to the load balancing apparatus D<b>17</b>. The load balancing apparatus D<b>17</b> allocates the packet to the Web server <b>19</b><i>c </i>or <b>19</b><i>d. </i>
P-0082[0082] Incidentally, in all of the cases described above, the router <b>13</b> merely routes the packet to the load balancing apparatus on the basis of the next hop address information but does not execute address translation.
P-0083[0083] As explained above, the router <b>13</b> according to this embodiment can allocate the access packet for the same application protocol to a plurality of load balancing apparatuses. In consequence, the load on the load balancing apparatuses and on the Web servers can be mitigated. To allocate the packet, the router <b>13</b> need not execute address translation that is made by the load balancing apparatus. Therefore, the router itself does not become a bottleneck of the Web system.
P-0084[0084] When the range of the hash value of the hash table <b>38</b> in the router <b>13</b> is appropriately set, allocation ratios of specific load balancing apparatuses and specific Web servers can be increased. Therefore, a system that is more flexible can be constituted.
P-0085[0085] As a result, the site providing the information service can cope with an abrupt increase of traffics and can continue servicing without deteriorating high-speed performance and high reliability.
P-0086[0086]FIG. 6 shows another construction of a Web system of a site providing information services. The Web system <b>100</b> shown in FIG. 6 is different from the Web system <b>10</b> shown in FIG. 1 in that it can use a security technology called “IPsec (IP security)”.
P-0087[0087] IPsec is a technology that encrypts the packet itself and executes communication. A firm security system can be constituted by use of IPsec. However, IPsec generally encrypts the packets as a whole. Therefore, the router cannot extract the destination port number (application protocol information) inside the TCP header, and cannot distribute the traffics for each application protocol such as HTTP and HTTPS.
P-0088[0088] The system shown in FIG. 6 can distribute the packets by handling the IPsec packet itself as an access packet for a kind of application protocol.
P-0089[0089] A concrete construction of this Web system <b>100</b> will be hereinafter explained.
P-0090[0090] The same reference numeral is used in FIG. 6 to identify the same constituent member as that of the Web system <b>10</b> shown in FIG. 1. In other words, the router <b>13</b>, the load balancing apparatuses A<b>14</b> to D<b>17</b> and the Web servers <b>19</b><i>a </i>to <b>19</b><i>d </i>are exactly the same as those shown in FIG. 1. The Web system <b>100</b> shown in FIG. 6 further includes Web servers <b>23</b><i>a </i>to <b>23</b><i>d </i>and an IPsec router B<b>22</b> that copes with IPsec. The Web servers <b>23</b><i>a </i>and <b>23</b><i>b </i>are connected to the load balancing apparatus A<b>14</b> and the Web servers <b>23</b><i>c </i>and <b>23</b><i>d </i>are connected to the load balancing apparatus B<b>15</b>. The IPsec router B<b>22</b> is connected to the router <b>13</b>. IPsec-associated client apparatuses <b>20</b><i>a </i>and <b>20</b><i>b </i>corresponding to IPsec and an IPsec router A<b>21</b> are connected to the Web system <b>100</b> through the Internet <b>12</b>. Client apparatuses <b>11</b><i>c </i>and <b>11</b><i>d </i>not corresponding to IPsec are connected to the IPsec router A<b>21</b>.
P-0091[0091]FIG. 7 shows an example of the content of each entry of the policy routing table <b>37</b> of the router <b>13</b> shown in FIG. 6. As shown in the drawing, the entries having the policy number <b>50</b> “1” and “2” among the entries contain “IPsec” as the protocol type <b>54</b>. These two entries are used when the router <b>13</b> routes the TCP/IP packet encrypted by use of IPsec.
P-0092[0092] Incidentally, information called “AH” (Authentication Header) is practically stored in the protocol type field inside the IP header of the TCP/IP packet encrypted by use of IPsec. To have the explanation more easily understood, however, the protocol type is expressed hereby as “IPsec”.
P-0093[0093] A concrete operation of the Web system <b>100</b> having the construction described above will be explained.
P-0094[0094] Explanation will be given first on the case of a communication form of IPsec called “transport mode”. In this transport mode, both client apparatus and Web server encrypt the TCP/IP packet and communicate with each other.
P-0095[0095] When the IPsec-associated client apparatus <b>20</b><i>a </i>makes HTTP packet, the IPsec-associated client apparatus <b>20</b><i>a </i>executes an encryption process for the TCP/IP packet (HTTP packet) carrying the HTTP information and transmits the encrypted IP packet to the Web system <b>100</b>.
P-0096[0096]FIG. 8 shows a construction of the encrypted TCP/IP packet. As shown in the drawing, the encrypted packet has an IP header <b>70</b> and a encrypted data <b>78</b>. The IP header <b>70</b> has the same construction as the construction shown in FIG. 5. The encrypted data <b>78</b> is created when the TCP header, data and other information created by the IPsec-associated client apparatus <b>20</b><i>a </i>for the HTTP packet are encrypted.
P-0097[0097] In this example, the IP header <b>70</b> of the encrypted IP packet includes a virtual address “10.0.0.1” of the IPsec-associated Web server as the destination IP address <b>73</b>, the address “192.10.0.100” of the client apparatus <b>20</b><i>a </i>as the source IP address and “IPsec” as the protocol type <b>75</b>.
P-0098[0098] The router <b>13</b> of the Web system <b>100</b> receives the encrypted IP packet in the reception unit <b>31</b> and stores it in the buffer <b>32</b>. The reception unit <b>31</b> extracts the IP header information of the reception packet and transfers it to the policy routing retrieval unit <b>34</b> inside the route search unit <b>30</b>. The policy routing retrieval unit <b>34</b> extracts the destination IP address <b>73</b>, the source IP address <b>74</b> and the protocol type <b>75</b> as the look-up key from the IP header information and retrieves the policy routing table <b>37</b>. The information so extracted from the IP header information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “1” of the policy routing table <b>37</b> shown in FIG. 7. Therefore, the policy routing retrieval unit <b>34</b> looks up the next hop address <b>57</b> of that entry. In this case, “hash table #1” is registered to the next hop address field <b>57</b>. The policy routing retrieval unit <b>34</b> reads out the hash table number “#1” from the policy routing table <b>37</b> and transfers it with the IP header information to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> first calculates the hash value from the IP header information. The IP header information contains “192.10.0.100” as the source IP address <b>74</b> and its lowest order byte is “100”. Therefore, the hash value is “100”. The hash table retrieval unit <b>35</b> retrieves the hash table <b>38</b><i>a </i>designated by the hash table number “#1”. In the hash table <b>38</b><i>a</i>, the next hop address <b>61</b> corresponding to the hash value “100” is “50.0.0.1” (IP address of load balancing apparatus A<b>14</b>) and the transmission interface <b>62</b> is “If1”. The hash table retrieval unit <b>35</b> reads out the routing destination information and transfers them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> delivers the routing destination information to the transmission unit <b>33</b> and gives an output instruction. Receiving the instruction, the transmission unit <b>33</b> reads out the encrypted IP packet from the buffer <b>32</b>, transmits it from “If1” to the line and routes the encrypted IP packet to the load balancing apparatus A<b>14</b>. Incidentally, the router <b>13</b> merely routes the packet on the basis of the next hop address information.
P-0099[0099] Receiving the encrypted IP packet, the load balancing apparatus A<b>14</b> operates as described above and allocates the packet to the IPsec-associated Web server <b>23</b><i>a </i>or <b>23</b><i>b</i>. However, only the IP header information can be acquired from the encrypted IP packet. Therefore, the look-up keys <b>112</b> and <b>122</b> of the pattern table <b>97</b> and the forwarding cache <b>99</b> of the load balancing apparatus A<b>14</b> further contain the protocol type information. The forwarding unit <b>94</b> uses the destination IP address <b>73</b>, the source IP address <b>74</b> and the protocol type <b>75</b> contained in the IP header information of the encrypted IP packet as the look-up keys, retrieves each table and acquires the transmission information.
P-0100[0100] Incidentally, FIG. 6 shows the case where the load balancing apparatus A<b>14</b> allocates the encrypted IP packet to the IPsec-associated Web server <b>23</b><i>a. </i>
P-0101[0101] Receiving the encrypted IP packet, the IPsec-associated Web server <b>23</b><i>a </i>deciphers it to the original TCP/IP packet (HTTP packet). The IPsec-associated Web server <b>23</b><i>a </i>then executes the process for the HTTP packet.
P-0102[0102] Next, it will be assumed that the IPsec-associated client apparatus <b>20</b><i>b </i>executes the HTTP packet.
P-0103[0103] Next, it will be assumed that the IPsec-associated client apparatus <b>20</b><i>b </i>makes the HTTP packet. In this case, the IPsec-associated client apparatus <b>20</b><i>b </i>sets the virtual address “10.0.0.1” as the destination IP address <b>73</b> of the IP header <b>70</b>, the address “192.10.0.200” of its own as the source IP address <b>74</b> and “IPsec” as the protocol type <b>75</b> and forwards the encrypted IP packet.
P-0104[0104] In this case, too, the reception unit <b>31</b> of the router <b>13</b> receives the encrypted IP packet in the same way as described above. The route search unit <b>30</b> decides the routing destination of the encrypted IP packet. The information contained in the IP header of the encrypted IP packet is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “1” in the policy routing table <b>37</b>. The hash value calculated from the IP header information is “200”. Therefore, the policy routing retrieval unit <b>34</b> reads out the hash table number “#1” as the next hop address <b>57</b> from the entry having the policy number <b>50</b> “1” in the policy routing table <b>37</b>. The policy routing retrieval unit <b>34</b> transfers the IP header information and the hash table number “#1” to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> reads out the next hop address <b>61</b> “60.0.0.1” (IP address of load balancing apparatus B<b>15</b>) corresponding to the hash value “200” and the transmission interface “If2” from the hash table <b>38</b><i>a </i>designated by the hash table number “#1”, and forwards them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> forwards the routing destination information to the transmission unit <b>33</b> and gives an output instruction. The transmission unit <b>33</b> transmits the encrypted IP packet from designated “If2” to the line. The router <b>13</b> routes in this way the encrypted IP packet to the load balancing apparatus B<b>15</b>.
P-0105[0105] The load balancing apparatus B<b>15</b> operates in the same way as the load balancing apparatus A<b>14</b> and forwards the encrypted IP packet to the IPsec-associated Web server <b>23</b><i>c </i>or <b>23</b><i>d. </i>
P-0106[0106]FIG. 6 shows the case where the load balancing apparatus B<b>15</b> outputs the encrypted IP packet to the IPsec-associated Web server <b>23</b><i>d. </i>
P-0107[0107] Receiving the encrypted IP packet, the IPsec-associated Web server <b>23</b><i>d </i>deciphers it to the original TCP/IP packet (HTTP packet) and executes a process for the HTTP packet.
P-0108[0108] As described above, the router <b>13</b> handles the packet encrypted by use of IPsec as a kind of the access packet of the application protocol and can allocate it to a plurality of load balancing apparatuses.
P-0109[0109] Next, explanation will be given on the case where a communication form of IPsec called “tunnel mode” is used. In this tunnel mode, the client apparatuses and the Web servers need not correspond to IPsec. Instead, an IPsec router associated with IPsec is interposed between them. The IPsec encrypts and deciphers the packets exchanged between the client apparatus and the Web server.
P-0110[0110] When the client apparatus <b>11</b><i>c </i>makes the HTTP packet, the client apparatus <b>11</b><i>c </i>transmits the TCP/IP packet (HTTP packet) carrying the HTTP information to the Web system <b>100</b> of the site. In this case, the destination IP address <b>73</b> contained in the TCP/IP packet transmitted by the client apparatus <b>11</b><i>c </i>is the virtual address “30.0.0.1” of the Web server, the source IP address <b>74</b> is “192.10.0.50” of its own, the protocol type <b>75</b> is “TCP” and the destination port number <b>76</b> is “HTTP”.
P-0111[0111] The IPsec router A<b>21</b> receives this TCP/IP packet. The IPsec router A<b>21</b> encrypts the reception packet and routes it to the Web system <b>100</b>. The construction of this encrypted packet is fundamentally the same as the construction shown in FIG. 8. However, encrypted data <b>78</b> is generated when the TCP/IP packet transmitted by the client apparatus <b>11</b><i>c </i>is encrypted as a whole. Therefore, the IPsec router A<b>21</b> encrypts the reception packet as a whole to generate the encrypted data <b>78</b>, adds the IP header and routes it.
P-0112[0112] In this example, the IP header <b>70</b> of the encrypted IP packet routed by the IPsec router A<b>21</b> contains the address “20.0.0.1” of the IPsec router B<b>22</b> as the destination IP address <b>73</b>, the address “192.0.0.10” of the IPsec router A<b>21</b> as the source IP address <b>74</b> and “IPsec” as the protocol type <b>75</b>.
P-0113[0113] The router <b>13</b> of the Web system <b>100</b> receives this encrypted IP packet in the reception unit <b>31</b> and stores it in the buffer <b>32</b>. The reception unit <b>31</b> extracts the IP header information of the reception packet and transfers it to the policy routing retrieval unit <b>34</b> inside the route search unit <b>30</b>. The policy routing retrieval unit <b>34</b> extracts the destination IP address <b>73</b>, the source IP address <b>74</b> and the protocol type <b>75</b> as the look-up key from the IP header information and retrieves the policy routing table <b>37</b>. The information so extracted from the IP header information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “2” of the policy routing table <b>37</b> shown in FIG. 7. Therefore, the policy routing retrieval unit <b>34</b> reads out the transmission interface <b>58</b> “If5” from that entry. The policy routing retrieval unit <b>34</b> transfers this information to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> indicates the transmission interface “If5” as the transmission to the transmission unit <b>33</b>. The transmission unit <b>33</b> reads out the encrypted IP packet from the buffer <b>32</b> and transmits it from designated “If5” to the line. The router <b>13</b> thus routes the encrypted IP packet to the IPsec router B<b>22</b>.
P-0114[0114] Receiving the encrypted IP packet, the IPsec router B<b>22</b> deciphers it to the original TCP/IP packet. The IPsec router B<b>22</b> again routes the deciphered TCP/IP packet to the router <b>13</b>.
P-0115[0115] The router <b>13</b> receives the deciphered TCP/IP packet in the reception unit <b>31</b> and stores it in the buffer <b>32</b>. The reception unit <b>31</b> extracts the header information of the reception packet and transfers it to the policy routing retrieval unit <b>34</b>. The header information of the deciphered TCP/IP packet contains the destination IP address “30.0.0.1”, the protocol type “TCP” and the destination port number <b>76</b> “HTTP”. Therefore, the content of this header information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “3” of the policy routing table <b>37</b>. Therefore, the policy routing retrieval unit <b>34</b> reads out the information “hash table #2” contained as the next hop address <b>57</b> in that entry. The policy routing retrieval unit <b>34</b> transfers this information and the hash table number information “#2” to the hash table retrieval unit <b>35</b>. The hash table retrieval unit <b>35</b> calculates the hash value from the header information. Since the source IP address <b>74</b> is “192.10.0.50”, the hash value is “50”. The hash table retrieval unit <b>35</b> reads out the next hop address <b>61</b> “70.0.0.1” (IP address of load balancing apparatus C<b>16</b>) corresponding to the hash value “50” and the transmission interface <b>62</b> “If3” from the hash table <b>38</b><i>b </i>designated by the hash table number information “#2”, and forwards them to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> forwards the routing destination information to the transmission unit <b>33</b> and gives an output instruction. The transmission unit <b>33</b> reads out the reception packet from the buffer <b>32</b> and transmits it from designated “If3” to the line. The packet is routed in this way to the load balancing apparatus C<b>16</b>.
P-0116[0116] The load balancing apparatus C<b>16</b> operates in the same was as the load balancing apparatus A<b>14</b> described above and forwards the packet to the Web server <b>19</b><i>a </i>or <b>19</b><i>b. </i>
P-0117[0117]FIG. 6 shows the case where the load balancing apparatus C<b>16</b> forwards the HTTP packet to the Web server <b>19</b><i>a. </i>
P-0118[0118] Incidentally, when communication is made in the tunnel mode, the policy routing table <b>37</b> must have an entry having a policy number <b>50</b> “4” as shown in FIG. 7. For, when the Web servers <b>19</b><i>a </i>to <b>19</b><i>d </i>forward the response packets to the client apparatuses, the IPsec router B<b>22</b> must encrypt from time to time the response packets.
P-0119[0119] The load balancing apparatuses C<b>16</b> and D<b>17</b> route the response packets transmitted from the Web servers <b>19</b><i>a </i>to <b>19</b><i>d </i>to the router <b>13</b> as described above. The header information of the response packet routed to the router <b>13</b> contains the virtual address “30.0.0.1” as the source IP address <b>74</b>, “TCP” as the protocol type <b>75</b> and “HTTP” as the source port number <b>77</b>. When the router <b>13</b> receives this response packet, the policy routing retrieval unit <b>34</b> retrieves the policy routing table <b>37</b> and judges that the information is coincident with the information of the look-up key contained in the entry having the policy number <b>50</b> “4”. The policy routing retrieval unit <b>34</b> reads out the transmission interface <b>58</b> “If5” from the entry and transfers it to the transmission control unit <b>36</b>. The transmission control unit <b>36</b> delivers the transmission interface information to the transmission unit <b>33</b>. The transmission unit <b>33</b> transmits the response packet from designated “If5”. The router <b>13</b> routes in this way the response packet to the IPsec router B<b>22</b>.
P-0120[0120] The IPsec router B<b>22</b> receives the response packet and encrypts it. The IPsec router B<b>22</b> adds the IP header to the encrypted data and again routes the encrypted IP packet to the router <b>13</b>. The encrypted IP packet contains in this case the address of the IPsec router A<b>21</b> as the destination IP address. Receiving this encrypted packet, the router <b>13</b> routes it to the IPsec router A<b>21</b> in accordance with its destination IP address.
P-0121[0121] As described above, the client apparatus and the Web server can conduct two-way communication in the tunnel mode.
P-0122[0122] In the Web system <b>100</b> shown in FIG. 6, the IPsec router B<b>22</b> is different from the router <b>13</b> and is connected to the router <b>13</b>. However, the router <b>13</b> may well have the function of the IPsec router. In this case, the router <b>13</b> discriminates the packet to be communicated in the IPsec tunnel mode and encrypts or deciphers the packet.
P-0123[0123] As described above, the router <b>13</b> in the Web system <b>100</b> shown in FIG. 6 can allocate the IPsec packets to a plurality of load balancing apparatuses irrespective of the form of communication using IPsec. As a result, the Web system <b>100</b> can provide services having improved security.
P-0124[0124] Incidentally, the Web systems shown in FIGS. <b>1</b> and <b>6</b> have been explained on the premise of the systems using 32-bit address as the IP address, that is, the IPv4 (Internet Protocol Version 4) address. However, communication can also be made by use of 128-bit address as the IP address, that is, the IPv6 (Internet Protocol Version 6) address, in accordance with IPv6.
P-0125[0125] It should be further understood by those skilled in the art that although the foregoing description has been made on embodiments of the invention, the invention is not limited thereto and various changes and modifications may be made without departing from the spirit of the invention and the scope of the appended claims.
Contents5
10 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10
Every citation, both ways
| Document | Relation | Office | Cited during |
|---|---|---|---|
| US8996614B2 | Cited by | United States of America | Search report |
| EP3522475A4 | Cited by | European Patent Office (EPO) | Search report |
| US11153225B2 | Cited by | United States of America | Applicant |
| US8190403B2 | Cited by | United States of America | Applicant |
| US2013067231A1 | Cited by | United States of America | Pre-grant |
| US12375373B2 | Cited by | United States of America | Applicant |
| US8595239B1 | Cited by | United States of America | Applicant |
| US9130760B2 | Cited by | United States of America | Applicant |
| US7865183B2 | Cited by | United States of America | Search report |
| US10110504B2 | Cited by | United States of America | Applicant |
| US11201760B2 | Cited by | United States of America | Search report |
| US9439129B2 | Cited by | United States of America | Applicant |
| WO2010019629A3 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US9954751B2 | Cited by | United States of America | Applicant |
| US9565074B2 | Cited by | United States of America | Applicant |
| US2011149733A1 | Cited by | United States of America | Pre-grant |
| US9565063B2 | Cited by | United States of America | Applicant |
| US2010271964A1 | Cited by | United States of America | Pre-grant |
| US9497611B2 | Cited by | United States of America | Applicant |
| US8725820B2 | Cited by | United States of America | Applicant |
| US11405289B2 | Cited by | United States of America | Search report |
| US2009111532A1 | Cited by | United States of America | Pre-grant |
| US2014016552A1 | Cited by | United States of America | Pre-grant |
| EP3944081A1 | Cited by | European Patent Office (EPO) | Search report |
| US9391716B2 | Cited by | United States of America | Applicant |
| US8660130B2 | Cited by | United States of America | Search report |
| US9306891B2 | Cited by | United States of America | Applicant |
| US9444692B2 | Cited by | United States of America | Applicant |
| WO2010138936A3 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US10990447B1 | Cited by | United States of America | Search report |
| US9083710B1 | Cited by | United States of America | Search report |
| US9755891B2 | Cited by | United States of America | Applicant |
| CN109408241A | Cited by | China | Search report |
| US9621402B2 | Cited by | United States of America | Search report |
| US2011134928A1 | Cited by | United States of America | Pre-grant |
| US7315541B1 | Cited by | United States of America | Search report |
| CN103812703A | Cited by | China | Search report |
| WO2010019629A2 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US9544751B2 | Cited by | United States of America | Applicant |
| US8943221B2 | Cited by | United States of America | Applicant |
| US2009141659A1 | Cited by | United States of America | Pre-grant |
| US2009006047A1 | Cited by | United States of America | Pre-grant |
| US2010250668A1 | Cited by | United States of America | Pre-grant |
| CN109547354A | Cited by | China | Search report |
| US9497039B2 | Cited by | United States of America | Applicant |
| US9173081B2 | Cited by | United States of America | Applicant |
| CN112468541A | Cited by | China | Search report |
| US7463886B2 | Cited by | United States of America | Search report |
| US2005075084A1 | Cited by | United States of America | Pre-grant |
| US11184431B2 | Cited by | United States of America | Search report |
| US8929859B2 | Cited by | United States of America | Applicant |
| CN105763617A | Cited by | China | Search report |
| WO2023151264A1 | Cited by | World Intellectual Property Organization (WIPO) | International search |
| US2024080307A1 | Cited by | United States of America | Search report |
| CN117376358A | Cited by | China | Search report |
| US7765405B2 | Cited by | United States of America | Search report |
| CN113961303A | Cited by | China | Search report |
| EP2466828A1 | Cited by | European Patent Office (EPO) | Search report |
| US2006195698A1 | Cited by | United States of America | Pre-grant |
| US2006031242A1 | Cited by | United States of America | Pre-grant |
| US8725896B2 | Cited by | United States of America | Applicant |
| US10038988B2 | Cited by | United States of America | Applicant |
| US10057180B2 | Cited by | United States of America | Applicant |
| US7660897B2 | Cited by | United States of America | Search report |
| US2013159150A1 | Cited by | United States of America | Pre-grant |
| US9641403B2 | Cited by | United States of America | Applicant |
| US9300531B2 | Cited by | United States of America | Applicant |
| US9602676B2 | Cited by | United States of America | Applicant |
| US10084751B2 | Cited by | United States of America | Search report |
| US8675659B2 | Cited by | United States of America | Applicant |
| US8824370B2 | Cited by | United States of America | Applicant |
| US11159571B2 | Cited by | United States of America | Applicant |
| US9450766B2 | Cited by | United States of America | Applicant |
| US8218561B2 | Cited by | United States of America | Search report |
| US11895193B2 | Cited by | United States of America | Applicant |
| US9356912B2 | Cited by | United States of America | Search report |
| US2013198411A1 | Cited by | United States of America | Pre-grant |
| US2009103710A1 | Cited by | United States of America | Pre-grant |
| US2005195834A1 | Cited by | United States of America | Pre-grant |
| US2002049840A1 | Cites | United States of America | Pre-grant |
| US2002097724A1 | Cites | United States of America | Pre-grant |
| US2003074395A1 | Cites | United States of America | Pre-grant |
| US2003074467A1 | Cites | United States of America | Pre-grant |
| US2003101276A1 | Cites | United States of America | Pre-grant |
| US2003193895A1 | Cites | United States of America | Pre-grant |
| US6567404B1 | Cites | United States of America | Pre-grant |
| US6744767B1 | Cites | United States of America | Pre-grant |
2 members in 2 offices; this record represents the family
Priority claims1
| Document | Office | Kind | Date |
|---|---|---|---|
| 2002084746 | Japan | A |
Members2
| Document | Office | Kind | |
|---|---|---|---|
| JP2003281109A | Japan | A | |
| US2003195919A1 | United States of America | A1 |
24 transactions on the USPTO file
Abandoned after 1 non-final rejection.
- Non-final rejections
- 1
- Final rejections
- 0
- RCEs
- 0
- Appeals
- 0
Over time
Point at a mark for the transactionTransactions
| Event | Code | |
|---|---|---|
| Mail Abandonment for Failure to Respond to Office ActionAbandonedMABN2 | MABN2 | |
| Aband. for Failure to Respond to O. A.AbandonedABN2 | ABN2 | |
| Mail Supplemental Non-Final ActionMSRNF | MSRNF | |
| Supplemental Non-Final ActionSRNF | SRNF | |
| Request for Foreign Priority (Priority Papers May Be Included)RQPR | RQPR | |
| Mail Non-Final RejectionNon-final rejectionMCTNF | MCTNF | |
| Non-Final RejectionNon-final rejectionCTNF | CTNF | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| IFW TSS Processing by Tech Center CompleteTSSCOMP | TSSCOMP | |
| Case Docketed to Examiner in GAUDOCK | DOCK | |
| Transfer Inquiry to GAUTI1050 | TI1050 | |
| Application Dispatched from OIPEOIPE | OIPE | |
| Application Is Now CompleteCOMP | COMP | |
| Additional Application Filing FeesADDFLFEE | ADDFLFEE | |
| A statement by one or more inventors satisfying the requirement under 35 USC 115, Oath of the ApplicOATHDECL | OATHDECL | |
| IFW Scan & PACR Auto Security ReviewSCAN | SCAN | |
| Information Disclosure Statement consideredIDSC | IDSC | |
| Information Disclosure Statement (IDS) FiledM844 | M844 | |
| Information Disclosure Statement (IDS) FiledWIDS | WIDS | |
| Initial Exam Team nnIEXX | IEXX |
2 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Information on status: application discontinuationABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTIONSTCB | STCB | |
| AssignmentAS | AS |
Numbers
- Application
- 39345103
Titles
- English
- Packet distributing system and method for distributing access packets to a plurality of server apparatuses
Classification
- CPC, 7
- H04L45/745
- H04L61/255
- H04L67/1014
- H04L67/1023
- H04L67/1001
- H04L67/56
- H04L67/63
- IPC, 7
- G06F17 30
- G06F9 00
- G06F9 50
- G06F13 00
- G06F15 177
- H04L29 08
- H04L29 12