JP5148441B2 - Communication path redundancy and switching method in computer interconnection network, server device realizing the method, server module thereof, and program thereof - Google Patents

Description

  In the PC clustering field, where multiple computers such as PCs (Personal Computers) are connected to build a single high-speed computer system, the configuration of the interconnection network connecting computers is made redundant to improve the fault tolerance of the communication path. The present invention relates to a highly reliable technique for PC cluster system internal communication.

  A technology called PC clustering that economically constructs a system that has the same processing capacity as one large, high-speed computer system by connecting many inexpensive computers such as PCs instead of low processing capacity There is. This PC clustering technology is used in a wide range of fields as a construction method for various servers in a scientific and technical computing system, a data center, etc., and a large-capacity storage system.

  In PC clustering, a physical transmission medium used for connection between PCs is called an interconnection network. There are various types of interconnected networks, and they are properly used by taking advantage of the features of their communication functions. Specific examples of the interconnection network include Myrinet, Fiber Channel, InfiniBand, and the like. Further, usually, even in one type of interconnection network, a plurality of types of communication systems (protocols) can be operated in the upper layer. FIG. 1 shows an outline of the PC cluster system.

  Among these interconnected networks, InfiniBand (Non-patent Document 1) is most widely used in recent PC cluster systems because of its high speed, low cost, ease of use, and the like. Various types of protocols such as TCP / IP, SDP, and MPI can be operated in the upper layer of InfiniBand. One of the typical protocols is uDAPL (user direct access programming library) (non- There is a patent document 2).

  uDAPL is a standard for making it possible to use RDMA (remote direct memory access) functions of some interconnected networks such as InfiniBand via a common software interface that does not depend on the type of interconnected network. A software system that provides an application interface (API) and a data transfer protocol. Here, RDMA refers to DMA (direct memory access), which is a technique for speeding up input / output (I / O) processing in a single computer, which has been widely used in the past. This is an extension to communications between remote computers connected by a combined network.

  In DMA, when data is input / output (I / O) to / from an external device directly connected to a computer, the controller of the external device directly accesses the main memory of the computer without using the CPU. By performing the transfer, the processing load on the computer is reduced and the data input / output speed is increased.

  Similarly, in RDMA, data transfer can be performed between the main memories of remote computers with almost no CPU, so that communication between computers can be performed with very low load and high speed. In fact, uDAPL is one of the fastest protocol groups operating in the upper layers of InfiniBand and is gaining in importance.

  In RDMA, an essential operation mode is that a main memory area of a computer that is a target of data transmission or reception is disclosed to a communication partner in advance, and communication is started after a data transfer path is established. In this regard, other general data transfer methods, that is, the transmitting side transmits data without recognizing the final data storage location on the receiving side, and the receiving side stores the data ad hoc each time it receives data. It is different from the method of determining the place.

  In the latter method, connectionless communication without establishing a communication path between computers in advance is possible, whereas in the former RDMA, connection type communication in which a communication path is established between computers in advance is assumed. Become.

  Further, it is necessary to explicitly specify a memory area to be subjected to data transmission / reception processing.

  Therefore, uDAPL is an API reflecting such an essential operation mode of RDMA, that is, an API centering on a connection setting mechanism, an abstraction mechanism of a main memory area that is a target of data transfer, and the like. The outline of the uDAPL API will be described below.

  First, main object groups used in the uDAPL API are shown in FIG. Host A (hereinafter referred to as a small computer such as a PC) equipped with InfiniBand HCA (Host Channel Adapter) and host B are connected via an InfiniBand switch, and data transfer by RDMA between the main memories of the host It is a figure premised on performing. Here, a case where the host A is an RDMA initiator, that is, a side that initiates data transfer by RDMA is shown.

  As described above, uDAPL employs a connection type communication system. An EP (End Point) in FIG. 2 is an object that abstracts an end point of a connection between application processes serving as communication subjects. Connections are established between EPs, and only one-to-one connections are possible. When the application process starts the communication process, it designates a specific one of the EPs held by the application process. As a result, the connection used for the communication (and the EP as the communication partner) is specified.

  In this specification, one physical connection line between the host HCA and the InfiniBand switch is called a communication link. A logical communication path established between EPs is called a connection.

  An LMR (Local Memory Region) is an object for abstractly expressing a main memory region that is a target of a communication operation. One LMR represents a single contiguous virtual memory area. The application process starts communication processing by designating all or part of the virtual continuous memory area represented by the LMR. Prior to its use, the LMR has determined the actual physical area of main memory to which it is mapped (mapped). Therefore, LMR designation means designation of a main memory area to be subjected to the communication process.

  RMR (Remote Memory Region) is an object for setting related to access restriction from a remote host to a main memory area mapped to LMR in its own host. The RMR can be bound to all or part of the LMR (the figure shows the case where one RMR is bound to the entire region of the LMR), and the main memory mapped to the bound region of the LMR The access restriction set in association with the RMR is applied to this area. That is, a memory window is provided for setting an access restriction flexibly by cutting out an arbitrary area from the LMR.

  A PZ (Protection Zone) is an object that provides a means for grouping various objects and excluding operations from objects outside the group. Each of the EP, LMR, and RMR is associated with only one PZ when it is generated. A communication operation executed via a certain EP can access only an area of the main memory mapped to the LMR or RMR belonging to the same PZ as the PZ to which the EP belongs.

  EVD (Event Dispatcher) is an object for notifying completion of various operations of uDAPL as an event. At the time of generation, the above-mentioned EP is a message reception completion notification EVD (receive EVD), a message transmission / RDMA read / RDMA write / RMR mapping completion notification EVD (request EVD), and an inter-EP connection establishment notification. It is associated with three EVDs of EVD (connect EVD). The application process recognizes completion of these operations performed through the EP by event notification from each EVD.

  Note that the EP, LMR, RMR, PZ, and EVD objects are associated with one InfiniBand HCA at the time of generation.

  This completes the description of the main object group used in the uDAPL API. Next, a procedure for actually communicating using these will be described. For example, the procedure in the case where the host A reads data in the main memory of the host B by RDMA and stores it in its own main memory is as follows (1) to (9). An outline of this procedure is shown in FIG.

  (1) The host A requests information regarding the memory area (target memory area) of the host B to be accessed by RDMA. The request message itself is stored in a request transmission buffer in the main memory of the host A, and a message transmission operation is started by designating the LMR and EP mapped to this buffer. (This operation is a normal message transmission without using the RDMA function. Note that host A does not know where in the main memory of host B the transmitted message is stored).

(2) The host A confirms that the message transmission has been normally completed through the transmission operation completion event notification from the request EVD.

  (3) The host A specifies the LMR and EP mapped in the memory information reception buffer, and starts the memory information reception operation from the host B.

(4) Host A waits for completion of reception of memory information from host B through notification from receive EVD.

  (5) The host B starts the memory request reception operation from the host A by designating the LMR and EP mapped in the request reception buffer in its own main memory.

(6) The host B waits for the completion of the reception of the memory information request from the host A through the notification from the receive EVD.

  (7) When the host B receives the memory information request from the host A, the host B confirms the contents of the memory request stored in the request reception buffer, and permits a memory access by RDMA from the host A (RDMA area) ), Specifically, a transmission operation of a triplet (rmr_triplet) of a start address, a length, and an access key is started. The contents to be transmitted at this time are stored in the memory information transmission buffer of the main memory, and message transmission is performed by designating the LMR and EP mapped in this area. (This operation is also normal message transmission different from RDMA).

(8) The host B waits for the transmission completion of the message through the transmission operation completion event notification from the request EVD. When this completion is confirmed, the operation returns to ( 5 ). (Host B does not sense the start / end of RDMA processing itself).

(9) When the host A confirms reception of the memory information from the host B through the event notification from the receive EVD, the host A uses the memory information (rmr_triplet) stored in the memory information reception buffer (7). ) RDMA operation start processing.

(10) Using the LMR mapped to the RDMA area on the main memory, the main memory area is designated as the data receiving area, the main memory area of the target host B is designated by the rmr_triplet, and EP And RDMA read processing is started. After this process is started, the end of RDMA read is waited using the same receive EVD used in (9).

  (11) Actual data transfer by RDMA is performed by the functions of the physical layer and link layer of InfiniBand. When the data in the RDMA area on the main memory of the host B is larger than the maximum data length (4 Kbytes) handled by InfiniBand, the data is divided into a plurality of packets and transferred to the host A. The host A stores the received packet in the RDMA area on the main memory. In the hosts A and B, these processes are performed almost without any CPU.

  (12) When the end of the RDMA read is confirmed by the notification from the EVD, the host A determines that all the data in the RDMA area corresponding to the LMR of the host B has been transferred to the RDMA area corresponding to the LMR of the host A To do. To continue RDMA transfer using another RDMA area, return to (1).

  The above is the outline of the RDMA communication procedure by uDAPL. Although the above is an explanation of RDMA read, the same operation procedure is used for RDMA write in which the initiator transmits data to the opposite host except that the data transfer direction is different.

  The present specification is directed to a communication method including an API having a connection setting mechanism and an abstraction mechanism of a main memory area that is a target of data transfer. At present, uDAPL is the only specific example.

INFINIBAND NETWORK ARCHITECTURE, MINDSHARE, INC., Tom Shanley, PC SYSTEM ARCHITECTURE SERIES, Addison-Wesley (2003), ISBN 0-321-11765-4. uDAPL: User Direct Access Programming Library, Version 2.0, DAT collaborative.

  Conventional uDAPL up to version 1.2 did not provide a mechanism for improving fault tolerance of a communication path, that is, a mechanism for redundancy of a communication path or switching of a communication path when a fault occurs. In the current uDAPL version 2.0, these functions are optionally supported as HA (High Availability) functions (Non-Patent Document 2, Pages 58-62). However, in the API, the specification method of redundancy of the EP connection setting is specified (Non-Patent Document 2, Pages 231, 233), but details of the operation in combination with the LMR are not specified. There are many unclear points.

  Therefore, at present, there is no way to achieve fault tolerance of the interconnection network other than by realizing an architecture that explicitly considers fault tolerance at the application software level. In that case, the following two obvious methods can be considered.

  (Method 1) The same data is always redundantly supplied to all of the redundant communication paths, but only data on one of the communication paths is actually used. When a failure occurs, the data to be used is switched to data on another communication path.

  (Method 2) Although the communication path is made redundant, data is transmitted through only one communication path for communication. When a failure occurs, the communication path to be used is switched, and data is sent only to the new communication path.

  FIG. 4 illustrates (Method 1). In this example, host A reads data from a disk as an external device into the main memory, and then transfers the data to the main memory of host B by RDMA, and host B distributes the received data to the network via the NIC. In order to improve the fault tolerance of the RDMA communication between the hosts A and B, redundancy is achieved by two communication paths corresponding to the connections 1 and 2.

  On the main memories of the hosts A and B, there are RDMA areas 1 and 2 corresponding to the communication path 1 and the communication path 2. These RDMA areas are divided into a plurality of segments, and one LMR is mapped to each segment. A set of LMRs mapped to each segment in the RDMA region 1 is called an LMR group 1, and a set of LMRs mapped to each segment in the RDMA region 2 is called an LMR group 2. That is, FIG. 4 is an example in which a plurality of RDMA areas and LMRs shown in FIG. 2 are held for each communication path.

  The host A reads data having the same size as the segment on the main memory and stores it in the segment in order. One data on the disk is always read twice, one is stored in a segment in the RDMA area 1 and the other is stored in a segment in the RDMA area 2. In this way, the same data is stored redundantly in the RDMA areas 1 and 2.

  In addition, using the LMR groups 1 and 2, the data stored in the RDMA areas 1 and 2 are transferred from the host A to the host B while using both the communication paths 1 and 2 at any time. Can be a host). Data in the RDMA area 1 of the host A is transferred to the RDMA area 1 of the host B via the communication path 1, and data in the RDMA area 2 of the host A is transferred to the RDMA area 2 of the host B via the communication path 2.

  On each communication path, data in one segment is transferred by one RDMA operation, and data transfer is continuously performed by repeating this. This data transfer operation is performed in parallel with the data read operation from the disk.

  As described above, all data read from the disk is made redundant and transferred between hosts via the redundant communication path.

  In parallel with the data reception operation from the host A, the host B performs an operation of distributing data from either one of the connections 1 and 2 to the network via the NIC.

  Here, for example, when a failure occurs on the communication path while data from the communication path 1 is being distributed to the network and communication becomes impossible, the host B sends data to be distributed to the network on the NIC. Switch to data from communication path 2. In this way, continuous data distribution to the network is continued.

  As can be seen from this example, in (Method 1), even if a failure occurs in a part of the communication paths made redundant between the hosts, the data transfer between the hosts is not interrupted. In the example of FIG. 4, normal operation as a system can be maintained by simply switching the communication path that is the source of data distributed to the network on the NIC of the host B. Since there is no need to switch the communication path through which the data flows, there is an advantage that data copying or moving processing between the RDMA areas accompanying the switching of the communication path as described later (method 2) is unnecessary.

However, in (Method 1), since data is always transferred in a redundant manner, there is a problem that the processing load of the host and the consumption of various resources used for communication increase. In the example of FIG.
Data read from the disk in the host A, data transmission in the host A, and data reception in the host B are always duplexed. Since the transmission processing of the host A and the reception processing of the host B are performed by RDMA, the load on the host CPU is small. However, the process of reading data from the disk may increase the processing load on the host. In addition, data input / output performance with external devices such as disks is often a governing term for overall system performance. Increasing the consumption of resources in this part such as the I / O bus increases overall system performance. It is often not a good idea to improve In particular, such a problem becomes more serious as the system performs high-speed and large-capacity data communication. The above is the problem of (Method 1).

  On the other hand, FIG. 5 illustrates (Method 2). As in FIG. 4, in this example, after the host A reads the data from the external disk into the main memory, it is transferred to the main memory of the host B by RDMA, and the host B sends the received data to the network via the NIC. Deliver on top. The communication path between the hosts A and B is also duplicated as in the example of FIG.

  4 is different from FIG. 4 in that data is transmitted through only one of the two communication paths for communication. Under normal conditions, data read from the disk of the host A is stored in the RDMA area 1 for the communication path 1 of the main memory, and then transferred to the host B using the communication path 1 by RDMA. The host B distributes the data received in the RDMA area 1 for the communication path 1 of the main memory to the network via the NIC. In FIG. 5, the data flow in this state is indicated by solid arrows.

  When a failure occurs on the communication path 1 during this operation, the host A and the host B switch the RDMA area and LMR group of the main memory to be used from the communication path 1 to the communication path 2. As a result, after switching, the data read from the disk is transferred to the host B via the communication path 2 and distributed to the network, as indicated by the dashed arrows in the figure.

  As described above, in (method 2), since the data transfer itself is not made redundant, there is no problem like (method 1) in which the processing load on the host and the consumption of various resources increase.

  However, in (Method 2), it is necessary to switch the communication path through which data flows when a failure occurs. At this time, data is copied (or moved) between RDMA areas on the main memory or newly used. In some cases, data must be read again from the external device to the RDMA area. For example, as shown in FIG. 6, when a failure occurs on the communication path 1, let us consider a case where data that has not yet been transferred to the host B remains in the RDMA area 1 of the host A. In this case, the host A needs to execute one of the following (A) and (B) so that the data can be transferred to the host B via the communication path 2.

  (A) Copy (or move) untransferred data in the RDMA area 1 to the RDMA area 2 (FIG. 7).

  (B) The same data as the untransferred data in the RDMA area 1 is read again from the disk and stored in the RDMA area 2 (FIG. 8).

  As shown in the figure, a failure occurred in the communication path 1 during the RDMA transfer of data in a data transfer unit (one segment in this example) in one RDMA operation, and the RDMA transfer ended abnormally in the middle. In this case, after completing either (A) or (B), the RDMA transfer is resumed from the beginning of the segment. This is because the progress status of each RDMA cannot be grasped by the application process, and the amount of data transferred before abnormal termination cannot be grasped accurately, so it is necessary to re-execute from the beginning.

  During the execution of (A) or (B), data transfer between the hosts A and B is stopped. Therefore, in order to prevent the data distribution from the host B to the network, it is necessary to execute the above operation on the host A side as fast as possible. However, particularly when the system handles high-speed and large-capacity data, the amount of data to be copied (moved) or re-read tends to increase, and it is not easy to execute the above operation at high speed.

  An object of the present invention is to solve the problems of the above-described conventional trivial communication path redundancy method while maintaining a rapid switching operation of a communication path when a failure occurs on the communication path.

  As means for solving the above problems, a single main memory area is prepared for data input / output with an external device and RDMA transfer between hosts, while redundant communication is provided. An LMR for performing communication on the communication path for each path is prepared, and a method of mapping the LMR corresponding to each communication link on the single main memory area is invented.

  An outline of the present invention is shown in FIG. In the present invention, the LMR group corresponding to each redundant communication path (here, two communication paths) is prepared in the conventional (method 1) described in [Problem to be Solved by the Invention], ( Similar to method 2), except that the RDMA area on the main memory provides a single area.

  During normal operation, data transfer between hosts using only one of a plurality of redundant communication paths as in the conventional (method 2) described in [Problems to be solved by the invention] I do. In the example of FIG. 9, in the host A, data having the same size as the segment on the main memory is read from the disk and stored in the segment. Are transferred to the host B (the RDMA initiator may be either host). In the host B, the data received from the communication path 1 using the LMR group 1 is stored in the main memory, and the data on the main memory is distributed to the network via the NIC.

  Here, as shown in FIG. 10, when a failure occurs in the communication path 1, the host A and the host B switch the LMR group used for RDMA from 1 to 2. Then, the data transfer by RDMA is resumed from the segment on the main memory that has not been transferred to the host B yet.

  When a failure occurs in the communication path 1 in the middle of RDMA transfer of data in a data transfer unit (one segment in this example) in one RDMA operation as shown in the figure, and the RDMA transfer ends abnormally in the middle After switching the LMR group to be used from 1 to 2, the RDMA transfer is executed again from the beginning of the segment. This is because the progress status of each RDMA cannot be grasped by the application process, and the amount of data transferred before abnormal termination cannot be grasped accurately, so it is necessary to re-execute from the beginning.

  Data reading operation from the disk to the main memory in the host A and data distribution operation from the main memory to the network in the host B can be continued without being affected by the switching of the LMR group.

  In the present invention, data reading from the disk need only be performed once for each piece of data in a single RDMA area. In data distribution to the network, data reading from a single RDMA area to the NIC need only be performed once for each data. Therefore, the problem of (method 1) described in [Problems to be solved by the invention], that is, the increase in processing load on the host due to redundant data transfer between the external device and the main memory. There is no problem of increasing the consumption of various resources.

  The present invention is the same as (Method 2) described in [Problem to be Solved by the Invention] in that the communication path through which data flows when a failure occurs is changed. However, in the present invention, the only operation necessary for switching the communication path is to switch the LMR group. In the example of FIG. 9, the data read operation from the disk in the host A to the main memory and the data distribution operation from the main memory to the network in the host B are not affected by the switching of the LMR group described above, This can be continued all the time using the RDMA area. Therefore, the problem of (Method 2) described in [Problems to be solved by the invention], that is, high-speed execution such as data copying (moving) between RDMA areas and data re-reading from a disk is difficult. There is also no problem that possible processing is involved.

  According to the present invention, the conventional trivial communication path redundancy system described in [Problems to be Solved by the Invention] is maintained while maintaining a rapid switching operation of a communication path when a failure occurs on the communication path. It is possible to solve the problems that you have.

[Best Mode for Carrying Out the Invention 1]
Hereinafter, the best mode 1 (hereinafter referred to as mode 1) for carrying out the present invention will be described. This Embodiment 1 is based on the method of Claims 1, 3, and 4 of [Claims], and corresponds to Claim 5 because it is a server device that embodies it. FIG. 11 shows the overall configuration of the apparatus according to the first embodiment.

The interconnection network redundant multimedia server apparatus shown in FIG. 11 is a server apparatus according to this embodiment. It has a function of distributing multimedia data having time continuity such as video data to an external device and a function of storing multimedia data received from the external device. Communication server module 1 to M 1 which communicates an external device _directly, storage device 1 to M 2 that multimedia data is _stored, the storage server module 1 to M _2, the server in a database for controlling the storage device, a communication server module storage A network switch between server modules used for multimedia data transfer between server modules, and a local area network connecting the communication server module, the storage server module, and the database in the server.

  The communication server module and the storage server module are modules composed of general-purpose computers, and the interconnected network redundant multimedia server apparatus is a cluster of these. That is, the communication server module and the storage server module correspond to the computer described in claim 1, and the interconnection network redundant multimedia server device corresponds to the server device described in claim 1.

  The interconnected network redundant multimedia server device is connected to an external device such as a terminal or a multimedia information input device via a terminal-side network. The terminal is a device that receives multimedia data from the server device. The multimedia information input device is a device that transmits multimedia data to be stored in the server device.

  The network switch between server modules in the interconnection network redundant multimedia server apparatus corresponds to the interconnection network according to claim 1. In the first embodiment, the switch is an InfiniBand switch and uDAPL is used for an upper layer. This uDAPL corresponds to “a communication system having a connection setting mechanism between computers and an abstraction mechanism of a main memory area of the computer”.

  In the first embodiment, the interconnection network is made redundant by using two switches. That is, this corresponds to the case where the value of D described in claim 1 is 2.

  A network between server modules consisting of one switch, a communication link extending from the switch, and all HCAs connected to the switch by the communication link is called a system, and the two systems in FIG. 1 system. In addition, in this 0 system or 1 system, a communication path comprising the same switch and a communication link connecting one communication server module and one storage server module is defined as a 0 system communication path, a 1 system communication path, Call like this.

  It is assumed that communication between arbitrary HCAs is possible within each system, but communication across systems is not possible.

  Each communication server module has one network interface card (NIC) for connection to the terminal-side network and an InfiniBand host adapter card (for connection to the network switch between the 0-system and 1-system server modules. Two HCA) are installed.

  Each storage server module has two InfiniBand HCAs for connection to the network switches between the 0-system and 1-system server modules, and a Fiber Channel host bus adapter (for connection with the storage device managed by itself) One HBA) is installed.

  The local area network in the interconnected network redundant multimedia server device is a network that connects the communication server module, the storage server module, and the database in the server. This network does not need to be able to execute a communication method having a connection setting mechanism between computers or an abstraction mechanism of the main memory area of the computer like the network between the 0 system and 1 system server module. A general network such as Ethernet used in an area network may be used.

  The intra-server database in the interconnected network redundant multimedia server device is a database that holds data relating to the storage location of programs stored in the server device and the usage status of each storage device. The communication server module and the storage server module of the server device access the in-server database via the local area network to obtain the above information. As an apparatus for realizing the database in the server, a storage apparatus capable of holding the above-mentioned data and having communication with the communication server module or storage server module may be used. It is easy to realize with. This internal database has two tables: a program storage information table and a storage device usage status table. These are shown in FIG.

The program storage information table is a table showing in which storage device the multimedia data of each program stored in the interconnection network redundant multimedia server apparatus is stored. The example of FIG. 12 shows that the program with ID 1 is stored in the storage device 5 and the program with ID ₂ is stored in the storage device M2. Each time a new program is stored in the server device, such an entry is created in the table. The storage device usage status table is a table showing the free capacity of each storage device of the server device. The free capacity of each storage device from storage device 1 to M ₂ is stored. Each time a new program is stored in the server device, the numerical value indicating the free space in the table is updated.

  In the first embodiment, multimedia data is distributed as follows. A terminal requesting delivery of multimedia data issues a delivery request for multimedia data to any one communication server module in the server device via the terminal-side network. The distribution request includes, as information, a request source ID for specifying the request source terminal and a program ID for specifying the multimedia data for which distribution is requested (the management method of the request source ID and the program ID is specified here). do not do).

  The communication server module that has received the distribution request accesses the database in the server via the local area network, searches for an entry including the program ID in the program storage information table in the database, and determines the multimedia data of the program. Is stored.

  Then, the multimedia data of the program is received from the storage server module that manages the storage device of the storage device number using one system that is operating normally in the network between the server modules, and the request source Deliver to the terminal corresponding to the ID.

  Further, the storage of multimedia data is performed as follows. A multimedia information input device that requests storage of multimedia data issues a request to store multimedia data to any one communication server module in the server device via the terminal-side network. The accumulation request includes, as information, a request source ID that identifies the request source multimedia information input device and a program ID that is newly allocated to the multimedia data to be accumulated.

  The communication server module that has received the storage request accesses the database in the server via the local area network, refers to the storage device usage status table in the database, and selects the storage device with the largest free space for the multimedia data. Determine the storage device.

  Then, the multimedia data received from the multimedia information input device is transmitted to the storage server module that manages the storage device, using one normally operating system among the network between server modules.

  The storage server module stores the received multimedia information in a storage device managed by the storage server module. When storage of all the multimedia information of the program in the storage device is completed, the storage server module accesses the database in the server, and a new entry consisting of the program ID and the storage device number storing the program ID is stored in the program storage information table. Furthermore, the free capacity of the storage device after storing the program is recorded in the storage device usage status table, and the process is terminated.

  Note that the multimedia information of one program is divided into fixed-length segments and stored in the storage device. Reading from the storage device, writing to the storage device, and transfer between the communication server module and the storage server module are all performed in units of this segment.

  Data transfer between the communication server module and the storage server module is performed by RDMA via uDAPL API. One of the segments is transferred between the two in a single RDMA operation. By repeating this transfer operation, the multimedia data of the program is transferred. In the first embodiment, it is assumed that the communication server module is an RDMA initiator.

  Note that the format used when transmitting / receiving data to / from the terminal-side network is not particularly specified.

  The above is the description of the apparatus configuration and the outline of the operation of the first embodiment. Based on these, a detailed description including the technique of the present invention will be given below. First, FIG. 13 shows details of the internal structure of the communication server module.

  The RDMA area is a buffer for storing multimedia information of a program, and is an area secured in the main memory using a mechanism provided by the operating system. Divided into N areas, each segment stores one segment of multimedia information.

  In the case of the distribution operation, the segment received from the storage server module by RDMA transfer is stored and functions as a buffer until it is distributed to the terminal side network.

  In the accumulation operation, the multimedia information received from the terminal-side network is stored and functions as a buffer until it is transferred to the accumulation server module in segments by RDMA transfer.

  A read pointer (rd_ptr), a write pointer (wr_ptr), a full flag (full_flag), and an empty flag (empty_flag) are software program variables for managing the storage status of segments in the RDMA area.

  rd_ptr is a pointer that holds a buffer number that is a read source when a segment already stored in the RDMA area is read. In the case of the distribution operation, it indicates a segment that is distributed to the terminal side network via the NIC, and in the case of the storage operation, it indicates a segment that is transferred to the storage server module by RDMA.

  wr_ptr is a pointer that holds a buffer number as a write destination when writing a segment to the RDMA area. In the case of the distribution operation, it indicates the storage destination buffer of the segment transferred by the RDMA from the storage server module, and in the case of the storage operation, it indicates the storage destination buffer of the multimedia data received from the terminal side network.

  The full_flag is a flag that holds a value of 1 if segments are stored in all buffers, and a value of 0 in other cases.

  empty_flag is a flag that holds a value of 1 if no segment is stored in any buffer, and a value of 0 otherwise.

  By using a finite-length RDMA area composed of N buffers as a virtual ring buffer using these pointers and flags, program data having a segment length longer than N can be handled.

  The request transmission buffer is an area secured on the main memory in the same manner as the RDMA area, and is a buffer for storing a memory information request to be transmitted to the storage server module before executing the RDMA transfer.

  The memory information receiving buffer is a buffer for receiving the memory information sent from the storage server module in response to the memory information request in an area secured on the main memory by the same method as described above.

  A combination of the RDMA area, the request transmission buffer, and the memory information reception buffer corresponds to the data storage means according to claim 1. Therefore, only one of them is ensured regardless of the number of redundant communication paths.

  Data transfer LMR groups 0 and 1 are uDAPL objects, which are LMRs for two redundant 0-system and 1-system communication paths, respectively. This corresponds to LMR groups 1 and 2 in FIG. For each buffer [i] (0 ≦ i ≦ N−1) in the RDMA area on the main memory, 2 of LMR [0] [i] for 0 system and LMR [1] [i] for 1 system Two LMRs are mapped.

  In the first embodiment, not only the transfer of multimedia data but also the transmission / reception of memory information associated with RDMA use two communication paths to improve reliability. For this purpose, LMRs for the 0-system and 1-system communication paths mapped to the request transmission buffer and the memory information reception buffer are prepared. S-LMR is an LMR corresponding to a request transmission buffer, and R-LMR is an LMR corresponding to a memory information reception buffer.

  The combination of one LMR group for data transfer and one LMR group for memory information exchange corresponds to the memory area abstraction means according to claim 1, and in the first embodiment, two memory area abstraction means are held. Corresponds to the case.

  The request EVD is a uDAPL object that notifies the completion of the transmission of the memory information request to be transmitted to the storage server module and the completion of one RDMA read / write operation. Prepare for each of the 0-system and 1-system communication paths.

  The receive EVD is a uDAPL object that notifies the completion of reception of memory information from the storage server module. Prepare for each of the 0-system and 1-system communication paths.

  The EP is a uDAPL object that is an end point of a connection established for communication with the storage server module. In the present embodiment 1, the connection is made redundant by establishing the connection between the communication server module and the storage server module on the two communication paths of the 0 system and the 1 system, and the communication between the two is made highly reliable. Therefore, two EPs for 0 system and 1 system are used. In other words, the present embodiment 1 realizes an example of the path redundancy means according to claim 1 that has a total of two redundant communication paths.

  It is assumed that the EPs of each system are associated with the request EVD and the receive EVD in the drawing at the time of generation. Thereby, the completion notification of the operation to be executed via the EP of a certain system is performed through the EVD in the same system.

  In the first embodiment, the LMR group and the EP are used only in a combination of things in the same system. For this reason, the LMR group and EP of each system are generated in association with one PZ in the system so that processing in which these resources belonging to different systems are mixed cannot be performed.

  The communication path monitoring unit is a software module that monitors the communication availability status of each communication path connecting the communication server module and the storage server module. One exists corresponding to each communication path of the 0 system and the 1 system. In addition, it has a status variable that is a software program variable for recording the state of the communication path. This corresponds to the communication path failure / recovery detecting means according to claim 4.

  The communication path monitoring unit of each system periodically exchanges messages with the communication path monitoring unit on the storage server module opposite to the communication path, and sets a predetermined time (the “message according to claim 3”). The occurrence of a failure in the communication path and recovery from the failure are detected based on the “replacement specified time”).

  If the message exchange cannot be completed within the specified time, it is considered that the communication path has failed and becomes unusable, and the value “OFF” is set in the status variable. If the message exchange is completed within the specified time, it is assumed that the communication path is usable, and the status variable value is set to “normal”.

  Note that this message exchange uses lightweight communication such as ICMP echo request / reply instead of message send / receive and RDMA provided by the uDAPL API.

  uDAPL does not have a function of notifying application software of an interruption in a communication path. However, as will be described later, when waiting for notification of completion of various processes via the EVD, a specified waiting time (corresponding to the “specified time for data transfer” according to claim 3) is set. Can do. Therefore, it is possible to detect a failure in the communication path using this specified time during data transfer.

  On the other hand, the present embodiment 1 uses the method according to claims 3 and 4, that is, a method for judging the state of the communication path by using both the specified time for data transfer and the communication path monitoring unit. . This is superior to the former in the following two points.

  -The data size of one segment of multimedia data is very large, and the variation (jitter) of time required for the transfer tends to increase. Therefore, in the method of detecting a failure in the communication path based on the specified time for data transfer, if the specified time is not long enough to absorb this jitter, the simple processing delay at the time of data transfer and the communication path failure are confused. The possibility to do. However, this does not allow for quick failure detection. In the method according to the third and fourth aspects, the fault detection can be speeded up without erroneous determination as described above.

  -A communication availability monitoring mechanism that operates independently of the data transfer operation can detect not only a failure but also a recovery from the failure.

  The above is the detailed description of the internal structure of the communication server module. Next, FIG. 14 shows details of the internal structure of the storage server module.

  Connections are established between the 0 system EP of the storage server module and the 0 system EP of the communication server module, and between the 1 system EP of the storage server module and the 1 system EP of the communication server module.

  The RDMA area in the main memory is a buffer for storing the multimedia information of the program. In the case of a distribution operation, the segment read from the storage device is stored, and is used as a buffer for transmission to the communication server module by RDMA. Function. In the accumulation operation, the segment received from the communication server module by RDMA transfer is stored and functions as a buffer until it is written to the accumulation device.

  This RDMA area is also used as a virtual ring buffer by using rd_ptr, wr_ptr, full_flag, and empty_flag, as in the communication server module.

  The memory information transmission buffer is a buffer for storing memory information to be transmitted in response to a memory information request from the communication server module.

  The request reception buffer is a buffer for receiving a memory information request from the communication server module.

  Since the other constituent elements are the same as the corresponding constituent elements of the communication server module, description thereof will be omitted.

  The above is the detailed description of the internal structure of the communication server module and the storage server module. Details of the operation of the interconnection network redundant multimedia server apparatus configured as described above will be described below.

  In the following, only the multimedia data distribution operation from the server apparatus to the terminal will be described. In the case of the accumulation operation, only the transfer direction of the multimedia data is reversed, and there is no essential difference.

  Also, the timing for generating each component in the communication / storage server module is not defined in the first embodiment, and it is assumed that each component exists before the start of all operations. Therefore, the connection between the communication server module / accumulation server module shown in FIGS. 13 and 14 is also made by connecting opposite server modules in the distribution operation described below, and presupposes existence.

  For simplification, description of EVD for connection establishment notification between EPs is omitted in FIGS. 13 and 14, and description of RMR corresponding to LMR for data transfer is omitted in FIG.

  First, FIG. 15 shows a flowchart showing processing of the communication server module in the first embodiment. The processing of the communication server module will be described according to S1 to S15 shown in FIG.

  (S1) A program distribution request is received from the terminal.

  (S2) The server database is accessed, the program storage information table in the database is searched using the program ID included in the distribution request as a key, and the storage device number for managing the storage device storing the program is obtained. .

  (S3) A distribution start notification including the program ID is issued to the storage server module that manages the storage device having the storage device number obtained in S2.

  (S4) Wait until a preprocessing completion notification is received from the storage server module. This pre-processing completion notification indicates that the storage server module side has completed initialization of various resources, and the segment of the program can be transferred.

  (S5) An initial delivery time to the terminal, that is, a time to start delivery of the program to the terminal is set. For example, if the initial delivery time is set to the time when the delivery request from the terminal is received, the delivery is started when the first segment of the program is received from the storage server module.

  (S6) The values of rd_ptr and wr_ptr are each initialized to 0 to point to the buffer [0] in the main memory. Since the buffer is still empty (no segment is stored), full_flag is initialized to 0 and empty_flag is initialized to 1.

  (S7) Of the two redundant communication paths, a variable I representing the system number of the communication path to be used is initialized to 0 (initially used from the 0 system communication path).

  (S8) “Segment distribution process to terminal” described later is started.

  (S9) The contents of the memory information request to be transmitted to the storage server module are written to the request transmission buffer on the main memory. This memory information request specifies the LMR of the storage server module that is the target of the RDMA read operation by including a number that specifies the segment that is the transfer target, but the format is not specified in this specification.

  (S10) Wait until the value of full_flag becomes 0. That is, the process waits until there is an empty buffer in the main memory and a segment can be received from the storage server module.

  (S11) One segment of the program is received from the storage server module. Details of this step will be described later.

  (S12) When one segment is received, the value is updated so that wr_ptr points to the storage destination buffer of the next segment. If the last buffer [N1] has already been indicated, the value is updated so that the first buffer [0] is indicated, otherwise the next buffer is indicated.

  (S13) Since it is certain that all the buffers in the main memory are not empty immediately after receiving the segment, the value of empty_flag is set to 0.

  (S14) As a result of updating the value of wr_ptr in S12, if it becomes equal to the value of rd_ptr, it means that all the buffers in the main memory are occupied by the segment data, so the value of full_flag is set to 1. Otherwise it does nothing.

  (S15) If the segment received from the storage server module in S11 is the last segment of the program, the process ends. Otherwise, return to S9.

  FIG. 16 is a flowchart of the “segment distribution process to terminal” started in S8 of FIG. This process will be described in accordance with S20 to S27 shown in FIG.

  (S20) Wait until the current time passes the set “delivery time to terminal”.

  (S21) Referring to empty_flag in the main memory, it is confirmed that the value is 0, that is, there is a segment to be distributed. If the value of empty_flag is 1, it waits until it becomes 0.

  (S22) The segment stored in the buffer [rd_ptr] is distributed to the terminal specified by the request source ID in the distribution request received in S1. In this specification, the data format for distribution to the terminal-side network is not specified.

  (S23) When one segment is distributed, the value is updated so that rd_ptr points to the buffer of the storage destination of the next segment. If the last buffer [N1] has already been indicated, the value is updated so that the first buffer [0] is indicated, otherwise the next buffer is indicated.

  (S24) Immediately after the segment is delivered, there is at least one empty buffer in the main memory in which no segment is stored, so the value of full_flag is set to 0.

  (S25) As a result of updating the value of rd_ptr in S23, if it becomes equal to the value of wr_ptr, it means that all the buffers in the main memory are empty, so the value of empty_flag is set to 1. Otherwise it does nothing.

  (S26) A value obtained by adding the playback time of one segment at the terminal to the currently set “delivery time to terminal” is set as the next delivery time.

  (S27) If the segment distributed in S22 is the last segment of the program, the process is terminated. Otherwise, the process returns to S20.

  FIG. 17 shows details of S11 “Segment reception processing from storage server module” in FIG. This processing will be described in accordance with S30 to S44 shown in FIG.

  (S30) The contents in the request transmission buffer are transmitted as a memory information request to the storage server module via the I-system communication path using S-LMR [I]. This memory information request uses not a RDMA but a simple message send mechanism prepared by uDAPL.

  (S31) If the message send in S30 ends normally within a predetermined time set in advance, the process proceeds to S35. If not completed within the specified time, the process proceeds to S32.

  (S32) The status variable value of the I-system communication path is confirmed. If the value is “normal”, there is a possibility that the processing of the message send is delayed, and the process returns to S31. If it is not “normal” (if “off”), the process proceeds to S33.

  (S33) It is determined that a failure has occurred in the I-system communication path, and it is determined whether there are other communication paths that can be used instead. If not, the process ends here (abnormal end). If so, the process proceeds to S34.

  (S34) The system number of the communication path to be newly used is determined. The smallest number among the system numbers of the communication paths whose status is “normal” is set in the variable I. Thereafter, the process returns to S30, and the memory information request is transmitted again on the new communication path.

  (S35) The memory information is received from the storage server module via the I-system communication path using R-LMR [I], and the contents are stored in the memory information reception buffer. This memory information is received by using a simple message receive mechanism prepared by uDAPL, not by RDMA.

  (S36) If the message receive in S35 ends normally within the specified time set in advance, the process proceeds to S40. If not finished within the specified time, the process proceeds to S37.

  (S37) The status variable value of the I-system communication link is confirmed. If the value is “normal”, there is a possibility that the message receive processing may be delayed, and the process returns to S36. If it is not “normal” (if “off”), the process proceeds to S38.

  (S38) It is determined that a failure has occurred in the I-system communication path, and it is determined whether there are other communication paths that can be used instead. If not, the process ends here (abnormal end). If there is, move to S39.

  (S39) The system number of the communication path to be newly used is determined. The smallest number among the system numbers of the communication paths whose status is “normal” is set in the variable I. Thereafter, the process returns to S35, and the memory information is received again on the new communication link.

  (S40) In order to receive the segment of the program information from the storage server module, LMR [I] [rd_ptr] is designated as the receiving LMR, and the buffer address of the storage server module designated by the memory information received in S36 is set. Issue the target RDMA read.

  (S41) If the RDMA read in S40 ends normally within a predetermined time set in advance and the segment can be received in the buffer [I] [rd_ptr], the process proceeds to S12 in FIG. If not completed within the specified time, the process proceeds to S42.

  (S42) The value of the status variable of the I-system communication path is confirmed. If the value is “normal”, there is a possibility that RDMA read processing is delayed, and the process returns to S41. If it is not “normal” (if “off”), the process proceeds to S43.

  (S43) It is determined that a failure has occurred in the I-system communication path, and it is determined whether there are other communication paths that can be used instead. If not, the process ends here (abnormal end). If there is, the process proceeds to S44.

  (S44) The system number of the communication path to be newly used is determined. The smallest number among the system numbers of the communication paths whose status is “normal” is set in the variable I. Thereafter, the process returns to S40, and RDMA read is performed again on the new communication path.

  FIG. 18 is a flowchart showing the processing of the 0-system and 1-system communication path monitoring unit in FIG. The communication path monitoring units of each system operate independently of each other and independently of the operations of the communication server modules shown in FIGS. 15, 16, and 17, respectively. The processing of the communication path monitoring unit will be described according to S50 to S60 of the same figure.

  (S50) The value of the status variable indicating the state of the own communication path is initialized to “normal”.

  (S51) An ICMP echo request packet is transmitted to the communication path monitoring unit corresponding to the same system on the opposite host.

  (S52) If an ICMP echo reply packet can be received within a predetermined time set in advance from the opposite communication path monitoring unit in S51, it is determined that there is no problem in the own communication path, and the process returns to S51 to continue monitoring. If not received, the process proceeds to S53.

  (S53) It is determined that a failure has occurred in the own communication path, and the value of the status variable is changed to “OFF”.

  (S54) In order to detect recovery of the own communication path, an ICMP echo request packet is transmitted to the opposite communication path monitoring unit in S51.

  (S55) If an ICMP echo reply packet can be received within a predetermined time set in advance from the opposite communication path monitoring unit in S54, it is determined that the own communication path has been recovered, and the process proceeds to S56. If not received, the process returns to S54 to continue the recovery detection operation of the own communication path.

  (S56) The connection between the EP of the own communication path and the EP on the opposite host is disconnected.

  (S57) The EP of the own communication path is deleted.

  (S58) A new EP for the own communication path is generated.

  (S59) Using the new EP generated in S58, a new connection is established with the opposite EP in the communication path.

  (S60) The value of the status variable of the own communication path is returned to “normal”, and the process proceeds to S51 to perform a normal line monitoring operation.

  InfiniBand has no function for explicitly canceling communication processing once issued to a communication path. For this reason, a process of recreating a new one after deleting the EP and connection without reusing the communication path recovered from the failure is performed from step 56 to step 59. This prevents a communication process that was being executed at the moment when a failure occurred in the communication path from being erroneously restarted after recovery from the failure.

  FIG. 19 is a flowchart showing processing of the storage server module in the first embodiment. The processing of the communication server module will be described according to S70 to S81 in the same figure.

  (S70) The communication server module receives the distribution start notification transmitted in S3 of FIG.

  (S71) The values of rd_ptr and wr_ptr are each initialized to 0 to indicate the buffer [0] in the main memory. Since the buffer is still empty (no segment is stored), full_flag is initialized to 0 and empty_flag is initialized to 1.

  (S72) Of the two redundant communication links, the variable I indicating the communication path to be used is initialized to 0 (initially used from the 0-system communication link).

  (S73) “Segment read processing from storage device” described later is started.

  (S74) Wait until the value of empty_flag becomes 0. That is, it waits until there is a segment read from the storage device and stored in the buffer on the main memory, and the segment can be transmitted to the communication server module.

  (S75) If the segment in the buffer [I] [rd_ptr] is the first segment of the program, the process proceeds to S76, and if not, the process proceeds to S77.

  (S76) A pre-processing completion notification is issued to the communication server module.

  (S77) One segment of the program is transmitted to the communication server module. Details of this step will be described later.

  (S78) When one segment is transmitted, the value is updated so that rd_ptr points to the storage buffer of the segment to be transmitted next. If the last buffer [N1] has already been indicated, the value is updated so that the first buffer [0] is indicated, otherwise the next buffer is indicated.

  (S79) Since it is certain that all the buffers on the main memory are not full immediately after the transmission of the segment, the value of full_flag is set to 0.

  (S80) As a result of updating the value of rd_ptr in S78, if it becomes equal to the value of wr_ptr, it means that all the buffers in the main memory have become empty, so the value of empty_flag is set to 1. Otherwise it does nothing.

  (S81) If the segment transmitted to the communication server module in S77 is the last segment of the program, the process ends. Otherwise, the process returns to S74.

  FIG. 20 is a flowchart showing the “segment reading process from the storage device” started in S73 of FIG. This processing will be described in accordance with S90 to S95 in FIG.

  (S90) Wait until the value of full_flag becomes zero. That is, it waits until there is an empty buffer in the main memory and storage of the segment read from the storage device becomes possible.

  (S91) One segment of the program is read from the storage device and stored in the buffer [wr_ptr] on the main memory.

  (S92) When one segment is stored in the buffer, the value is updated so that wr_ptr points to the storage buffer of the next segment read from the storage device. If the last buffer [N1] has already been indicated, the value is updated so that the first buffer [0] is indicated, otherwise the next buffer is indicated.

  (S93) Immediately after storing the segment read from the storage device in the buffer on the main memory, it is certain that all the buffers are not empty, so the value of empty_flag is set to 0.

  (S94) As a result of updating the value of wr_ptr in S93, if it becomes equal to the value of rd_ptr, it means that all the buffers in the main memory are occupied by the segment data, so the value of full_flag is set to 1. Otherwise it does nothing.

  (S95) If the segment read from the storage device in S91 is the last segment of the program, the process ends. Otherwise, the process returns to S90.

  FIG. 21 shows details of S77 “Segment transmission process to communication server module” in FIG. This processing will be described in accordance with S100 to S111 shown in FIG.

  (S100) A memory information request is received from the communication server module via the I-system communication path using R-LMR [I], and the contents are stored in the request reception buffer. The memory information request is received not by RDMA but by using a simple message receive mechanism prepared by uDAPL.

  (S101) If the message receive in S100 ends normally within a predetermined time set in advance, the process proceeds to S105. If not completed within the specified time, the process proceeds to S102.

  (S102) The status variable value of the I-system communication path is confirmed. If the value is “normal”, there is a possibility that the process of message receive may be delayed, and the process returns to S101. If it is not “normal” (if “off”), the process proceeds to S103.

  (S103) It is determined that a failure has occurred in the I-system communication path, and it is determined whether there are other communication paths that can be used instead. If not, the process ends here (abnormal end). If there is, the process proceeds to S104.

  (S104) The system number of the communication path to be newly used is determined. The smallest number among the system numbers of the communication paths whose status is “normal” is set in the variable I. Thereafter, the process returns to S100, and the memory information is received again on the new communication path.

  (S105) The contents of the memory information to be transmitted to the communication server module are written into the memory information transmission buffer. The memory information includes rmr_triplet (start address, length, access key) corresponding to the buffer [rd_ptr].

  (S106) Wait until the segment read from the storage device is stored in the buffer [rd_ptr] on the main memory.

  (S107) The memory information regarding the buffer [rd_ptr] is transmitted to the communication server module via the I-system communication path using S-LMR [I]. The memory information is transmitted using a simple message send mechanism prepared by uDAPL instead of RDMA.

  (S108) If the message send in S106 ends normally within a predetermined time set in advance, the process proceeds to S78 in FIG. If not completed, the process proceeds to S108.

  (S109) The status variable value of the I-system communication path is confirmed. If the value is “normal”, there is a possibility that the processing of the message send is delayed, and the process returns to S107. If it is not “normal” (if “off”), the process proceeds to S109.

  (S110) It is determined that a failure has occurred in the I-system communication path, and it is determined whether there are other communication paths that can be used instead. If not, the process ends here (abnormal end). If there is, the process proceeds to S110.

  (S111) A system number of a communication path to be newly used is determined. The smallest number among the system numbers of the communication paths whose status is “normal” is set in the variable I. Thereafter, the process returns to S106, and the memory information is retransmitted on the new communication link.

  Note that the 0-system and 1-system communication path monitoring units in FIG. 14 are independent of each other, and independent of the operation of the storage server modules shown in FIGS. To work. The operation is the same as the operation of the communication path monitoring unit of the communication server module shown in FIG.

  Since this embodiment 1 uses two server module network switches, it has two connections. However, when D (≧ 2) server module network switches are used, it has D connections. Needless to say, the data transfer LMR group and the like are also provided in each module in correspondence with this.

[Best Mode for Carrying Out the Invention 2]
The best mode 2 (hereinafter referred to as mode 2) for carrying out the present invention will be described below. The present embodiment 2 is based on the method of claims 2, 3 and 4 described in [Claims], and corresponds to claim 6 because it is a server device embodying the method. The configuration of the entire server device according to the second embodiment is shown in FIG.

  The difference from the form 1 shown in FIG. 11 is that a communication link for connection between the 0 system and the 1 system exists. This corresponds to the interconnecting network communication link according to claim 2. With this communication link, communication between the 0-system HCA and the 1-system HCA is possible in the second embodiment.

  FIG. 23 shows details of the internal structure of the communication server module and the storage server module in FIG.

  The data storage means on the communication server module corresponds to that described in claim 1, and is a simplified description of the buffer, request transmission buffer, and memory information reception buffer of FIG. The memory area abstraction means 1 to 4 correspond to the memory area abstraction means described in claim 1, and the data transfer LMR group and the memory information exchange LMR group of FIG. It is a thing. The communication path monitoring units 1 to 4 correspond to the communication path failure / recovery detection means according to claim 4. Further, as in FIG. 13, the description of including PZ and each EVD corresponding to each connection is omitted. Further, as in FIG. 13, a read pointer, a write pointer, a full flag, and an empty flag are provided, but the description is omitted.

  The data storage means on the storage server module corresponds to that described in claim 1, and is a simplified description of the buffer, memory information transmission buffer, and request reception buffer of FIG. Other parts are omitted from FIG. 14 as in the case of the communication server module.

  The difference from Form 1 is that the communication server module and the storage server module have a total of four connections including those crossing the 0-system communication link and the 1-system communication link, and the memory area abstraction means, EP, communication path monitoring section, In addition, PZ and EVD which are omitted from description are provided with four computers corresponding to these four connections.

  As a result, the fault tolerance of communication between computers is higher than that in the first embodiment. In the first mode, for example, when a total of two places, that is, the 0-system communication link of the communication server module and the 1-system communication link of the storage server module are disconnected, communication between both computers cannot be performed. In this case, even in such a case, communication can be continued using the connection 3 using the 1-system communication link of the communication server module and the 0-system communication link of the storage server module.

  The database in the server is also exactly the same as in the form 1, and has the same information as that shown in FIG.

  The details of the operation flow of each part in FIG.

  In the second embodiment, the communication path corresponding to connection 1 is referred to as communication path 1, the communication path corresponding to connection 2 is referred to as communication path 2, and so on.

  Also, with respect to the assignment of numbers to the connection, EP, memory area abstraction means, and communication path monitoring unit corresponding to each communication path, the communication server module side that is the RDMA initiator is used as a reference in FIG. Thus, numbers 1 to 4 are assigned, and on the storage server module side, the same numbers as the corresponding resource numbers on the communication server module side are assigned. Thus, in the communication path switching process (the process included in FIGS. 17 and 21 in the first embodiment), it is used between the communication server module and the storage server module in the same manner as FIGS. 17 and 21 in the first embodiment. Communication paths can always be synchronized.

Since this form 2 has two communication links, it has four connections, but when it has D (≧ 2) communication links, it has D ^two connections and abstracts the memory area. It goes without saying that the computerization means and the like are also provided with ^two D's in each computer.

  As is apparent from the description of each embodiment of the present invention, the interconnected network redundant multimedia server apparatus of the present invention has the following advantages.

  First, when the multimedia information of a program is distributed from the server device, the segment of the multimedia information needs to be read out for each segment only once for each segment. In data distribution to the network, data reading from a single RDMA area to the NIC need only be performed once for each data.

  Therefore, the problem of (method 1) described in [Problems to be solved by the invention], that is, the increase in processing load on the host due to redundant data transfer between the external device and the main memory. And the problem of increased consumption of various resources can be avoided. This can also be said when the program multimedia information is stored in the server device.

  Further, the present invention is the same as (Method 2) described in [Problems to be solved by the invention] in that the communication link through which data flows is changed when a communication link failure occurs. However, in the present invention, the only operation necessary for switching the communication link is to switch the LMR group. The switching process is hardly loaded and ends immediately.

  Therefore, when distributing the multimedia data of the program from the server device, the data distribution operation from the main memory to the network in the host B can be normally continued without being interrupted and hardly affected by the switching of the LMR group. . Even when the multimedia information of the program is stored in the server device, the time during which the segment cannot be transferred from the host B to the host A is very short. Therefore, the multimedia information of the program transmitted from the multimedia information input device is not missed.

  In (Method 2) described in [Problems to be Solved by the Invention], it is necessary to copy (move) data between RDMA areas and reread data from a disk. In order to seamlessly execute the distribution operation as described above and the reception of multimedia information from the outside, it is necessary to execute these operations at high speed, but this is very difficult. However, in the present invention, such a problem is also solved.

  As described above, according to the present invention, the conventional obviousness described in [Problems to be Solved by the Invention] is maintained while maintaining a quick switching operation of a communication link when a failure occurs on the communication link. It is possible to solve the problem of the redundant communication link redundancy method.

  The apparatus of the present invention can be realized by a computer and a program, and the program can be recorded on a recording medium or provided through a network.

It is a figure which shows the structural example of the PC cluster system which is background art. It is a figure which shows the main object groups used by API of uDAPL which is background art. It is a figure which shows the procedure at the time of performing communication between two computers by RDMA read of uDAPL. It is a figure explaining (example 1) which is an example of a prior art. It is a figure explaining the outline | summary of (system 2) which is an example of a prior art. It is a figure explaining the detail of operation | movement of (system 2) which is an example of a prior art. It is a figure explaining the detail of operation | movement of (system 2) which is an example of a prior art. It is a figure explaining the detail of operation | movement of (system 2) which is an example of a prior art. It is a figure explaining the outline | summary of the means for solving the subject of this invention. It is a figure explaining the detail of operation | movement of the means for solving the subject of this invention. It is a figure which shows the structure of the whole apparatus of Embodiment 1 of this invention. It is a figure explaining the information which the database in a server in Embodiment 1 of this invention manages. It is a block diagram which shows the structure of the communication server module in Embodiment 1 of this invention. It is a block diagram which shows the structure of the storage server module in Embodiment 1 of this invention. It is a flowchart which shows the process of the communication server module in Embodiment 1 of this invention. It is a flowchart which shows the segment delivery process to the terminal of the communication server module in Embodiment 1 of this invention. It is a flowchart which shows the detail of step 11 of FIG. It is a flowchart which shows the process of the communication path monitoring part of the communication server module and storage server module in Embodiment 1 of this invention. It is a flowchart which shows the process of the storage server module in Embodiment 1 of this invention. It is a flowchart which shows the segment read-out process from the storage device of the storage server module in Embodiment 1 of this invention. It is a flowchart which shows the detail of step 77 of FIG. It is a figure which shows the structure of the whole apparatus of Embodiment 2 of this invention. It is a block diagram which shows the structure of the communication server module and storage server module in Embodiment 2 of this invention.

Claims (10)

A plurality of computers are connected using an interconnection network capable of executing a communication method having a connection setting mechanism between computers and an abstraction mechanism of the main memory area of the computer, and the interconnection network is D (≧ 2). A method for redundancy and switching of communication paths in a server apparatus configured to have D communication links made redundant to a single system and each computer connecting to each system,
Each of the computers is
A path redundancy means for setting a connection between its own communication link i (1 ≦ i ≦ D) and the communication link i of the opposite computer to secure a total of D redundant communication paths;
One data storage means secured in the main memory as a storage area for data to be communicated;
D memory area abstraction means corresponding to the communication paths and all corresponding to the data storage means,
The communication path during the transfer of the data contained in the data storage means to and from the opposite computer using a communication path corresponding to the memory area abstraction means via a certain one In the event of a failure, simply switching the memory area abstraction means used to a memory area abstraction means corresponding to another communication path, switching the communication path used for the data transfer, and continuing the data transfer. Communication path redundancy and switching method. In the communication path redundancy and switching method according to claim 1,
In the server device in which a communication link between interconnection networks is added to the server device so that communication can be performed between arbitrary systems of the interconnection network redundant to D (≧ 2) systems,
For each of the computers, a connection is established between the communication link i (1 ≦ i ≦ D) of the own computer and all the communication links of the opposite computers to secure a total of ^two redundant communication paths. The communication path redundancy and switching method, wherein the path redundancy means is changed as described above, and the memory area abstraction means is changed to have D ² corresponding to each communication path. In the communication path redundancy and switching method according to claim 1 or 2,
Wherein each computer corresponds to each communication path, performs a computer and the message exchange of the counter via the communication path, a slightly longer time than the time normally required for data transfer to the prescribed time for data transfer, further If the message exchange is completed within the specified time, the shorter time is set as the specified time for the message exchange. After the normal state is recorded as the state of the communication path, the same processing is repeated and within the specified time. If it could not be terminated, it has a communication path failure detection means having a function of recording the abnormal state as the state of the communication path and terminating the process,
If the data transfer cannot be completed within the specified time, refer to the status of the communication path, and if it is normal, wait for the end of the data transfer until the next specified time elapses. A communication path redundancy and switching method in which the data transfer is continued by switching to another communication path in a normal state if it is in a state. In the communication path redundancy and switching method according to claim 3,
Each of the computers can continue the message exchange even after recording an abnormal state as a communication path state to the communication path failure detection means, and can complete the message exchange again within the specified time. In this case, it has a communication path failure / recovery detection means whose function is changed to return the communication path to the normal state.
A communication path redundancy and switching method that makes it possible to reuse a communication path whose communication has been recovered after a failure once. A plurality of computers are connected using an interconnection network capable of executing a communication method having a connection setting mechanism between computers and an abstraction mechanism of the main memory area of the computer, and the interconnection network is D (≧ 2). A server apparatus configured to have D communication links that are redundant to each system and each computer connects to each system;
Each of the computers is
A path redundancy means for setting a connection between its own communication link i (1 ≦ i ≦ D) and the communication link i of the opposite computer to secure a total of D redundant communication paths;
One data storage means secured in the main memory as a storage area for data to be communicated;
D memory area abstraction means corresponding to the communication paths and all corresponding to the data storage means,
The communication path during the transfer of the data contained in the data storage means to and from the opposite computer using a communication path corresponding to the memory area abstraction means via a certain one In the event of a failure, simply switching the memory area abstraction means used to a memory area abstraction means corresponding to another communication path, switching the communication path used for the data transfer, and continuing the data transfer. Server device. The server device according to claim 5,
In the server device in which a communication link between interconnection networks is added to the server device so that communication can be performed between arbitrary systems of the interconnection network redundant to D (≧ 2) systems,
For each of the computers, a connection is established between the communication link i (1 ≦ i ≦ D) of the own computer and all the communication links of the opposite computers to secure a total of ^two redundant communication paths. In this way, the path redundancy means is changed, and the memory area abstraction means is changed to have D ² corresponding to each communication path. A plurality of computers are connected using an interconnection network capable of executing a communication method having a connection setting mechanism between computers and an abstraction mechanism of the main memory area of the computer, and the mutual connection network is D (≧ 2 ) Redundant to each system, each computer has D communication links for connecting to each system, and the plurality of computers have a communication server module that communicates directly with an external device and a storage that controls the storage device A communication server module in a server device composed of server modules,
A path redundancy means for setting a connection between its own communication link i (1 ≦ i ≦ D) and the communication link i of the storage server module to secure a total of D redundant communication paths;
One data storage means secured in the main memory as a storage area for data to be communicated;
D memory area abstraction means corresponding to the communication paths and all corresponding to the data storage means,
The communication path during the transfer of the data contained in the data storage means to and from the opposite computer using a communication path corresponding to the memory area abstraction means via a certain one In the event of a failure, simply switching the memory area abstraction means used to a memory area abstraction means corresponding to another communication path, switching the communication path used for the data transfer, and continuing the data transfer. Communication server module. A plurality of computers are connected using an interconnection network capable of executing a communication method having a connection setting mechanism between computers and an abstraction mechanism of the main memory area of the computer, and the mutual connection network is D (≧ 2 ) Redundant to each system, each computer has D communication links for connecting to each system, and the plurality of computers have a communication server module that communicates directly with an external device and a storage that controls the storage device A storage server module in a server device composed of server modules,
Path redundancy means for setting a connection between its own communication link i (1 ≦ i ≦ D) and the communication link i of the communication server module to secure a total of D redundant communication paths;
One data storage means secured in the main memory as a storage area for data to be communicated;
D memory area abstraction means corresponding to the communication paths and all corresponding to the data storage means,
The communication path during the transfer of the data contained in the data storage means to and from the opposite computer using a communication path corresponding to the memory area abstraction means via a certain one In the event of a failure, simply switching the memory area abstraction means used to a memory area abstraction means corresponding to another communication path, switching the communication path used for the data transfer, and continuing the data transfer. Storage server module.   A program for causing a computer to function as the communication server module according to claim 7.   A program for causing a computer to function as the storage server module according to claim 8.

Images