Data transfer device and data transfer method and data transfer program

The present invention relates to a data transfer device, a data transfer method and a data transfer program, and more particularly to a data transfer device, a data transfer method and a data transfer program capable of compressing and transferring data at a plurality of compression levels.

As one method of transferring a large amount of data at high speed, a method of compressing the data before transfer is known. Data compression differs in data size after compression depending on the type of data, but for example, text data and CAD data can be compressed to about 1/10 the size. Therefore, the method of compressing the data before transfer can increase the transfer speed by 10 times by simple calculation.

As a known example, it is known that compression and transfer efficiency are improved by having a user specify an acceptable waiting time and performing compression that can be performed within that time (see, for example, Patent Document 1). ). Further, as a known example, it is known that the time required for transferring the compressed data is predicted, and when the time is longer than the set time, the determination of whether to perform or cancel the transfer is inquired (for example, Patent Document). 2). The above-mentioned known example is characterized in that since the target data type is an image or a video, the degree of compression can be considerably increased and the data can be reduced.

Further, in a known example in recent years, data transfer processing and compression processing are performed at the same time to realize high-speed transfer, or compression processing and transfer processing are controlled to have the same processing time. , Achieve more efficient transfer (see, for example, Patent Document 3).<patcit num="1"><text>JP-A-2007-28646</text></patcit><patcit num="2"><text>Japanese Unexamined Patent Publication No. 10-136189</text></patcit><patcit num="3"><text>Japanese Unexamined Patent Publication No. 2002-112257</text></patcit>

<p> In many of the above known examples, the target data type is video (video data). Since the compression of video data is lossy compression, it is possible to increase the compression rate by lowering the quality of the video. On the other hand, general data compression is lossless compression, so there is a limit to the compression rate.</p><p> Furthermore, in recent years, there have been an increasing number of cases in which multiple processors are installed in a single personal computer, such as multi-core and multi-CPU. Since the transfer process depends on the thickness of the network (transfer speed), it cannot be expected to increase the speed depending on the number of processors. However, the compression process can be accelerated by making good use of multiple processors.</p><p> One embodiment of the present invention has been made in view of the above points, and an object of the present invention is to provide a data transfer device, a data transfer method, and a data transfer program that minimizes the total time of the estimated compression time and the estimated transfer time. And.</p>

<p> In order to solve the above problems, one embodiment of the present invention is a data transfer device capable of compressing and transferring data at a plurality of compression levels, and the compression rate and compression rate of the data are registered for each of the plurality of compression levels. The compression rate of data is acquired for each compression level from the compressed information database, and the estimated time required for compressing the data to be transferred is calculated for each compression level based on the compression rate of the data for each compression level. The compression rate of data is acquired for each compression level from the time calculation means and the compression information database, and the estimated size of the data to be transferred after compression is calculated for each compression level based on the compression rate of the data for each compression level. With a transfer estimated time calculation means that calculates and calculates the estimated time required for transferring the data to be transferred for each compression level.<u style="single">、</u>The compression level determining means for determining the compression level in which the estimated time required for compression of the data to be transferred and the estimated time required for transfer are added for each compression level and the added estimated time is the shortest, and the data to be transferred are described. It is characterized by having a compression means for compressing at the determined compression level and a transfer means for transferring the data to be transferred to the transfer destination.</p><p> In addition, it is also effective as an aspect of the present invention that an arbitrary combination of the constituent elements, expressions or constituent elements of one embodiment of the present invention is applied to a method, an apparatus, a system, a computer program, a recording medium, a data structure and the like. ..</p>

<p> As described above, according to one embodiment of the present invention, the total time of the estimated compression time and the estimated transfer time can be minimized.</p>

Next, the best mode for carrying out the present invention will be described with reference to the drawings based on the following examples.

The data transfer system of this embodiment has parallel processing functions of data compression, encryption, and transfer. The data transfer device constituting the data transfer system of this embodiment has one or more CPUs (processors) and efficiently schedules compression, encryption, and transfer processing.

The data transfer system of this embodiment efficiently parallelizes data compression processing, encryption processing, and transfer processing in a data transfer device having a plurality of processors when transferring a large amount of data between remote locations. By doing so, it is possible to suppress the occurrence of waiting time for these processes and realize a high transfer speed.

In the data transfer system of this embodiment, the compression, encryption, and transfer processing performed separately are considered together, and the total time required for the compression, encryption, and transfer processing is minimized. For example, in a conventional data transfer system, compression, encryption, and transfer processing are considered separately, so that the total time required for compression, encryption, and transfer processing is not always the shortest.

FIG. 1 is a configuration diagram of the data transfer system of this embodiment. In the data transfer system, the data transfer device 1 and the data receiving device 2 are connected to each other via a data transferable network 3. When transferring data from the data transfer device 1 to the data receiving device 2, the data transfer system efficiently parallelizes the data compression process, encryption process, and transfer process in the data transfer device 1 having a plurality of processors. By doing so, it is possible to suppress the occurrence of waiting time for data compression processing, encryption processing, and transfer processing, and to increase the transfer speed.

The data transfer device 1 is realized by, for example, a hardware configuration as shown in FIG. FIG. 2 is a hardware configuration diagram of an example of a data transfer device. The data transfer device 1 in FIG. 2 is an input device 11, an output device 12, a drive device 13, an auxiliary storage device 14, a main storage device 15, and one or more arithmetic processing units 16-, which are connected to each other by bus B, respectively. It is configured to have 1 to 16-n and an interface device 17.

The input device 11 is composed of a keyboard, a mouse, and the like, and is used for inputting various signals. The output device 12 is composed of a display device or the like, and is used for displaying various windows, data, and the like. The interface device 17 is composed of a modem, a LAN card, and the like, and is used for connecting to the network 3.

The data transfer program of this embodiment is at least a part of various programs that control the data transfer device 1. The data transfer program is provided, for example, by distributing the recording medium 18 or downloading from the network 3.

The recording medium 18 on which the data transfer program is recorded is a recording medium such as a CD-ROM, a flexible disk, a magneto-optical disk, or the like that optically, electrically, or magnetically records information, a ROM, a flash memory, or the like. Various types of recording media such as a semiconductor memory for electrically recording the data can be used.

When the recording medium 18 on which the data transfer program is recorded is set in the drive device 13, the data transfer program is installed in the auxiliary storage device 14 from the recording medium 18 via the drive device 13. The data transfer program downloaded from the network 3 is installed in the auxiliary storage device 14 via the interface device 17.

The auxiliary storage device 14 stores the installed data transfer program and also stores necessary files, data, and the like. The main storage device 15 reads and stores the data transfer program from the auxiliary storage device 14 when the computer is started up or when the user requests the start-up. Then, one or more arithmetic processing units 16-1 to 16-n realize various processes as described later according to the data transfer program stored in the main storage device 15.

When one or more arithmetic processing units 16-1 to 16-n are collectively referred to, they are simply referred to as arithmetic processing units 16. The data transfer program realizes the processing block as shown in FIG. 3 by information processing using the hardware resources of the data transfer device 1 shown in FIG. FIG. 3 is a processing block configuration diagram of an example of a data transfer device. The data transfer device 1 of FIG. 3 has a configuration including a storage unit 31, a compression unit 32, an encryption unit 33, a determination unit 34, a prediction unit 35, a database (DB) 36, a transfer unit 37, and a measurement unit 38.

The storage unit 31 stores data or the like to be transferred to the data receiving device 2. The compression unit 32 uses one or more arithmetic processing units 16 to perform compression processing according to the compression level. The encryption unit 33 performs encryption processing using one or more arithmetic processing units 16. The determination unit 34 determines the compression level so that the time required for the compression process, the encryption process, and the transfer process is the shortest. The details of the process for determining the compression level will be described later.

The prediction unit 35 predicts the time required for the compression process, the encryption process, and the transfer process by using the compression information table stored in the DB 36. The details of the compression information table will be described later. The transfer unit 37 transfers the data compressed at the compression level determined by the determination unit 34 to the data receiving device 2. When the measuring unit 38 transfers data to the data receiving device 2, it measures the current transfer speed of the line used and notifies the forecasting unit 35. Next, the processing of the data transfer device 1 will be described.

(In the case of one data) Here, an example is shown in which the data to be transmitted is 17.3 MB of CAD data. The LZMA algorithm is used as the compression algorithm. The compression level ranges from "0" without compression to "5" with the highest compression ratio. When the data to be transmitted is compressed at the compression level "5", for example, 17.3MB of CAD data is compressed to 5.4MB. The time required for the compression process is 20 seconds. Similarly, if the data to be transmitted is compressed at the compression level "1", for example, 17.3 MB of CAD data will be compressed to 7.3 MB. The time required for the compression process is 7 seconds.

If only the data size after compression is compared, the higher the compression level is, the better. However, it is meaningless if the time required for the compression process (compression time) is longer than the time required for the transfer process (transfer time) that can be reduced by the compression process.

For example, if the transfer speed of the line used is 1 Mbps, it takes 138.9 seconds to transfer uncompressed 17.3 MB of CAD data. Also, when the compression level is "5", the data size of the CAD data is 5.4MB, so it takes 43.7 seconds to transfer the compressed CAD data. Similarly, when the compression level is "1", the data size of the CAD data is 7.3MB, so it takes 59.2 seconds to transfer the compressed CAD data.

When the compression level is "0", the time required for the compression process and the transfer process is 138.9 seconds, which is the transfer time of 139.9 seconds plus the compression time of 0 seconds. When the compression level is "5", the time required for the compression process and the transfer process is 63.7 seconds, which is the transfer time of 43.7 seconds plus the compression time of 20 seconds. When the compression level is "1", the time required for the compression process and the transfer process is 66.2 seconds, which is the transfer time of 59.2 seconds plus the compression time of 7 seconds.

That is, it can be seen that the time required for the compression process and the transfer process is the shortest when the compression level is 5. In this way, in the case of a line with a transfer speed of 1 Mbps, it is better to set the compression level to "5".

Next, consider the case where the transfer speed of the line used is 10 Mbps. When the transfer speed of the line used is 10 Mbps, it takes 13.9 seconds to transfer uncompressed 17.3 MB of CAD data. Also, when the compression level is "5", the data size of the CAD data is 5.4MB, so it takes 4.4 seconds to transfer the compressed CAD data. Similarly, when the compression level is "1", the data size of the CAD data is 7.3MB, so it takes 5.9 seconds to transfer the compressed CAD data.

When the compression level is "0", the time required for the compression process and the transfer process is 13.9 seconds, which is the transfer time of 13.9 seconds plus the compression time of 0 seconds. When the compression level is "5", the time required for the compression process and the transfer process is 24.4 seconds, which is the transfer time of 4.4 seconds plus the compression time of 20 seconds. When the compression level is "1", the time required for the compression process and the transfer process is 12.9 seconds, which is the transfer time of 5.9 seconds plus the compression time of 7 seconds.

That is, it can be seen that the time required for the compression process and the transfer process is the shortest when the compression level is "1". In this way, in the case of a line with a transfer speed of 10 Mbps, it is better to set the compression level to "1".

In the data transfer system of this embodiment, the time required for the compression process and the transfer process is calculated for each compression level, and the compression level that minimizes the time required for the compression process and the transfer process is selected to select the compression level at which compression level. Determining if compression is the most efficient.

The compression level is L, the data size of CAD data is V, the compression time is C (L, V), the data size after compression is S (L, V), and the transfer time is T (S (L, V)). In this case, the data transfer system of this embodiment finds L that minimizes C (L, V) + T (S (L, V)) before transferring the CAD data, and transfers the CAD data at the compression level L. It is compressed and then transferred.

(For multiple data) Conventionally, when the transfer target is a plurality of data, for example, when the data A (hereinafter, simply referred to as A) is compressed and the data B (hereinafter, simply referred to as B) is transferred after being transferred, B is compressed while A is being transferred.

If A is larger than B, the compression time of B can be wasted. On the contrary, when A has a smaller size than B, there is a problem that the compression time of B becomes longer. Therefore, the data transfer system of this embodiment solves the problem as follows.

Unlike image data, when transferring multiple data sequentially such as text files and application files, perform compression processing and transfer processing in parallel, and optimize the time until all data is transferred. Then, the time that can be applied to the compression process for each data is limited.

For example, when transferring files A, B, C, ... (hereinafter simply referred to as A, B, C, ...) in the order of A, B, C, ..., the first A is described above. The optimum compression level LA may be determined by the method for a single piece of data, and the data may be compressed at that compression level LA before being transferred.

The next B needs to be compressed while A is being transferred. When A is small data, the transfer time t is short, so it is necessary to determine the optimum compression level LB that can be compressed within the time t applied to the compression process of B. However, in reality, when the efficiency of the compression process is good, there are cases where it is more efficient to transfer the data after the compression process is performed by spending a time t or more.

FIG. 4 is a schematic diagram showing the relationship between the previous data transfer process and the current data compression process when transferring a plurality of data. FIG. 4A shows a state in which the optimum compression level LB that can be compressed within the transfer time t of A, which is the previous data, is determined, and B is compressed at that compression level LB. In the case of FIG. 4A, since the compression process of B is shorter than the transfer time t of A, the transfer process of B can be started immediately after the transfer process of A is completed.

On the other hand, FIG. 4 (B) determines the optimum compression level LB to be compressed by taking a time equal to or longer than the transfer time t of the previous data A, and compresses and transfers B at the compression level LB. It shows the situation. In the case of FIG. 4B, since the compression process of B is longer than the transfer time t of A, the transfer process of B cannot be started immediately after the transfer process of A is completed. However, in the case of Fig. 4 (B), the efficiency of the compression process of B is high, and the data size of B is smaller than that of Fig. 4 (A), so the transfer process of B is faster than that of Fig. 4 (A). It is finished.

In the following, as an example, consider the case of transferring two data on a line with a transfer rate of 20 Mbps. The data transfer device 1 compresses the second data with a data size of 17.3 MB during the transfer process of the first data, and the transfer process of the first data and the compression process of the second data are performed. After both are completed, the second data transfer process shall be started.

FIG. 5 is a graph showing the correspondence between the transfer time of the first data and the time from the start of the transfer process of the first data to the end of the transfer process of the second data. .. In the graph of FIG. 5, the horizontal axis is the transfer time of the first data of 0 to 15 seconds, and the vertical axis is the transfer process of the second data after the transfer process of the first data is started. The time to finish is shown for each compression level.

As can be seen from FIG. 5, when the transfer time of the first data is 0 to 2 seconds, the transfer time can be shortened by not compressing the second data. If the transfer time of the first data is 2 to 9 seconds, the transfer time can be shortened by compressing the second data at the compression level "1". Furthermore, if the transfer time of the first data is 9 seconds or more, there is almost no difference, but the transfer time can be shortened by compressing the second data at the compression level "3".

When the data size of the first data is V1, for the compression levels L1, L2, L3, ..., Ln, the value of the following equation (1) is minimized before the transfer of the first data. The compression level L to be performed is obtained, and the first data is compressed at the compression level L.

<maths num="1"><img file="JP2010141515A_D0001.tif" /></maths>

The compression level of the first data at this time is set to L'1. When the size of the mth data to be transferred for the second and subsequent data is Vm, the value of the following equation (2) is minimized for each compression level L1, L2, L3, ..., Ln. The compression level L is obtained, and the mth data is compressed at the compression level L.

<maths num="2"><img file="JP2010141515A_D0002.tif" /></maths>

If the number of data to be transferred and the data size are all known before the transfer is started, each compression level L1, L2, L3, ..., Ln that minimizes the value of the following equation (3) It is also possible to start the data transfer after finding all the values of.

<maths num="3"><img file="JP2010141515A_D0003.tif" /></maths>

(Learning) In the above equation, it is difficult to obtain the values of C (L, V), S (L, V), and T (S (L, V)) at the time of transfer. For example, the data transfer time is proportional to the data size and can be obtained from the line transfer speed and the data size. However, the data size and compression time after compression cannot be known until the compression process is performed.

For example, a method of obtaining the compression time and the compression rate by compressing a part of the data can be considered. However, since the compression time and compression rate are almost determined by the type of data, the compression time and compression rate are stored in the compression information table of database 36 for each type of data in advance, and the compression of database 36 is performed when the value is required. It was decided to acquire the compression time and compression rate from the information table.

For example, the compression efficiency differs as shown in Fig. 6 depending on the type of data. FIG. 6 is an explanatory diagram showing the difference in compression efficiency depending on the type of data. In the compression information table of the database 36, for example, the compression time of data of a unit size (for example, 1 MB) and the data size after compression are recorded for each type of data classified by the extension of the data name.

For data of different sizes, the data compression time and the compressed data size are calculated by multiplying the compression time of the unit size data in the compression information table by the compressed data size. The data compression time depends on the specifications of the PC. In addition, the compression time of data varies depending on the application that created it, even if the data has the same extension.

Therefore, each time the data transfer device 1 compresses the data, the data transfer device 1 updates the compression information table with the result of the compression as the average value. Further, since the transfer speed of the line is often not constant, the data transfer device 1 obtains the transfer speed of the current line each time data is transferred and uses it.

(For multiprocessors) In recent years, in the data transfer device 1, a multiprocessor environment in which a plurality of processors can be used at the same time has become common. By performing the compression process using such a plurality of processors, the compression time in the data transfer device 1 is shortened as shown in FIG. 7. FIG. 7 is an explanatory diagram showing the difference in compression time depending on the number of processors. Since the compression time is shortened by performing the compression process using a plurality of processors, it is necessary to take into consideration the number of available processors when obtaining the above optimum compression level or when learning.

When the data size of the first data is V1, the first for the number of available processors N1, N2, N3, ..., Nn, compression level L1, L2, L3, ..., Ln Before transferring data, the compression level L and the number of processors N that minimize the value of the following equation (4) are obtained, and the first data is compressed at the compression level L and the number of processors N.

<maths num="4"><img file="JP2010141515A_D0004.tif" /></maths>

At this time, the compression level of the first data is set to L'1, and the number of processors used to compress the first data is set to N'1. For the second and subsequent data When the size of the mth data to be transferred is Vm, the number of available processors N1, N2, N3, ..., Nn, compression level L1, L2, L3, ... For, Ln, the compression level L and the number of processors N that minimize the value of the following equation (5) are obtained, and the mth data is compressed at the compression level L and the number of processors N.

<maths num="5"><img file="JP2010141515A_D0005.tif" /></maths>

(encryption) In order to transfer data securely, it is important to encrypt the data before transfer. When there are two or more processors, compression processing, encryption processing, and transfer processing can be processed in parallel, and it is necessary to adjust the compression level in a balance between compression processing, encryption processing, and transfer processing. Become. When the compression process, the encryption process, and the transfer process are performed in parallel, the compression process, the encryption process, and the transfer process are performed at the same time as shown in FIG. FIG. 8 is a schematic diagram showing parallelization of compression processing, encryption processing, and transfer processing.

Since there is no particular level for encryption, the number of processors that perform encryption processing is M, the time required for encryption processing (encryption time) is E (V, M), and the data size after encryption is S2 (V). ), In the case of the first data, find the values of L, N, and M that minimize the value of the following equation (6).

<maths num="6"><img file="JP2010141515A_D0006.tif" /></maths>

The number of processors used to encrypt the first data at this time is set to M'1. In the case of the second data, as shown in Fig. 8, since there is no data to be transferred even when the second data is compressed, L and N that minimize the value of the following equation (7) And find the value of M.

<maths num="7"><img file="JP2010141515A_D0007.tif" /></maths>

In the case of the third and subsequent data, find the values of L, N, and M that minimize the value of the following equation (8).

<maths num="8"><img file="JP2010141515A_D0008.tif" /></maths>

The data transfer device 1 can learn the encryption time as well as the compression time to improve the accuracy. If there is a process that can be parallelized other than the encryption process, the data transfer time can be shortened by changing the above equations (6) to (8) in the same way and obtaining the optimum compression level.

(Processing procedure of data transfer device 1) In the following, the processing procedure of the data transfer device 1 will be described separately for the case of transmitting the first data and the case of transmitting the second and subsequent data with reference to the flowchart. FIG. 9 is a flowchart showing the processing procedure when the first data is transmitted.

Proceeding to step S1, the prediction unit 35 of the data transfer device 1 acquires the data size V of the data to be transmitted, and determines the type of data to be transmitted from the extension of the data name. Proceeding to step S2, the prediction unit 35 attempts to acquire a compressed information table as described later from DB36 according to the type (extension) of the data to be transmitted.

If there is no compression information table according to the type of data to be transmitted to DB36, the prediction unit 35 proceeds to step S3 and compresses a part or all of the data to be transmitted at compression levels L1, L2, L3, ..., Ln. Then, after creating a compression information table from the obtained information (compression speed, compression rate) and registering it in DB36, the process returns to step S2.

If there is a compression information table according to the type of data to be transmitted to the DB 36, the prediction unit 35 acquires the compression information table according to the type of data to be transmitted, and proceeds to step S4. In step S4, a plurality of small data for measuring the transfer speed of the line to be used are transmitted from the transfer unit 37 to the data receiving device 2. When the data receiving device 2 receives small data for measuring the transfer speed of the line to be used, the data receiving device 2 notifies the data transfer device 1 of the reception time of the small data.

When the data receiving device 2 notifies the reception time of the small data, the measuring unit 38 proceeds to step S5 and back-calculates the transfer speed of the line to be used from the transmitting time and the receiving time of the small data. If the transfer speed of the line to be used is specified, the processing of steps S4 and S5 may be omitted.

Proceeding to step S6, the prediction unit 35 estimates the total time of the time required for transfer preparation such as compression processing and encryption processing and the time required for transfer processing using the acquired compression information table. The determination unit 34 determines the most efficient compression level at which the total time of the time required for transfer preparation such as compression processing and encryption processing and the time required for transfer processing is the shortest, and the compression processing and encryption are performed at that compression level. Judgment processing is performed so that the compression unit 32 and the encryption unit 33 perform transfer preparation such as conversion processing.

The determination process in this embodiment is set to be negligibly smaller than the time required for transfer preparation such as compression processing and encryption processing and the time required for transfer processing. Following step S6, the process proceeds to step S7, and the determination unit 34 causes the transfer unit 37 to transmit the data for which transfer preparation has been completed to the data receiving device 2.

Therefore, the data transfer device 1 is optimized so that the processing time related to the transfer of one data is minimized.

The details of the determination process in step S6 will be described separately for a standard case, a case where there are a plurality of processors, and a case where there are a plurality of processors and encryption is performed. FIG. 10 is a flowchart showing the determination processing procedure in the standard case. In the flowchart of FIG. 10, the compression information table as shown in FIG. 11 is used.

FIG. 11 is a configuration diagram of a compressed information table used in the determination process in the standard case. In the compression information table of FIG. 11, the compression speed and the compression rate are registered for each combination of data type and compression level.

Proceeding to step S11 of FIG. 10, the prediction unit 35 uses the compression information table of FIG. 11 to estimate the time required for the compression process of the data to be transmitted for the compression levels L1, L2, L3, ..., Ln. Expected compression time) Find C1, C2, C3, ..., Cn.

Proceeding to step S12, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S13, the determination unit 34 adds the estimated compression times C1, C2, C3, ..., Cn and the estimated transfer times T1, T2, T3, ..., Tn for each compression level, and the added value. Finds the smallest compression level Lm. Proceeding to step S14, the compression unit 32 compresses the data to be transmitted at the compression level Lm obtained by the determination unit 34.

Therefore, in the data transfer system of the present embodiment, in the data transfer device 1 having one processor, the data compression process and the transfer process are efficiently parallelized, thereby suppressing the occurrence of waiting time for these processes. The transfer speed can be increased.

Further, FIG. 12 is a flowchart showing a determination processing procedure when there are a plurality of processors. In the flowchart of FIG. 12, the compression information table as shown in FIG. 13 is used.

FIG. 13 is a configuration diagram of a compression information table used in the determination process when there are a plurality of processors. In the compression information table of FIG. 13, the compression rate is registered for each combination of data type and compression level, and the compression speed is registered for each combination of data type, compression level, and number of processors.

Proceeding to step S21 of FIG. 12, the prediction unit 35 uses the compression information table of FIG. 13 to obtain the compression levels L1, L2, L3, ..., Ln and the number of available processors N1, N2, N3, ···, For each combination with Np, the estimated time (expected compression time) C11, C12, ···, Cnp-1, Cnp required for the compression process of the data to be transmitted is obtained.

Proceeding to step S22, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S23, the determination unit 34 adds the estimated compression times C11, C12, ..., Cnp-1, Cnp and the estimated transfer times T1, T2, T3, ..., Tn for each compression level and adds them. Find the combination of the compression level Lm with the smallest value and the number of processors Nq. Proceeding to step S24, the compression unit 32 compresses the data to be transmitted with the compression level Lm obtained by the determination unit 34 and the number of processors Nq.

Therefore, in the data transfer system of the present embodiment, in the data transfer device 1 having a plurality of processors, the data compression process and the transfer process are efficiently parallelized, thereby suppressing the occurrence of waiting time for these processes. The transfer speed can be increased.

Further, FIG. 14 is a flowchart showing a determination processing procedure when there are a plurality of processors and encryption is performed. In the flowchart of FIG. 14, the compression information table as shown in FIG. 15 is used.

FIG. 15 is a configuration diagram of a compression information table used in the determination process when there are a plurality of processors and encryption is performed. In the compression information table of FIG. 15, the compression rate is registered for each combination of data type and compression level, and the compression speed and encryption speed are registered for each combination of data type, compression level and number of processors. ..

Proceeding to step S31 of FIG. 14, the prediction unit 35 uses the compression information table of FIG. 15 to obtain the compression levels L1, L2, L3, ..., Ln and the number of available processors N1, N2, N3, ···, For each combination with Np, the estimated time (expected compression time) C11, C12, ···, Cnp-1, Cnp required for the compression process of the data to be transmitted is obtained.

Further, in step S32, the prediction unit 35 uses the compression information table shown in FIG. 15 to set the compression levels L1, L2, L3, ..., Ln and the number of processors M1, M2, M3 that perform encryption processing. For each combination with, ..., Mk, the estimated time (estimated encryption time) E11, E12, ..., Enk-1, Enk required for the encryption process of the data to be transmitted is obtained.

Proceeding to step S33, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S34, the determination unit 34 determines the estimated compression time C11, C12, ..., Cnp-1, Cnp and the estimated encryption time E11, E12, ..., Enk-1, Enk and the estimated transfer time T1, T2, T3, ..., Tn are added for each compression level, and the combination of the compression level Lm with the smallest added value and the number of processors Nq, Mi is obtained. Proceeding to step S35, the compression unit 32 compresses the data to be transmitted with the compression level Lm obtained by the determination unit 34 and the number of processors Nq and Mi. Proceeding to step S36, the encryption unit 33 encrypts the data to be transmitted with the number of processors Mi.

Therefore, in the data transfer system of the present embodiment, in the data transfer device having a plurality of processors, the data compression process, the encryption process, and the transfer process are efficiently parallelized, so that a waiting time for these processes occurs. Can be suppressed and the transfer speed can be increased.

Next, the processing procedure of the data transfer device 1 will be described with reference to the flowchart of the case where the second and subsequent data are transmitted. FIG. 16 is a flowchart showing a processing procedure when transmitting the second and subsequent data.

In step S41, the previous data transfer process is performed. For example, when the previous data is the first data, the data transfer processing is performed by the processing procedure of FIG. 9 described above. If the previous data is the second and subsequent data, the data transfer processing is performed according to the processing procedure shown in the flowchart of FIG.

Proceeding to step S42, the measurement unit 38 obtains and records the transfer speed of the line to be used from the time taken for the transfer of the previous data. Proceeding to step S43, the prediction unit 35 of the data transfer device 1 acquires the data size V of the data to be transmitted next, and determines the type of data to be transmitted next from the extension of the data name. Proceeding to step S44, the prediction unit 35 attempts to acquire the above-mentioned compressed information table from the DB 36 according to the type (extension) of the data to be transmitted next.

If there is no compression information table according to the type of data to be transmitted to DB36, the prediction unit 35 proceeds to step S45 and compresses a part or all of the data to be transmitted at compression levels L1, L2, L3, ..., Ln. Then, after creating a compression information table from the obtained information (compression speed, compression rate) and registering it in DB36, the process returns to step S44. If there is a compression information table according to the type of data to be transmitted to the DB 36, the prediction unit 35 acquires the compression information table according to the type of data to be transmitted, and proceeds to step S46.

Proceeding to step S46, the prediction unit 35 uses the acquired compression information table to take longer time to prepare for transfer such as compression processing and encryption processing of the data to be transmitted next and transfer of the previous data. Estimate the total time of and the time required for the next data transfer process. The determination unit 34 determines the time required for transfer preparation such as compression processing and encryption processing of the data to be transmitted next, the longer time required for transferring the previous data, and the time required for the next data transfer processing. The most efficient compression level at which the total time of the above is the shortest is determined, and at that compression level, a determination process is performed in which the compression unit 32 and the encryption unit 33 perform transfer preparation such as compression processing and encryption processing. Following step S46, the process proceeds to step S47, and the determination unit 34 causes the transfer unit 37 to transmit the data for which transfer preparation has been completed to the data receiving device 2.

Therefore, the data transfer device 1 is optimized so that the processing time related to the transfer of a plurality of data is minimized.

The details of the determination process in step S46 will be described separately for a standard case, a case where there are a plurality of processors, and a case where there are a plurality of processors and encryption is performed. FIG. 17 is a flowchart showing the determination processing procedure in the standard case. In the flowchart of FIG. 17, the compression information table shown in FIG. 11 is used.

Proceeding to step S51 of FIG. 17, the prediction unit 35 predicts that the compression processing of the data to be transmitted next is performed for the compression levels L1, L2, L3, ..., Ln by using the compression information table of FIG. Find the time (expected compression time) C1, C2, C3, ..., Cn.

Proceeding to step S52, the prediction unit 35 predicts the compression estimated time C1, C2, C3, ..., Cn when the estimated transfer time of the previous data is larger than the estimated compression time C1, C2, C3, ... Replace Cn with the estimated transfer time of the previous data. As shown in FIG. 4, the process of step S52 is for dealing with the fact that the transfer process of the data to be transmitted next cannot be started unless the transfer process of the previous data is completed.

Proceeding to step S53, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S54, the determination unit 34 adds the estimated compression times C1, C2, C3, ..., Cn and the estimated transfer times T1, T2, T3, ..., Tn for each compression level, and the added value. Finds the smallest compression level Lm. Proceeding to step S55, the compression unit 32 compresses the data to be transmitted at the compression level Lm obtained by the determination unit 34.

Therefore, in the data transfer system of the present embodiment, in the data transfer device 1 having one processor, the data compression process and the transfer process are efficiently parallelized, thereby suppressing the occurrence of waiting time for these processes. The transfer speed can be increased.

Further, FIG. 18 is a flowchart showing a determination processing procedure when there are a plurality of processors. In the flowchart of FIG. 18, the compression information table as shown in FIG. 13 is used.

Proceeding to step S61 of FIG. 18, the prediction unit 35 uses the compression information table of FIG. 13 to obtain the compression levels L1, L2, L3, ..., Ln and the number of available processors N1, N2, N3, ···, For each combination with Np, the estimated time (expected compression time) C11, C12, ···, Cnp-1, Cnp required for the compression process of the data to be transmitted is obtained.

Proceeding to step S62, the prediction unit 35 predicts the compression estimated time C1, C2, C3, ..., Cn when the estimated transfer time of the previous data is larger than the estimated compression time C1, C2, C3, ... Replace Cn with the estimated transfer time of the previous data. As shown in FIG. 4, the process of step S62 is for dealing with the fact that the transfer process of the data to be transmitted next cannot be started unless the transfer process of the previous data is completed.

Proceeding to step S63, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S64, the determination unit 34 adds the estimated compression times C11, C12, ..., Cnp-1, Cnp and the estimated transfer times T1, T2, T3, ..., Tn for each compression level, and adds them. Find the combination of the compression level Lm with the smallest value and the number of processors Nq. Proceeding to step S65, the compression unit 32 compresses the data to be transmitted with the compression level Lm obtained by the determination unit 34 and the number of processors Nq.

Therefore, in the data transfer system of the present embodiment, in the data transfer device 1 having a plurality of processors, the data compression process and the transfer process are efficiently parallelized, thereby suppressing the occurrence of waiting time for these processes. The transfer speed can be increased.

Further, FIGS. 19 and 20 are flowcharts showing a determination processing procedure when there are a plurality of processors and encryption is performed. In the flowcharts of FIGS. 19 and 20, the compression information table as shown in FIG. 15 is used. Note that FIG. 19 shows the determination processing procedure when the second data is transmitted. FIG. 20 shows the determination processing procedure when the third data is transmitted.

Proceeding to step S71 of FIG. 19, the prediction unit 35 uses the compression information table of FIG. 15 to obtain the compression levels L1, L2, L3, ..., Ln and the number of available processors N1, N2, N3, ···, For each combination with Np, the estimated time (expected compression time) C11, C12, ···, Cnp-1, Cnp required for the compression process of the data to be transmitted is obtained.

Proceeding to step S72, the prediction unit 35 performs the estimated compression time C1, C2, C3, ..., Cn when the estimated encryption time of the previous data is larger than the estimated compression time C1, C2, C3, ..., Replace Cn with the estimated encryption time of the previous data. As shown in FIG. 8, the process of step S72 is for dealing with the fact that the encryption process of the data to be transmitted next cannot be started unless the encryption process of the previous data is completed.

Further, in step S73, the prediction unit 35 uses the compression information table of FIG. 15 to set the compression levels L1, L2, L3, ..., Ln and the number of processors M1, M2, M3 that perform encryption processing. For each combination with, ..., Mk, the estimated time (estimated encryption time) E11, E12, ..., Enk-1, Enk required for the encryption process of the data to be transmitted is obtained.

Proceeding to step S74, the prediction unit 35 indicates the estimated encryption time E11, E12, ..., Enk-1, Enk when the estimated transfer time of the previous data is larger, the estimated encryption time E11, E12, ... Replace Enk-1, Enk with the estimated transfer time of the previous data. As shown in FIG. 8, the process of step S74 is for dealing with the fact that the transfer process of the data to be transmitted next cannot be started unless the transfer process of the previous data is completed.

Proceeding to step S75, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S76, the determination unit 34 determines the estimated compression time C11, C12, ..., Cnp-1, Cnp and the estimated encryption time E11, E12, ..., Enk-1, Enk and the estimated transfer time T1, T2, T3, ..., Tn are added for each compression level, and the combination of the compression level Lm with the smallest added value and the number of processors Nq, Mi is obtained. Proceeding to step S77, the compression unit 32 compresses the data to be transmitted with the compression level Lm obtained by the determination unit 34 and the number of processors Nq. Proceeding to step S78, the compression unit 32 encrypts the data to be transmitted with the number of processors Mi obtained by the determination unit 34.

Proceeding to step S81 of FIG. 20, the prediction unit 35 uses the compression information table of FIG. 15 to obtain the compression levels L1, L2, L3, ..., Ln and the number of available processors N1, N2, N3, ···, For each combination with Np, the estimated time (expected compression time) C11, C12, ···, Cnp-1, Cnp required for the compression process of the data to be transmitted is obtained.

Proceeding to step S82, the prediction unit 35 indicates that the estimated compression time C1, C2, C3, ..., Cn is larger than the estimated encryption time of the previous data or the estimated transfer time of the previous data. , Expected compression time C1, C2, C3, ..., Cn is replaced with the longer of the estimated encryption time of the previous data or the estimated transfer time of the previous data. As shown in FIG. 8, the process of step S72 cannot start the encryption process of the data to be transmitted next unless the encryption process of the previous data and the transfer process of the previous data are completed. It is for dealing with it.

Further, in step S83, the prediction unit 35 uses the compression information table shown in FIG. 15 to set the compression levels L1, L2, L3, ..., Ln and the number of processors M1, M2, M3 that perform encryption processing. For each combination with, ..., Mk, the estimated time (estimated encryption time) E11, E12, ..., Enk-1, Enk required for the encryption process of the data to be transmitted is obtained.

Proceeding to step S84, the prediction unit 35 indicates the estimated encryption time E11, E12, ..., Enk-1, Enk, when the estimated transfer time of the previous data is larger, the estimated encryption time E11, E12, ... Replace Enk-1, Enk with the estimated transfer time of the previous data. As shown in FIG. 8, the process of step S84 is for dealing with the fact that the transfer process of the data to be transmitted next cannot be started unless the transfer process of the previous data is completed.

Proceeding to step S85, the prediction unit 35 uses the compression information table to estimate the estimated time (estimated transfer time) required for the transfer process at the compressed data size for the compression levels L1, L2, L3, ..., Ln. ) Find T1, T2, T3, ..., Tn.

Proceeding to step S86, the determination unit 34 determines the estimated compression time C11, C12, ..., Cnp-1, Cnp and the estimated encryption time E11, E12, ..., Enk-1, Enk and the estimated transfer time T1, T2, T3, ..., Tn are added for each compression level, and the combination of the compression level Lm with the smallest added value and the number of processors Nq, Mi is obtained. Proceeding to step S87, the compression unit 32 compresses the data to be transmitted with the compression level Lm obtained by the determination unit 34 and the number of processors Nq. Proceeding to step S88, the compression unit 32 encrypts the data to be transmitted with the number of processors Mi obtained by the determination unit 34.

Therefore, in the data transfer system of the present embodiment, in the data transfer device having a plurality of processors, the data compression process, the encryption process, and the transfer process are efficiently parallelized, so that a waiting time for these processes occurs. Can be suppressed and the transfer speed can be increased.

(Summary) As described above, the data transfer device 1 of the present embodiment performs the compression process, the encryption process, and the transfer process in parallel, and changes the compression level (compression rate) according to the condition of the line to be transferred. When there are a plurality of processors, the number of processors that perform compression processing and encryption processing is dynamically changed. The optimum compression level (compression rate) can be determined by learning the compression time and compression rate for each extension of the file (data) to be transmitted.

The present invention may have a configuration as described in the appendix below. (Appendix 1) A data transfer device that can compress and transfer data at multiple compression levels. The compression rate of data is acquired for each compression level from the compression information database in which the compression rate and compression rate of data are registered for each of a plurality of compression levels, and the data to be transferred is transferred based on the compression rate of data for each compression level. An estimated compression time calculation means that calculates the estimated time required for compression for each compression level, and The compression rate of data is acquired for each compression level from the compression information database, and the expected size of the data to be transferred after compression is calculated for each compression level based on the compression rate of the data for each compression level, and the data is transferred. An estimated transfer time calculating means for calculating the estimated time required for transferring data for each compression level, and an estimated transfer time calculation means. A compression level determining means that adds the estimated time required for compression of the data to be transferred and the estimated time required for transfer for each compression level, and determines the compression level having the shortest added estimated time. A compression means that compresses the data to be transferred at the determined compression level, and With a transfer means to transfer the data to be transferred to the transfer destination Data transfer device with. (Appendix 2) The estimated time required to compress the data to be transferred and the estimated time required to transfer the previous data are compared for each compression level, and the estimated time required to transfer the previous data is used to compress the data to be transferred. The data transfer device according to Appendix 1, further comprising a means for changing the estimated compression time, in which the estimated time required for compressing the data to be transferred is set as the estimated time required for transferring the previous data when the estimated time required is larger than the expected time. (Appendix 3) The expected compression time calculation means determines the compression level and the number of processors from a compression information database in which the compression rate of data for each of a plurality of compression levels and the compression rate corresponding to the number of processors performing compression processing are registered. The compression rate of data is acquired for each combination, and the estimated time required for compressing the data to be transferred is determined by the compression level and the number of processors based on the compression rate acquired for each combination of the compression level and the number of processors. Calculated for each combination The compression level determining means adds the estimated time required for compression of the data to be transferred and the estimated time required for transfer for each combination of the compression level and the number of the processors, and the added estimated time is the shortest. And the combination of the number of the processors is determined, The compression means compresses the data to be transferred by the determined number of processors at the determined compression level. The data transfer device described in Appendix 1. (Appendix 4) The compression estimated time changing means sets the estimated time required for compressing the data to be transferred calculated for each combination of the compression level and the number of processors and the estimated time required for transferring the previous data to the compression level. When the estimated time required for transferring the previous data is larger than the expected time required for compressing the data to be transferred, the estimated time required for compressing the data to be transferred is set to one. The data transfer device described in Appendix 2, which is the estimated time required to transfer the previous data. (Appendix 5) From the compression information database in which the compression rate of data for each of a plurality of compression levels and the compression speed and the encryption speed according to the number of processors performing the compression process and the encryption process are registered, the compression level and the number of the processors. The data encryption speed is obtained for each combination of the above, and the estimated time required for encrypting the data to be transferred is determined based on the compression level and the obtained encryption speed for each combination of the number of processors. A means of calculating the estimated compression time for each combination of the number of processors, With encryption means to encrypt the data to be transferred Have, The compression level determining means adds the estimated time required for compression of the data to be transferred, the estimated time required for encryption, and the estimated time required for transfer for each combination of the compression level and the number of the processors, and the added prediction. Determine the combination of the compression level and the number of processors with the shortest time, The compression means compresses the data to be transferred at the determined compression level by the determined number of processors. The data transfer device according to Appendix 1 or 3, wherein the encryption means encrypts data to be transferred by the determined number of processors. (Appendix 6) The estimated time required to encrypt the data to be transferred calculated for each combination of the compression level and the number of the processors and the estimated time required to transfer the previous data are the combination of the compression level and the number of the processors. When the estimated time required to transfer the previous data is larger than the estimated time required to encrypt the data to be transferred, the estimated time required to encrypt the data to be transferred is the estimated time required to encrypt the previous data. It also has a means to change the estimated encryption time, which is the estimated time required for transfer. The expected compression time changing means includes an estimated time required for compressing the data to be transferred calculated for each combination of the compression level and the number of processors, and an estimated time required for encrypting the previous data. The estimated time required for data transfer is compared for each combination of the compression level and the number of processors, and the estimated time required for encrypting the previous data or the estimated time required for transferring the previous data is compared. If the estimated time required to compress the data to be transferred is larger than the expected time required to compress the data to be transferred, the estimated time required to compress the data to be transferred is the estimated time required to encrypt the previous data or the estimated time required to transfer the previous data. The larger of the time The data transfer device according to Appendix 2 or 4. (Appendix 7) The data transfer device according to any one of Supplementary note 1 to 6, wherein the compression rate and compression rate of data are registered for each combination of a plurality of compression levels and types of data to be transferred. (Appendix 8) The data transfer device according to any one of Supplementary note 1 to 7, wherein the compression information database reflects the compression rate and the compression speed when the data to be transferred is actually compressed by the compression means. (Appendix 9) A data transfer method in a data transfer device that can compress and transfer data at multiple compression levels. The estimated compression time calculation means acquires the data compression rate for each compression level from the compression information database in which the data compression rate and compression rate are registered for each of a plurality of compression levels, and the data compression rate for each compression level. Based on the step of calculating the estimated time required to compress the data to be transferred for each compression level, The estimated transfer time calculation means acquires the compression rate of data for each compression level from the compression information database, and based on the compression rate of the data for each compression level, determines the estimated size of the data to be transferred after compression at the compression level. A step of calculating for each compression level and calculating the estimated time required for transferring the data to be transferred for each compression level. A step in which the compression level determining means adds the estimated time required for compression of the data to be transferred and the estimated time required for transfer for each compression level, and determines the compression level having the shortest added estimated time. A step in which the compression means compresses the data to be transferred at the determined compression level. The step that the transfer means transfers the data to be transferred to the transfer destination Data transfer method having. (Appendix 10) A computer used as a data transfer device that can compress and transfer data at multiple compression levels, The data compression rate is acquired for each compression level from the compression information database in which the data compression rate and compression rate are registered for each of a plurality of compression levels, and the data to be transferred is transferred based on the data compression rate for each compression level. An estimated compression time calculation means that calculates the estimated time required for compression for each compression level, and The compression rate of data is acquired for each compression level from the compression information database, and the expected size of the data to be transferred after compression is calculated for each compression level based on the compression rate of the data for each compression level, and the data is transferred. An estimated transfer time calculating means for calculating the estimated time required for transferring data for each compression level, and an estimated transfer time calculation means. A compression level determining means that adds the estimated time required for compression of the data to be transferred and the estimated time required for transfer for each compression level, and determines the compression level having the shortest added estimated time. A compression means that compresses the data to be transferred at the determined compression level, and With a transfer means to transfer the data to be transferred to the transfer destination A data transfer program to make it work.

The present invention is not limited to the specifically disclosed examples, and various modifications and modifications can be made without departing from the scope of claims. For example, the data transfer device 1 of this embodiment is composed of one housing, but may be composed of a plurality of housings. As a data transfer system in which the data transfer device 1 is composed of a plurality of housings, a storage unit 31 for storing data to be transmitted and a processing unit other than the storage unit 31 may be provided in separate housings.

Further, if the data transfer device 1 and the data reception device 2 of the present embodiment are configured to be included in one housing, a device capable of transferring and receiving data can be realized.

<figref num="1">It is a block diagram of the data transfer system of this Example.</figref><figref num="2">It is a hardware block diagram of an example of a data transfer device.</figref><figref num="3">It is a processing block block diagram of an example of a data transfer apparatus.</figref><figref num="4">It is a schematic diagram showing the relationship between the transfer process of the previous data and the compression process of the current data when transferring a plurality of data.</figref><figref num="5">It is a graph which showed the correspondence between the transfer time of the first data and the time from the start of the transfer process of the first data to the end of the transfer process of the second data.</figref><figref num="6">It is explanatory drawing which showed the difference of compression efficiency depending on the type of data.</figref><figref num="7">It is explanatory drawing which showed the difference of compression time by the number of processors.</figref><figref num="8">It is a schematic diagram which showed the parallelization of a compression process, an encryption process, and a transfer process.</figref><figref num="9">It is a flowchart which showed the processing procedure at the time of transmitting the first data.</figref><figref num="10">It is a flowchart which showed the judgment processing procedure in a standard case.</figref><figref num="11">It is a block diagram of the compression information table used in the judgment process in a standard case.</figref><figref num="12">It is a flowchart which showed the determination processing procedure when there are a plurality of processors.</figref><figref num="13">It is a block diagram of the compression information table used in the determination process when there are a plurality of processors.</figref><figref num="14">It is a flowchart which showed the determination processing procedure at the time of having a plurality of processors and encrypting.</figref><figref num="15">It is a block diagram of the compression information table used in the determination process at the time of having a plurality of processors and encrypting.</figref><figref num="16">It is a flowchart which showed the processing procedure at the time of transmitting the second and subsequent data.</figref><figref num="17">It is a flowchart which showed the judgment processing procedure in a standard case.</figref><figref num="18">It is a flowchart which showed the determination processing procedure when there are a plurality of processors.</figref><figref num="19">It is a flowchart which showed the determination processing procedure at the time of having a plurality of processors and encrypting.</figref><figref num="20">It is a flowchart which showed the determination processing procedure at the time of having a plurality of processors and encrypting.</figref>

Code description

1 Data transfer device 2 Data receiver 3 Data transferable network 11 Input device 12 Output device 13 Drive device 14 Auxiliary storage 15 Main memory 16,16-1 ~ 16-n One or more arithmetic processing units 17 Interface device 18 Recording medium 31 Memory 32 Compressor 33 Encryption Department 34 Judgment unit 35 Forecast Department 36 Database (DB) 37 Transfer section 38 Measuring unit