System and methods for providing fast cacheable access to a key-value device through a filesystem interface
20 claims: 9 independent, 11 dependent
- 1キーバリューシステム における 格納装置へのキャッシュアクセスシステムであって、 コンピュータと、 前記コンピュータ内のプロセッサと、 前記コンピュータ内のメモリと、 前記メモリに格納されて前記プロセッサ上で動作し、ページキャッシュを含むファイルシステムと、 前記メモリに格納され て前記プロセッサ上で動作する キーバリューファイルシステム(KVFS)と、 前記メモリに格納されて前記プロセッサ上で動作するKVFSシムと、 を備え、 前記KVFSは、 ファイルを識別するファイル の 名称を含むファイルシステムコマンドを前記ファイルシステムから受信する受信ユニットと、 前記ファイルシステムコマンドをキーバリューシステムコマンドにマッピングするマッピングユニットと、 前記キーバリューシステムコマンドを 前記 格納装置に伝送するコマンドユニットと、 前記キーバリューシステムコマンドの結果を前記ファイルシステムに返還する返還ユニットと、を含 み、 前記KVFSシムは、 客体の名称を含む第2キーバリューシステムコマンドをアプリケーションから受信する第2受信ユニットと、 前記第2キーバリューシステムコマンドを前記ファイルシステムコマンドにマッピングする第2マッピングユニットと、 前記ファイルシステムコマンドを前記ファイルシステムに伝送する第2コマンドユニットと、を含む ことを特徴とするシステム。
- 2前記格納装置は、前記ファイルに対するメタデータ客体及びデータ客体を格納することを特徴とする請求項1に記載のシステム。
- 3前記KVFSシムは、前記ファイルに対応するファイルディスクリプタを格納するファイルディスクリプタルックアップテーブルを更に含むことを特徴とする請求項 1 に記載のシステム。
- 4前記第2コマンドユニットは、前記ファイルシステムに前記ファイルディスクリプタ及び前記ファイルシステムコマンドを伝送するように動作することを特徴とする請求項 3 に記載のシステム。
- 5前記KVFSシムは、前記客体 の 名称から前記ファイル の 名称を生成する名称生成ユニットを更に含むことを特徴とする請求項 3 に記載のシステム。
- 6前記KVFSは、KVFSキャッシュを更に含むことを特徴とする請求項1に記載のシステム。
- 7前記KVFSキャッシュは、前記格納装置からの客体に対するデータのコピーを格納するように動作することを特徴とする請求項 6 に記載のシステム。
- 8前記返還ユニットは、前記ファイルシステムに前記KVFSキャッシュに格納された前記客体に対する前記データの一部のみを返還するように動作することを特徴とする請求項 7 に記載のシステム。
- 9前記KVFSは、前記格納装置からの客体に対するメタデータを格納するアイノードを更に含むことを特徴とする請求項1に記載のシステム。
- 10前記客体に対するデータを格納する前記ページキャッシュの少なくとも一部、及び前記客体に対するデータを要請する前記ファイルシステムコマンドに基づいて、前記ファイルシステムは、前記ファイルシステムコマンドを前記KVFSに伝送することなく、前記ページキャッシュから前記客体に対するデータを変換することを特徴とする請求項1に記載のシステム。
- 11キーバリューシステム における 格納装置へのキャッシュアクセス システムの動作 方法であって、 前記キャッシュアクセスシステムは、コンピュータ内のメモリに格納されてプロセッサ上で動作するファイルシステム、キーバリューファイルシステム(KVFS)、及びKVFSシムを備え、 前記方法は、前記KVFSによって、 ファイルを識別する ファイルの名称を含む ファイルシステムコマンドを 前記ファイルシステムから 受信する段階と、 前記ファイルに 対 する 情報を格納している アイノードにアクセスする段階と、 前記アイノードから 前記 格納装置に格納された データ 客体を識別する ための 客体 の 名称を識別する段階と、 前記ファイルシステムコマンドをキーバリューシステムコマンドにマッピングする段階と、 前記 KVFS に含まれるKVFS キャッシュの客体に 対してキーバリューシステム コマンドを遂行する段階と、 前記キーバリューシステムコマンドを前記格納装置に伝送する段階と、 前記格納装置から受信した 前記 キーバリューシステム コマンドの結果を 前記ファイルシステムに 返還する段階と、 を有し、 前記方法は、前記KVFSシムによって、 客体の名称を含む第2キーバリューシステムコマンドをアプリケーションから受信する段階と、 前記第2キーバリューシステムコマンドを前記ファイルシステムコマンドにマッピングする段階と、 前記ファイルシステムコマンドを前記ファイルシステムに伝送する段階と、 を更に有することを特徴とする方法。
- 12前記KVFSキャッシュの客体に 対してキーバリューシステム コマンドを遂行する段階は、 前記KVFSキャッシュで前記客体 の 名称を検索する段階と、 前記客体 の 名称と共に前記客体が前記KVFSキャッシュに存在する場合、前記KVFSキャッシュの客体に対するコマンドを遂行する段階と、を含むことを特徴とする請求項11に記載の方法。
- 13前記KVFSキャッシュの客体に 対してキーバリューシステム コマンドを遂行する段階は、 前記客体 の 名称が前記KVFSキャッシュに存在しない場合、前記ファイルシステムコマンドをキーバリューシステムコマンドにマッピングする段階と、 前記客体 の 名称と共に前記客体に対する前記キーバリューシステムコマンドを前記格納装置に伝送する段階と、 前記格納装置から前記客体を受信する段階と、 前記客体を前記KVFSキャッシュに格納する段階と、を含むことを特徴とする請求項12に記載の方法。
- 14前記 キーバリューシステム コマンドの結果を 前記ファイルシステムに 返還する段階は、 前記客体に対するデータの一部のみにアクセスする段階と、 前記キーバリューシステムコマンドの結果として前記データの一部を返還する段階と、を含むことを特徴とする請求項13に記載の方法。
- 15前記ファイルに対応するアイノードが発見されない場合、前記格納装置にファイル の 名称によって識別されるメタデータ客体を要請する段階と、 前記格納装置から前記ファイルに対するメタデータを含む前記メタデータ客体を受信する段階と、 前記メタデータ客体から前記メタデータを取出す段階と、 前記メタデータを使用して前記アイノードを生成する段階と、を更に含むことを特徴とする請求項11に記載の方法。
- 16前記KVFSキャッシュの客体に 対してキーバリューシステム コマンドを遂行する段階は、前記ファイルシステムコマンドに応答して前記アイノードを変形する段階を含むことを特徴とする請求項15に記載の方法。
- 17前記KVFSキャッシュの客体に 対してキーバリューシステム コマンドを遂行する段階は、 前記格納装置からの前記メタデータ客体を削除する段階と、 前記格納装置に交替メタデータ客体を格納する段階と、を更に含むことを特徴とする請求項16に記載の方法。
- 18前記第2キーバリューシステムコマンドを前記ファイルシステムコマンドにマッピングする段階は、第2客体の名称からファイルの名称を生成する段階を含み、 前記第2客体の名称からファイルの名称を生成する段階は、 前記 第2 客体 の 名称にハッシュ機能を適用して 前記 ファイル の 名称を生成する段階 を含む ことを特徴とする 請求項11に記載の 方法。
- 19前記第2客体の名称にハッシュ機能を適用してファイルの名称を生成する段階は、 前記ハッシュ機能によるハッシュ値のASCII表示として前記ファイルの名称を生成する段階 を 含むことを特徴とする請求項18に記載の方法。
- 20前記ハッシュ機能によるハッシュ値のASCII表示として前記ファイル の 名称を生成する段階は、前記ハッシュ値のASCII表示と衝突インデックスとを組合せる段階を含むことを特徴とする請求項19に記載の方法。
Independent claims20
200 paragraphs, as filed
The present invention relates to key-value storage, and more particularly to a cache access providing method and system for accessing a key-value system storage device using an operating system cache.
Existing operating systems embody a variety of methods to cache file system data in memory and improve performance. In particular, the page cache (or buffer cache) caches a lot of frequently accessed data to improve overall file system performance. The page cache itself does not require the file system to be present on the block device, but most configurations that actually utilize the page cache to improve the performance of the file system require the file system to be present on the block device. do.
Key-value Solid State Drives are the latest technology to provide even better storage performance. However, this key-value system for use in SSD like is to export the object symbol (object a semantics) in place of the block symbol (block a semantics), can not be commonly connected to the page cache. Using a key-value SSD requires one of the following: bypassing the entire current file system or using the file system without the advantages of page cache. In either case, the data from the key-value SSD is not cached in the operating system's page cache or buffer cache.
This requires creating a performance cliff and embodying the caching method of the user program itself to restore reasonable performance. Embodying the cache in the user program increases software development costs and complexity for the user. Moreover, when user space caching is used, other programs cannot easily share their cache, and when the program is terminated, the entire cache content is lost.
There is a demand for how systems, including key-value SSDs, can take advantage of page caches.
<p><patcit num="1"><text>U.S. Pat. No. 8,407,403</text></patcit><patcit num="2"><text>U.S. Pat. No. 8,880,787</text></patcit><patcit num="3"><text>U.S. Patent Application Publication No. 2013/0042060</text></patcit><patcit num="4"><text>U.S. Patent Application Publication No. 2013/0250686</text></patcit></p>
<p> The present invention has been made in view of the above-mentioned conventional problems, and an object of the present invention is to provide a cache access method and a system for accessing a key-value system storage device through a file system interface.</p>
<p> A cache access system to a key-value system storage device according to an aspect of the present invention made to achieve the above object is stored in a computer, a processor in the computer, a memory in the computer, and the memory. The KVFS comprises a file system running on the processor and including a page cache and a key-value file system (KVFS) stored in the memory, wherein the KVFS issues a file system command containing a file name that identifies the file. The receiving unit received from the file system, the mapping unit that maps the file system command to the key value system command, the command unit that transmits the key value system command to the storage device, and the result of the key value system command are filed in the file. Includes return units to return to the system.</p><p> The cache access method to the key-value system storage device according to one aspect of the present invention made to achieve the above object is a step of receiving a file system command for identifying a file and a step of accessing an inode corresponding to the file. And the stage of identifying the object name that identifies the object stored in the storage device from the inode, the stage of executing the command for the object of the key value file system (KVFS) cache, and the stage of returning the result of the command. , Have.</p><p> The cache access method to the key-value system storage device according to another aspect of the present invention made in order to achieve the above object includes the stage of receiving the object name for identifying the object stored in the storage device and the object name. It has a stage of applying a hash function and generating a file name.</p>
<p> INDUSTRIAL APPLICABILITY According to the present invention, high-speed cache access to a key-value system storage device can be provided through a file system interface.</p>
<figref num="1">It is a figure which shows the system which uses the page cache of the operating system when accessing the key value system storage device by one Embodiment of this invention.</figref><figref num="2">It is an additional detailed block diagram of the computer shown in FIG.</figref><figref num="3A">It is a figure which shows the flow of data and a command through the hierarchy of the computer shown in FIG.</figref><figref num="3B">It is a figure which shows the flow of data and a command through the hierarchy of the computer shown in FIG.</figref><figref num="4">It is a detailed block diagram of the key-value file system (KVFS) shown in FIG.</figref><figref num="5">It is a detailed block diagram of the KVFS shim shown in FIG.</figref><figref num="6">It is a detailed block diagram of the name generation unit shown in FIG.</figref><figref num="7">It is a detailed block diagram of the file descriptor look-up table shown in FIG.</figref><figref num="8">It is a detailed block diagram which shows an example of the structure of the metadata object shown in FIG.</figref><figref num="9A">It is a flowchart which shows an example of the procedure for processing a command using the computer shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="9B">It is a flowchart which shows an example of the procedure for processing a command using the computer shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="9C">It is a flowchart which shows an example of the procedure for processing a command using the computer shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="9D">It is a flowchart which shows an example of the procedure for processing a command using the computer shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="9E">It is a flowchart which shows an example of the procedure for processing a command using the computer shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="10A">It is a flowchart which shows an example of the procedure of the operation of the KVFS shim shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="10B">It is a flowchart which shows an example of the procedure of the operation of the KVFS shim shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="11A">It is a flowchart which shows an example of the procedure of the operation of KVFS shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="11B">It is a flowchart which shows an example of the procedure of the operation of KVFS shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="12A">It is a flowchart which shows an example of the procedure which uses the KVFS cache shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="12B">It is a flowchart which shows an example of the procedure which uses the KVFS cache shown in FIG. 1 by one Embodiment of this invention.</figref><figref num="13">It is a flowchart which shows an example of the procedure which generates the file name from the object name using the name generation unit shown in FIG. 5 by one Embodiment of this invention.</figref><figref num="14">It is a flowchart which shows an example of the procedure which transforms the metadata object shown in FIG. 1 by one Embodiment of this invention.</figref>
Hereinafter, specific examples of embodiments for carrying out the present invention will be described in detail with reference to the drawings. In the following detailed description, various detailed descriptions are for the purpose of assisting the understanding of the technical idea of the present invention. However, those skilled in the art can carry out the ideas of the present invention without such detailed explanation. Also, widely known methods, procedures, configurations, circuits, and networks are not described so that embodiments are not unnecessarily obscured.
The terms first, second, etc. are used herein to describe various elements, but such elements are not limited to such terms. Such terms are used simply to distinguish one from one element to the other. For example, without departing from the idea of the present invention, the first module may indicate the second module, and similarly, the second module may indicate the first module.
The terms used herein are solely for the purpose of describing particular embodiments and are not intended to limit the invention. As used herein and in the claims, the singular term is intended to include multiple forms unless the context explicitly refers to something else. As used herein, the term "and / or" includes or refers to one or more possible combinations of the listed inventory. The term "contains" as used in the detailed description expresses the presence of the characteristic, number, stage, behavior, and / or component referred to, and one or more other characteristics, number, stage, behavior, configuration. Does not exclude the existence or addition of elements and / or these groups. The features and composition of the drawings are not necessarily shown at a constant scale.
Embodiments of the present invention include a method of accessing key values that accelerate data access using an operating system page cache (or buffer cache). Key-value requests (or key-value system commands) are converted to file system requests that utilize the page cache. Further, the embodiment of the present invention converts a file system request into a key value request. For example, an embodiment of the present invention converts a page cache Read Page command into a key-value system GET and PUT command. To implement such a transformation, a key-value file system (KVFS) contains its own internal page cache that reduces the number of times an access request is generated for the storage device, including a partial read and write (partial). reads and writes). The storage device also stores a metadata object (object) that supports the file system interface and functionality while requiring minimal overhead.
By leveraging an existing operating system page cache, embodiments of the present invention improve the data access performance of key-value applications. Such results have the added benefit of allowing multiple applications to share the page cache and allow the cached data to continue throughout the application restart.
Embodiments of the invention use standard operating system page caches and buffer caches without requiring changes in the generic part of the operating system. In order to achieve such results, embodiments of the present invention include two new configurations:
1) A new key-value file system (KVFS) shim is introduced in the user space of the operating system. The KVFS shim is transparent to the application, ignoring some of the ways the application is concatenated.
2) A KVFS driver (called KVFS hierarchy or KVFS) is installed in the file system hierarchy of the operating system. The KVFS driver translates file system requests into key-value system requests according to the standard file system interface required by the operating system (BSD®'s Vnode or Linux®'s VFS interface).
FIG. 1 is a diagram showing a system that uses an operating system page cache when accessing a key-value system storage device according to an embodiment of the present invention. In FIG. 1, computer 105 includes processor 110, memory 115, and storage device 120. The processor 110 is a variety of processors, such as Intel Xeon (Intel Xeon®) or Intel Celeron Processor (Intel Celeron® processor). The memory 115 is a variety of memories such as non-volatile memory (eg, flash memory) or SRAM (static Random Access Memory), or may be general DRAM. The storage device 120 is a variety of storage devices that do not use the conventional block interface. This embodiment provides a solid state drive (SSD) that provides a key-value (object) interface. drive) is included. However, in other embodiments, it supports other types of storage devices, other types of interfaces, or both. In the following detailed description, in the context of commands, interfaces or other contexts, the term "key-value" is generally replaced by other alternatives that are compatible with the other dedicated storage device 120.
Memory 115 includes application 125, which is a variety of applications. In one embodiment, application 125 is an application designed to take advantage of the key-value interface of storage device 120. However, in another embodiment, application 125 is an application that utilizes a conventional file system. As described with reference to FIG. 3B below, embodiments of the present invention allow an application utilizing conventional file system commands to access a storage device that provides a key-value interface, such as storage device 120. To.
The memory 115 further includes an operating system 130 including a file system 135. The file system 135 is a conventional file system in the same way that the operating system 130 is a conventional operating system that includes a page cache. The term "page cache" includes caches provided by the operating system to store data for applications, such as the more general buffer cache or the more up-to-date Linux (R) type page cache. For conversion between traditional file system commands and key-value system commands, operating system 130 includes key-value file system (KVFS) shims 140 and KVFS145. The KVFS Sim 140 converts key-value system commands into file system commands that can be processed further by the file system. The KVFS145 translates the file system commands back into key-value system commands and communicates with the storage device 120 (which provides the key-value system interface instead of the traditional block interface, as described above). The KVFS Sim 140 is used as a feature that ignores library features commonly called by applications.
The implementation changes through other devices because the detailed configuration that embodies the KVFS Sim 140 and KVFS145 depends on variables that include the commands applied by the storage device 120, the file system 135, and the specifics of the operating system 130. do. In one embodiment, the KVFS Sim 140 and KVFS145 are pluggable. Implemented using functions), all of the KVFS Sim 140 and KVFS145 include a complete set of all possible functions. For a particular embodiment, a particular function is activated and the rest of the function remains in the deactivated state. For example, the KVFS Sim 140 and KVFS145 have the ability to manage all possible file system commands for all possible file systems and all possible key-value system commands for all possible storage devices 120. include. When the KVFS Sim 140 and KVFS 145 are installed on the computer, the ability to process specific commands recognized by the file system 135 and the storage device 120 is activated to provide the specific KVFS Sim 140 and KVFS 145 required for the computer 105. use.
The operating system 130 itself contains a page cache, and additional extensions reduce the need for the computer 105 to access the data in the storage device 120. For example, KVFS145 contains KVFS cache 150. The KVFS cache 150 stores a copy of data 155 and a copy of metadata 160. The copy (155, 160) is a copy of the data object 165 and the metadata object 170 stored in the storage device 120. As will be described in more detail with reference to FIGS. 3A-8 below, the data object 165 stores the underlying data and the metadata object 170 stores the metadata of the file. That is, the data object 165 and the metadata object 170 both constitute the file 175.
One reason to include the KVFS cache 150 is to manage partial reads and writes. Key-value semantics indicate that the object is entirely read or written, and partial data reading or writing is not allowed. That is, when data is required from the data object 165 stored in the storage device 120, the entire data object 165 must be read out. Similarly, if data is written to the data object 165 stored in the storage device 120, the entire data object 165 must be written.
However, the file system symbol allows partial data reading and writing. For example, a file system command may only want to read a data field from the data object 165. The key-value system symbol requires the entire data object 165 to be read, regardless of how much data was actually used, and the rest of the data is cached somewhere if needed in the future, the data object. Prevents the need to read 165 again. However, the file system command from operating system 130 only requests the specific data requested by the application, so the page cache in operating system 130 does not cache the rest of the data from the data object 165.
That is, the KVFS cache 150 provides a means of storing the data to be removed, even though it may be needed at a given point in the future.
Of course, this means that the KVFS cache 150 is a cache with restrictions that exist for any cache. The KVFS cache 150 contains a finite size determined by the space allocated to the KVFS cache 150. If the KFVS cache 150 is required to store more data than is allocated, the KVFS cache 150 must cycle through the data output of the KVFS cache 150. The KVFS cache 150 uses the appropriate algorithm to remove old data and create space for new data, such as LFU (Least Frequently Used), LRU (Least Recently Used), or other predetermined schedules.
One result of the KVFS cache 150 that strips the previous data is that for some objects, the KVFS cache 150 contains only a portion of the data. For example, suppose the data is requested by a database that is 200MB in size. Since the object is read and written as whole data from the key-value system storage device, a single object, which is approximately 200MB in size, stores the database. If part of the database is read, the entire 200MB of the database will be loaded into the KFVS cache 150. After that, a request to read a file with a size of 10KB is received, but it is assumed that the KVFS cache 150 is in the full state. For some reason, the KVFS cache 150 decides to remove 10KB of database space and creates space for the requested files.
Suppose other requests for data from the database are received. If a database larger than 199MB is in the KVFS cache 150, it is likely that the requested data is still in the KVFS cache 150. If so, the request is fulfilled from the KVFS cache 150 without accessing the storage device 120. However, if smaller data is read, and if the requested data is part of the data removed from the KVFS cache 150, the KVFS145 will need to request the entire 200MB database object again.
Data writing is managed in the same way. When data is written, if the replaced data is stored in the KVFS cache 150, the data in the KVFS cache 150 is updated and the KVFS145 returns the result. The KVFS 145 then writes the data from the KVFS cache 150 to the storage device 120 so that the data is updated in a more permanent storage. After that, the data in the KVFS cache 150 is cleared. Of course, if the KVFS cache 150 is full and new data is loaded into the KVFS cache 150, the KVFS cache 150 will store the data when the data page is removed from the KVFS cache 150. It is necessary to know whether or not the data has been written to the storage device 120 so that it can be flushed to. Therefore, the KVFS cache 150 needs to keep track of dirty bits for each page of the KVFS cache 150. Of course, another alternative is to allow the data object to be written to the storage device 120 before the KVFS145 returns the result of the data write operation. In this situation, the KVFS cache 150 is confident that the data will be safely removed.
The data object 165 includes the object name 180. The object name 180 is the data used to uniquely position the data object 165 in the storage device 120. In a similar manner, the metadata object 170 itself contains a name, as described with reference to FIGS. 5 and 6 below. The name of the metadata object 170 is derived from the name 180 of the data object 165. By generating the name of the metadata object 170 by inducing the object name 180, the metadata object 170 is always placed to recognize the object name 180.
File 175 contains file name 185. The file name 185 is independent of the object name 180, and the file name 185 can be changed without changing the object name 180 and vice versa.
FIG. 2 is an additional detailed configuration diagram of the computer 105 shown in FIG. Referring to FIG. 2, in general, one or more machines 105 is one or more processors 110 including a memory controller 215 and a clock 220 used to organize the operation of the configuration of one or more machines 105. including. Processor 110 is concatenated to memory 115, including, for example, random access memory (RAM), read-only memory (ROM), and other state storage media. The processor 110 is connected to the storage device 120 and the network connector 230. For example, the network connector 215 is an Ethernet® connector or wireless connector. The processor 110 is connected to the bus 240. The input / output interface port and user interface 245 managed using the I / O (input / output) engine 250, among other components, are connected to bus 240.
3A and 3B are diagrams showing the flow of data and commands through the hierarchy of the computer 105 shown in FIG. FIG. 3A illustrates an embodiment of the invention in which the application 125 issues a key-value system command recognized by the storage device 120. If application 125 issues the key-value system command 305, application 125 is a library function (library). function) is used. Such library functionality is ignored by the KVFS shim 140, which receives the key-value system command 305. The KVFS sim 140 maps the key-value system command 305 to the file system command 310. File system command 310 is a file system command similar to key-value system command 305, but is processed by file system 135, which is part of operating system 130 in FIG. The file system 135 (or the operating system 130 of FIG. 1, which depends on the implementation of the file system 135 and the operating system 130 of FIG. 1) accesses the page cache 315 and executes the file system command 310. When the page cache 315 executes the file system command 310, the file system 135 (or the operating system 130 in Figure 1) returns the result 320 (result) to the KVFS sim 140. The KVFS shim 140 maps the result 320 to the form expected by application 125. Application 125 expects a result for the key-value system command 305, which has a different form than the result for the file system command 310.
If file system 135 (or operating system 130 in Figure 1) fails to execute file system command 310 using page cache 315, file system 135 transmits file system command 310 to KVFS145. The KVFS145 uses the KVFS cache 150 to execute file system command 310. When KVFS145 uses KVFS cache 150 to execute file system command 135, KVFS returns result 325. File system 135 (or operating system 130 in Figure 1) generates the required update to page cache 315 and returns result 325 (shown as result 320 in Figure 3A) to KVFS Sim 140. The process continues as described above.
KVFS145 requires an update of containment device 120. For example, if file system command 310 updates the metadata for file 175 in FIG. 1, KVFS145 updates the metadata object 170 as stored in storage device 120. However, whether or not KVFS145 requires a change to storage device 120 depends on the specific ideas of KVFS145, storage device 120, and file system command 310, and is not necessarily required for all file system commands 310. ..
If KVFS145 cannot perform file system command 310 using KVFS cache 150, KVFS 145 maps file system command 310 to key value system command 330. The key-value system command 330 is expected to be normally identical to the key-value system command 305 issued by application 125. However, the key-value system command 330 may differ from the key-value system command 305 in some way. The KVFS145 receives from the storage device 120 the result that the KVFS145 returns to the file system 135 (or the operating system 130 in FIG. 1) as in the result 335. After that, the process continues as described above. The KVFS145 updates the KVFS cache 150 in FIG. 1 based on the result 335 received from the storage device 120. For example, if file system command 310 involves renaming file 175 in Figure 1, and KVFS cache 150 in Figure 1 does not pre-store the metadata object 170 in Figure 1, KVFS 145 will use key-value system command 330. To retrieve the metadata object 170 of FIG. 1, store a copy 160 of the metadata object 170 in the KVFS cache 150 of FIG. 1, and store the metadata object 170 of FIG. 1 stored in the KVFS cache 150 of FIG. Update copy 160 of. The KVFS145 issues an additional second key-value system command 330 to remove the metadata object 170 of FIG. 1 from the storage device 120 and stores the alternate metadata object 170 of FIG. 1 in the storage device 120. Therefore, the storage device 120 contains updated metadata.
FIG. 3B is similar to FIG. 3A. However, in another embodiment, application 125 issues file system command 310 instead of key-value system command 305 in FIG. 3A. For example, application 125 is an application that is not designed to take advantage of the key-value interface provided by containment device 120, but expects to use traditional file system commands instead.
The KVFS Sim 140 does not need to convert key-value system commands to file system commands because application 125 issues traditional file system commands. As a result, file system 135 (or operating system 130 in Figure 1) leverages page cache 315 based on file system command 310. However, KVFS145 still maps file system command 310 to key-value system command 330. If the actual vault 120 uses an object vault, the KVFS145 will have the vault 120 in the file system 135 by mapping the file system command 310 to the key-value system command 310 using KVFS145. Make it look like you are using. In such an embodiment of the invention, the application 125 affects the advantages of the page cache 315 even though the storage device 120 does not use the conventional block storage. The behavior of file system 135 (and / or operating system 130 in FIG. 1), KVFS145 (and KVFS cache 150 in FIG. 1), and storage device 120 is identical to that described in FIG. 3A.
Although FIGS. 3A and 3B are shown as alternative embodiments of the invention, the embodiments of the invention described in FIGS. 3A and 3B are combined with each other. For example, an embodiment of the invention operates as shown in FIG. 3A when application 125 issues a key-value system command such as the key-value system command 305 of FIG. 3A, and application 125 operates the file system command of FIG. 3B. If you issue a filesystem command like 310, it works as shown in Figure 3B. As a result, the KVFS cache 150 and page cache 315 in Figure 1 are affected by using one of the key-value system or file system commands through Application 125, and the data is the KVFS cache 150 and page in Figure 1. Shared through such application 125 within cache 315.
FIG. 4 is a detailed configuration diagram of the key-value file system (KVFS) 145 shown in FIG. In FIG. 4, the KVFS cache 150 described with reference to FIGS. 1, 3A, and 3B is excluded, and the KVFS145 includes a receive unit 405, a mapping unit 410, a command unit 415, and a return unit 420. Receiving unit 405 receives commands from other levels of operating system 130 in Figure 1, such as File System 135 in Figure 1. Mapping unit 410 maps file system commands to key-value system commands. The command unit 415 issues a key-value system command to the storage device 120 in FIG. The return unit 420 is calling the operating system 130 of Figure 1, such as File System 135 of Figure 1. Return the result to level). Not all units need to respond to all file system commands. For example, if a file system command is executed from data residing in the KVFS cache 150, mapping unit 410 and command unit 415 do not require access to information from storage device 120 in FIG.
The mapping of file system commands to key-value system commands has been described with reference to FIGS. 3A and 3B. To achieve such a mapping, the mapping unit 410 includes the appropriate mapping from file system commands to key-value system commands. For example, mapping unit 410 contains a table showing what the given file system command and the corresponding key-value system command are. Association is one-to-may. A single file system command contains multiple key-value system commands. For example, on a flash SSD, the data will not be overwritten. Modifying the data involves invalidating the original data (which, when appropriate, is subject to garbage collection by the SSD) and writing a new data object. That is, changing the metadata for a file means that the KVFS145 deletes the metadata object 170 of FIG. 1 (more precisely, the KVFS145 invalidates the metadata object 170 of FIG. 1 in the containment device 120. And the storage device 120 performs garbage collection to secure the space occupied by the previous object), requesting that the alternate metadata object be stored.
It is worth noting that the difference is divided between the names of the various data elements in the system. Referring again to FIG. 1, the data object 165 contains the object name 180 and the file 175 contains the file name 185. (The metadata object 170 also includes the object name as described with reference to FIGS. 5-8 below. However, since the name of the metadata object 170 is strictly internal to the operation of the computer 105, it is meta. The name of the data object 170 is not fully described herein.) The object name 180 identifies the data object 165. File name 185 identifies file 175. The file name 185 itself is the metadata stored in the metadata object 170, and the display shown in FIG. 1 is merely symbolic. The file 175 is an efficient element for the file system 135 and the data object 165 is an efficient element for the key-value system of the storage device 120. It is very rare for the object name 180 and the file name 185 to be the same if the object name 180 and the file name 185 are separated and not completely impossible.
In addition, the object name 180 and the file name 185 are transformed without affecting each other. For example, if application 125 decides to rename file name 185, such changes affect the content of metadata object 170 but do not change object name 180. Alternatively, if the object name 180 is changed, this will affect the data object 165 (and since the object name for the metadata object 170 will be changed, it will have an indirect effect on the metadata object 170. ,) The file name 185 remains unchanged. That is, it is very important to keep the content of the object name and the file name separate, which is a related but classified concept.
Referring again to FIG. 4, KVFS145 further includes an inode 425. Inode 425 is a typical data structure of a file. The inode 425 is a common inode used in Unix-based systems, or the inode 425 is a new data structure. The inode 425 stores information for a file such as file 175 in Figure 1. In particular, the inode 425 stores file metadata such as file name 185, file generation time and date, file owner, and so on. However, the Inode 425 contains additional information as appropriate for implementation.
FIG. 5 is a detailed configuration diagram of the KVFS shim 140 shown in FIG. In FIG. 5, the KVFS shim 140 includes a receive unit 505, a mapping unit 510, a command unit 515, and a return unit 520. The receiving unit 505 receives a command from application 125 in FIG. The mapping unit 510 maps key-value system commands to file system commands. Command unit 515 issues file system commands to file system 135 (or operating system 130 in Figure 1) in Figure 1. Return unit 520 returns the result to application 125 in Figure 1. Unlike the KVFS 145 in Figure 4, the KVFS Sim 140 does not perform key-value system commands by itself, but transmits the file system commands to the file system 135 in Figure 1 (or the operating system 130 in Figure 1).
The mapping of key-value system commands to file system commands has been described with reference to FIGS. 3A and 3B. To achieve such a mapping, the mapping unit 510 includes the appropriate mapping from key-value system commands to file system commands. For example, the mapping unit 510 contains a table showing what the file system command corresponds to the given key-value system command. However, in contrast to the mapping unit 410 in FIG. 4, the mapping unit 510 of the KVFS shim 140 generally contains a simpler embodiment. While there are many file system commands issued to the file system 135 in FIG. 1, there are only three key value system commands (GET, PUT, and DELETE) issued to the key value storage device. The GET command reads data from the storage device, the PUT command writes data to the storage device, and the DELETE command invalidates the data in the storage device. That is, if the number of commands issued to the key-value storage device is small, the implementation of the mapping unit 510 tends to be simple. Also, since the file system 135 of FIG. 1 generally includes similar commands for reading, writing, and deleting data, the mapping from key-value system commands to file system commands is relatively simple. Nevertheless, the single key-value system command maps to multiple file system commands according to the detailed configuration of storage 120 in FIG. 1 and operating system 130 in FIG.
The KVFS sim 140 also includes a file descriptor lookup table. File descriptor (file descriptor) is an internal method for accessing (one of writing or reading) the data in a file. The KVFS shim 140 stores identifiers for file descriptors in the file descriptor lookup table 525. The placed file descriptor is transmitted to the file system 135 in FIG. 1 as an argument to the file system command. In the absence of the file descriptor lookup table 525, the KVFS Sim 140 queries the operating system 130 in Figure 1 for the file descriptor each time a file is accessed or opened, and executes all required commands. Requires closing the file for the key-value system command of. However, all of these approaches are time-intensive. By storing the file descriptor in the file descriptor lookup table 525, the KVFS Sim 140 can quickly determine the appropriate file descriptor for the file system command corresponding to the received key-value system command. The file descriptor lookup table 525 is described in more detail with reference to FIG.
The KVFS shim 140 also includes a name generation unit 530. As mentioned above, the metadata object contains the names needed to access the object. However, the metadata object name is important only when it is converted from the object to a file. Therefore, the name of the metadata object is important only in KVFS Sim 140 and KVFS 145 in Figure 1. As a result, almost all suitable algorithms are used to generate names for the metadata objects.
There are some good features for the procedure for generating a name for a metadata object. First, the procedure must be deterministic. Given the same data, the same metadata name must always occur. Second, the procedure must be able to avoid collisions. Given other data, other metadata names must occur. Third, because the object name has a given length, the procedure needs to process the potential length data. This is all the features present in the name generation unit 530 that generates the name for the metadata object 170 in FIG. 1 given the object name 180.
FIG. 6 is a detailed configuration diagram of the name generation unit 530 shown in FIG. 5, and shows a configuration for generating a name from the object name 180 in FIG. 1 to the metadata object 170 in FIG. By starting with the object name 180, the name generation unit 530 avoids the problem of consistently generating the same name for the metadata object 170 of FIG. 1 from conflicting inputs. In FIG. 6, the name generation unit 530 includes a hash unit 605, an ASCII display unit 610, and a collision index unit 615. Hash unit 605 performs a hash for the object name 180. Cryptographic hash algorithms like SHA-1 provide most of the features appropriate for name generation. Therefore, it is the best choice to generate a name for a metadata object. However, there are some characters that are not part of the filename (like the slash (/) that is sometimes used to separate files from the container). Since the result of the cryptographic hash algorithm is not mandatory as a file name, ASCII display unit 610 acquires the result of hash unit 605 and produces an ASCII display of the result. ASCII display unit 610 removes problematic characters from the results of hash unit 605.
Of course, cryptographic hash algorithms do not guarantee that there are no conflicts between hash results. For example, SHA-1 produces 160-bit hash results regardless of the size of the input data. That is, when SHA-1 receives input data of 160 bits or more, SHA-1 generates a 160-bit hash. There is less likely, but the possibility of collisions still exists, as there are more possible inputs than outputs for inputs larger than 160 bits. To resolve this possibility, the collision index unit 615 adds the collision index to the ASCII display in the event of a collision. The combination of the collision index and the ASCII representation of the result of the hash unit 605 avoids possible collisions in the generation of names for the metadata object 170 in Figure 1.
If a name is generated for the metadata object 170 of FIG. 1, the KVFS shim 140 of FIG. 1 and the KVFS 145 of FIG. 1 use such a name to access the metadata object 170 of FIG. By transmitting a PUT, GET, or DELETE request to the storage device 120 in FIG. 1 with the name generated for the metadata object 170 in FIG. 1, the KVFS shim 140 in FIG. 1 and the KVFS 145 in FIG. 1 are ensured. Then, continuously access and use the metadata object 170 in Fig. 1.
FIG. 7 is a detailed configuration diagram of the file descriptor lookup table 525 shown in FIG. As mentioned above with reference to Figure 5, the file descriptor lookup table 525 provides a way for the KVFS shim 140 in Figure 1 to access the file descriptor for a given file. In Figure 7, the file descriptor lookup table 525 contains various associations for hashes and file descriptors. For example, the first hash 705 is associated with the first file descriptor 710, the second hash 715 is associated with the second file descriptor 720, and the third hash 725 is associated with the third file descriptor 730. In FIG. 7, the file descriptor lookup table 525 shows three relationships, but this embodiment supports various relationships. Given a hash value, if the corresponding file descriptor exists in the file descriptor lookup table 525, the KVFS Sim 140 looks for the corresponding file descriptor.
Hashes (705, 715, 725) store file descriptors for files as managed by operating system 130 in Figure 1. If the file descriptor has not yet been opened, the KVFS shim 140 in Figure 1 opens the file and receives the file descriptor again. The KVFS shim 140 in Figure 1 adds hash values and file descriptors to the file descriptor lookup table 525 for future use.
The KVFS shim 140 in Figure 1 uses the name for the metadata object 170 in Figure 1 as a hash to retrieve the file descriptor lookup table 525. Two other file descriptors have the same hash value in the file descriptor lookup table 525 because the name for the metadata object 170 in Figure 1 is generated by using the hash unit 605 in Figure 6 (along with other procedures). There is virtually no possibility of being associated with.
FIG. 8 is a detailed configuration diagram showing an example of the structure of the metadata object 170 shown in FIG. In Figure 8, the metadata object 170 contains various data fields. Such data fields are file name 185, date 805 when file 175 of FIG. 1 was generated, (the term "date" as used herein is considered to include all dates and times) of Figure 1. File 175 was recently modified date 810, file 175 in Figure 1 was recently accessed 815, type 820 for file 175 in Figure 1 (eg, executable, document, text file, or something else), Figure 1. Includes file 175 size 825, container 830 for storing file 175 in Figure 1, and owner 835 for file 175 in Figure 1.
The metadata object 170 further includes the object name 180. The inclusion of the object name 180 provides a system for access to the metadata object 170 to return to the data object 165 again. (Remembering that the name for the metadata object 170 is generated from the object name 180) In one embodiment, the metadata object 170 directly includes the object name 180. The metadata object 170 has a fixed size for efficient access to the metadata object 170. The fixed size means that the space allocated to the object name 180 is fixed in advance. Including the object name 180 within the metadata object 170 can be problematic because the object name is infinitely long. It is necessary that the object name 180 is not longer than the size of the field assigned to the object name 180 in the metadata object 170. In particular, this is unlikely to be a problem. The field assigned for name 180 contains the intended number of characters (200, 1000, 10000, or more). However, there is a possibility of field excess, which causes an error in operating system 130 in Figure 1.
As an alternative, as shown in FIG. 8, the metadata object 170 includes a pointer 840 indicating the location where the object name 180 is stored. If the system knows where the object name 180 is stored and the length 845 of the object name 180, the system retrieves the object name 180. The reason is that the metadata object 170, which contains a pointer to the name length 845, is more efficient at reading data of a fixed size than reading data of an unknown size. FIG. 8 shows the name length 845 stored with the object name 180, but in another embodiment the name length 845 is stored in the metadata object 170.
The metadata object 170 further includes a pointer 850 to permissions 855 (permissions). Authority 855 identifies what authority exists for the data object 165 in FIG. The structure of privilege 855 changes according to operating system 130 in Figure 1. For example, in a Unix-based system, permission 855 identifies whether it is the owner of file 175 in Figure 1 or another user in the group that includes the owner of file 175 in Figure 1. Specifies whether the user of is reading, writing, and executing the file. Other operating systems specify privilege 855 for other methods. FIG. 8 shows the privilege 855 accessed from the metadata object 170 through the pointer 850, but in other embodiments, the privilege 855 is stored within the metadata object 170.
9A-9E are flowcharts showing an example of a procedure for processing a command using the computer 105 shown in FIG. 1 according to the embodiment of the present invention. Referring to FIG. 9A, at stage 903, the KVFS shim 140 in FIG. 1 receives the key-value system command 305 in FIG. 3A from application 125 in FIG. At stage 906, the KVFS shim 140 in Figure 1 maps the key-value system command 305 in Figure 3A to the file system command 310 in Figure 3A. At stage 909, the KVFS Sim 140 looks up the file descriptor lookup table 525 in Figure 5 to find out if the intended file was previously opened. As described with reference to FIG. 6, such a search uses the name for the metadata object 170 of FIG. 1 generated by the name generation unit 530 of FIG.
At step 912 in FIG. 9B, the KVFS shim 140 in FIG. 1 determines whether the file descriptor lookup table 525 in FIG. 5 contains the intended file descriptor. If the file descriptor lookup table 525 in Figure 5 contains the intended file descriptor, then at 915, the KVFS Sim 140 in Figure 1 accesses the intended file descriptor from the file descriptor lookup table 525 in Figure 5. Otherwise, at stage 918, the KVFS Sim 140 requests a new file descriptor from the operating system 130 in Figure 1 by opening the intended file. At stage 921, the KVFS shim 140 in Figure 1 receives a new file descriptor. At stage 924, the KVFS shim 140 in Figure 1 adds a new file descriptor to the file descriptor lookup table 525 in Figure 5 for future use.
In either case, if the KVFS Sim 140 in Figure 1 contains the intended file descriptor, then at stage 927 in Figure 9C, the KVFS 145 in Figure 1 is from File System 135 in Figure 1, and more specifically in Operating System 130 in Figure 1. Receives the file system command 310 in Figure 1 from. Such a step involves the KVFS shim 140 of FIG. 1 transmitting the file system command 310 of FIG. 3A to the operating system 130 of FIG. 1 and using the page cache 315 of FIG. 3A to fulfill the request. If the page cache 315 in Figure 3A fails to fulfill the request, the operating system 130 in Figure 1 propagates the file system command 310 in Figure 3A to the KVFS145 in Figure 1.
At stage 930, KVFS145 determines if there is an inode 425 in Figure 4 that stores the intended metadata. In the absence of the inode 425, at stage 933, KVFS145 in FIG. 1 requests metadata for file 175 in FIG. 1 from storage device 120 in FIG. More specifically, KVFS145 in FIG. 1 requests the metadata object 170 in FIG. 1 from the storage device 120 in FIG. At stage 936, the KVFS145 in FIG. 1 receives the metadata object 170 for the file 175 in FIG. 1 from the storage device 120 in FIG. At stage 939, the KVFS145 in Figure 1 stores the metadata in the Inode 425 in Figure 4.
At stage 942, regardless of whether or not there is an eye node 425 in FIG. 4 that stores the intended metadata, the KVFS145 in FIG. 1 is the object name from the inode 425 in FIG. 4 or the metadata object 170 in FIG. Access 180. At stage 945, the KVFS145 in Figure 1 maps the file system command 310 in Figure 3A to the key-value system command 330 in Figure 3A.
At stage 948 in Figure 9D, if the file system command 310 in Figure 3A somehow transforms the metadata in File 175 in Figure 1, KVFS145 in Figure 1 transforms the inode 425 in Figure 4. At stage 951, the KVFS145 in Figure 1 uses the KVFS cache 150 in Figure 1 to execute the key-value system command 330 in Figure 3A.
At stage 954, the KVFS 145 in Figure 1 searches the KVFS cache 150 in Figure 1 to determine if the KVFS cache 150 in Figure 1 stores the intended data. At stage 957, the KVFS 145 in FIG. 1 determines whether the KVFS cache 150 in FIG. 1 stored the intended data. If the data object 165 in Figure 1 (or a suitable part of the data object 165 in Figure 1) is not stored in the KVFS cache 150 in Figure 1, at 960, the KVFS145 in Figure 1 issues the key-value system command 330 in Figure 3A. It is transmitted to the storage device 120 of FIG. 1 and the data object 165 of FIG. 1 is collected. At stage 963, the KVFS 145 of FIG. 1 receives the data object 165 of FIG. 1 from the storage device 120, and at stage 966, the KVFS 145 of FIG. 1 stores a copy 155 of the data object 165 of FIG. 1 in the KVFS cache 150 of FIG. do. Of course, the storage step involves removing some data from the KVFS cache 150 in Figure 1 to create space for new data. The KVFS 145 in Figure 1 uses the intended algorithm to select the data to be removed from the KVFS cache 150 in Figure 1.
In this respect, the KVFS 145 of FIG. 1 ensures that the KVFS cache 150 of FIG. 1 stores the intended data. At stage 969 of FIG. 9E, the KVFS 145 of FIG. 1 accesses the data or part of the data from the copy 155 of the data object 165 of FIG. 1 from the KVFS cache 150 of FIG. When data is written, such an access operation can be done by removing the data object 160 of FIG. 1 from the storage device 120 of FIG. 1 and writing a new data object, or by browsing the page of the KVFS cache 150 of FIG. Includes one of the simple indications later as dirty (so that the page is flushed to the containment device 120 in Figure 1). At stage 972, the KVFS145 in Figure 1 returns the result 335 in Figure 3A to the operating system 130 in Figure 1. At stage 975, the operating system 130 in Figure 1 propagates to the application 125 in Figure 1 as a result of Figure 3A.
The above description is very complex as it shows all levels of operation of the operating system 130, including the KVFS shim 140 of FIG. 1, the file system 135 of FIG. 1, and the KVFS 145 of FIG. It is advantageous to consider the operation individually with the KVFS shim 140 in Figure 1 and the KVFS 145 in Figure 1. An analysis of the behavior of the file system of FIG. 1 is not provided below because the file system 135 of FIG. 1 is maintained unchanged in this embodiment.
10A and 10B are flowcharts showing an example of the procedure for operating the KVFS shim 140 shown in FIG. 1 according to the embodiment of the present invention. At stage 1005 in Figure 10A, receive unit 505 in Figure 5 receives the key-value system command 305 in Figure 3A from application 125 in Figure 1. At 1010, the mapping unit 510 in Figure 5 maps the key-value system command 305 in Figure 3A to the file system command 310 in Figure 3A. This involves generating a name for the metadata object 170 in FIG. 1, as described with reference to FIG. 13 below. At 1015, the KVFS shim 140 in Figure 1 looks up the file descriptor lookup table 525 to determine if there is a file descriptor for file 175 in Figure 1.
At stage 1020 in Figure 10B, the KVFS shim 140 in Figure 1 determines if a file descriptor for file 175 in Figure 1 was found in the file descriptor lookup table 525 in Figure 5. If a file descriptor for file 175 in Figure 1 is found in the file descriptor lookup table 525 in Figure 5, the file descriptor is accessed in 1025 steps. Otherwise, at stage 1030, the KVFS shim 140 in Figure 1 requests a new file descriptor for file 175 in Figure 1. Such a request includes a request for file system 135 of FIG. 1 to open file 175 of FIG. At 1035, the KVFS shim 140 in Figure 1 receives a new file descriptor from the file system 135 in Figure 1. At 1040, the KVFS shim 140 in Figure 1 adds a new file descriptor (and its name to the metadata object 170 in Figure 1) to the file descriptor lookup table 525 in Figure 5.
In either case, if the KVFS shim 140 in Figure 1 contains a file descriptor for file 175 in Figure 1, then at 1045, the KVFS Sim 140 in Figure 1 issues file system command 310 through the operating system 130 in Figure 1 in Figure 1. It is transmitted to the storage device 120. Then, at 1050, the KVFS shim 140 in Figure 1 returns the result 325 in Figure 3A received from the operating system 130 in Figure 3A to the application 125 in Figure 1.
The KVFS sim 140 in Figure 1 is responsible for converting key-value system commands to file system commands so that the original page cache 315 of the operating system 130 in Figure 3A is affected. If application 125 issues a file system command instead of a key-value system command, the KVFS shim 140 in Figure 1 is passed bypass and the file system command is propagated directly to the operating system 130 in Figure 1. The result 325 in Figure 3B is then returned directly to the application in Figure 1.
11A and 11B are flowcharts showing an example of the procedure for the operation of KVFS145 shown in FIG. 1 according to the embodiment of the present invention. At stage 1105 in Figure 11A, the KVFS145 in Figure 1 receives the file system command 310 in Figure 3A. At stage 1110, KVFS145 in Figure 1 searches for inode 425 in Figure 4, which contains metadata for file 175 in Figure 1, identified by the file system command 310 in Figure 3A. At stage 1115, the KVFS145 in FIG. 1 determines whether the inode 425 in FIG. 4 was discovered. If the inode 425 of FIG. 4 is discovered, at stage 1120, the KVFS145 of FIG. 1 accesses the inode 425 of FIG. 4, and at stage 1125, the KVFS145 of FIG. 1 accesses the object name 180 from the inode 425 of FIG. ..
On the other hand, at stage 1115, if KVFS145 in FIG. 1 does not find the inode 425 in FIG. 4, at stage 1130 in FIG. 11B, KVFS145 in FIG. 1 requests the storage device 120 in FIG. 1 for the metadata object 170 in FIG. .. At stage 1135, the KVFS145 of FIG. 1 receives the metadata object 170 of FIG. 1 from the storage device 120. At stage 1140, KVFS145 in Figure 1 retrieves metadata from the metadata object 170 in Figure 1. At stage 1145, the KVFS145 in Figure 1 accesses the object name 180 in Figure 1 from the metadata object 170 in Figure 1. Such retrieval is a direct operation when the metadata object 170 directly stores the object name 180. Alternatively, such retrieval is an indirect operation of first fetching the pointer of the object name 180 (and the name length 845) before the KVFS145 of FIG. 1 loads the object name 180. At 1150 stages, the KVFS145 in Figure 1 produces the inode 425 in Figure 4.
If the KVFS cache 150 of FIG. 1 is present at stage 1155, regardless of whether the inode 425 of FIG. 4 was created or discovered, the KVFS145 of FIG. 1 becomes a copy 155 of FIG. 1 of the data object 165 of FIG. Execute a file system command against it. Finally, at 1160, the KVFS145 in Figure 1 returns the result of the command.
The content for FIGS. 11A and 11B assumes that the KVFS145 in FIG. 1 does not transmit key-value system commands to the storage device 120 in FIG. Such results may not be accurate as described with reference to FIGS. 12A and 12B below, which elaborate on step 1155 of FIG. 11B.
12A and 12B are flowcharts showing an example of a procedure using the KVFS cache 150 shown in FIG. 1 according to an embodiment of the present invention. At stage 1205 in FIG. 12A, the KVFS145 in FIG. 1 searches the KVFS cache 150 in FIG. 1 and a copy (155, 160) of the data object 165 and the metadata object 170 in FIG. Determine if it exists. At 1210, the KVFS 145 of FIG. 1 determines whether the KVFS cache 150 of FIG. 1 has stored copies (155, 160) of the data object 165 and the metadata object 170 of FIG. As used herein, the term "storing a copy" means storing the entire copy (155, 160) of the data object 165 and the metadata object 170 of FIG. 1 or the data object 165 and the meta of FIG. It does not necessarily mean to store all parts of the data object 170. Mandatory is that the KVFS cache 150 in FIG. 1 stores a copy of the data object 165 and / or part of the metadata object 170 in FIG. 1 to which the file system command is applied. If the KVFS cache 150 of FIG. 1 stores all copies of the appropriate portion of the data object 165 and / or the metadata object 170 of FIG. 1, the copy of FIG. 1 (155, 160) is the data object 165 of FIG. 1 and / or The KVFS 145 in FIG. 1 determines that the KVFS cache 150 in FIG. 1 stores a copy (155, 160) of the data object 165 and the metadata object 170 in FIG. 1, even if it is not an entire copy of the metadata object 170.
If the KVFS cache 150 in FIG. 1 stores a copy (155, 160) of the data object 165 and the metadata object 170 in FIG. 1, at 1215 steps, the KVFS 145 in FIG. 1 is relative to the copy in FIG. 1 (155, 160). And execute the file system command 310 in Figure 3A. If the file system command 310 contains a data change to one of the data object 165 in Figure 1 or the metadata object 170 in Figure 1, the KVFS145 in Figure 1 will dirty the page affected by the KVFS cache 150 in Figure 1. Mark as as to ensure that the modified data is eventually flushed to the containment device 120, or the KVFS145 in Figure 1 immediately removes the currently affected object and puts a replacement copy of the modified object into the containment device 120. Perform one of the things to store.
If the KVFS cache 150 in Figure 1 does not store a copy 155 of the data object 165 in Figure 1, then at stage 1220 in Figure 12B, the KVFS145 in Figure 1 changes the file system command 310 in Figure 3A to the key-value system command 330 in Figure 3A. Map. At 1225, the KVFS145 in Figure 1 transmits the key-value system command 330 in Figure 3A to the storage device 120 in Figure 1. At 1230, the KVFS145 in Figure 1 receives a copy of the object affected by the key-value system command 330 in Figure 3A from the storage device 120. At 1235, the KVFS 145 of FIG. 1 stores a copy (155 and / or 160) of the data object 165 and / or the metadata object 170 of FIG. 1 received from the storage device 120 of FIG. 1 into the KVFS cache 150 of FIG. do.
At stage 1240, the command is executed even if the KVFS cache 150 of FIG. 1 contains or does not store the data object 165 and / or the metadata object 170 of FIG. 1 or the copy 155 and / or 160 of FIG. Will be done. At 1240, the KVFS145 in Figure 1 transforms the Inode 425 in Figure 4 to reflect all the changes nominated by the file system command 330 in Figure 3A. At 1245, the KVFS145 in Figure 1 accesses the appropriate portion of the data from one or all of the copies 155 and / or 160 in Figure 1 from the KVFS cache 150 in Figure 1. At stage 1250, the KVFS145 in Figure 1 returns some of the data that accessed the operating system 130 in Figure 1.
FIG. 13 is a flowchart showing an example of a procedure for generating a file name from an object name 180 by using the name generation unit 530 shown in FIG. 5 according to an embodiment of the present invention. At stage 1305 in Figure 13, the KVFS Sim 140 in Figure 1 receives the object name 180. A file name is generated from the object name 180. At the 1310 stage, the hash unit 605 of FIG. 6 applies a hash algorithm to the object name 180 of FIG. 1 to generate a hash value. At stage 1315, the ASCII display unit 610 in Figure 6 produces an ASCII display of the hash value, thereby generating a valid filename in the file system 135 in Figure 1. At 1320, the collision index unit 615 combines the ASCII representation of the hash value with the collision index to generate a name for the metadata object 170 in FIG. The name for the metadata object 170 in FIG. 1 is guaranteed to be unique in the operating system 130 in FIG. 1 or at least in the folder containing the file 175 in FIG.
FIG. 14 is a flowchart showing an example of a procedure for transforming the metadata object 170 shown in FIG. 1 according to an embodiment of the present invention. It is recalled that if the storage device 120 in FIG. 1 is a flash-based storage device, the data will not be overwritten. Instead, the original data is invalidated (later subject to garbage collection) to transform the data, and a new data object containing the transformed data is written. At 1410 in FIG. 14, KVFS145 in FIG. 1 removes the metadata object 170 in FIG. 1 from the storage device 120 in FIG. At 1415, the KVFS145 in FIG. 1 stores the alternate metadata object in the storage device 120 in FIG.
An embodiment of the present invention is shown with reference to FIGS. 9A to 14. However, one of ordinary skill in the art may modify the order of any of the steps, omit one of the steps, or include a concatenation not shown in the drawings to allow other embodiments. All such modifications of the flow chart, whether explicitly stated or not, are considered to be embodiments of the present invention.
The following description provides a general description of a machine or suitable machine that embodies a particular idea of the invention. One or more machines, at least in part, are input from common input devices such as keyboards, mice, etc., as well as commands received from other machines, virtual reality environment, It is controlled by biofeedback or other input signals. As used herein, the term "machine" is intended to broadly include a single machine, a virtual machine, or a system in which a machine, a virtual machine, or a device operating together is connected by communication. .. Illustrative machines include computing devices such as personal computers, workstations, servers, portable computers, small computers, telephones, tablets, etc., as well as mass transit, such as transportation such as automobiles, trains, taxis, etc. Includes equipment.
One or more machines include an internal controller such as a programmable or non-programmable logic device or array, an application specific integrated circuit (ASIC), an internal computer, a smart card, and the like. One or more machines utilize one or more remote devices and one or more connections, such as network interfaces, modems, or other communication connections. The devices are interconnected by physical and / or logical network methods such as intranet, internet, LAN (Local Area Network), WAN (Wide Area Network) and the like. We have a wide variety of network communications such as radio frequency (RF), satellite, wavelength, IEEE (Institute of Electrical and Electronics Engineers) 802.11, Bluetooth®, light, infrared, cable, laser, etc. Recognize the availability of wired or wireless short or long wavelength carriers and protocols.
Embodiments of the invention relate to features, procedures, data structures, application programs that, when accessed by a machine, allow the machine to perform work or define summary data types or low-level hardware configurations. The data is referenced or described in connection with it. Relevant data may be stored in volatile or non-volatile memory (eg, RAM, ROM), or hard drives, floppy (registered trademark) disks, optical storage, tapes, flash memory, memory sticks (registered trademark), It is stored in related storage media including digital video disks, bio-storage, etc. and other storage devices. Relevant data is provided in the form of packets, series data, parallel data, radio signals, etc. on transmission environments, including physical or logical networks, and is used in compressed or encrypted form. .. The relevant data is used in a distributed environment and is stored logically or remotely for device access.
Embodiments of the invention include a type of non-transitory machine-readable medium (tangible, non-transitory machine-readable medium) containing instructions executed by one or more processors. The command word includes a command word for carrying out the components of the present invention as described herein.
Although the theory of the invention has been described with reference to embodiments, embodiments are modified without departing from the technical ideas of the invention and combined with other suitable embodiments. The detailed description is then focused on a particular embodiment, but other configurations are considered. In particular, although expressions such as "according to embodiments of the invention" have been used herein, such expressions generally mean the possibility of reference embodiments, wherein the invention is of a particular embodiment. Not intended to be limited to configuration. As described herein, such terms refer to the same or other embodiments that are combined with other embodiments.
The embodiments described above do not limit the present invention. Although some embodiments have been described, one of ordinary skill in the art can readily carry out the various variations of the invention without departing from the advantages and features of the present specification. Therefore, such modifications are intended to be included within the ideas of the invention as defined in the claims.
The embodiments of the present invention are extended without limitation to the following statements.
Statement 1. Embodiments of the present invention include a cache access system to a key-value system storage device. The system is stored in computer 105, processor 110 contained in computer 105, memory 115 contained in computer 105, and stored in memory 115 to run on processor 110, file system 135 including page cache 315, and memory. It has a stored key-value file system (KVFS). The KVFS145 has a receiving unit 405 that receives a file system command 310 containing a file name 185 that identifies the file 175 from the file system 135, a mapping unit 410 that maps the file system command 310 to the key value system command 330, and a key value system. It includes a command unit 415 that transmits the command 330 to the storage device 120 and a return unit 420 that returns the result of the key-value system command 330 to the file system 135.
Statement 2. Embodiments according to the invention include a system according to statement 1. The storage device 120 stores the metadata object 170 and the data object 165 for the file 175.
Statement 3. Embodiments according to the invention include a system according to statement 1. Embodiments of the present invention further include an application 125 that is stored in memory 115 and operates on processor 110 to issue file system commands 310.
Statement 4. Embodiments according to the invention include a system according to statement 1. Embodiments according to the invention further include a KVFS shim 140 stored in memory. The KVFS shim 140 has a second receiving unit 505 that receives the second key-value system command 305 containing the object name 180, a second mapping unit 510 that maps the second key-value system command 305 to the file system command 310, and a file. Includes a second command unit 515, which transmits system commands 310 to file system 135.
Statement 5. Embodiments according to the invention include a system according to statement 1. The KVFS shim 140 further includes a file descriptor lookup table 525 that stores the file descriptors (710, 720, 730) corresponding to file 175.
Statement 6. Embodiments according to the invention include a system according to statement 5. The second command unit 515 operates to transmit the file system command 310 and the file descriptors (710, 720, 730) to the file system 135.
Statement 7. Embodiments according to the invention include a system according to statement 5. The KVFS shim 140 further includes a name generation unit 530 that generates the file name 185 from the object name 180.
Statement 8. Embodiments according to the invention include a system according to statement 7. The name generation unit 530 includes a hash unit 605 that hashes the object name 180 to generate a hash value.
Statement 9. Embodiments according to the invention include a system according to statement 8. The name generation unit 530 further includes an ASCII display unit 610 that generates the file name 185 as an ASCII display of the hash value.
Statement 10. Embodiments according to the invention include a system according to statement 9. The name generation unit 530 further includes a collision index unit 615 that combines the file name 185 and the collision index.
Statement 11. Embodiments according to the invention include a system according to statement 1. KVFS145 includes KVFS cache 150.
Statement 12. Embodiments according to the invention include a system according to statement 11. The KVFS cache 150 operates to store a copy 155 of data from the storage device 120 to the object (165, 170).
Statement 13. Embodiments according to the invention include a system according to statement 12. The return unit 140 operates to return only a part of the data for the object (165, 170) stored in the KVFS cache 150 to the file system 135.
Statement 14. Embodiments according to the invention include a system according to statement 1. The KVFS145 includes an inode 425 that stores metadata for the object 165 from the storage device 120.
Statement 15. Embodiments of the present invention include memory 115. An embodiment of the present invention includes a data structure stored in memory 115. The data structures are named 185 for file 175, generated date 805 for file 175, modified date 810 for file 175, access date 815 for file 175, type 820 for file 175, and size 830 for file 175. Includes the owner for file 175 and the name 180 for the object 165 where the data is stored.
Statement 16. Embodiments of the invention include memory 115 according to statement 15. The name 180 for the object 165 that stores the data includes a pointer 840 to the name 180 for the object 165 that stores the data.
Statement 17. Embodiments of the invention include memory 115 according to statement 16. The name 180 for the object 165 that stores the data includes the size 845 of the name 180 for the object 165 that stores the data.
Statement 18. Embodiments of the invention include memory 115 according to statement 15. The data structure contains multiple permissions 855 on the object 165 that stores the data.
Statement 19. Embodiments of the invention include memory 115 according to statement 18. The plurality of permissions 855 on the object 165 that stores the data includes a pointer 850 to a second data structure that stores the multiple permissions 855 on the object 165 that stores the data.
Statement 20. An embodiment of the present invention includes a cache access method to a key-value system storage device. The method is to receive a file system command 310 containing a file name 185 that identifies the file 175, and at least one key-value system command that is executed with at least one object name 180 that identifies the file system command 310 to the object 165. The stage of mapping to 330, the stage of satisfying at least one key-value system command 330 using the key-value file system (KVFS) cache 150, and the key of at least one key-value system command 330. It includes the stage of receiving the value system result 335 and the stage of returning the key value system result 325.
Statement 21. Embodiments of the invention include a method according to statement 20. The stage of receiving the file system command 310 is the stage of receiving the second key value system command 305 including the second object name 180 identifying the second object 165 (903) and the stage of receiving the second key value system command 305 in the file system. Includes step (906) to map to command 310.
Statement 22. Embodiments of the invention include methods according to statement 21. The step of receiving the second key-value system command 305 (903) includes the step of receiving the second key-value system command 305 (903) obtained from the set including the PUT command, the GET command, and the DELETE command.
Statement 23. Embodiments of the invention include methods according to statement 21. The step of mapping the second key-value system command 305 to the file system command 310 (906) includes the step of generating the file name 185 from the second object name 180 (1310, 1315, 1320).
Statement 24. Embodiments of the invention include methods according to statement 23. The stage of generating the file name 185 from the second object name 180 (1310, 1315, 1320) includes the stage of applying the hash function to the second object name 180 to generate the file name 185 (1310).
Statement 25. Embodiments of the invention include methods according to statement 24. The stage of applying the hash function to the second object name 180 to generate the file name 185 (1310) is the stage of applying the hash function to the second object name 180 to generate the hash value (1310), and the stage of the hash value. Includes the stage (1315) of generating filename 185 as an ASCII display.
Statement 26. Embodiments of the invention include methods according to statement 25. The step of generating the file name 185 from the second object name 180 (1315) further includes the step of combining the ASCII display of the hash value and the collision index (1320).
Statement 27. Embodiments of the invention include methods according to statement 21. The step (927) of receiving the file system command 310 includes the step (909) of searching the file descriptor lookup table 525 for the file descriptors (710, 720, 730) associated with file name 185.
Statement 28. Embodiments of the invention include methods according to statement 27. The stage of receiving the file system command 310 (927) is the stage of receiving the file descriptor (710, 720, 730) for the file 175 (921) and the file if the file descriptor lookup table 525 does not contain the file name 185. It further includes the step (924) of adding the descriptors (710, 720, 730) and the file name 185 to the file descriptor lookup table 525.
Statement 29. Embodiments of the invention include methods according to statement 27. At the stage of receiving the file system command 310 (927), if the file descriptor lookup table 525 contains the file name 185, the file descriptor (710, 720) for the file 175 from the file descriptor lookup table 525 using the file name 185 is used. , 730) further includes the stage of accessing (915).
Statement 30. Embodiments of the invention include methods according to statement 23. The stage of receiving the file system command 310 (927) is the stage of requesting the metadata object 170 identified by the file name 185 from the storage device 120 (933), and the metadata including the metadata from the storage device 120 to the file 175. Further includes the stage of receiving the object 170 (936).
Statement 31. Embodiments of the invention include a method according to statement 30. The step of mapping the file system command 310 to at least one key-value system command 330 (945) includes accessing the second object name 180 from the metadata object 170 (942). The stage (951) of using the KVFS cache 150 to satisfy at least one key-value system command 330 uses the KVFS cache 150 to satisfy at least one key-value system command 330 using the metadata object 170. (951) and the KVFS cache 150 to satisfy at least one key-value system command 330 using the second object 165 (951).
Statement 32. Embodiments of the invention include methods according to statement 31. The stage of accessing the second object name 180 from the metadata object 170 (942) is the stage of accessing the pointer 840 to the second object name 180 from the metadata object 170 to the second object 165 (942), and the second object. Includes the stage (942) of retrieving the second object name 180 using the pointer 840 to name 180.
Statement 33. Embodiments of the invention include a method according to statement 32. The stage of accessing the pointer 840 from the metadata object 170 to the second object name 180 for the second object 165 (942) is from the metadata object 170 to the name length 845 and the second object name 180 for the second object name 180. Includes step (942) to access pointer 840. The stage (942) of retrieving the second object name 180 using the pointer 840 to the second object name 180 uses the name length 845 for the second object name 180 and the pointer 840 to the second object name 180. Includes the stage of recovering the second object name 180 (942).
Statement 34. Embodiments of the invention include a method according to statement 30. The stage of receiving the file system command 310 (927) includes storing the metadata from the metadata object 170 in the inode 425 (939).
Statement 35. Embodiments of the invention include methods according to statement 34. The stage (951) of satisfying at least one key-value system command 330 using the key-value file system (KVFS) cache 150 transforms the inode 425 in response to at least one key-value system command 330. Includes stage (948).
Statement 36. Embodiments of the invention include methods according to statement 35. The step of transforming the inode 425 in response to at least one key-value system command 330 (948) is to remove the metadata object 170 from the containment device 120 (1410) and to put the alternate metadata object 170 into the containment device 120. Includes the storage stage (1415) and.
Statement 37. Embodiments of the invention include a method according to statement 20. The stage (951) of satisfying at least one key-value system command 330 using the key-value file system (KVFS) cache 150 includes the stage of searching the object 150 in the KVFS cache 150 (954). ..
Statement 38. Embodiments of the invention include methods according to statement 37. The stage of receiving the key-value system result 335 of at least one key-value system command 330 (972) is the stage of accessing data from the KVFS cache 150 to the object 165 (969) if the KVFS cache 150 stores the object 165. include.
Statement 39. Embodiments of the invention include methods according to statement 38. The stage of receiving the key-value system result 335 of at least one key-value system command 330 (972) is the stage of transmitting the key-value system command 330 to the storage device 120 (960) if the KVFS cache 150 does not store the object 165. It includes a step of receiving data for the object 165 from the storage device 120 (963) and a step of storing the data in the KVFS cache 150 (966).
Statement 40. Embodiments of the invention include methods according to statement 37. The stage of receiving the key-value system result 335 of at least one key-value system command 330 (972) is the stage of receiving only a part of the data for the object 165 (969) and the stage of receiving a part of the data as the key-value system result 325. The stage of returning (972), including.
Statement 41. An embodiment of the present invention includes a cache access method to a key-value system storage device. The method stores the key-value system command 330 (1005) containing the object name 180 that identifies the object 165, the stage 310 that maps the key-value system command 330 to the file system command 310, and the file system command 310. Includes a step (1045) of transmission to device 120.
Statement 42. Embodiments of the invention include methods according to statement 41. The step of receiving the key-value system command 330 (1005) includes the step of receiving the key-value system command 330 obtained from the set including the PUT command, the GET command, and the DELETE command (1005).
Statement 43. Embodiments of the invention include methods according to statement 41. The step of mapping the key-value system command 330 to the file system command 310 (1010) includes the step of generating the file name 185 from the object name 180 (1310, 1315, 1320).
Statement 44. Embodiments of the invention include a method according to statement 43. The stage of generating the file name 185 from the object name 180 (1310, 1315, 1320) includes the stage of applying the hash function to the object name 180 to generate the file name 185 (1310).
Statement 45. Embodiments of the invention include methods according to statement 44. The stage of applying the hash function to the object name 180 to generate the file name 185 (1310) is the stage of applying the hash function to the object name 180 to generate the hash value (1310), and the stage of generating the hash value as an ASCII display of the file. Includes the stage (1315) to generate the name 185.
Statement 46. Embodiments of the invention include methods according to statement 45. The step of generating the file name 185 from the object name 180 (1315) includes the step of combining the ASCII display of the hash value and the collision index (1320).
Statement 47. Embodiments of the invention include methods according to statement 41. The step of mapping the key-value system command 330 to the file system command 310 (1010) includes searching the file descriptor lookup table 525 for the file descriptor (710, 720, 730) associated with file name 185 (1015).
Statement 48. Embodiments of the invention include methods according to statement 47. At the stage of mapping the key-value system command 330 to the file system command 310 (1010), if the file descriptor lookup table 525 does not contain the file name 185, the file descriptor (710, 720,) for the file 175 corresponding to the object 165 It further includes a step of receiving the 730) (1035) and a step of adding the file descriptors (710, 720, 730) and the file name 185 to the file descriptor lookup table 525 (1040).
Statement 49. Embodiments of the invention include methods according to statement 47. At the stage of mapping the key-value system command 330 to the file system command 310 (1010), if the file descriptor lookup table 525 contains the file name 185, the file name 185 is used from the file descriptor lookup table 525 to the file 175. It further includes the stage (1025) of accessing the file descriptors (710, 720, 730).
Statement 50. Embodiments of the invention include methods according to statement 41. The method further includes the step (1050) of returning the result 320 of the file system command 310.
Statement 51. An embodiment of the present invention includes a cache access method to a key-value system storage device. The method is to receive the file system command 310 to identify the file 175 (1105), to access the eye node 425 corresponding to the file 175 (1120), and to identify the object 165 stored in the storage device 120. The stage of accessing the name 180 from the eye node 425 (1125), the stage of executing a command to the object 165 of the key-value file system (KVFS) cache 150 (1155), and the stage of command result 325. Includes the return stage (1160).
Statement 52. Embodiments of the invention include methods according to statement 51. The stage of receiving the file system command 310 (1105) includes the stage of receiving the file system command 310 (1105) including the file descriptors (710, 720, 730). The step of accessing the inode 425 corresponding to file 175 (1120) includes the step of accessing the inode 425 corresponding to the file descriptor (710, 720, 730) (1120).
Statement 53. Embodiments of the invention include methods according to statement 51. The key-value file system (KVFS) The stage of executing a command for the object 165 of the cache 150 (1155) is the stage of searching the object name 180 in the KVFS cache 150 (1205) and the object name 180. If an object 165 with is present in the KVFS cache 150, the step (1215) of executing a command against the object 165 of the KVFS cache 150 is included.
Statement 54. Embodiments of the invention include a method according to statement 53. The stage of executing a command for the object 165 of the KVFS cache 150 (1215) is the stage of mapping the file system command 310 to the key value system command 330 if the object 165 with the object name 180 is not included in the KVFS cache 150. (1220), the stage of transmitting the key-value system command 330 for the object 165 having the object name 180 to the storage device 120 (1225), the stage of receiving the object 165 from the storage device 120 (1230), and the KVFS cache 150. Includes the stage of storing the object 165 (1235).
Statement 55. Embodiments of the invention include methods according to statement 54. The stage of returning the result 320 of the key value system command 330 (1160) is the stage of accessing only a part of the data for the object 165 (1245) and the stage of returning a part of the data as the key value system result 325 (1250). ) And, including.
Statement 56. Embodiments of the invention include methods according to statement 51. The method is to request the metadata object 170 identified by the file name 185 from the vault 120 if the eye node 425 corresponding to the file descriptor (710, 720, 730) cannot be found (1130) and from the vault 120. Receiving metadata object 170 containing metadata for file 175 (1135), retrieving metadata from metadata object 170 (1140), and using metadata to generate inode 425 (1150). And further include.
Statement 57. Embodiments of the invention include methods according to statement 56. The stage of retrieving metadata from the metadata object 170 (1140) includes accessing the pointer 840 from the metadata object 170 to the object name 180 for the object 165 (1145).
Statement 58. Embodiments of the invention include methods according to statement 57. The stage of accessing the pointer 840 from the metadata object 170 to the object name 180 to the object 165 (1145) is the stage of accessing the pointer to the name length 845 and the object name 180 from the metadata object 180 to the object name 180 (1145). including.
Statement 59. Embodiments of the invention include methods according to statement 51. The key-value file system (KVFS) The stage of executing a command against the object 165 of the cache 150 (1155) includes the stage of transforming the inode 425 in response to the file system command 310.
Statement 60. Embodiments of the invention include a method according to statement 59. The key-value file system (KVFS) The stage of executing a command for the object 165 of the cache 150 (1155) is the stage of deleting the metadata object 170 from the storage device 120 (1410) and the storage device. Includes a stage (1415) in which the alternate metadata object 170 is stored in 120.
Statement 61. An embodiment of the present invention includes a cache access method to a key-value system storage device. The method is to receive the object name 180 that identifies the object 170 stored in the storage device 120 (1305), and to apply the hash function to the object name 180 to generate the file name 185 (1310). include.
Statement 62. Embodiments of the invention include methods according to statement 61. The method further includes the step (1315) of generating the file name 185 as an ASCII representation of the hash value.
Statement 63. Embodiments of the invention include a method according to statement 62. The step of generating the file name 185 as an ASCII representation of the hash value (1315) includes the step of combining the ASCII representation of the hash value with the collision index (1320).
Statement 64. Embodiments of the invention include a product (or article) comprising a type of storage medium comprising a non-temporary command word. When a non-temporary directive is executed on machine 105, it receives (927) a file system command 310 containing file name 185 that identifies file 175, and file system command 310 at least one object that identifies object 165. Map to at least one key-value system command 330 for name 180 (945) and satisfy at least one key-value system command 330 using the key-value file system (KVFS) cache 150 (951). It is executed to receive (975) the key-value system result 335 of at least one key-value system command 330 and return the key-value system result 325.
Statement 65. Embodiments of the invention include products according to Statement 64. Receiving the file system command 310 (927) receives the second key-value system command 305 containing the second object name 180 identifying the second object 165 (903) and the second key-value system command 305. To map the file system command 310 (906), and include.
Statement 66. Embodiments of the invention include products according to Statement 65. Receiving a second key-value system command 305 (903) includes receiving a second key-value system command 305 (903) obtained from a set containing a PUT command, a GET command, and a DELETE command.
Statement 67. Embodiments of the invention include products according to Statement 65. Mapping the second key-value system command 305 to the file system command 310 (906) involves generating the file name 185 from the second object name 180 (1310, 1315, 1320).
Statement 68. Embodiments of the invention include products according to Statement 23. Generating the file name 185 from the second object name 180 (1310, 1315, 1320) includes applying the hash function to the second object name 180 to generate the file name 185 (1310).
Statement 69. Embodiments of the invention include products according to Statement 68. Applying the hash function to the second object name 180 to generate the file name 185 (1310) is to apply the hash function to the second object name 180 to generate the hash value (1310), and to generate the hash value. Includes generating filename 185 as an ASCII display (1315).
Statement 70. Embodiments of the invention include products according to Statement 69. Generating the file name 185 from the second object name 180 (1315) further includes combining the ASCII representation of the hash value with the collision index (1320).
Statement 71. Embodiments of the invention include products according to Statement 65. Receiving the file system command 310 (927) involves searching the file descriptor lookup table 525 (909) for the file descriptors (710, 720, 730) associated with file name 185.
Statement 72. Embodiments of the invention include products according to Statement 71. Receiving the file system command 310 (927) means receiving the file descriptors (710, 720, 730) for file 175 (921) and the file if the file descriptor lookup table 525 does not contain the file name 185. Further includes adding descriptors (710, 720, 730) and file name 185 to the file descriptor lookup table 525 (924).
Statement 73. Embodiments of the invention include products according to Statement 71. At the stage of receiving the file system command 310 (927), if the file descriptor lookup table 525 contains the file name 185, the file descriptor (710, 720) for the file 175 from the file descriptor lookup table 525 using the file name 185 is used. , 730) further include accessing (915).
Statement 74. Embodiments of the invention include products according to Statement 67. Receiving the file system command 310 (927) requests the metadata object 170 identified by the file name 185 from the storage device 120 (933) and the metadata containing the metadata from the storage device 120 to the file 175. Further including receiving object 170 (936).
Statement 75. Embodiments of the invention include products according to Statement 74. Mapping the file system command 310 to at least one key-value system command 330 (945) involves accessing the second object name 180 from the metadata object 170 (942). Satisfying at least one key-value system command 330 using KVFS cache 150 (951) uses KVFS cache 150 to satisfy at least one key-value system command 330 using metadata object 170. Includes doing (951) and using the KVFS cache 150 to satisfy at least one key-value system command 330 using the second object 165 (951).
Statement 76. Embodiments of the invention include products according to Statement 75. Accessing the second object name 180 from the metadata object 170 (942) means accessing the pointer 840 from the metadata object 170 to the second object name 180 for the second object 165 (942) and the second object. Includes retrieving the second object name 180 using the pointer 840 to name 180 (942).
Statement 77. Embodiments of the invention include products according to Statement 76. Accessing the pointer 840 from the metadata object 170 to the second object name 180 for the second object 165 (942) is from the metadata object 170 to the name length 845 and the second object name 180 for the second object name 180. Includes accessing pointer 840 (942). Retrieving the second object name 180 using the pointer 840 to the second object name 180 (942) uses the name length 845 for the second object name 180 and the pointer 840 to the second object name 180. Includes the recovery of the second object name 180 (942).
Statement 78. Embodiments of the invention include products according to Statement 74. Receiving the file system command 310 (927) involves storing the metadata from the metadata object 170 in the inode 425 (939).
Statement 79. Embodiments of the invention include products according to Statement 79. Satisfying at least one key-value system command 330 using the key-value file system (KVFS) cache 150 (951) responds to at least one key-value system command 330 with the inode 425. Includes transforming (948).
Statement 80. Embodiments of the invention include products according to Statement 79. Transforming the inode 425 in response to at least one key-value system command 330 (948) removes the metadata object 170 from the containment device 120 (1410) and puts the alternate metadata object 170 into the containment device 120. Includes storing (1415) and.
Statement 81. Embodiments of the invention include products according to Statement 64. Satisfying at least one key-value system command 330 using the key-value file system (KVFS) cache 150 includes searching for the object 150 in the KVFS cache 150 (954). ..
Statement 82. Embodiments of the invention include products according to Statement 81. Receiving the key-value system result 335 of at least one key-value system command 330 (972) means accessing data from the KVFS cache 150 to the object 165 (969) if the KVFS cache 150 stores the object 165. include.
Statement 83. Embodiments of the invention include products according to Statement 82. Receiving the key-value system result 335 of at least one key-value system command 330 (972) is to transmit the key-value system command 330 to the storage device 120 if the KVFS cache 150 does not store the object 165 (960). And to receive the data for the object 165 from the storage device 120 (963) and to store the data in the KVFS cache 150 (966).
Statement 84. Embodiments of the invention include products according to Statement 81. Receiving key-value system result 335 of at least one key-value system command 330 (972) receives only part of the data for object 165 (969) and part of the data as key-value system result 325. To return (972), including.
Statement 85. Embodiments of the invention include a product (or article) comprising a type of storage medium comprising a non-temporary command word. When a non-temporary instruction is executed on machine 105, it receives a key-value system command 330 (1005) containing an object name 180 that identifies the object 165, and the key-value system command 330 becomes a file system command 310. Mapping 310 and transmitting the file system command 310 to the storage device 120 (1045) are performed.
Statement 86. Embodiments of the invention include products according to Statement 85. Receiving the key-value system command 330 (1005) includes receiving the key-value system command 330 (1005) obtained from a set containing the PUT, GET, and DELETE commands.
Statement 87. Embodiments of the invention include products according to Statement 88. Mapping the key-value system command 330 to the file system command 310 (1010) involves generating the file name 185 from the object name 180 (1310, 1315, 1320).
Statement 88. Embodiments of the invention include products according to Statement 87. Generating the file name 185 from the object name 180 (1310, 1315, 1320) includes applying the hash function to the object name 180 to generate the file name 185 (1310).
Statement 89. Embodiments of the invention include products according to Statement 88. Applying the hash function to the object name 180 to generate the file name 185 (1310) is to apply the hash function to the object name 180 to generate the hash value (1310), and the file is displayed as an ASCII display of the hash value. Includes producing the name 185 (1315).
Statement 90. Embodiments of the invention include products according to Statement 89. Generating the file name 185 from the object name 180 (1315) involves combining the ASCII representation of the hash value with the collision index (1320).
Statement 91. Embodiments of the invention include products according to Statement 85. Mapping the key-value system command 330 to the file system command 310 (1010) involves searching the file descriptor lookup table 525 for the file descriptor (710, 720, 730) associated with file name 185 (1015).
Statement 92. Embodiments of the invention include products according to Statement 91. Mapping the key-value system command 330 to the file system command 310 (1010) means that if the file descriptor lookup table 525 does not contain the file name 185, then the file descriptor (710, 720, etc.) for the file 175 corresponding to the object 165 It further includes receiving (1035) the file descriptor (710, 720, 730) and adding the file name 185 to the file descriptor lookup table 525 (1040).
Statement 93. Embodiments of the invention include products according to Statement 91. Mapping key-value system command 330 to file system command 310 (1010) means that if file descriptor lookup table 525 contains file name 185, then file name 185 is used from file descriptor lookup table 525 to file 175. Further includes accessing (1025) file descriptors (710, 720, 730).
Statement 94. Embodiments of the invention include products according to Statement 85. When the instruction word is executed by machine 105, returning the result 320 of the file system command 310 (1050) is executed.
Statement 95. Embodiments of the invention include a product (or article) comprising a type of storage medium comprising a non-temporary command word. When a non-temporary command is executed on machine 105, it receives a file system command 310 that identifies file 175 (1105), accesses inode 425 that corresponds to file 175 (1120), and stores it. Accessing the object name 180 that identifies the object 165 stored in the device 120 from the eye node 425 (1125) and executing commands for the object 165 in the key-value file system (KVFS) cache 150. To do (1155) and to return the result of the command 325 (1160).
Statement 96. Embodiments of the invention include products according to Statement 95. Receiving a file system command 310 (1105) includes receiving a file system command 310 (1105) containing a file descriptor (710, 720, 730). Accessing the inode 425 corresponding to file 175 (1120) includes accessing the inode 425 corresponding to the file descriptor (710, 720, 730) (1120).
Statement 97. Embodiments of the invention include products according to Statement 95. To execute a command for the object 165 of the key-value file system (KVFS) cache 150 (1155) is to search the object name 180 in the KVFS cache 150 (1205) and the object name 180. If an object 165 with is present in the KVFS cache 150, it includes executing a command against the object 165 in the KVFS cache 150 (1215).
Statement 98. Embodiments of the invention include products according to Statement 97. Performing a command on object 165 of KVFS cache 150 (1215) maps file system command 310 to key-value system command 330 if object 165 with object name 180 is not included in KVFS cache 150. (1220), transmitting the key-value system command 330 for the object 165 with the object name 180 to the storage device 120 (1225), receiving the object 165 from the storage device 120 (1230), and to the KVFS cache 150. Includes storing object 165 (1235).
Statement 99. Embodiments of the invention include products according to Statement 98. Returning the result 320 of the key-value system command 330 (1160) means accessing only part of the data for the object 165 (1245) and returning part of the data as the key-value system result 325 (1250). ) And, including.
Statement 100. Embodiments of the invention include products according to Statement 95. When the command word is executed by machine 105, if it cannot find the eye node 425 corresponding to the file descriptor (710, 720, 730), it requests the metadata object 170 identified by the file name 185 from the storage device 120 ( 1130), receiving metadata object 170 containing metadata for file 175 from storage device 120 (1135), retrieving metadata from metadata object 170 (1140), and using metadata to inode To generate 425 (1150) and is executed.
Statement 101. Embodiments of the invention include products according to Statement 100. Retrieving metadata from a metadata object 170 (1140) involves accessing a pointer 840 from the metadata object 170 to the object name 180 for the object 165 (1145).
Statement 102. Embodiments of the invention include products according to Statement 101. Accessing the pointer 840 from the metadata object 170 to the object name 180 to the object 165 (1145) is accessing the pointer to the name length 845 and the object name 180 from the metadata object 180 to the object name 180 (1145). including.
Statement 103. Embodiments of the invention include products according to Statement 95. Performing a command on object 165 of the key-value file system (KVFS) cache 150 involves transforming the inode 425 in response to file system command 310.
Statement 104. Embodiments of the invention include products according to statement 103. Performing a command on object 165 in the key-value file system (KVFS) cache 150 (1155) removes the metadata object 170 from storage 120 (1410) and the storage device. Includes (1415), and storing alternate metadata object 170 in 120.
Statement 105. Embodiments of the invention include a product (or article) comprising a type of storage medium comprising a non-temporary command word. When a non-temporary command word is executed on the machine 105, it receives the object name 180 that identifies the object 170 stored in the storage device 120 (1305), and applies the hash function to the object name 180 to create a file. Generating name 185 (1310) and is executed.
Statement 106. Embodiments of the invention include products according to statement 105. When the instruction word is executed on machine 105, it is executed to generate filename 185 as an ASCII representation of the hash value (1315).
Statement 107. Embodiments of the invention include products according to statement 106. Generating the file name 185 as an ASCII representation of the hash value (1315) involves combining the ASCII representation of the hash value with the collision index (1320).
Although the embodiments of the present invention have been described in detail with reference to the drawings, the present invention is not limited to the above-described embodiments and is variously modified within the range not departing from the technical scope of the present invention. It is possible to carry out.
105 Computer 110 Processor 115 Memory 120 Storage 125 Application 130 Operating System 135 File System 140 KVFS Sim 145 Key Value File System (KVFS) 150 KVFS Cache 155 Data Object (Copy) 160 Metadata Object (Copy) 165 Data Object 170 Metadata Object 175 File 180 Object Name 185 File Name 215 Memory Controller 220 Clock 230 Network Connector 240 Bus 245 User Interface 250 I / O (Input / Output) Engine 305, 330 Key Value System Command 310 File System Command 315 Page Cache 320, 325 , 335 Result 405, 505 Receiving unit 410, 510 Mapping unit 415, 515 Command unit 420, 520 Return unit 425 Inode 525 File descriptor Look-up table 530 Name generation unit 605 Hash unit 610 ASCII display unit 615 Collision index unit 705, 715, 725 1st to 3rd hashes 710, 720, 730 1st to 3rd file descriptors 805 Generated date 810 Modified date 815 Accessed date 820 Type 825 Size 830 Container 835 Owner 840, 850 Pointer 845 Length 855 Privileges
22 sheets
Sheet 1 Sheet 2 Sheet 3 Sheet 4 Sheet 5 Sheet 6 Sheet 7 Sheet 8 Sheet 9 Sheet 10 Sheet 11 Sheet 12 Sheet 13 Sheet 14 Sheet 15 Sheet 16 Sheet 17 Sheet 18 Sheet 19 Sheet 20 Sheet 21 Sheet 22
Every citation, both ways
| Document | Relation | Office |
|---|---|---|
| JP2015153014A | Cites | Japan |
| JP2014048741A | Cites | Japan |
12 members in 5 offices
Priority claims10
| Document | Office | Kind | Date |
|---|---|---|---|
| 201662298987 | United States of America | P | |
| 201662298987 | United States of America | P | |
| 62298987 | United States of America | – | |
| 15143504 | United States of America | – | |
| 201615143504 | United States of America | A | |
| 201615143504 | United States of America | A | |
| 15143504 | – | – | – |
| 62298987 | – | – | – |
| US201615143504 | – | – | – |
| US201662298987P | – | – | – |
Members12
| Document | Office | Kind | |
|---|---|---|---|
| US2017242867A1 | United States of America | A1 | |
| CN107103021A | China | A | |
| EP3211547A1 | European Patent Office (EPO) | A1 | |
| JP2017151974A | Japan | A | |
| KR20170099351A | Republic of Korea | A | |
| CN107103021B | China | B | |
| JP6974009B2This record | Japan | B2 | |
| EP3211547B1 | European Patent Office (EPO) | B1 | |
| US11301422B2 | United States of America | B2 | |
| US2022300456A1 | United States of America | A1 | |
| US12197388B2 | United States of America | B2 | |
| KR102847544B1 | Republic of Korea | B1 |
10 legal events, as the office reported them to INPADOC
Over the term
Point at a mark for the eventEvents
| Event | Code | |
|---|---|---|
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| Receipt of annual feesJAPANESE INTERMEDIATE CODE: R250R250 | R250 | |
| First payment of annual fees (during grant procedure)JAPANESE INTERMEDIATE CODE: A61A61 | A61 | |
| Certificate of patent or registration of utility modelJAPANESE INTERMEDIATE CODE: R150R150 | R150 | |
| Written decision to grant a patent or to grant a registration (utility model)JAPANESE INTERMEDIATE CODE: A01A01 | A01 | |
| Decision of grant or rejection writtenTRDD | TRDD | |
| Request for written amendment filedJAPANESE INTERMEDIATE CODE: A523A521 | A521 | |
| Notification of reasons for refusalJAPANESE INTERMEDIATE CODE: A131A131 | A131 | |
| Report on retrievalJAPANESE INTERMEDIATE CODE: A971007A977 | A977 | |
| Written request for application examinationJAPANESE INTERMEDIATE CODE: A621A621 | A621 |
Numbers
- Publication
- 6974009
- Publication, DOCDB
- 6974009
- Publication, EPODOC
- JP6974009B
- Application
- 20161
- Application, DOCDB
- 2017020161
- Application, EPODOC
- JP20170020161
Titles2
- Japanese
- キーバリューシステムにおける格納装置への高速キャッシュアクセス方法及びシステム
- English
- High-speed cache access method and system to storage device in key-value system
Classification
- CPC, 9
- G06F16/152
- G06F16/188
- G06F16/172
- G06F16/17
- G06F16/13
- G06F16/84
- G06F16/162
- G06F12/0882
- G06F2212/603
- IPC, 3
- G06F16 172
- G06F3 06
- G06F16 28
