System and method for capturing facial and body motion

The present invention relates to 3D graphics and animation, and more particularly to a motion capture system that allows both facial and physical movements to be captured simultaneously in a volume that can accommodate multiple actors.

This application claims priority in accordance with 35 USC 120 as a partial continuation of US Patent Application No. 10 / 427,114, filed May 1, 2003, entitled "System and Method for Capturing Facial and Body Motion." To do.

Motion capture systems are used to capture the motion of a real object and map it onto a computer-generated object. Such systems are often used in the production of movies and video games to create digital representations of people, which are used as source data for creating computer graphics (CG) animations. .. In a typical system, the actor wears clothing with markers attached in various positions (eg, with small reflective markers attached to the torso and limbs), and while the markers are illuminated, the digital camera , Record the movement of the actor from various angles. The system then analyzes the image to determine the position (eg, as spatial coordinates) and orientation of the markers on the actor's clothing within each frame. By tracking the position of the marker, the system creates a spatial representation of the marker over time, constructing a digital representation of the actor in motion. The movement can then be applied to the digital model, and then the digital model can be textured and rendered to generate a complete CG representation of the actor and / or acting. This technique has been used by special effects companies and has produced incredibly realistic animations in many popular movies.

Motion capture systems are also used to track the movements of actor's facial features to create representations of the actor's facial movements and facial expressions (eg, laughter, crying, smiles, etc.). Similar to body motion capture, a marker is attached to the actor's face and a camera records the actor's facial expression. Face markers are usually much smaller than the corresponding body markers, as facial movement involves smaller muscles compared to the larger muscles involved in body movement, and cameras are usually commonly used for body motion capture. It has a higher resolution than the camera. The camera is usually aligned in a common plane with the actor's physical movements, which is restricted to keep the camera in focus on the actor's face. A face motion capture system can be incorporated into a helmet or other device physically attached to the actor to uniformly illuminate the face markers and minimize the degree of relative movement between the camera and the face. .. For this, facial and body movements are usually captured in separate steps. The captured facial movement data is then later combined with the captured body movement data as part of the subsequent animation process.

The advantage of motion capture systems over traditional animation techniques such as key framing is the ability to visualize in real time. The production team can examine the spatial representation of the actor's movements in real time or near real time, allowing the actor to modify his physical performance to capture optimal data. In addition, motion capture systems detect subtle nuances of physical motion that cannot be easily reproduced using other animation techniques, thereby generating data that accurately reflects natural motion. As a result, animations created using source material collected using a motion capture system show a truly squeaky look.

Despite these advantages of motion capture systems, capturing facial and body movements separately often results in non-trivial animation data. Facial and body movements are intricately linked, thereby enhancing facial expressions by the corresponding body movements. For example, an actor uses constant body movements (ie, body language), such as using arm movements when speaking with excitement or shrugging when in a bad mood, to convey emotions and the corresponding facial expressions. May emphasize facial expressions. This link between facial and physical movements is lost when the movements are captured separately, making it difficult to synchronize these separately captured movements with each other. When facial and physical movements are combined, the resulting animation often looks noticeably strange. Separation of facial and body movements represents a significant drawback of traditional motion capture systems, as the purpose of motion capture is to enable the creation of increasingly realistic animations.

Another drawback of traditional motion capture systems is that the actor's motion data can be obstructed by other objects such as props and other actors. Specifically, when a part of the body marker or the face marker is blocked from the field of view of the digital camera, the data regarding the body part or the face part is not collected. As a result, occlusions or holes occur in the operation data. Occlusions can later be filled using traditional computer graphics techniques during post-production, but the filling data lacks the quality of the actual operating data, resulting in the audience There are poor animations that can be identified. To avoid this problem, conventional motion capture systems limit the number of objects that can be captured at one time to, for example, one actor. Since the quality of an actor's performance often depends on the interaction with other actors and objects, this also tends to impair the realism of the motion data. Moreover, it is difficult to combine these separate performances together in a natural-looking way.

Yet another drawback of traditional motion capture systems is that audio is not recorded at the same time as motion capture. In animation, it is common to record the audio track first and then animate the character to match the audio track. During face motion capture, the actor will move his lips in sync with the recorded audio track. This inevitably further reduces the visual quality of the motion data, as it is difficult for the actor to perfectly synchronize the facial motion with the audio track. In addition, body movements often affect the way speech is spoken, and by incorporating body movements and facial movements separately, the audio tracks are synchronized to produce a cohesive end product. Difficulty increases.

<p> Therefore, it is desirable to provide a motion capture system that overcomes the above and other drawbacks of the prior art. More specifically, it is desirable to provide a motion capture system that enables simultaneous capture of both body and face movements within a volume that can accommodate multiple actors. It is also desirable to provide a motion capture system that enables audio recording at the same time as body motion capture and face motion capture.</p>

<p> According to the teachings of the present invention, there are provided motion capture systems and methods that allow both body and facial movements to be captured simultaneously in a volume that can accommodate a plurality of actors. This motion capture system and method also enables audio recording at the same time as body motion capture and face motion capture.</p><p> More specifically, the motion capture system and method is adapted to include at least one actor having a body marker that defines multiple body points and a face marker that defines multiple face points. Includes capture volume. The motion capture volume can have any desired geometry such as an ellipse, a circle, a rectangle, a polygon, and the like. Multiple motion cameras are placed around the motion capture volume. The motion cameras are arranged so that all laterally exposed surfaces of the actor operating in the motion capture volume are within the field of view of at least one of the multiple motion cameras at almost all times. A motion capture processor is combined with multiple motion cameras to generate a digital model that reflects the actor's combined body / face movements. At least one microphone can be oriented to pick up audio from the motion capture volume.</p><p> In one embodiment of the invention, the motion capture volume further includes a rectangular area subdivided into sections. The plurality of motion cameras further include a first subgroup of cameras oriented in each section. The plurality of motion cameras may further include a second subgroup of cameras that are oriented into each section and spatially separated from the first subgroup of cameras. The cameras in the first subgroup are oriented with respect to the cameras in the second subgroup so that their fields of view substantially overlap. The cameras in the first and second subgroups can each further include at least three cameras. The plurality of motion cameras are respectively arranged in the corners of the motion capture volume, and can further include a third subgroup of cameras that are generally oriented at the center of the motion capture volume.</p><p> By considering the detailed description of the preferred embodiments below, one of ordinary skill in the art will gain a more complete understanding of the systems and methods of incorporating physical and facial movements, as well as their additional benefits and objectives. References are made to the accompanying drawings, which are briefly described first.</p>

As further described below, the present invention meets the need for a motion capture system that allows both facial and physical movements to be captured simultaneously in a volume that can accommodate multiple actors. Further, the present invention also meets the need for a motion capture system that allows audio recording at the same time as body motion capture and face motion capture. In the detailed description below, similar element numbers are used to describe similar elements shown in one or more of the drawings.

First, referring to FIG. 1, the block diagram shows a motion capture system 10 according to an embodiment of the present invention. Motion capture system 10 is a plurality of face motion cameras 14₁~14_NAnd multiple body motion cameras 16₁~16_NIncludes a motion capture processor 12 adapted to communicate with. The motion capture processor 12 may further include a programmable computer having a data storage device 20 adapted to allow storage of associated data files. One or more computer workstations using the network 18₁~18_NCan be combined with motion capture processor 12 to allow multiple graphic artists to work with stored data files in the process of creating computer graphics animations. Face motion camera 14₁~14_NAnd body motion camera 16₁~16_NIs placed on the motion capture volume (described below) to capture the combined action of one or more actors acting within the motion capture volume.

Each actor's face and body are face motion camera 14 while the actor is acting in a motion capture volume₁~14_NAnd body motion camera 16₁~16_NMarked with a marker detected by. The marker may be a reflective element or an illuminated element. Specifically, the body of each actor can be marked by a plurality of reflective markers arranged at various body positions including the head, legs, and torso. The actor can wear a bodysuit made of non-reflective material to which the marker is attached. The actor's face is also marked with multiple markers. Face markers are generally smaller than body markers and more face markers are used than body markers. It is expected that a large number of face markers (eg, more than 100) will be used to capture facial movements at sufficient resolution. In one exemplary implementation, 152 smaller face markers and 64 larger body markers are attached to the actor. Body markers can have a width or diameter in the range of 5 to 9 millimeters, and face markers can have a width or diameter in the range of 2 to 4 millimeters.

To ensure the alignment of the face markers, it is possible to form a mask for each actor's face with holes in the appropriate positions corresponding to the desired marker positions. A mask can be placed on the actor's face and the position of the holes can be marked directly on the face using a suitable pen. The face marker can then be attached to the actor's face at the marked position. Face markers can be applied to the actor's face using suitable materials well known in the theater field, such as make-up glue. In this way, motion capture works over a very long period of time (eg months) can obtain fairly consistent motion data about the actor, even if the markers are attached and removed daily.

Motion capture processor 12 is a face motion camera 14₁~14_NAnd body motion camera 16₁~16_NProcesses the 2D image received from and generates a 3D digital representation of the captured behavior. Specifically, the motion capture processor 12 receives two-dimensional data from each camera as part of the image capture process, and stores the data in the data storage device 20 in the form of a plurality of data files. Then, as part of the image processing process, the 2D data files are decomposed into a single set of 3D coordinates linked to each other in the form of trajectory files that represent the motion of individual markers. The image processing process uses images from one or more cameras to determine the position of each marker. For example, markers may only be visible to a subset of cameras due to occlusion by the actor's facial features or body parts in the motion capture volume. In that case, image processing uses images from other cameras that do not block the view of the marker to determine the position of the marker in space.

By finding the position of the marker using images from multiple cameras, the image processing process evaluates the image information from multiple angles and uses the triangulation process to find the spatial position. The locus file is then subjected to motion calculations to generate a digital representation that reflects the body and face movements that correspond to the actor's performance. Using spatial information over time, the calculation finds its progress as each marker moves through space. An appropriate data management process can be used to control the storage of a large number of files related to the entire process in data storage 20 and the retrieval from data storage 20. Motion capture processor 12 and workstation 18₁~18_NCan use commercially available software packages, such as those available from Vicon Motion Systems or Motion Analysis Corp., to perform the above and other data processing functions.

The motion capture system 10 further includes the ability to record audio in addition to motion. Multiple microphones 24₁~24_NCan be placed around the motion capture volume to pick up audio (eg, spoken dialogue) during actor performance. Motion capture processor 12 directly or through audio interface 22 microphone 24₁~24_NCan be combined with. Microphone 24₁~24_NCan be fixed in place, or movable on the boom to follow movements, or the actor can use the microphone 24₁~24_NCarrying a microphone 24₁~24_NCan communicate wirelessly with the motion capture processor 12 or audio interface 22. The motion capture processor 12 receives the recorded audio in the form of a digital file, along with a time track or other data that allows synchronization with movement, and stores it on the data storage device 20.

Figures 2 and 3 show an exemplary motion capture volume 30 surrounded by multiple motion capture cameras. The motion capture volume 30 includes the peripheral edge 32. The motion capture volume 30 is shown as a rectangular area subdivided by grid lines. It should be understood that the motion capture volume 30 actually includes a three-dimensional space with a grid that defines the floor for the motion capture volume. The movement is captured in the three-dimensional space above the floor. In a preferred embodiment of the invention, the motion capture volume 30 comprises a floor area approximately 10 feet square and approximately 6 feet high above the floor. Advantageously, motion capture volumes of other sizes and shapes such as ellipses, circles, rectangles, polygons, etc. can also be used to suit the specific needs of the production.

FIG. 2 shows a top view of the motion capture volume 30 in which a plurality of motion capture cameras are arranged in a generally circular pattern around the peripheral edge 32. It should be appreciated that individual cameras are graphically represented as triangles with acute angles that represent the orientation of the camera's lenses, and therefore multiple cameras are directed towards the motion capture volume 30 from multiple distinct directions. More specifically, multiple motion capture cameras are multiple body motion cameras 16₁~16₈And multiple face motion cameras 14₁~14_NIncluding further. It should be understood that many are unsigned in view of the large number of facial motion cameras in Figure 2. In this embodiment of the invention, there are more face motion cameras than body motion cameras. Body motion camera 16₁~16₈Approximately two are placed on each side of the motion capture volume 30, and the face motion camera 14₁~14_NApproximately 12 are arranged on each side of the motion capture volume 30. Face motion camera 14₁~14_NAnd body motion camera 16₁~16_NAre nearly identical except that the focusing lens of the face motion camera is chosen to provide a narrower field of view than the body motion camera.

FIG. 3 shows a side view of the motion capture volume 30 in which a plurality of motion capture cameras are arranged on approximately three floors above the floor of the motion capture volume. On the lower floor, there are multiple face motion cameras 14 arranged approximately 8 on each side of the motion capture volume 30.₁~14₃₂including. In one embodiment of the present invention, the lower floor face motion camera 14₁~14₃₂Each of them points slightly upward so that the camera on the opposite side of the motion capture volume 30 is not included in the field of view. A motion capture camera generally includes a light source (eg, an array of light emitting diodes) used to illuminate the motion capture volume 30. Since the light source looks like a bright reflectance that overwhelms the data from the reflection marker to the motion capture camera, it is desirable that the motion capture camera "does not" see the light source of another motion capture camera. On the middle floor, there are multiple body motion cameras 16 arranged approximately two on each side of the motion capture volume 30.₃~16₇including. As discussed above, body motion cameras have a wider field of view than face motion cameras, allowing each camera to contain a larger amount of motion capture volume 30 within its respective field of view.

Upstairs are multiple face motion cameras (eg 14) with approximately 5 arranged on each side of the motion capture volume 30.₃₃~14₅₂)including. In one embodiment of the present invention, the upper floor face motion camera 14₃₃~14₅₂Each of them points slightly downward so that the camera on the opposite side of the motion capture volume 30 is not included in the field of view. As shown on the left side of Figure 2, several face motion cameras focused on the front edge of the motion capture volume 30 (eg 14).₅₃~14₆₀) Is also included in the middle floor. Since the actor's performance is generally directed towards the front edge of the motion capture volume 30, the number of cameras in that area is increased and the amount of data lost to occlusion is reduced. In addition, some face motion cameras focused on the corners of motion capture volume 30 (eg 14)₆₁~14₆₄) Is included in the middle floor. These cameras also play a role in reducing the amount of data lost to occlusion.

In a preferred embodiment of the invention, the body and face motion cameras record the image of the marked actor from a number of different angles, so that almost all aspects of the actor are on at least one camera at all times. On the other hand, it is exposed. More specifically, it is preferred that the placement of the cameras allows almost all sides of the actor to be exposed to at least three cameras at all times. By arranging the cameras at multiple heights, it is possible to model irregular surfaces as the actor moves within the motion capture field 30. Thereby, the motion capture system 10 records the physical movement of the actor at the same time as the facial movement (that is, the facial expression). As discussed above, audio recording can also be performed at the same time as motion capture. FIG. 4 is a top view of the motion capture volume 30 showing an exemplary arrangement of the face motion camera. The motion capture volume 30 is graphically divided into quadrants labeled a, b, c, and d. Face motion cameras are grouped into clusters 36 and 38, with each camera cluster representing multiple cameras. For example, one such camera cluster can include two face motion cameras located downstairs and one face motion camera located upstairs, advantageously the cameras in the cluster. Other arrangements can also be used. The two camera clusters 36, 38 are physically adjacent to each other, but are horizontally displaced from each other by a recognizable distance. The two camera clusters 36 and 38, respectively, focus on the front edge of quadrant d from an angle of approximately 45 °. The first camera cluster 36 has a field of view extending from a part of the front edge of quadrant c to the right edge of the front edge of quadrant d. The second camera cluster 38 has a field of view extending from the left edge of the front edge of quadrant d to a part of the right edge of quadrant d. Therefore, the fields of view of the first and second camera clusters 36, 38 overlap over a considerable length of the front edge of quadrant d. A similar arrangement of camera clusters is included for each of the other outer edges (corresponding to margin 32) in quadrants a, b, c, and d.

FIG. 5 is a top view of the motion capture volume 30 showing an exemplary arrangement of the face motion camera. Similar to FIG. 4, the motion capture volume 30 is graphically divided into quadrants a, b, c, and d. Face motion cameras are grouped into clusters 42, 44, where each camera cluster represents multiple cameras. Similar to the embodiment of FIG. 4, the cluster can include one or more cameras arranged at various heights. In this arrangement, the camera clusters 42 and 44 are arranged at the corners of the motion capture volume 30 and face the motion capture volume. These corner camera clusters 42, 44 record images of actors that are not picked up by other cameras due to occlusion etc. Other similar camera clusters are also located at other corners of the motion capture volume 30.

Have a height and angle of the various camera relative motion capture volume 30, the motion capture volume serves to increase the available data captured from the actors in the volume, to reduce the likelihood of data occlusion. It is also possible to motion capture multiple actors at the same time within the motion capture volume 30. In addition, a large number of different cameras allow the motion capture volume 30 to be significantly larger than in the prior art, which allows for a wider range of motion within the motion capture volume, thus allowing for more complex performances. It becomes. It should be appreciated that a number of alternative arrangements of body and face motion cameras can also be used to advantage. For example, more or fewer separate floors can be used, and the actual height of each camera within each floor can be changed.

In the description of the preferred embodiment of the invention described above, the body and face motion cameras remain in place. In this way, the motion capture processor 12 has a fixed reference point on which the movements of the body and face markers can be measured. The disadvantage of this arrangement is that the size of the motion capture volume 30 is limited. If you want to capture performance movements that require more space (for example, a scene where a character is running over a long distance), you must divide the performance into multiple segments that are motion-captured separately. I have to. In an alternative embodiment of the invention, one part of the camera remains fixed and the other camera moves to follow the movement. Movable cameras can be made movable using computer controlled servomotors or can be manually moved by a human camera operator. The motion capture processor 12 tracks the movement of the camera and removes this movement in subsequent processing of the captured data to generate a three-dimensional digital representation that reflects the physical and facial movements that correspond to the actor's performance.

FIG. 6 is a perspective view of the motion capture volume 30 showing the motion capture data reflecting the two actors 52 and 54 in the motion capture volume. FIG. 6 reflects what the motion capture data looks like to the workstation 18 operator described above with respect to FIG. Similar to FIGS. 2 and 3 (above), FIG. 6 shows the camera 14 located downstairs.₁~14₁₂And the camera 14 placed upstairs₃₃~14₄₀And the camera 14 placed in the corner of the motion capture volume 30₆₀、14₆₂A plurality of face motion cameras including and are further shown. The two actors 52, 54 appear as a group of dots corresponding to reflective markers on the body and face. As illustrated and discussed above, there are far more markers on the actor's face than on the actor's body. The actor's body and facial movements are tracked by the motion capture system 10 in much the same way as described above.

Next, with reference to FIGS. 7 and 8, the motion capture data is shown to be viewed by the workstation 18 operator. Similar to FIG. 6, the motion capture data reflects the two actors 52, 54, with the high density dots reflecting the actor's face and the other dots reflecting the body points. The motion capture data further includes three occlusion regions 62, 64, 66 shown as elliptical shapes. Occlusion regions 62, 64, 66 represent locations where reliable motion data was not captured because light from one camera entered the field of view of the other camera. This light overwhelms the illumination from the reflective markers and is interpreted by the motion capture processor 12 as a body marker or face marker. The image processing process performed by the motion capture processor 12 creates a virtual mask that removes camera illumination by defining the occlusion areas 62, 64, 66 shown in FIGS. 7 and 8. The production company can try to control the actor's performance to physically avoid the movements hidden by the occlusion area. Nevertheless, as shown in Figure 8, some data capture loss is inevitable. In Figure 8, the face of actor 54 is almost completely hidden by physical movement into the occlusion area 64.

FIG. 9 shows an embodiment of a motion capture system that reduces the occlusion problem. Specifically, FIG. 9 shows cameras 84 and 74 physically located at both ends of a motion capture volume (not shown) with respect to each other. Cameras 84, 74 include light sources 88, 78 adapted to illuminate the field of view of the camera, respectively. Cameras 84, 74 further include polarizing filters 86, 76 disposed in front of the camera lens. As will be apparent from the following description, the polarizing filters 86 and 76 are arranged (ie, rotated) so as to be out of phase with each other. The light source 88 emits light polarized by the polarizing filter 86. The polarized light reaches the polarizing filter 76 of the camera 74, but the polarized light is reflected or absorbed by the polarizing filter 76 instead of passing through the camera 74. As a result, the camera 84 "does not see" the illumination from the camera 74, thereby avoiding the formation of occlusion regions and eliminating the need for virtual masking.

Although the above description has referred to the use of optical sensing of physical markers affixed to the body and face to track movement, those skilled in the art can advantageously use alternative methods of tracking movement. I want to be understood. For example, instead of pasting markers, the actor's physical features (eg nose or eye shape) can be used as natural markers to track movement. Such feature-based motion capture systems eliminate the need to attach markers to actors before each performance. In addition, alternative media other than light can be used to detect the corresponding markers. For example, the marker can include an ultrasonic or electromagnetic radiator detected by a corresponding receiver placed around the motion capture volume. In this regard, it should be understood that the cameras described above are merely optical sensors, and advantageously other types of sensors can also be used.

Next, referring to FIG. 10, the block diagram shows a motion capture system 100 according to an alternative embodiment of the present invention. The motion capture system 100 has a considerably improved data capacity as compared with the above-described embodiment, and is suitable for capturing a considerably large amount of data related to the expanded motion capture volume. The motion capture system 100 includes three separate networks joined together by a master server 110 that acts as a repository for the collected data. The networks are data network 120, artist network 130, and reconstruction render network. network) 140 is included. The master server 110 provides central control and data storage for the motion capture system 100. The data network 120 communicates the two-dimensional (2D) data captured during the performance to the master server 110. The artist network 130 and the rebuild render network 140 can then access these same 2D data files from the master server 110. The master server 110 further includes a memory 112 system suitable for storing large amounts of data.

The data network 120 provides an interface with a motion capture camera to provide initial data processing for captured motion data, which is then supplied to the master server 100 and stored in memory 112. More specifically, multiple motion captures in which the data network 120 is placed against the motion capture volume (described below) to capture the combined motion of one or more actors acting within the motion capture volume. Camera 122₁~122_NCombined with. The data network 120 also includes multiple microphones 126, either directly or via the appropriate audio interface 124, to capture audio associated with the performance (eg, dialogue).₁~126_NCan also be combined. One or more user workstations 128 can be combined with the data network 120 to provide operation, control, and monitoring of the functions of the data network. In one embodiment of the invention, the data network 120 includes a plurality of motion capture data processing stations available from Vicon Motion Systems, Motion Analysis Corp, etc., and a plurality of slave processing stations that collate the captured data as a 2D file. Can be realized with.

Artist network 130, suitable workstation 132₁~132_NUse to provide a fast infrastructure for multiple data checkers and animators. The data checker accesses the 2D data file from the master server 110 and verifies the acceptability of the data. For example, the data checker reviews the data and verifies that important aspects of acting have been captured. If important aspects of the performance are not captured, such as when some of the data is blocked, the performance can be repeated as needed until the captured data is considered acceptable. Data checkers and associated workstations 132 to facilitate communication with actors and / or scene directors 132₁~132_NCan be placed physically close to the motion capture volume.

The Reconstruction Render Network 140 provides a high-speed data processing computer suitable for performing automatic reconstruction of 2D data files and rendering the 2D data files into three-dimensional (3D) animation files stored by the master server 110. .. 1 or more user workstations 142₁~142_NCan be combined with the reconstructed render network 140 to provide manipulation, control, and monitoring of data network functionality. Animators accessing Artist Network 130 also access 3D animation files while producing the final computer graphics animation.

FIG. 11 shows a top view of the exemplary motion capture volume 150. Similar to the above embodiment, the motion capture volume 150 is generally a rectangular area subdivided by grid lines. In this embodiment, the motion capture volume 150 is intended to represent a significantly larger space and can be further subdivided into four sections or quadrants (A, B, C, D). Each section has a size approximately equal to the size of the motion capture volume 30 described above, thus the motion capture volume 150 has four times the surface area of the embodiment described above. An additional section E is centered in space and partially overlaps with each of the other sections. The grid lines further include numerical coordinates (1-5) along the vertical axis and alphabetic coordinates (AE) along the horizontal axis. In this way, a specific position on the motion capture volume can be defined in alphanumeric coordinates such as region 4A. Such designation allows management of the motion capture volume 150 in that it gives the actor instructions as to where to perform the actor's performance and / or where to place the props. For the benefit of actors and / or scene directors, grid lines and alphanumeric coordinates can be physically marked on the floor of motion capture volume 150. It should be understood that these grid lines and alphanumeric coordinates are not included in the 2D data file.

In a preferred embodiment of the invention, each section A through E has a square shape with dimensions of 10 feet square and has a total area of 400 square feet, or approximately four times the motion capture volume of the above embodiment. .. It should be appreciated that other shapes and sizes can be used for the motion capture volume 150 to advantage.

Next, referring to FIGS. 12A to 12C, the motion capture camera 122₁~122_NAn exemplary arrangement of is shown for the peripheral area around the motion capture volume 150. The peripheral area provides a scaffolding arrangement to support cameras, lights, and other equipment, area 152.₁~152₄Shown as. Motion capture camera 122₁~122_NHowever, at various camera heights and angles, the area around the motion capture volume 150 is 152.₁~152₄They are generally evenly distributed in each of them. In addition, motion capture camera 122₁~122_NAre each oriented to focus on individual sections of the motion capture volume 150 sections rather than the entire motion capture volume. In an exemplary embodiment of the invention, there are a total of 200 motion capture cameras, and a group of 40 individual cameras is dedicated to each of the five sections A through E of the motion capture volume 150. ..

Specifically, the motion capture camera 122 depending on the distance from the motion capture volume and the height of the motion capture volume 150 from the floor.₁~122_NArrangement can be defined. Figure 12A shows the first group of motion capture cameras 122 oriented at the furthest distance from the motion capture volume 150 and generally at the lowest height.₁~122₈₀An exemplary arrangement of is shown. Illustrative area 152₁With reference to (the other areas are about the same), row 1 172 is located radially outward with respect to the motion capture volume 150 at the highest height (eg 6 feet) from the floor and row 2. The 174 is placed at a slightly lower height (eg 4 feet) and the 3rd row 176 is placed at the lowest height (eg 1 foot) radially inward with respect to the 1st and 2nd rows. There are three lines of cameras. In an exemplary embodiment, there are a total of 80 motion capture cameras in this first group.

FIG. 12B shows the motion capture camera 122 of the second group, which is closer to the motion capture volume 150 than the first group and is oriented at a higher height than the first group.₈₁~122₁₆₀Indicates the arrangement of. Illustrative area 152₁With reference to (the other areas are about the same), row 1 182 is located radially outward with respect to the motion capture volume at the highest height (eg 14 feet) from the floor, row 2 184. Is placed at a slightly lower height (eg 11 feet) and the third row 186 is placed at the lowest height (eg 9 feet) radially inward with respect to the first and second rows. , There are 3 lines of cameras. In an exemplary embodiment, there are a total of 80 motion capture cameras in this second group.

Figure 12C shows the motion capture camera 122 of the third group, which is closer to the motion capture volume 150 than the second group and is oriented at a higher height than the second group.₁₆₁~122₂₀₀Indicates the arrangement of. Illustrative area 152₁With reference to (the other areas are about the same), row 1 192 is located radially outward with respect to the motion capture volume at the highest height (eg 21 feet) from the floor, row 2 194. Is placed at a slightly lower height (eg 18 feet) and the third row 196 is placed at the lowest height (eg 17 feet) radially inward with respect to the first and second rows. , There are 3 lines of cameras. In an exemplary embodiment, there are a total of 40 motion capture cameras in this third group. It should be appreciated that other arrangements of motion capture cameras and different numbers of motion capture cameras can also be used to advantage.

The motion capture camera is focused on each section of the motion capture volume 150, as described above with respect to FIG. For each of the sections A to E of the motion capture volume 150, the motion capture camera is focused from each of the four sides to the section. For example, the first group of cameras farthest from the motion capture volume can focus on the section of the motion capture volume closest to it. Conversely, the third group of cameras closest to the motion capture volume can focus on the section of the motion capture volume farthest from it. A camera at one end of each side can focus on the other end section. In a more specific example, the section A of the motion capture volume 150 is set to the peripheral area 152.₁A camera with a low height in rows 1 182 and 3 186 and a peripheral area 152₄1st row 182 and 3rd row 186 low-height cameras and peripheral area 152₃Medium-height cameras in rows 2 184 and 3 186 and peripheral area 152₂Can be covered by a combination of medium height cameras in rows 2 184 and 3 rows 186. Figures 12A and 12B reveal that the motion camera is more concentrated in the center of the peripheral region due to the capture of motion within the central section E.

By giving different angles and heights and focusing more cameras on each section of the motion capture volume 150, it is much more likely to capture the entire performance while minimizing unwanted occlusion. Given the large number of cameras used in this arrangement, it is advantageous to place a light shield around each camera to reduce the detection of external light from another camera located on the opposite side of the motion capture volume. is there. In this exemplary embodiment of the invention, the same camera can be used to capture both facial and physical movements at the same time, thus eliminating the need for separate body and facial motion cameras. Markers of different sizes can be used for actors to distinguish between facial and physical movements, and given a large motion capture volume, large markers are commonly used to ensure data capture. Will be done. For example, a 9 mm marker can be used for the body and a 6 mm marker can be used for the face.

Having thus described preferred embodiments of systems and methods that incorporate body and facial movements, it should be apparent to those skilled in the art that some of the advantages of the present invention have been achieved. It should also be appreciated that various modifications, adaptations, and alternative embodiments of the invention can be made within the scope and spirit of the invention. The present invention is further defined by the appended claims.

<figref num="1">It is a block diagram which shows the motion capture system by one Embodiment of this invention.</figref><figref num="2">It is a top view of the motion capture volume including a plurality of motion capture cameras arranged in the peripheral part of the motion capture volume.</figref><figref num="3">It is a side view of the motion capture volume including a plurality of motion capture cameras arranged in the peripheral part of the motion capture volume.</figref><figref num="4">It is a top view of the motion capture volume which shows the exemplary arrangement of the face motion camera with respect to the quadrant of the motion capture volume.</figref><figref num="5">It is a top view of the motion capture volume which shows the exemplary arrangement of the face motion camera with respect to the corner of the motion capture volume.</figref><figref num="6">It is a perspective view of the motion capture volume which shows the motion capture data which reflects two actors in a motion capture volume.</figref><figref num="7">It is a figure which reflects the two actors in a motion capture volume, and shows the motion capture data which shows the occlusion area of data.</figref><figref num="8">It is a figure which shows the motion capture data similar to FIG. 7 in which one of two actors is blocked by an occlusion area.</figref><figref num="9">It is a block diagram which shows the alternative embodiment of the motion capture camera used in the motion capture system.</figref><figref num="10">It is a block diagram which shows the motion capture system by another embodiment of this invention.</figref><figref num="11">It is the top view of the extended motion capture volume which defines a plurality of performance areas.</figref><figref num="12A">It is a top view of the extended motion capture volume of FIG. 11 which shows the exemplary arrangement of a motion capture camera.</figref><figref num="12B">It is a top view of the extended motion capture volume of FIG. 11 which shows the exemplary arrangement of a motion capture camera.</figref><figref num="12C">It is a top view of the extended motion capture volume of FIG. 11 which shows the exemplary arrangement of a motion capture camera.</figref>