Communication management apparatus, image communication system, communication management method, and recording medium

CROSS-REFERENCE TO RELATED APPLICATIONS

0001This patent application is based on and claims priority pursuant to 35 U.S.C. § 119(a) to Japanese Patent Application No. 2020-166189, filed on Sep. 30, 2020, in the Japan Patent Office, the entire disclosure of which is hereby incorporated by reference herein.

BACKGROUND

Technical Field

0002The present disclosure relates to a communication management apparatus, an image communication system, a communication management method, and a recording medium.

Description of the Related Art

0003An image capturing device capable of capturing an omnidirectional scene using a plurality of wide-angle lenses or fish-eye lenses is known. Also known is a system capable of distributing image data of an image captured using such an image capturing device in real time such that a distribution site where the image capturing device is disposed is viewable at a different site in real time.

0004For example, there is a system in which images captured by a plurality of cameras at a remote location are displayed on a terminal such that a user can grasp the situation at the remote location.

0005In the existing method, however, a user who desires to view a scene at a predetermined position in an area where a plurality of image capturing devices are disposed, individually operates a plurality of image capturing devices in an area including the predetermined position.

SUMMARY

0006Example embodiments include a communication management apparatus for communicating with a communication terminal that displays images captured by a plurality of image capturing devices. The communication management apparatus includes circuitry that receives, from the communication terminal, predetermined-position information indicating a predetermined position in an area where the plurality of image capturing devices are provided and number-of-division information indicating the number of divisions for dividing the area from the predetermined position. The circuitry further transmits, to the communication terminal, first predetermined-area information indicating a predetermined area including the predetermined position in an image captured by a first image capturing device among the plurality of image capturing devices, and second predetermined-area information indicating a predetermined area including the predetermined position in an image captured by a second image capturing device among the plurality of image capturing devices. The first image capturing device is arranged at a shortest distance from the predetermined position corresponding to the predetermined-position information. the second image capturing device is arranged such that an angle between the second image capturing device and the first image capturing device is closest to an angle obtained by dividing the area in accordance with the number of divisions indicated by the number-of-division information.

0007Example embodiments include an image communication system including the above-described communication management apparatus, and a communication terminal configured to display images captured by a plurality of image capturing devices.

0008Example embodiments include a communication management method executed by a communication management apparatus, the communication management apparatus is communicable with a communication terminal that displays images captured by a plurality of image capturing devices. The communication management method includes: receiving, from the communication terminal, predetermined-position information indicating a predetermined position in an area where the plurality of image capturing devices are provided and number-of-division information indicating the number of divisions for dividing the area from the predetermined position; and transmitting, to the communication terminal, first predetermined-area information indicating a predetermined area including the predetermined position in an image captured by a first image capturing device among the plurality of image capturing devices, and second predetermined-area information indicating a predetermined area including the predetermined position in an image captured by a second image capturing device among the plurality of image capturing devices. The first image capturing device is arranged at a shortest distance from the predetermined position corresponding to the predetermined-position information. The second image capturing device is arranged such that an angle between the second image capturing device and the first image capturing device is closest to an angle obtained by dividing the area in accordance with the number of divisions indicated by the number-of-division information.

0009Example embodiments include a non-transitory recording medium which, when executed by one or more processors, cause a communication management method.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

A more complete appreciation of the disclosure and many of the attendant advantages and features thereof can be readily obtained and understood from the following detailed description with reference to the accompanying drawings, wherein:

<figref idref="DRAWINGS">FIGS. 1A, 1B, and 1C</figref> are a side view, a front view, and a plan view of an image capturing device according to an embodiment of the present disclosure, respectively;

<figref idref="DRAWINGS">FIG. 2</figref> is an illustration for explaining how a user uses the image capturing device, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIGS. 3A, 3B, and 3C</figref> are views illustrating a hemispherical image (front side) captured by the image capturing device, a hemispherical image (back side) captured by the image capturing device, and an image in equirectangular projection, respectively, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 4A</figref> is a conceptual diagram illustrating how a surface of a sphere is covered with the equirectangular projection image according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 4B</figref> is a view illustrating a spherical image according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 5</figref> is a view illustrating positions of a virtual camera and a predetermined area in a case in which the spherical image is of a three-dimensional solid sphere according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 6A</figref> is a perspective view of the virtual camera and the predetermined area illustrated in <figref idref="DRAWINGS">FIG. 5</figref> according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 6B</figref> is a view illustrating an image of the predetermined area displayed on a display according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 7</figref> is a view illustrating a relationship between predetermined-area information and the image of the predetermined area according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 8</figref> is a view illustrating a point in a three-dimensional Euclidean space according to spherical coordinates, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 9</figref> is a diagram illustrating the general arrangement of an image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 10</figref> is a view illustrating an overview of a distribution site in the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 11</figref> is a diagram illustrating an example hardware configuration of the image capturing device according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 12</figref> is a diagram illustrating an example hardware configuration of a distribution terminal, a communication management apparatus, and a communication terminal according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 13</figref> is a diagram illustrating an example functional configuration of the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 14</figref> is a diagram illustrating an example functional configuration of the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 15A</figref> is a conceptual diagram illustrating an example image capturing device management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 15B</figref> is a conceptual diagram illustrating an example image type management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 16A</figref> is a conceptual diagram illustrating an example session management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 16B</figref> is a conceptual diagram illustrating an example image type management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 17A</figref> is a conceptual diagram illustrating an example predetermined-area management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 17B</figref> is a conceptual diagram illustrating an example arrangement information management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 18</figref> is a conceptual diagram illustrating an example distribution-site management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 19A</figref> is a conceptual diagram illustrating an example weighting coefficient management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 19B</figref> is a conceptual diagram illustrating an example specific weighting coefficient management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 20A</figref> is a conceptual diagram illustrating an example image type management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 20B</figref> is a conceptual diagram illustrating an example predetermined-area management table according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 21</figref> is a view schematically describing an example relationship between a point of interest and image capturing devices when the number of divisions is two, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 22</figref> is a view schematically describing an example selection process based on an angle between image capturing devices when the number of divisions is two, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 23</figref> is a view schematically describing an example relationship between a point of interest and image capturing devices when the number of divisions is three, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 24</figref> is a view schematically describing an example selection process based on an angle between image capturing devices when the number of divisions is three, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 25</figref> is a view schematically describing an example selection process based on an angle between image capturing devices when the number of divisions is three, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 26</figref> is a view schematically describing an example relationship between a point of interest and image capturing devices when the number of divisions is four, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 27</figref> is a view schematically describing an example relationship between a point of interest and image capturing devices in irregular positions, according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 28</figref> is a sequence diagram illustrating an example process for participating in a specific communication session in the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 29</figref> is a view illustrating an example screen for selecting a communication session according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 30</figref> is a sequence diagram illustrating an example process for managing image type information in the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 31</figref> is a sequence diagram illustrating an example process for communicating captured image data and audio data in the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 32</figref> is a sequence diagram illustrating an example process for displaying a point of interest in the image communication system according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 33</figref> is a flowchart illustrating an example process for selecting an image capturing device according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 34</figref> is a view illustrating an example site display screen displayed on the communication terminal according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 35</figref> is a flowchart illustrating an example process for selecting a supplementary image capturing device according to an embodiment of the present disclosure;

<figref idref="DRAWINGS">FIG. 36</figref> is a view illustrating an example display screen on which a predetermined-area image is displayed, according to an embodiment of the present disclosure; and

<figref idref="DRAWINGS">FIG. 37</figref> is a sequence diagram illustrating a modification of the process for displaying a point of interest in the image communication system.

0055The accompanying drawings are intended to depict embodiments of the present invention and should not be interpreted to limit the scope thereof. The accompanying drawings are not to be considered as drawn to scale unless explicitly noted. Also, identical or similar reference numerals designate identical or similar components throughout the several views.

DETAILED DESCRIPTION

0056In describing embodiments illustrated in the drawings, specific terminology is employed for the sake of clarity. However, the disclosure of this specification is not intended to be limited to the specific terminology so selected and it is to be understood that each specific element includes all technical equivalents that have a similar function, operate in a similar manner, and achieve a similar result.

0057Referring now to the drawings, embodiments of the present disclosure are described below. As used herein, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise.

0058Hereinafter, one or more embodiments of the present disclosure will be described with reference to the drawings. In the drawings, any redundant descriptions thereof will be omitted.

Method for Generating Spherical Image

0059A method for generating a spherical image according to one or more embodiments will be described with reference to <figref idref="DRAWINGS">FIGS. 1A to 8</figref>.

0060First, the external appearance of an image capturing device <b>10</b> will be described with reference to <figref idref="DRAWINGS">FIGS. 1A to 1C</figref>. The image capturing device <b>10</b> is a digital camera for capturing images from which a 360-degree spherical image is generated. <figref idref="DRAWINGS">FIGS. 1A, 1B, and 1C</figref> are a side view, a front view, and a plan view of the image capturing device <b>10</b>, respectively.

0061As illustrated in <figref idref="DRAWINGS">FIG. 1A</figref>, the image capturing device <b>10</b> has a size such that a person can hold the image capturing device <b>10</b> with one hand. As illustrated in <figref idref="DRAWINGS">FIGS. 1A, 1B, and 1C</figref>, the image capturing device <b>10</b> includes an imaging element <b>103</b><i>a </i>and an imaging element <b>103</b><i>b </i>in an upper portion thereof such that the imaging element <b>103</b><i>a </i>is disposed on the front side and the imaging element <b>103</b><i>b </i>is disposed on the back side. The imaging elements (image sensors) <b>103</b><i>a </i>and <b>103</b><i>b </i>are used together with an optical member (e.g., lenses <b>102</b><i>a </i>and <b>102</b><i>b </i>described below) capable of capturing a hemispherical image (with an angle of view of 180 degrees or more). As illustrated in <figref idref="DRAWINGS">FIG. 1B</figref>, the image capturing device <b>10</b> further includes an operation device <b>115</b> such as a shutter button on the back surface of the image capturing device <b>10</b>.

0062Next, a situation in which the image capturing device <b>10</b> is used will be described with reference to <figref idref="DRAWINGS">FIG. 2</figref>. <figref idref="DRAWINGS">FIG. 2</figref> is an illustration for explaining how a user uses the image capturing device <b>10</b>. As illustrated in <figref idref="DRAWINGS">FIG. 2</figref>, the image capturing device <b>10</b> is used to, for example, capture an image of objects around the user when the user holds the image capturing device <b>10</b> with one hand. The imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>illustrated in <figref idref="DRAWINGS">FIGS. 1A to 1C</figref> capture the objects surrounding the user to obtain two hemispherical images.

0063Next, an overview of a process for creating a spherical image from images captured by the image capturing device <b>10</b> will be described with reference to <figref idref="DRAWINGS">FIGS. 3A to 4B</figref>. <figref idref="DRAWINGS">FIG. 3A</figref> is a view illustrating a hemispherical image (front side) captured by the image capturing device <b>10</b>, <figref idref="DRAWINGS">FIG. 3B</figref> is a view illustrating a hemispherical image (back side) captured by the image capturing device <b>10</b>, and <figref idref="DRAWINGS">FIG. 3C</figref> is a view illustrating an image in equirectangular projection (hereinafter referred to as “equirectangular projection image”). <figref idref="DRAWINGS">FIG. 4A</figref> is a conceptual diagram illustrating how a surface of a sphere is covered with the equirectangular projection image, and <figref idref="DRAWINGS">FIG. 4B</figref> is a view illustrating a spherical image.

0064As illustrated in <figref idref="DRAWINGS">FIG. 3A</figref>, an image obtained by the imaging element <b>103</b><i>a </i>is a curved hemispherical image (front side) captured through the lens <b>102</b><i>a </i>described below. Also, as illustrated in <figref idref="DRAWINGS">FIG. 3B</figref>, an image captured by the imaging element <b>103</b><i>b </i>is a curved hemispherical image (back side) captured through the lens <b>102</b><i>b </i>described below. The image capturing device <b>10</b> combines the hemispherical image (front side) and the hemispherical image (back side), which is flipped by 180 degrees, to create an equirectangular projection image EC illustrated in <figref idref="DRAWINGS">FIG. 3C</figref>.

0065Then, as illustrated in <figref idref="DRAWINGS">FIG. 4A</figref>, the image capturing device <b>10</b> uses Open Graphics Library for Embedded Systems (OpenGL ES) to map the equirectangular projection image EC onto the surface of the sphere so as to cover the surface of the sphere to create a spherical image (spherical panoramic image) CE illustrated in <figref idref="DRAWINGS">FIG. 4B</figref>. In other words, the spherical image CE is represented such that the equirectangular projection image EC corresponds to a surface facing the center of the sphere. OpenGL ES is a graphics library used for visualizing two-dimensional (2D) and three-dimensional (3D) data. The spherical image CE may be either a still image or a moving image.

0066As described above, the spherical image CE is an image mapped onto a sphere surface so as to cover the sphere surface and may look strange to the human eye. The image capturing device <b>10</b> displays the spherical image CE such that a predetermined area T included in the spherical image CE is represented as a flat image having fewer curves. Thus, it is less likely that a person viewing the spherical image CE feels strange. An image of the predetermined area T is hereinafter referred to as “predetermined-area image”. The display of the predetermined-area image will be described with reference to <figref idref="DRAWINGS">FIGS. 5 to 8</figref>.

0067<figref idref="DRAWINGS">FIG. 5</figref> is a view illustrating positions of a virtual camera IC and the predetermined area T in a case in which the spherical image CE is of a three-dimensional solid sphere. The virtual camera IC is at a position of a point of view of a user who is viewing the spherical image CE displayed on the surface of the three-dimensional solid sphere. <figref idref="DRAWINGS">FIG. 6A</figref> is a perspective view of the virtual camera IC and the predetermined area T illustrated in <figref idref="DRAWINGS">FIG. 5</figref>, and <figref idref="DRAWINGS">FIG. 6B</figref> is a view illustrating an image of the predetermined area T displayed on a display. In <figref idref="DRAWINGS">FIG. 6A</figref>, the spherical image CE illustrated in <figref idref="DRAWINGS">FIG. 5</figref> is represented by a three-dimensional solid sphere CS. Assuming that the spherical image CE generated in the way described above is represented by the solid sphere CS, the virtual camera IC is located inside the spherical image CE, as illustrated in <figref idref="DRAWINGS">FIG. 5</figref>. The predetermined area T in the spherical image CE is an imaging area of the virtual camera IC and is specified by predetermined-area information indicating an imaging direction and an angle of view of the virtual camera IC in a three-dimensional virtual space including the spherical image CE. Zooming of the predetermined area T may be represented by moving the virtual camera IC toward or away from the spherical image CE. A predetermined-area image Q is an image of the predetermined area T in the spherical image CE. Accordingly, the predetermined area T can be specified by an angle of view α and a distance f from the virtual camera IC to the spherical image CE (see <figref idref="DRAWINGS">FIG. 7</figref>).

0068The predetermined-area image Q illustrated in <figref idref="DRAWINGS">FIG. 6A</figref> is displayed on a predetermined display as an image of the imaging area of the virtual camera IC, as illustrated in <figref idref="DRAWINGS">FIG. 6B</figref>. The image illustrated in <figref idref="DRAWINGS">FIG. 6B</figref> is a predetermined-area image represented by predetermined-area information that is set by default. In the following, a description will be given using the imaging direction (ea, aa) and the angle of view (α) of the virtual camera IC. The predetermined area T may be indicated by, instead of the angle of view α and the distance f, the imaging area (X, Y, Z) of the virtual camera IC, which is the predetermined area T.

0069Next, a relationship between the predetermined-area information and the image of the predetermined area T will be described with reference to <figref idref="DRAWINGS">FIG. 7</figref>. <figref idref="DRAWINGS">FIG. 7</figref> is a view illustrating a relationship between the predetermined-area information and the image of the predetermined area T. As illustrated in <figref idref="DRAWINGS">FIG. 7</figref>, “ea” denotes an elevation angle, “aa” denotes an azimuth angle, and “a” denotes an angle of view of the virtual camera IC. The position of the virtual camera IC is changed such that the point of gaze of the virtual camera IC, which is indicated by the imaging direction (ea, aa), matches a central point CP(x, y) of the predetermined area T serving as the imaging area of the virtual camera IC. As illustrated in <figref idref="DRAWINGS">FIG. 7</figref>, the central point CP(x, y) of the predetermined area T, whose diagonal angle of view is represented by the angle of view α of the virtual camera IC and is denoted by α, is used as a parameter (x, y) of the predetermined-area information. The predetermined-area image Q is an image of the predetermined area T in the spherical image CE. The distance from the virtual camera IC to the central point CP(x, y) of the predetermined area T is denoted by “f”. The distance between the center point CP and a given vertex of the predetermined area T is denoted by “L” (<b>2</b>L is a diagonal line). In <figref idref="DRAWINGS">FIG. 7</figref>, a trigonometric function equation typically expressed by Equation (1) below is satisfied.

0000<maths id="MATH-US-00001" num="00001"><math overflow="scroll"><mtable><mtr><mtd><mrow><mrow><mi>L</mi><mo>/</mo><mi>f</mi></mrow><mo>=</mo><mrow><mi>tan</mi><mo>⁡</mo><mrow><mo>(</mo><mrow><mi>α</mi><mo>/</mo><mn>2</mn></mrow><mo>)</mo></mrow></mrow></mrow></mtd><mtd><mrow><mo>(</mo><mn>1</mn><mo>)</mo></mrow></mtd></mtr></mtable></math></maths><img file="US2022103751A1_D0001.tif" />

0070The image capturing device <b>10</b> described above is an example of an image capturing device capable of acquiring a wide-angle view image, and the spherical image CE is an example of the wide-angle view image. The wide-angle view image is typically an image captured using a wide-angle lens, such as a lens capable of capturing an image of a wider range than that the human eye can perceive. Further, the wide-angle view image is typically an image taken with a lens having a focal length of 35 mm or less in terms of 35 mm film.

0071<figref idref="DRAWINGS">FIG. 8</figref> is a view illustrating a point in a three-dimensional Euclidean space according to spherical coordinates. The position coordinates of the central point CP when expressed by the spherical polar coordinate system is defined as (r, θ, ϕ). The coordinates (r, θ, ϕ) represent the radial distance, the polar angle, and the azimuth angle, respectively. The radial distance r is a distance from the origin of a three-dimensional virtual space including the spherical image CE to the central point CP and is thus equal to the distance f. <figref idref="DRAWINGS">FIG. 8</figref> illustrates a relationship among the radial distance r, the polar angle θ, and the azimuth angle ϕ. In the following, a description will be given using the position coordinates (r, θ, ϕ) of the virtual camera IC.

Overview of Image Communication System

0072Next, an overview of an image communication system according to this embodiment will be described with reference to <figref idref="DRAWINGS">FIG. 9</figref>. <figref idref="DRAWINGS">FIG. 9</figref> is a diagram illustrating an example of the general arrangement of an image communication system <b>1</b>. The image communication system <b>1</b> illustrated in <figref idref="DRAWINGS">FIG. 9</figref> is a system in which captured images such as video images distributed from a plurality of distribution sites are displayed at a plurality of viewing sites to provide real-time viewing of wide-range images (e.g., spherical images) obtained by capturing scenes at the distribution sites. In this embodiment, the distribution site is also referred to as “area”.

0073As illustrated in <figref idref="DRAWINGS">FIG. 9</figref>, the image communication system <b>1</b> includes image capturing devices <b>10</b> (image capturing devices <b>10</b>A and <b>10</b>B) and distribution terminals <b>30</b> (distribution terminals <b>30</b>A and <b>30</b>B) located at a plurality of distribution sites (distribution site A (area A) and distribution site B (area B)), a communication management apparatus <b>50</b>, and communication terminals <b>70</b> (communication terminals <b>70</b>C and <b>70</b>D) located at a plurality of viewing sites (viewing sites C and D). The image capturing devices <b>10</b>A and <b>10</b>B are hereinafter referred to collectively as “image capturing devices <b>10</b>” or individually as “image capturing device <b>10</b>” unless distinguished. The distribution terminals <b>30</b>A and <b>30</b>B are hereinafter referred to collectively as “distribution terminals <b>30</b>” or individually as “distribution terminal <b>30</b>” unless distinguished. The communication terminals <b>70</b>C and <b>70</b>D are hereinafter referred to collectively as “communication terminals <b>70</b>” or individually as “communication terminal <b>70</b>” unless distinguished.

0074The distribution terminals <b>30</b>, the communication management apparatus <b>50</b>, and the communication terminals <b>70</b> of the image communication system <b>1</b> can communicate with each other via a communication network <b>100</b>. The communication network <b>100</b> is constructed by the Internet, a mobile communication network, a local area network (LAN), or the like. The communication network <b>100</b> may include, in addition to a wired communication network, a network based on a wireless communication standard such as third generation (3G), fourth generation (4G), fifth generation (5G), Wireless Fidelity (Wi-Fi) (Registered Trademark), Worldwide Interoperability for Microwave Access (WiMAX), or Long Term Evolution (LTE).

0075As described above, the image capturing device <b>10</b> is a special digital camera configured to capture an image of an object or surroundings such as scenery to obtain two hemispherical images, from which a spherical image is generated. The captured image obtained by the image capturing device <b>10</b> may be a moving image or a still image or may include both a moving image and a still image. Further, the captured image may include an image and audio. The distribution terminal <b>30</b> is configured to acquire an image from the image capturing device <b>10</b> via a wired cable such as a Universal Serial Bus (USB) cable and distribute the acquired image to the communication terminal <b>70</b> via the communication management apparatus <b>50</b>. In one example, the image capturing device <b>10</b>A and the distribution terminal <b>30</b>A are located at the same site, namely, the distribution site A. The image capturing device <b>10</b>B and the distribution terminal <b>30</b>B are located at the same site, namely, the distribution site B. The number of distribution sites used is not limited to two, and one distribution site or three or more distribution sites may be used. In addition, the image capturing device <b>10</b> and the distribution terminal <b>30</b> may be connected wirelessly using short-range wireless communication or the like, instead of using a wired cable.

0076The communication management apparatus <b>50</b> controls communication between the distribution terminals <b>30</b> and the communication terminals <b>70</b> and manages types of image data (e.g., general image and special image) to be transmitted and received. In one example, the special image is a spherical image. The communication management apparatus <b>50</b> is arranged in a service company or the like that provides image communication services. The communication management apparatus <b>50</b> has a server function and provides identification information of image data (image data ID), the IP address of an image capturing device, image type information representing the type of the image data, and the like in response to a request from the communication terminal <b>70</b>.

0077The communication management apparatus <b>50</b> may be constructed by a single computer or a plurality of computers that are assigned to divided components (functions or means) as appropriate. All or some of the functions of the communication management apparatus <b>50</b> may be implemented by a server computer existing in a cloud environment or a server computer existing in an on-premise environment.

0078The communication terminal <b>70</b> is a computer such as a personal computer (PC), which is used by a user at each viewing site. The communication terminal <b>70</b> displays an image (a still image and/or a moving image) distributed from the distribution terminal <b>30</b>. The communication terminal <b>70</b> acquires a spherical image, which is an image captured by the image capturing device <b>10</b>, via the communication network <b>100</b>. The communication terminal <b>70</b> has installed therein OpenGL ES and is capable of creating predetermined-area information indicating a partial area of a spherical image sent from the distribution terminal or creating a predetermined-area image from the spherical image. In one example, the communication terminal <b>70</b>C is placed at the viewing site C where a user C<b>1</b> is located, and the communication terminal <b>70</b>D is placed at the viewing site D where a user D<b>1</b> is located.

0079The arrangement of the terminals and devices (i.e., the communication terminals <b>70</b>, the image capturing devices <b>10</b>, and the distribution terminals <b>30</b>) and the users C<b>1</b> and D<b>1</b> illustrated in <figref idref="DRAWINGS">FIG. 9</figref> is an example, and another example may be used. Each of the communication terminals <b>70</b> is not limited to a PC and may be, for example, a tablet terminal, a smartphone, a wearable terminal, a projector (PJ), an Interactive White Board (IWB), which is an electronic whiteboard with mutual communication capability, a telepresence robot, or the like.

0080A distribution site in the image communication system <b>1</b> will now be schematically described with reference to <figref idref="DRAWINGS">FIG. 10</figref>. <figref idref="DRAWINGS">FIG. 10</figref> is a view illustrating an overview of a distribution site in the image communication system <b>1</b>. <figref idref="DRAWINGS">FIG. 10</figref> illustrates the image capturing device <b>10</b>A and the distribution terminal <b>30</b>A at the distribution site A. While <figref idref="DRAWINGS">FIG. 10</figref> illustrates an example of the distribution site A, the same applies to other distribution sites such as the distribution site B.

0081The distribution site A illustrated in <figref idref="DRAWINGS">FIG. 10</figref> is, for example, a space such as an office floor, in which a plurality of image capturing devices <b>10</b>A (image capturing devices <b>10</b>A-<b>1</b> to <b>10</b>A-<b>8</b>) and a distribution terminal <b>30</b>A capable of communicating with the image capturing devices <b>10</b>A are arranged. The distribution site is not limited to the office floor and may be any space for which a user (or viewer) at a viewing site desires to remotely grasp the situation, and examples of the distribution site include a school, a factory, a warehouse, a construction site, a server room, and a store. For example, the image capturing devices <b>10</b>A are arranged at predetermined intervals on desks on the office floor. The plurality of image capturing devices <b>10</b>A are used to capture images of the entire distribution site A. The distribution terminal <b>30</b>A receives captured image data from the image capturing devices <b>10</b>A and distributes the received captured image data to the communication terminals <b>70</b> at viewing sites. The numbers of image capturing devices <b>10</b>A and distribution terminals <b>30</b>A arranged at the distribution site A are not limited to those illustrated in <figref idref="DRAWINGS">FIG. 10</figref>. That is, as illustrated in <figref idref="DRAWINGS">FIG. 9</figref>, one image capturing device <b>10</b>A and one distribution terminal <b>30</b>A may be arranged at the distribution site A. As illustrated in <figref idref="DRAWINGS">FIG. 10</figref>, image capturing devices <b>10</b>A and distribution terminals <b>30</b>A having many-to-one relationship may be arranged. The same applies to the distribution site B.

0082In an existing system in which images captured by a plurality of image capturing devices arranged at a remote site are viewable at viewing sites, a user who desires to view the situation at a specific portion in the distribution site does not know which of the images captured by the image capturing devices to view. In addition, in a case in which users at different viewing sites desire to view different portions in the distribution site, operations on the image capturing devices (e.g., pan-tilt-zoom (PTZ) operations) may conflict with each other. As a result, it may be difficult to provide an exclusive viewing that allows a plurality of users who view the same distribution site to view different portions. Further, even in a case in which spherical images, as described above, are captured by the image capturing devices, a user performs individual operations on a plurality of spherical images to perform a process for displaying an intended portion, and it is difficult for a user to perform an intuitive operation. To address this inconvenience, the image communication system <b>1</b> allows a user to perform an intuitive operation on a display screen such that a plurality of users are able to view, with interest, different portions in the distribution site using a captured image acquired from the same image capturing device.

Hardware Configuration of Image Communication System

0083Next, the hardware configuration of the devices or terminals of the image communication system <b>1</b> according to an embodiment will be described with reference to <figref idref="DRAWINGS">FIGS. 11 and 12</figref>. In the hardware configurations illustrated in <figref idref="DRAWINGS">FIGS. 11 and 12</figref>, certain hardware elements may be added or deleted as appropriate.

Hardware Configuration of Image Capturing Device

0084First, the hardware configuration of the image capturing device <b>10</b> will be described with reference to <figref idref="DRAWINGS">FIG. 11</figref>. <figref idref="DRAWINGS">FIG. 11</figref> is a diagram illustrating an example hardware configuration of the image capturing device <b>10</b>. In the following description, the image capturing device <b>10</b> is a spherical (omnidirectional) image capturing device including two imaging elements. However, the image capturing device <b>10</b> may include any suitable number of imaging elements greater than or equal to two imaging elements. In addition, the image capturing device <b>10</b> is not necessarily a device dedicated to capturing of an omnidirectional image. Alternatively, a typical digital camera, smartphone, or the like may be equipped with an external omnidirectional image capturing unit to implement an image capturing device having substantially the same functions as those of the image capturing device <b>10</b>.

0085As illustrated in <figref idref="DRAWINGS">FIG. 11</figref>, the image capturing device <b>10</b> includes an imaging device <b>101</b>, an image processor <b>104</b>, an imaging controller <b>105</b>, a microphone <b>108</b>, an audio processor <b>109</b>, a central processing unit (CPU) <b>111</b>, a read only memory (ROM) <b>112</b>, a static random access memory (SRAM) <b>113</b>, a dynamic random access memory (DRAM) <b>114</b>, the operation device <b>115</b>, an input/output interface (I/F) <b>116</b>, a short-range communication circuit <b>117</b>, an antenna <b>117</b><i>a </i>of the short-range communication circuit <b>117</b>, an electronic compass <b>118</b>, a gyro sensor <b>119</b>, an acceleration sensor <b>120</b>, and a network I/F <b>121</b>.

0086The imaging device <b>101</b> includes wide-angle lenses (so-called fish-eye lenses) <b>102</b><i>a </i>and <b>102</b><i>b</i>, each having an angle of view greater than or equal to 180 degrees so as to form a hemispherical image. The imaging device <b>101</b> further includes the two imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>corresponding to the wide-angle lenses <b>102</b><i>a </i>and <b>102</b><i>b</i>, respectively. The imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>each include an image sensor such as a complementary metal oxide semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor, a timing generation circuit, and a group of registers. Each of the image sensors converts an optical image formed by a corresponding one of the wide-angle lenses <b>102</b><i>a </i>and <b>102</b><i>b </i>into an electric signal to output image data. Each of the timing generation circuits generates a horizontal or vertical synchronization signal, a pixel clock, and the like for a corresponding one of the image sensors. Each of the groups of registers has set therein various commands, parameters, and the like to be used for operations of a corresponding one of the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b. </i>

0087The imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>of the imaging device <b>101</b> are connected to the image processor <b>104</b> via respective parallel I/F buses. In addition, the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>of the imaging device <b>101</b> are connected to the imaging controller <b>105</b> via respective serial I/F buses such as inter-integrated circuit (I2C) buses. The image processor <b>104</b>, the imaging controller <b>105</b>, and the audio processor <b>109</b> are connected to the CPU <b>111</b> via a bus <b>110</b>. The ROM <b>112</b>, the SRAM <b>113</b>, the DRAM <b>114</b>, the operation device <b>115</b>, the input/output I/F <b>116</b>, the short-range communication circuit <b>117</b>, the electronic compass <b>118</b>, the gyro sensor <b>119</b>, the acceleration sensor <b>120</b>, and the network I/F <b>121</b> are also connected to the bus <b>110</b>.

0088The image processor <b>104</b> acquires respective pieces of image data output from the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>via the parallel I/F buses and performs predetermined processing on the pieces of image data. Thereafter, the image processor <b>104</b> combines the pieces of image data, which are subjected to the predetermined processing, to generate data of an equirectangular projection image as illustrated in <figref idref="DRAWINGS">FIG. 3C</figref>.

0089The imaging controller <b>105</b> usually functions as a master device while the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>usually function as slave devices. The imaging controller <b>105</b> sets commands and the like in the groups of registers of the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>via the respective I2C buses. The imaging controller <b>105</b> receives various commands from the CPU <b>111</b>. Further, the imaging controller <b>105</b> acquires status data and the like of the groups of registers of the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>via the respective I2C buses. The imaging controller <b>105</b> sends the acquired status data and the like to the CPU <b>111</b>.

0090The imaging controller <b>105</b> instructs the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>to output image data at a time when a shutter button of the operation device <b>115</b> is pressed (or tapped). Such a pressing or tapping operation is hereinafter referred to simply as “operation”. In some cases, the image capturing device <b>10</b> has a function of displaying a preview image on a display (e.g., a display of an external terminal such as a smartphone that performs short-range communication with the image capturing device <b>10</b> through the short-range communication circuit <b>117</b>) or displaying a moving image. In the case of displaying a moving image, the imaging elements <b>103</b><i>a </i>and <b>103</b><i>b </i>continuously output image data at a predetermined frame rate (frames per minute).

0091Further, as described below, the imaging controller <b>105</b> operates in cooperation with the CPU <b>111</b> to also function as a synchronization controller for synchronizing the time when the imaging element <b>103</b><i>a </i>outputs image data and the time when the imaging element <b>103</b><i>b </i>outputs image data. Although the image capturing device <b>10</b> does not include a display in this embodiment, the image capturing device <b>10</b> may include a display. The microphone <b>108</b> converts sounds into audio data (signal). The audio processor <b>109</b> acquires the audio data output from the microphone <b>108</b> via an I/F bus and performs predetermined processing on the audio data.

0092The CPU <b>111</b> controls the entire operation of the image capturing device <b>10</b> and also performs certain processing. The ROM <b>112</b> stores various programs for the CPU <b>111</b>. The SRAM <b>113</b> and the DRAM <b>114</b> each operate as a work memory to store programs to be executed by the CPU <b>111</b> or data being processed. More specifically, the DRAM <b>114</b> stores image data being processed by the image processor <b>104</b> or data of the equirectangular projection image on which processing has been performed.

0093The operation device <b>115</b> generally refers to various operation keys, a power switch, a shutter button, a touch panel having both the display and operation functions, and the like. The user operates the operation device <b>115</b> to input various image capturing modes, image capturing conditions, or the like.

0094The input/output I/F <b>116</b> generally refers to an interface circuit such as a USB I/F that allows the image capturing device <b>10</b> to communicate with an external medium such as a Secure Digital (SD) card or a personal computer. The input/output I/F <b>116</b> may be either wired or wireless. The data of the equirectangular projection image, which is stored in the DRAM <b>114</b>, is stored in an external medium via the input/output I/F <b>116</b> or transmitted to an external terminal (or apparatus) via the input/output I/F <b>116</b>, as appropriate.

0095The short-range communication circuit <b>117</b> communicates with an external terminal (or apparatus) via the antenna <b>117</b><i>a </i>of the image capturing device <b>10</b> using short-range wireless communication technology such as near-field communication (NFC), Bluetooth (registered trademark), or Wi-Fi (registered trademark). The short-range communication circuit <b>117</b> is capable of transmitting the data of the equirectangular projection image to an external terminal (or apparatus).

0096The electronic compass <b>118</b> calculates an orientation of the image capturing device from the Earth's magnetism and outputs orientation information. The orientation information is an example of related information (metadata) in compliance with Exchangeable Image File Format (EXIF) and is used for image processing such as image correction of captured images. The related information also includes data such as the date and time when the image is captured, and the data size of the image data. The gyro sensor <b>119</b> detects a change in angle of the image capturing device <b>10</b> (roll angle, pitch angle, and yaw angle) with movement of the image capturing device <b>10</b>. The change in angle is an example of related information (metadata) in compliance with EXIF and is used for image processing such as image correction of captured images. The acceleration sensor <b>120</b> detects acceleration in three axial directions. The image capturing device <b>10</b> calculates the position (an angle with respect to the direction of gravity) of the image capturing device <b>10</b>, based on the acceleration detected by the acceleration sensor <b>120</b>. The image capturing device <b>10</b> provided with the acceleration sensor <b>120</b> improves the accuracy of image correction. The network I/F <b>121</b> is an interface for performing data communication using the communication network <b>100</b> such as the Internet.

Hardware Configuration of Distribution Terminal

0097<figref idref="DRAWINGS">FIG. 12</figref> is a diagram illustrating an example hardware configuration of the distribution terminal <b>30</b>. The hardware components of the distribution terminal <b>30</b> are denoted by reference numerals in the <b>300</b><i>s</i>. The distribution terminal <b>30</b> is constructed by a computer. As illustrated in <figref idref="DRAWINGS">FIG. 12</figref>, the distribution terminal <b>30</b> includes a CPU <b>301</b>, a ROM <b>302</b>, a random access memory (RAM) <b>303</b>, a hard disk (HD) <b>304</b>, a hard disk drive (HDD) controller <b>305</b>, a display <b>306</b>, an external device connection I/F <b>308</b>, a network I/F <b>309</b>, a bus line <b>310</b>, a keyboard <b>311</b>, a pointing device <b>312</b>, a digital versatile disk rewritable (DVD-RW) drive <b>314</b>, a media I/F <b>316</b>, an audio input/output I/F <b>317</b>, a microphone <b>318</b>, a speaker <b>319</b>, and a short-range communication circuit <b>320</b>.

0098The CPU <b>301</b> controls the entire operation of the distribution terminal <b>30</b>. The ROM <b>302</b> stores a program used for driving the CPU <b>301</b>, such as an initial program loader (IPL). The RAM <b>303</b> is used as a work area for the CPU <b>301</b>. The HD <b>304</b> stores various data such as a program. The HDD controller <b>305</b> controls reading or writing of various data from or to the HD <b>304</b> under the control of the CPU <b>301</b>. The display <b>306</b> displays various kinds of information such as a cursor, a menu, a window, characters, and an image. The display <b>306</b> is an example of a display device. In one example, the display <b>306</b> is a touch panel display provided with an input device. The external device connection I/F <b>308</b> is an interface for connecting to various external devices. Examples of the external devices include, but are not limited to, a USB memory and a printer. The network I/F <b>309</b> is an interface for performing data communication using the communication network <b>100</b>. The bus line <b>310</b> is an address bus, a data bus, or the like for electrically connecting the hardware elements illustrated in <figref idref="DRAWINGS">FIG. 12</figref>, such as the CPU <b>301</b>.

0099The keyboard <b>311</b> is a type of input device provided with a plurality of keys for inputting characters, numerals, various instructions, or the like. The pointing device <b>312</b> is a type of input device for selecting or executing various instructions, selecting a processing target, or moving a cursor being displayed. The input device is not limited to the keyboard <b>311</b> and the pointing device <b>312</b> and may be a touch panel, a voice input device, or the like. The DVD-RW drive <b>314</b> controls reading or writing of various data from or to a DVD-RW <b>313</b>, which is an example of a removable recording medium. The removable recording medium is not limited to the DVD-RW and may be a digital versatile disk recordable (DVD-R), a Blu-ray Disc (registered trademark), or the like. The media I/F <b>316</b> controls reading or writing (storing) of data from or to a recording medium <b>315</b> such as a flash memory. The microphone <b>318</b> is a type of built-in sound collector for receiving input sounds. The audio input/output I/F <b>317</b> is a circuit that processes input and output of an audio signal between the microphone <b>318</b> and the speaker <b>319</b> under the control of the CPU <b>301</b>. The short-range communication circuit <b>320</b> is a communication circuit for performing communication with an external terminal (or apparatus) using short-range wireless communication technology such as NFC, Bluetooth, or Wi-Fi.

Hardware Configuration of Communication Management Apparatus

0100<figref idref="DRAWINGS">FIG. 12</figref> is a diagram illustrating an example hardware configuration of the communication management apparatus <b>50</b>. The hardware components of the communication management apparatus <b>50</b> are denoted by reference numerals in the <b>500</b><i>s </i>in parentheses. The communication management apparatus <b>50</b> is constructed by a computer. As illustrated in <figref idref="DRAWINGS">FIG. 12</figref>, since the communication management apparatus <b>50</b> has a configuration similar to that of the distribution terminal <b>30</b>, the description of the hardware components will be omitted.

Hardware Configuration of Communication Terminal

0101<figref idref="DRAWINGS">FIG. 12</figref> is a diagram illustrating an example hardware configuration of the communication terminal <b>70</b>. The hardware components of the communication terminal <b>70</b> are denoted by reference numerals in the <b>700</b><i>s </i>in parentheses. The communication terminal <b>70</b> is constructed by a computer. As illustrated in <figref idref="DRAWINGS">FIG. 12</figref>, since the communication terminal <b>70</b> has a configuration similar to that of the distribution terminal <b>30</b>, the description of the hardware components will be omitted.

0102Further, each of the programs described above may be recorded in a file in an installable or executable format on a computer-readable recording medium for distribution. Examples of the recording medium include a compact disc recordable (CD-R), a digital versatile disk (DVD), a Blu-ray Disc, an SD card, and a USB memory. In addition, the recording medium may be provided in the form of a program product to users within a certain country or outside that country. For example, in the communication management apparatus <b>50</b>, a program according to an embodiment of the present disclosure is executed to implement a communication management method according to an embodiment of the present disclosure.

Functional Configuration of Image Communication System

0103Next, the functional configuration of the image communication system <b>1</b> according to an embodiment will be described with reference to <figref idref="DRAWINGS">FIGS. 13 to 20B</figref>. <figref idref="DRAWINGS">FIGS. 13 and 14</figref> are diagrams illustrating an example functional configuration of the image communication system <b>1</b>. <figref idref="DRAWINGS">FIGS. 13 and 14</figref> illustrate devices and terminals related to the processes or operations described below among the devices and terminals illustrated in <figref idref="DRAWINGS">FIG. 9</figref>.

Functional Configuration of Image Capturing Device

0104First, the functional configuration of the image capturing device <b>10</b> will be described with reference to <figref idref="DRAWINGS">FIG. 13</figref>. The image capturing device <b>10</b> includes a communication unit <b>11</b>, an acceptance unit <b>12</b>, an imaging unit <b>13</b>, a sound collection unit <b>14</b>, and a storing and reading unit <b>19</b>. The communication unit <b>11</b>, the acceptance unit <b>12</b>, the imaging unit <b>13</b>, the sound collection unit <b>14</b>, and the storing and reading unit <b>19</b> are functions or means implemented by any one of the hardware elements illustrated in <figref idref="DRAWINGS">FIG. 11</figref> operating in accordance with instructions from the CPU <b>111</b> according to an image capturing device program loaded onto the DRAM <b>114</b> from the SRAM <b>113</b>. The image capturing device <b>10</b> further includes a storage unit <b>1000</b>. The storage unit <b>1000</b> is constructed by the ROM <b>112</b>, the SRAM <b>113</b>, and the DRAM <b>114</b> illustrated in <figref idref="DRAWINGS">FIG. 11</figref>. The storage unit <b>1000</b> stores the globally unique identifier (GUID) of the image capturing device <b>10</b>.

0105The communication unit <b>11</b> is mainly implemented by processing performed by the CPU <b>111</b> and communicates various data or information to another apparatus or terminal. The communication unit <b>11</b> performs, for example, data communication with another apparatus or terminal through the short-range communication circuit <b>117</b> using short-range wireless communication technology. Further, the communication unit <b>11</b> performs, for example, data communication with another apparatus or terminal through the input/output I/F <b>116</b> via various cables or the like. The communication unit <b>11</b> further performs data communication with another apparatus or terminal through the network I/F <b>121</b> via the communication network <b>100</b>.

0106The acceptance unit <b>12</b> is mainly implemented by processing performed by the CPU <b>111</b> on the operation device <b>115</b> and accepts various selections or inputs from a user. The imaging unit <b>13</b> is mainly implemented by processing performed by the CPU <b>111</b> on the imaging device <b>101</b>, the image processor <b>104</b>, and the imaging controller <b>105</b> and captures an object such as scenery to acquire captured image data. The sound collection unit <b>14</b> is mainly implemented by processing performed by the CPU <b>111</b> on the microphone <b>108</b> and the audio processor <b>109</b> and collects sounds around the image capturing device <b>10</b>.

0107The storing and reading unit <b>19</b> is mainly implemented by processing performed by the CPU <b>111</b> and stores various data (or information) in the storage unit <b>1000</b> or reads various data (or information) from the storage unit <b>1000</b>.

Functional Configuration of Distribution Terminal

0108Next, the functional configuration of the distribution terminal <b>30</b> will be described with reference to <figref idref="DRAWINGS">FIG. 13</figref>. The distribution terminal <b>30</b> includes a transmitting/receiving unit <b>31</b>, an acceptance unit <b>32</b>, an image and audio processing unit <b>33</b>, a display control unit <b>34</b>, a determination unit <b>35</b>, a creation unit <b>36</b>, a communication unit <b>37</b>, and a storing and reading unit <b>39</b>. The transmitting/receiving unit <b>31</b>, the acceptance unit <b>32</b>, the image and audio processing unit <b>33</b>, the display control unit <b>34</b>, the determination unit <b>35</b>, the creation unit <b>36</b>, the communication unit <b>37</b>, and the storing and reading unit <b>39</b> are functions or means implemented by any one of the hardware elements illustrated in <figref idref="DRAWINGS">FIG. 12</figref> operating in accordance with instructions from the CPU <b>301</b> according to a distribution terminal program loaded onto the RAM <b>303</b> from the HD <b>304</b>. The distribution terminal <b>30</b> further includes a storage unit <b>3000</b>. The storage unit <b>3000</b> is constructed by the ROM <b>302</b>, the RAM <b>303</b>, and the HD <b>304</b> illustrated in <figref idref="DRAWINGS">FIG. 12</figref>.

0109The transmitting/receiving unit <b>31</b> is mainly implemented by processing performed by the CPU <b>301</b> on the network I/F <b>309</b> and transmits and receives various data or information to and from another apparatus or terminal via the communication network <b>100</b>.

0110The acceptance unit <b>32</b> is mainly implemented by processing performed by the CPU <b>301</b> on the keyboard <b>311</b> or the pointing device <b>312</b> and accepts various selections or inputs from a user.

0111The image and audio processing unit <b>33</b> is mainly implemented by processing performed by the CPU <b>301</b> and performs image processing on captured image data acquired by the image capturing device <b>10</b> capturing an object. The image and audio processing unit <b>33</b> further performs audio processing on audio data of a voice signal, which is obtained by converting the voice of the user using the microphone <b>318</b>. For example, the image and audio processing unit <b>33</b> performs image processing on captured image data received from the image capturing device <b>10</b>, based on image type information such as the source name so that the display control unit <b>34</b> causes the display <b>306</b> to display an image. Specifically, when the image type information indicates the special image, the image and audio processing unit <b>33</b> converts the captured image data (e.g., data of hemispherical images as illustrated in <figref idref="DRAWINGS">FIGS. 3A and 3B</figref>) into spherical image data as illustrated in <figref idref="DRAWINGS">FIG. 4B</figref> to create spherical image data. Further, the image and audio processing unit <b>33</b> outputs a voice signal of audio data distributed from another terminal via the communication management apparatus <b>50</b> to the speaker <b>319</b> and outputs a voice from the speaker <b>319</b>.

0112The display control unit <b>34</b> is mainly implemented by processing performed by the CPU <b>301</b> and causes the display <b>306</b> to display various images, characters, or the like. The determination unit <b>35</b> is implemented by processing performed by the CPU <b>301</b> and performs various determinations. For example, the determination unit <b>35</b> determines the image type of captured image data received from the image capturing device <b>10</b>.

0113The creation unit <b>36</b> is mainly implemented by processing performed by the CPU <b>301</b> and creates a source name, which is an example of the image type information, in accordance with a naming rule, based on the general image or special image (that is, the spherical image) determined by the determination unit <b>35</b>. For example, if the determination unit <b>35</b> determines that the image type is the general image, the creation unit <b>36</b> creates the source name “Video” indicating the general image. By contrast, if the determination unit <b>35</b> determines that the image type is the special image, the creation unit <b>36</b> creates the source name “Video_Omni” indicating the special image.

0114The communication unit <b>37</b> is mainly implemented by processing performed by the CPU <b>301</b> on the short-range communication circuit <b>320</b> and communicates with the communication unit <b>11</b> of the image capturing device <b>10</b> using short-range wireless communication technology such as NFC, Bluetooth, or WiFi. In the foregoing description, the communication unit <b>37</b> and the transmitting/receiving unit <b>31</b> are configured as separate communication units. In another example, the communication unit <b>37</b> and the transmitting/receiving unit <b>31</b> may share a single communication unit.

0115The storing and reading unit <b>39</b> is mainly implemented by processing performed by the CPU <b>301</b> and stores various data (or information) in the storage unit <b>3000</b> or reads various data (or information) from the storage unit <b>3000</b>.

Image Capturing Device Management Table

0116<figref idref="DRAWINGS">FIG. 15A</figref> is a conceptual diagram illustrating an example image capturing device management table. The storage unit <b>3000</b> includes an image capturing device management database (DB) <b>3001</b>. The image capturing device management DB <b>3001</b> is implemented by an image capturing device management table illustrated in <figref idref="DRAWINGS">FIG. 15A</figref>. The image capturing device management table stores and manages a vendor ID and a product ID in the GUID of an image capturing device capable of obtaining two hemispherical images from which a spherical image is generated. Examples of the GUID include a vendor ID (VID) and a product ID (PID), which are used by a USB device. The vendor ID and the product ID may be stored when the distribution terminal <b>30</b> is shipped from the factory or may be additionally stored after the distribution terminal <b>30</b> is shipped from the factory, for example.

Image Type Management Table

0117<figref idref="DRAWINGS">FIG. 15B</figref> is a conceptual diagram illustrating an example image type management table. The storage unit <b>3000</b> includes an image type management DB <b>3002</b>. The image type management DB <b>3002</b> is implemented by an image type management table illustrated in <figref idref="DRAWINGS">FIG. 15B</figref>. The image type management table stores, for each image data ID, an Internet Protocol (IP) address of an image capturing device, which is an example of an address of an image capturing device, and a source name (image type information) in association with each other. The image data ID is an example of image data identification information for identifying image data of an image to be distributed. The IP address of the image capturing device indicates the IP address of the image capturing device <b>10</b> that has captured the image data indicated by the associated image data ID. The source name is a name for specifying the image capturing device <b>10</b> that has captured the image data indicated by the associated image data ID, and is an example of image type information. The source name is a name created by the distribution terminal <b>30</b> in accordance with a predetermined naming rule.

0118The illustrated example indicates that four image capturing devices having the IP addresses “2.1.2.3”, “2.1.1.5”, “2.1.5.4”, and “2.1.5.6” have transmitted image data indicated by image data IDs “RS001”, “RS002”, “RS003”, and “RS004”, respectively. It is also indicated that the image types indicated by the source names of the four image capturing devices are “Video_Omni”, “Video_Omni”, “Video”, and “Video”, which indicate the image types “special image”, “special image”, “general image”, and “general image”, respectively. In this embodiment, the special image is the spherical image. The IP address is an example of address information, and the address information may be a Media Access Control (MAC) address, a terminal identification (ID), or the like. While the IP address is a simplified representation of the Internet Protocol version 4 (IPv4) address, the FP address may be an Internet Protocol version 6 (IPv6) address. In addition, data other than image data may be managed in association with the image data ID. Examples of the data other than image data include audio data, and document data to be used to share the screen between the distribution site and the viewing sites.

Functional Configuration of Communication Management Apparatus

0119Next, the functional configuration of the communication management apparatus <b>50</b> will be described with reference to <figref idref="DRAWINGS">FIG. 14</figref>. The communication management apparatus <b>50</b> includes a transmitting/receiving unit <b>51</b>, a selection unit <b>53</b>, a determination unit <b>55</b>, a generation unit <b>56</b>, a distribution site management unit <b>57</b>, and a storing and reading unit <b>59</b>. The transmitting/receiving unit <b>51</b>, the selection unit <b>53</b>, the determination unit <b>55</b>, the generation unit <b>56</b>, the distribution site management unit <b>57</b>, and the storing and reading unit <b>59</b> are functions or means implemented by any one of the hardware elements illustrated in <figref idref="DRAWINGS">FIG. 12</figref> operating in accordance with instructions from the CPU <b>501</b> according to a communication management apparatus program loaded onto the RAM <b>503</b> from the HD <b>504</b>. The communication management apparatus <b>50</b> further includes a storage unit <b>5000</b>. The storage unit <b>5000</b> is constructed by the ROM <b>502</b>, the RAM <b>503</b>, and the HD <b>504</b> illustrated in <figref idref="DRAWINGS">FIG. 12</figref>.

0120The transmitting/receiving unit <b>51</b> is mainly implemented by processing performed by the CPU <b>501</b> on the network I/F <b>509</b> and transmits and receives various data or information to and from another apparatus via the communication network <b>100</b>.

0121The selection unit <b>53</b> is mainly implemented by processing performed by the CPU <b>501</b> and selects an image capturing device <b>10</b> in response to a request from the communication terminal <b>70</b>. Specifically, the selection unit <b>53</b> selects a specific image capturing device <b>10</b> among the plurality of image capturing devices <b>10</b>, based on, for example, coordinate information of a point of interest P, which is accepted by the communication terminal <b>70</b>, arrangement position information of the plurality of image capturing devices <b>10</b>, and number-of-division information indicating the number of divisions of an area accepted by the communication terminal <b>70</b>. The selected specific image capturing device <b>10</b> is referred to as “supplementary image capturing device”, in particular. In this embodiment, the supplementary image capturing device functions as an example of a second image capturing device. In this embodiment, the selection unit <b>53</b> functions as an example of selection means.

0122The determination unit <b>55</b> is mainly implemented by processing performed by the CPU <b>501</b> and performs various determinations.

0123The generation unit <b>56</b> is mainly implemented by processing performed by the CPU <b>501</b> and generates an image data ID and predetermined-area information. The generation unit <b>56</b> generates, for example, predetermined-area information. The predetermined-area information indicates a predetermined area (e.g., the predetermined area T illustrated in FIG. and the like) in an image captured by the image capturing device <b>10</b> selected by the selection unit <b>53</b>. An image in which the entire captured image is displayed (e.g., the spherical image CE illustrated in <figref idref="DRAWINGS">FIG. 5</figref> and the like) is also referred to as “entire image”. The distribution site management unit <b>57</b> is mainly implemented by processing performed by the CPU <b>501</b> and manages distribution site information indicating the state of the distribution site.

0124The storing and reading unit <b>59</b> is mainly implemented by processing performed by the CPU <b>501</b> and stores various data (or information) in the storage unit <b>5000</b> or reads various data (or information) from the storage unit <b>5000</b>.

Session Management Table

0125<figref idref="DRAWINGS">FIG. 16A</figref> is a conceptual diagram illustrating an example session management table. The storage unit <b>5000</b> includes a session management DB <b>5001</b>. The session management DB <b>5001</b> is implemented by a session management table illustrated in <figref idref="DRAWINGS">FIG. 16A</figref>. The session management table stores, for each session ID, a site ID and an IP address of a participant communication terminal in association with each other. The session ID is an example of session identification information for identifying a communication session for implementing image communication. The session ID is generated for each virtual floor. The session ID is also managed by the communication terminals <b>70</b> and is used when each of the communication terminals <b>70</b> selects a communication session. The site ID is an example of site identification information for identifying a distribution site. The IP address of the participant communication terminal indicates the IP address of a communication terminal <b>70</b> participating in a virtual floor indicated by the associated session ID.

Image Type Management Table

0126<figref idref="DRAWINGS">FIG. 16B</figref> is a conceptual diagram illustrating an example image type management table. The storage unit <b>5000</b> includes an image type management DB <b>5002</b>. The image type management DB <b>5002</b> is implemented by an image type management table illustrated in <figref idref="DRAWINGS">FIG. 16B</figref>. The image type management table stores the information managed by the image type management table illustrated in <figref idref="DRAWINGS">FIG. 15B</figref> and the same session ID as the session ID managed in the session management table illustrated in <figref idref="DRAWINGS">FIG. 16A</figref> in association with each other. The communication management apparatus <b>50</b> stores an image data ID, an IP address of an image capturing device, and image type information, which are the same as those managed in the distribution terminal <b>30</b> and the communication terminal <b>70</b>, because, for example, when a new communication terminal <b>70</b> enters a virtual floor, the communication management apparatus <b>50</b> transmits information including the image type information to a communication terminal <b>70</b> that is already in video communication and the new communication terminal <b>70</b>, which has newly participated in the video communication. As a result, the communication terminal <b>70</b> that is already in the video communication and the communication terminal <b>70</b> that has newly participated in the video communication do not have to transmit and receive such information including the image type information.

Predetermined-Area Management Table

0127<figref idref="DRAWINGS">FIG. 17A</figref> is a conceptual diagram illustrating an example predetermined-area management table. The storage unit <b>5000</b> includes a predetermined-area management DB <b>5003</b>. The predetermined-area management DB <b>5003</b> is implemented by a predetermined-area management table illustrated in <figref idref="DRAWINGS">FIG. 17A</figref>. The predetermined-area management table stores, for the IP address of each image capturing device (image source) from which captured image data is transmitted, an IP address of a communication terminal (image destination) to which the captured image data is transmitted, and predetermined-area information in association with each other. The predetermined-area information indicates a parameter of a predetermined-area image currently displayed on the communication terminal to which the captured image data is transmitted. As illustrated in <figref idref="DRAWINGS">FIGS. 6A, 6B, and 7</figref>, the predetermined-area information is a conversion parameter for converting the captured image into the image of the predetermined area T (the predetermined-area image Q) in the captured image.

0128For example, the information managed in the first row of the predetermined-area management table illustrated in <figref idref="DRAWINGS">FIG. 17A</figref> indicates that after the distribution terminal <b>30</b> receives captured image data obtained by the image capturing device <b>10</b> having the IP address “2.1.2.3”, the captured image data is transmitted from the distribution terminal <b>30</b> to the communication terminal <b>70</b> having the IP address “1.2.1.3” via the communication management apparatus <b>50</b>. It is also managed that the distribution terminal <b>30</b> is a distribution terminal from which the four pieces of predetermined-area information in the first to fourth rows of the predetermined-area management table illustrated in <figref idref="DRAWINGS">FIG. 17A</figref> are transmitted.

0129When predetermined-area information including IP addresses in the same set as an already managed set of the IP address of an image capturing device from which captured image data is transmitted and the IP address of a communication terminal to which the captured image data is transmitted is newly received by the transmitting/receiving unit <b>51</b>, the storing and reading unit <b>59</b> rewrites the already managed predetermined-area information to the newly received predetermined-area information.

Arrangement Information Management Table

0130<figref idref="DRAWINGS">FIG. 17B</figref> is a conceptual diagram illustrating an example arrangement information management table. The storage unit <b>5000</b> includes an arrangement information management DB <b>5004</b>. The arrangement information management DB <b>5004</b> is implemented by an arrangement information management table illustrated in <figref idref="DRAWINGS">FIG. 17B</figref>. The arrangement information management table stores, for each site ID for identifying a distribution site, an IP address of an image capturing device <b>10</b> arranged in the distribution site, coordinate values indicating the position at which the image capturing device <b>10</b> is arranged, and the arrangement direction of the image capturing device <b>10</b> in association with each other. The coordinate values are coordinate values on the map of the distribution site where the image capturing device <b>10</b> is arranged. The arrangement direction of the image capturing device <b>10</b> indicates the direction (angle) on the map in which the front surface of the image capturing device <b>10</b> faces. The arrangement direction is set to a desired direction (angle) by the administrator of the distribution terminal <b>30</b> or the distribution site.

Distribution-Site Management Table

0131<figref idref="DRAWINGS">FIG. 18</figref> is a conceptual diagram illustrating an example distribution-site management table. The storage unit <b>5000</b> includes a distribution site management DB <b>5005</b>. The distribution site management DB <b>5005</b> is implemented by a distribution-site management table illustrated in <figref idref="DRAWINGS">FIG. 18</figref>. The distribution-site management table stores, for each site ID for identifying a distribution site, a site name and distribution site information in association with each other. The distribution site information indicates the state of the distribution site. The distribution site information includes a uniform resource locator (URL) for accessing map image data indicating a map of the distribution site and site coordinate information indicating the coordinate values of the distribution site. The communication terminal <b>70</b> accesses the URL transmitted from the communication management apparatus <b>50</b> to acquire the map image data and the site coordinate information of the distribution site. The URL is an example of storage destination information. The storage destination information is not limited to the URL and may be a uniform resource identifier (URI) or the like.

Weighting Coefficient Management Table

0132<figref idref="DRAWINGS">FIG. 19A</figref> is a conceptual diagram illustrating an example weighting coefficient management table. The storage unit <b>5000</b> includes a weighting coefficient management DB <b>5006</b>. The weighting coefficient management DB <b>5006</b> is implemented by a weighting coefficient management table illustrated in <figref idref="DRAWINGS">FIG. 19A</figref>. The weighting coefficient management table stores, for each reference image capturing device name indicating the name of a reference image capturing device, a coefficient and a supplementary image capturing device name, which indicates the name of a supplementary image capturing device, in association with each other. The reference image capturing device is an image capturing device <b>10</b> arranged at the position closest to a point of interest P (predetermined position) designated by a user. The supplementary image capturing device is an image capturing device <b>10</b> capable of capturing from the most different aspects a portion of the point of interest P for which an image is difficult for the reference image capturing device to capture. The coefficient is a value indicating the arrangement relationship between the reference image capturing device and the supplementary image capturing device.

0133The coefficient is set to a desired value in advance by the administrator who arranges the image capturing devices <b>10</b> at the distribution site A and the like or the administrator who manages the communication management apparatus <b>50</b>. For example, for some pairs of supplementary image capturing device and the reference image capturing device, a coefficient of 10.0 is set due to presence of wall therebetween (specifically, CAM<b>1</b> and CAM<b>3</b>, CAM <b>2</b> and CAM <b>3</b>). In another example, for some pairs of supplementary image capturing device and the reference image capturing device, a coefficient of 5.0 is set due to presence of a partial wall therebetween.

Specific Weighting Coefficient Management Table

0134<figref idref="DRAWINGS">FIG. 19B</figref> is a conceptual diagram illustrating an example specific weighting coefficient management table. The storage unit <b>5000</b> includes a specific weighting coefficient management DB <b>5007</b>. The specific weighting coefficient management DB <b>5007</b> is implemented by a specific weighting coefficient management table illustrated in <figref idref="DRAWINGS">FIG. 19B</figref>. The specific weighting coefficient management table stores, for each reference image capturing device name indicating the name of a reference image capturing device, a specific coefficient and a supplementary image capturing device name, which indicates the name of a supplementary image capturing device, in association with each other. The specific coefficient is given as a value larger than any other coefficient in the weighting coefficient management table illustrated in <figref idref="DRAWINGS">FIG. 19A</figref> when at least one of the following conditions is satisfied: (1) an obstacle (such as a wall) is present between the reference image capturing device and the supplementary image capturing device; and (2) the supplementary image capturing device has any abnormality or the like. The specific coefficient is also set to a desired value in advance by the administrator who arranges the image capturing devices <b>10</b> at the distribution site A and the like or the administrator who manages the communication management apparatus <b>50</b>.

Functional Configuration of Communication Terminal

0135Next, the functional configuration of the communication terminal <b>70</b> will be described with reference to <figref idref="DRAWINGS">FIG. 14</figref>. The communication terminal <b>70</b> includes a transmitting/receiving unit (acquisition unit) <b>71</b>, an acceptance unit <b>72</b>, an image and audio processing unit <b>73</b>, a display control unit <b>74</b>, a determination unit <b>75</b>, a creation unit <b>76</b>, a point-of-interest specifying unit <b>77</b>, a selection unit <b>81</b>, a generation unit <b>82</b>, and a storing and reading unit <b>79</b>. The transmitting/receiving unit <b>71</b>, the acceptance unit <b>72</b>, the image and audio processing unit <b>73</b>, the display control unit <b>74</b>, the determination unit <b>75</b>, the creation unit <b>76</b>, the point-of-interest specifying unit <b>77</b>, the selection unit <b>81</b>, the generation unit <b>82</b>, and the storing and reading unit <b>79</b> are functions or means implemented by any one of the hardware elements illustrated in <figref idref="DRAWINGS">FIG. 12</figref> operating in accordance with instructions from the CPU <b>701</b> according to a communication terminal program loaded onto the RAM <b>703</b> from the HD <b>704</b>. The communication terminal <b>70</b> further includes a storage unit <b>7000</b>. The storage unit <b>7000</b> is constructed by the ROM <b>702</b>, the RAM <b>703</b>, and the HD <b>704</b> illustrated in <figref idref="DRAWINGS">FIG. 12</figref>.

0136The transmitting/receiving unit (acquisition unit) <b>71</b> is mainly implemented by processing performed by the CPU <b>701</b> on the network I/F <b>709</b> and transmits and receives various data or information to and from another apparatus or terminal via the communication network <b>100</b>. The transmitting/receiving unit (acquisition unit) <b>71</b> receives, for example, captured image data distributed from the distribution terminal <b>30</b> via the communication management apparatus <b>50</b>. Further, the transmitting/receiving unit (acquisition unit) <b>71</b> functions as, for example, an acquisition unit and acquires predetermined-area information. The predetermined-area information indicates a predetermined area including the point of interest P in an image captured by a specific image capturing device <b>10</b> selected based on the coordinate information (predetermined-position information) of the point of interest P, which is accepted by the acceptance unit <b>72</b>, and the arrangement positions (arrangement position information) of the image capturing devices <b>10</b>.

0137The acceptance unit <b>72</b> is mainly implemented by processing performed by the CPU <b>701</b> on the keyboard <b>711</b> or the pointing device <b>712</b> and accepts various selections or inputs from a user. For example, the acceptance unit <b>72</b> accepts input of the point of interest P, which is designated by the user, in a distribution site and the number of divisions into which a predetermined range (e.g., a circle having a radius r) centered on the point of interest P is divided from the point of interest P. In this embodiment, the acceptance unit <b>72</b> functions as, or has means for functioning as, an example of acceptance means.

0138The image and audio processing unit <b>73</b> is mainly implemented by processing performed by the CPU <b>701</b> and performs image processing on captured image data distributed from the distribution terminal <b>30</b>. The image and audio processing unit <b>73</b> further performs audio processing on audio data distributed from the distribution terminal <b>30</b>. For example, to display on the display <b>706</b> an image of a predetermined area corresponding to predetermined-area information received by the transmitting/receiving unit (acquisition unit) <b>71</b>, the image and audio processing unit <b>73</b> applies perspective projection conversion to the captured image (spherical image) using the predetermined-area information to generate a predetermined-area image corresponding to the predetermined-area information. Further, the image and audio processing unit <b>73</b> outputs a voice signal of audio data distributed from the distribution terminal <b>30</b> via the communication management apparatus <b>50</b> to the speaker <b>719</b> and outputs a voice from the speaker <b>719</b>.

0139The display control unit <b>74</b> is mainly implemented by processing performed by the CPU <b>701</b> and causes the display <b>706</b> to display various images, characters, or the like. For example, the display control unit <b>74</b> causes the display <b>706</b> to display the predetermined-area image generated by the image and audio processing unit <b>73</b>.

0140The determination unit <b>75</b> is implemented by processing performed by the CPU <b>701</b> and performs various determinations. In this embodiment, the display control unit <b>74</b> functions as, or has means for functioning as, an example of display control means.

0141The creation unit <b>76</b> is mainly implemented by processing performed by the CPU <b>701</b> and implements functions similar to those of the creation unit <b>36</b>. The point-of-interest specifying unit <b>77</b> is mainly implemented by processing performed by the CPU <b>701</b> and specifies the point of interest P in the distribution site where the image capturing devices <b>10</b> are arranged. For example, the point-of-interest specifying unit <b>77</b> specifies the coordinate information of the point of interest P designated by the user on the map image of the distribution site.

0142The selection unit <b>81</b> is mainly implemented by processing performed by the CPU <b>701</b> and implements functions similar to those of the selection unit <b>53</b>. The generation unit <b>82</b> is mainly implemented by processing performed by the CPU <b>701</b> and generates predetermined-area information. For example, the generation unit <b>82</b> generates predetermined-area information indicating a predetermined area in an image captured by an image capturing device <b>10</b> selected by the selection unit <b>81</b>. Further, the generation unit <b>82</b> functions as, for example, an acquisition unit and acquires predetermined-area information. The predetermined-area information indicates a predetermined area including the point of interest P in an image captured by a specific image capturing device <b>10</b> selected based on the coordinate information of the point of interest P, which is accepted by the acceptance unit <b>72</b>, and the arrangement positions of the image capturing devices <b>10</b>.

0143The storing and reading unit <b>79</b> is mainly implemented by processing performed by the CPU <b>701</b> and stores various data (or information) in the storage unit <b>7000</b> or reads various data (or information) from the storage unit <b>7000</b>.

Image Type Management Table

0144<figref idref="DRAWINGS">FIG. 20A</figref> is a conceptual diagram illustrating an example image type management table. The storage unit <b>7000</b> includes an image type management DB <b>7001</b>. The image type management DB <b>7001</b> is implemented by an image type management table illustrated in <figref idref="DRAWINGS">FIG. 20A</figref>. Since the image type management table has substantially the same data configuration as that of the image type management table illustrated in <figref idref="DRAWINGS">FIG. 15B</figref>, the description thereof will be omitted.

Predetermined-Area Management Table

0145<figref idref="DRAWINGS">FIG. 20B</figref> is a conceptual diagram illustrating an example predetermined-area management table. The storage unit <b>7000</b> includes a predetermined-area management DB <b>7002</b>. The predetermined-area management DB <b>7002</b> is implemented by a predetermined-area management table illustrated in <figref idref="DRAWINGS">FIG. 20B</figref>. The predetermined-area management table stores, for the IP address of each image capturing device that has obtained captured image data, predetermined-area information in association with each other. The predetermined-area information indicates a currently displayed predetermined-area image. As illustrated in <figref idref="DRAWINGS">FIGS. 6A, 6B, and 7</figref>, the predetermined-area information is a conversion parameter for converting the captured image into the image of the predetermined area T (the predetermined-area image Q) in the captured image.

0146The relationship between the point of interest P and a plurality of image capturing devices <b>10</b> arranged at the distribution site A will now be described with reference to <figref idref="DRAWINGS">FIGS. 21 to 27</figref>.

0000Relationship Between Point of Interest P and Image Capturing Devices when Number of Divisions is Two

0147<figref idref="DRAWINGS">FIG. 21</figref> is a view schematically describing an example relationship between the point of interest P and the image capturing devices <b>10</b> when the number of divisions is two. For example, at the distribution site A (area A) in the image communication system <b>1</b> illustrated in <figref idref="DRAWINGS">FIG. 10</figref>, the plurality of image capturing devices <b>10</b>A are arranged regularly in a grid pattern, as illustrated in <figref idref="DRAWINGS">FIG. 21</figref>. Based on the arrangement of the plurality of image capturing devices <b>10</b>A, the generation unit <b>56</b> generates a two-dimensional arrangement image (map image) based on the coordinate values of each of the image capturing devices <b>10</b>A, for example. At this time, the user who uses the communication terminal <b>70</b> operates a “point of interest” button <b>605</b> and a “number of divisions” button <b>670</b> (see <figref idref="DRAWINGS">FIG. 34</figref>). The “point of interest” button <b>605</b> indicates a predetermined position. The “number of divisions” button <b>670</b> is used to divide a predetermined range (e.g., a circle having a radius r) centered on the point of interest P from the point of interest P. The user can operate the “point of interest” button <b>605</b> while referring to a schematic image <b>650</b> corresponding to a two-dimensional arrangement image displayed on a site display screen <b>600</b> of the communication terminal <b>70</b>, which will be described below. The selection unit <b>53</b> selects, among the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> arranged in the circle having the radius r, the image capturing device <b>10</b>A-<b>1</b>, which is arranged at the position closest to the point of interest P.

0148Then, the selection unit <b>53</b> extends a virtual line segment connecting the point of interest P (predetermined position) and the image capturing device <b>10</b>A-<b>1</b> to divide the two-dimensional arrangement image generated by the generation unit <b>56</b> into two areas. The two-dimensional arrangement image obtained in the way described above is an example in the case of the number of divisions being set to two.

0000Process for Selecting Image Capturing Device when Number of Divisions is Two

0149<figref idref="DRAWINGS">FIG. 22</figref> is a view schematically describing an example selection process based on an angle between image capturing devices <b>10</b>A when the number of divisions is two. As illustrated in the example in <figref idref="DRAWINGS">FIG. 22</figref>, the angle (azimuth angle) between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>2</b> is 101 degrees. The angle between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>3</b> is 121 degrees. The angle between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>4</b> is 163 degrees. The distribution site A (area A) is divided into two areas by a thick broken line obtained by extending the virtual line segment connecting the point of interest P and the image capturing device <b>10</b>A-<b>1</b>.

0000Relationship Between Point of Interest P and Image Capturing Devices when Number of Divisions is Three

0150<figref idref="DRAWINGS">FIG. 23</figref> is a view schematically describing an example relationship between the point of interest P and the image capturing devices <b>10</b> when the number of divisions is three. For simplicity of description, the arrangement of the image capturing devices <b>10</b>A at the distribution site A illustrated in <figref idref="DRAWINGS">FIG. 23</figref> is similar to that in <figref idref="DRAWINGS">FIG. 21</figref>. In this state, in response to the user who uses the communication terminal <b>70</b> operating the “point of interest” button <b>605</b> and the “number of divisions” button <b>670</b>, the selection unit <b>53</b> extends a virtual line segment connecting the point of interest P (predetermined position) and the image capturing device <b>10</b>A-<b>1</b> to divide the two-dimensional arrangement image generated by the generation unit <b>56</b> into three areas such that the two-dimensional arrangement image is separated by 120 degrees. The two-dimensional arrangement image obtained in the way described above is an example in the case of the number of divisions being set to three.

0000Process for Selecting Image Capturing Device when Number of Divisions is Three

0151<figref idref="DRAWINGS">FIG. 24</figref> is a view schematically describing an example selection process based on an angle between image capturing devices <b>10</b>A when the number of divisions is three. The arrangement of the image capturing devices <b>10</b>A illustrated in <figref idref="DRAWINGS">FIG. 24</figref> is similar to that of the image capturing devices <b>10</b>A illustrated in <figref idref="DRAWINGS">FIG. 22</figref>. In <figref idref="DRAWINGS">FIG. 24</figref>, the distribution site A (area A) is divided into three areas by thick broken lines based on the virtual line segment (thick broken line) connecting the point of interest P and the image capturing device <b>10</b>A-<b>1</b> such that the thick broken lines each have an angle of 120 degrees.

0000Process for Selecting Image Capturing Device when Number of Divisions is Three (in Presence of Obstacle)

0152<figref idref="DRAWINGS">FIG. 25</figref> is a view schematically describing an example selection process based on an angle between image capturing devices <b>10</b>A when the number of divisions is three. In the example illustrated in <figref idref="DRAWINGS">FIG. 25</figref>, a wall is present between the image capturing device <b>10</b>A-<b>3</b> and the point of interest P.

0000Relationship Between Point of Interest P and Image Capturing Devices when Number of Divisions is Four

0153<figref idref="DRAWINGS">FIG. 26</figref> is a view schematically describing an example relationship between the point of interest P and the image capturing devices <b>10</b> when the number of divisions is four. The arrangement of the image capturing devices <b>10</b>A at the distribution site A illustrated in <figref idref="DRAWINGS">FIG. 26</figref> is also similar to that in <figref idref="DRAWINGS">FIGS. 21 and 23</figref>. In this state, in response to the user who uses the communication terminal <b>70</b> operating the “point of interest” button <b>605</b> and the “number of divisions” button <b>670</b>, the selection unit <b>53</b> extends a virtual line segment connecting the point of interest P (predetermined position) and the image capturing device <b>10</b>A-<b>1</b> to divide the two-dimensional arrangement image generated by the generation unit <b>56</b> into four areas such that the two-dimensional arrangement image is separated by 90 degrees. The two-dimensional arrangement image obtained in the way described above is an example in the case of the number of divisions being set to four.

0154<figref idref="DRAWINGS">FIG. 27</figref> is a view schematically describing an example relationship between the point of interest P and image capturing devices <b>10</b> arranged in irregular positions. As illustrated in <figref idref="DRAWINGS">FIG. 27</figref>, the image capturing devices <b>10</b> may be arranged in irregular positions at the distribution site A. Also in this case, the selection unit <b>53</b> can divide, in accordance with the number of divisions given by the user, that is, two, three, four, or the like, the area into the given number of divisions, on the basis of the positional relationship between the image capturing devices <b>10</b> and the point of interest P in an area having the radius r.

Process or Operation According to Embodiment

Session Participation Process

0155Next, a process or operation of the image communication system <b>1</b> according to an embodiment will be described with reference to <figref idref="DRAWINGS">FIGS. 28 to 36</figref>. In the following description, a captured image of the distribution site A is distributed to the communication terminal <b>70</b>, by way of example. However, similar processing is performed for distribution of an image from any other distribution site such as the distribution site B. First, a process for participating in a specific communication session will be described with reference to <figref idref="DRAWINGS">FIGS. 28 and 29</figref>. <figref idref="DRAWINGS">FIG. 28</figref> is a sequence diagram illustrating an example process for participating in a specific communication session in the image communication system <b>1</b>. <figref idref="DRAWINGS">FIG. 29</figref> is a view illustrating an example screen for selecting a communication session.

0156First, a user (e.g., the user C<b>1</b>) at the viewing site C performs an operation of displaying a selection screen for selecting a communication session. In response to the acceptance unit <b>72</b> accepting the operation of displaying the selection screen, the display control unit <b>74</b> of the communication terminal <b>70</b>C causes the display <b>706</b> to display a selection screen <b>800</b> illustrated in <figref idref="DRAWINGS">FIG. 29</figref> (step S<b>11</b>). The selection screen <b>800</b> illustrated in <figref idref="DRAWINGS">FIG. 29</figref> displays selection buttons <b>810</b><i>a</i>, <b>810</b><i>b</i>, <b>810</b><i>c</i>, etc. indicating floors A<b>1</b>, B<b>1</b>, B<b>2</b>, etc. to be selected, respectively. The selection button <b>810</b><i>a </i>and the other selection buttons are associated with respective session IDs.

0157When the user C<b>1</b> selects a selection button for the desired virtual floor that is a distribution site (here, the selection button <b>810</b><i>a</i>), the acceptance unit <b>72</b> accepts selection of a communication session (step S<b>12</b>). Then, the transmitting/receiving unit (acquisition unit) <b>71</b> transmits to the communication management apparatus <b>50</b> a participation request to participate in the communication session with the distribution site (step S<b>13</b>). Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the participation request. The participation request includes a session ID indicating the communication session for which selection is accepted in step S<b>12</b>, and the IP address of the communication terminal <b>70</b>C, which is the request sender terminal.

0158Then, the storing and reading unit <b>59</b> of the communication management apparatus <b>50</b> adds, in the session management table (the session management DB <b>5001</b>, see <figref idref="DRAWINGS">FIG. 16A</figref>), the IP address received in step S<b>13</b> to the “IP address of participant terminal” field for the record of the same session ID as the session ID received in step S<b>13</b> to perform a process for participating in the communication session (step S<b>14</b>). Further, the storing and reading unit <b>59</b> reads, from the session management DB <b>5001</b>, the site ID associated with the session ID of the communication session for which the participation process is performed. Then, the storing and reading unit <b>59</b> searches the distribution-site management table (the distribution site management DB <b>5005</b>, see <figref idref="DRAWINGS">FIG. 18</figref>) using the site ID read in step S<b>14</b> as a search key to read the distribution site information associated with the same site ID as the read site ID (step S<b>15</b>). Then, the transmitting/receiving unit <b>51</b> transmits a participation request response to the communication terminal <b>70</b>C (step S<b>16</b>). Thus, the transmitting/receiving unit (acquisition unit) <b>71</b> of the communication terminal <b>70</b>C receives the participation request response. The participation request response includes the session ID received in step S<b>13</b>, the distribution site information read in step S<b>15</b>, and a participation process result. The participation process result includes the site ID read in step S<b>14</b>. In the following, a description will be given of a case in which the participation process is successful. Through a process similar to the process illustrated in <figref idref="DRAWINGS">FIG. 28</figref>, the communication terminal <b>70</b>D at the viewing site D performs a process for participating in a communication session.

Process for Managing Image Type Information

0159Next, a process for managing the image type information will be described with reference to <figref idref="DRAWINGS">FIG. 30</figref>. <figref idref="DRAWINGS">FIG. 30</figref> is a sequence diagram illustrating a process for managing image type information in the image communication system <b>1</b>.

0160First, when a user at the distribution site A connects the image capturing device <b>10</b>A to the distribution terminal <b>30</b>A, the storing and reading unit <b>19</b> of the image capturing device <b>10</b>A reads the GUID of the image capturing device <b>10</b>A from the storage unit <b>1000</b>. Then, the communication unit <b>11</b> of the image capturing device <b>10</b>A transmits the GUID of the image capturing device <b>10</b>A to the distribution terminal <b>30</b>A (step S<b>31</b>). Thus, the communication unit <b>37</b> of the distribution terminal <b>30</b>A receives the GUID of the image capturing device <b>10</b>A.

0161Then, the determination unit <b>35</b> of the distribution terminal <b>30</b>A determines whether the same vendor ID and product ID as the vendor ID and product ID in the GUID received in step S<b>31</b> are managed in the image capturing device management table (the image capturing device management DB <b>3001</b>, see <figref idref="DRAWINGS">FIG. 15A</figref>) to determine the image type (step S<b>32</b>). Specifically, if the same vendor ID and product ID are managed in the image capturing device management DB <b>3001</b>, the determination unit <b>35</b> determines that the image capturing device <b>10</b>A is configured to capture a special image (here, a spherical image). By contrast, if the same vendor ID and product ID are not managed in the image capturing device management table (the image capturing device management DB <b>3001</b>, see <figref idref="DRAWINGS">FIG. 15A</figref>), the determination unit <b>35</b> determines that the image capturing device <b>10</b>A is configured to capture a general image.

0162Then, the storing and reading unit <b>39</b> stores, in the image type management table (the image type management DB <b>3002</b>, see <figref idref="DRAWINGS">FIG. 15B</figref>), the IP address of the image capturing device <b>10</b>A and image type information indicating the determination result obtained in step S<b>32</b> in association with each other (step S<b>33</b>). In this state, no image data ID is associated with the IP address of the image capturing device <b>10</b>A and the image type information. The image type information is, for example, a source name determined in accordance with a predetermined naming rule or an image type (general image or special image).

0163Then, the transmitting/receiving unit <b>31</b> transmits to the communication management apparatus <b>50</b> an additional request for the image type information (step S<b>34</b>). The additional request for the image type information includes the IP address of the image capturing device <b>10</b>A and the image type information, which are stored in step S<b>33</b>, and the site ID of the distribution site A. Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the additional request for the image type information.

0164Then, the storing and reading unit <b>59</b> of the communication management apparatus <b>50</b> searches the session management table (the session management DB <b>5001</b>, see <figref idref="DRAWINGS">FIG. 16A</figref>) using the site ID received in step S<b>34</b> as a search key to read the corresponding session ID (step S<b>35</b>).

0165Then, the generation unit <b>56</b> generates a unique image data ID (step S<b>36</b>). Then, the storing and reading unit <b>59</b> stores, in the image type management table (the image type management DB <b>5002</b>, see <figref idref="DRAWINGS">FIG. 16B</figref>), the session ID read in step S<b>35</b>, the image data ID generated in step S<b>36</b>, and the IP address of the image capturing device <b>10</b>A and the image type information received in step S<b>34</b> in association with each other as a new record (step S<b>37</b>). Then, the transmitting/receiving unit <b>51</b> transmits the image data ID generated in step S<b>36</b> to the distribution terminal <b>30</b>A (step S<b>38</b>). Thus, the transmitting/receiving unit <b>31</b> of the distribution terminal <b>30</b>A receives the image data ID.

0166Then, the storing and reading unit <b>39</b> of the distribution terminal <b>30</b>A stores, in the image type management table (the image type management DB <b>3002</b>, see <figref idref="DRAWINGS">FIG. 15B</figref>), the image data ID received in step S<b>38</b> in association with the IP address of the image capturing device <b>10</b>A and the image type information stored in step S<b>33</b> (step S<b>39</b>).

0167On the other hand, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> transmits an image type information addition notification to the communication terminal <b>70</b>C (step S<b>40</b>). Thus, the transmitting/receiving unit (acquisition unit) <b>71</b> of the communication terminal <b>70</b>C receives the image type information addition notification. The image type information addition notification includes the image data ID generated in step S<b>36</b>, and the IP address of the image capturing device <b>10</b>A and the image type information stored in step S<b>37</b>.

0168Then, the storing and reading unit <b>79</b> of the communication terminal <b>70</b>C stores, in the image type management table (the image type management DB <b>7001</b>, see <figref idref="DRAWINGS">FIG. 20A</figref>), the image data ID, the IP address of the image capturing device <b>10</b>A, and the image type information, which are received in step S<b>40</b>, in association with each other as a new record (step S<b>41</b>). Accordingly, the distribution terminal <b>30</b>A and the communication terminal <b>70</b>C can share the same information in the image type management DBs <b>3002</b> and <b>7001</b>, respectively. The image type information addition notification is also transmitted to another communication terminal, namely, the communication terminal <b>70</b>D, and is stored in the image type management DB <b>7001</b> of the communication terminal <b>70</b>D.

Process for Communicating Captured Image Data

0169Next, a process for transmitting the captured image data and audio data obtained at the distribution site A to the communication terminals <b>70</b> (i.e., the communication terminals <b>70</b>C and <b>70</b>D) via the communication management apparatus <b>50</b> will be described with reference to <figref idref="DRAWINGS">FIG. 31</figref>. <figref idref="DRAWINGS">FIG. 31</figref> is a sequence diagram illustrating an example process for communicating captured image data and audio data in the image communication system <b>1</b>. <figref idref="DRAWINGS">FIG. 31</figref> illustrates an example in which captured image data acquired by one image capturing device <b>10</b> is distributed to the communication terminals <b>70</b>. However, similar processing is performed when a plurality of pieces of captured image data acquired by other image capturing devices <b>10</b> arranged in the distribution site are distributed.

0170First, the communication unit <b>11</b> of the image capturing device <b>10</b>A transmits to the distribution terminal <b>30</b>A captured image data acquired by capturing an object or surroundings such as scenery and audio data acquired by collecting sounds (step S<b>51</b>). Thus, the communication unit <b>37</b> of the distribution terminal <b>30</b>A receives the captured image data and the audio data. In this case, since the image capturing device <b>10</b>A is capable of obtaining two hemispherical images from which a spherical image is generated, as illustrated in <figref idref="DRAWINGS">FIGS. 3A and 3B</figref>, the captured image data includes data of two hemispherical images.

0171Then, the transmitting/receiving unit <b>31</b> of the distribution terminal <b>30</b>A transmits to the communication management apparatus <b>50</b> the captured image data and the audio data sent from the image capturing device <b>10</b>A (step S<b>52</b>). Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the captured image data, the audio data, and the image data ID. An image data ID for identifying the captured image data to be transmitted and received is also transmitted and received.

0172Then, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> transmits the captured image data and the audio data to the communication terminals (the communication terminals <b>70</b>C and <b>70</b>D) participating in the same session as the session in which the distribution terminal <b>30</b>A is participating (steps S<b>53</b> and S<b>54</b>). Thus, the transmitting/receiving unit (acquisition unit) <b>71</b> of each of the communication terminals <b>70</b>C and <b>70</b>D receives the captured image data and the audio data. The image data ID for identifying the captured image data to be transmitted and received is also transmitted and received.

Process for Displaying Point of Interest P

0173Next, a process for displaying an image of the point of interest P in a distribution site, which is designated by a user at a viewing site, will be described with reference to <figref idref="DRAWINGS">FIGS. 32 to 36</figref>. <figref idref="DRAWINGS">FIG. 32</figref> is a sequence diagram illustrating an example process for displaying the point of interest P in the image communication system <b>1</b>. <figref idref="DRAWINGS">FIG. 32</figref> illustrates an example in which the communication terminal <b>70</b>C at the viewing site C displays a captured image distributed from the distribution terminal <b>30</b>A. Similar processing is performed when the captured image is displayed on the communication terminal <b>70</b>D at the viewing site D, which is another viewing site.

0174First, the display control unit <b>74</b> of the communication terminal <b>70</b>C uses the distribution site information received in step S<b>16</b> to cause the display <b>706</b> to display a site display screen <b>600</b> indicating the situation of the distribution site A (step S<b>71</b>). <figref idref="DRAWINGS">FIG. 34</figref> is a view illustrating an example site display screen displayed on the communication terminal <b>70</b>C. The site display screen <b>600</b> illustrated in <figref idref="DRAWINGS">FIG. 34</figref> displays a map image indicating the situation of a distribution site.

0175The site display screen <b>600</b> illustrated in <figref idref="DRAWINGS">FIG. 34</figref> includes a tour image <b>610</b> for viewing a spherical image of the distribution site to enable remote viewers to see the distribution site, a point-of-view change icon <b>630</b> for changing the point of view to a predetermined image capturing point in the tour image <b>610</b>, a cross (or a star or any other suitable mark) <b>635</b> indicating the point of interest P, a schematic image <b>650</b> indicating a schematic diagram of the distribution site, the “point of interest” button <b>605</b>, a “cancel” button <b>609</b>, and the “number of divisions” button <b>670</b>. The “point of interest” button <b>605</b> is operated to display the point of interest P in the distribution site. The “cancel” button <b>609</b> is operated to terminate the viewing of the distribution site. The “number of divisions” button <b>670</b> is used to divide an area of the distribution site centered on the point of interest P in the distribution site (from the point of interest P). The “number of divisions” button <b>670</b> is operated to display a number-of-divisions setting screen <b>675</b>, which is a pop-up window in this example, near the “number of divisions” button <b>670</b>. The number-of-divisions setting screen <b>675</b> includes an input field that follows “Number of divisions=” to enter the number of divisions. The user enters the desired value (e.g., 2, 3, or the like) in the input field and then operates an “OK” button. The user can operate a “cancel” button to change or remove the entered value.

0176The tour image <b>610</b> and the schematic image <b>650</b> are displayed using the map image data included in the distribution site information received in step S<b>16</b>. The tour image <b>610</b> is an image of the distribution site, which is captured by the image capturing device <b>10</b> in advance. The user C<b>1</b> can operate the point-of-view change icon <b>630</b> to understand the general arrangement and the like of the distribution site. The tour image <b>610</b> is described herein as being displayed using the map image data included in the distribution site information received in step S<b>16</b>. However, the tour image <b>610</b> may be configured to display captured image data of the distribution site, which is received in real time.

0177The cross <b>635</b> in the schematic image <b>650</b> functions as a point-of-view position icon indicating the current display position of the tour image <b>610</b>. For example, the user C<b>1</b> operates the point-of-view change icon <b>630</b> using a pointer p<b>1</b> while viewing the tour image <b>610</b> and the schematic image <b>650</b>. As a result, the user C<b>1</b> is able to view a desired portion in the distribution site.

0178Then, the user C<b>1</b> operates the point-of-view change icon <b>630</b> using the pointer p<b>1</b> and operates the “point of interest” button <b>605</b> such that the acceptance unit <b>72</b> accepts the designation (input) of the point of interest P (step S<b>72</b>). The acceptance unit <b>72</b> may accept the designation of the point of interest P in response to a click or double-click operation of the user C<b>1</b> using a mouse, which is an example of an input device.

0179The user C<b>1</b> further operates the pointing device <b>712</b> to operate the “number of divisions” button <b>670</b>. As a result, the display control unit <b>74</b> displays the number-of-divisions setting screen <b>675</b> on the site display screen <b>600</b>. In response to the display of the number-of-divisions setting screen <b>675</b>, the user C<b>1</b> enters the number of divisions n (e.g., 2, 3, or the like) and then operates the “OK” button such that the acceptance unit <b>72</b> accepts the input of the number of divisions (step S<b>73</b>).

0180Then, the point-of-interest specifying unit <b>77</b> specifies point-of-interest coordinates, which are the coordinates of the point of interest P for which the designation is accepted in step S<b>72</b> (step S<b>74</b>). The point-of-interest coordinates specified by the point-of-interest specifying unit <b>77</b> are the coordinates of the central point of the tour image <b>610</b> being displayed when the designation of the point of interest P is accepted in step S<b>72</b>. The point-of-interest coordinates may be coordinates indicating the position of the pointer p<b>1</b> in the tour image <b>610</b>.

0181The transmitting/receiving unit (acquisition unit) <b>71</b> transmits to the communication management apparatus <b>50</b> an image-capturing-device selection request indicating a request for selecting an image capturing device <b>10</b> (step S<b>75</b>). Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the image-capturing-device selection request transmitted from the communication terminal <b>70</b>C. The image-capturing-device selection request includes the point-of-interest coordinates specified in step S<b>74</b>, the number of divisions n received in step S<b>73</b>, and the site ID received in step S<b>16</b>.

Process for Selecting Image Capturing Device

0182Then, the communication management apparatus <b>50</b> executes a process for selecting an image capturing device <b>10</b> in response to the image-capturing-device selection request received in step S<b>75</b> (step S<b>76</b>). The process in step S<b>76</b> will be described in detail with reference to <figref idref="DRAWINGS">FIGS. 33 and 21 to 26</figref>. <figref idref="DRAWINGS">FIG. 33</figref> is a flowchart illustrating an example process for selecting an image capturing device.

0183First, the storing and reading unit <b>59</b> searches the distribution-site management table (the distribution site management DB <b>5005</b>, see <figref idref="DRAWINGS">FIG. 18</figref>) using the site ID received in step S<b>75</b> as a search key to read the distribution site information associated with the same site ID as the received site ID (step S<b>101</b>).

0184Then, the selection unit <b>53</b> divides the distribution site centered on the point of interest P (predetermined position) into a plurality of areas (from the point of interest P), based on the point-of-interest coordinates and the number of divisions n both received in step S<b>75</b> (step S<b>102</b>). Specifically, the selection unit <b>53</b> divides the entire area of the distribution site into two areas using the received point-of-interest coordinates as a starting point, based on the site coordinate information included in the distribution site information read in step S<b>101</b> and the number-of-division information (the number of divisions n) accepted in step S<b>73</b>. In the example illustrated in <figref idref="DRAWINGS">FIG. 21</figref>, the point of interest P designated by the user C<b>1</b> is indicated by a black circle.

0185Then, the storing and reading unit <b>59</b> searches the arrangement information management table (the arrangement information management DB <b>5004</b>, see <figref idref="DRAWINGS">FIG. 17B</figref>) using the site ID received in step S<b>75</b> as a search key to read the arrangement information associated with the same site ID as the received site ID (step S<b>103</b>).

0186Then, the selection unit <b>53</b> selects image capturing devices <b>10</b> arranged near the point-of-interest coordinates indicating the coordinates of the point of interest P, based on the coordinate values indicated by the arrangement information read in step S<b>103</b> (step S<b>104</b>). Specifically, the selection unit <b>53</b> extends a virtual line segment connecting the point of interest P (predetermined position) and the image capturing device <b>10</b>A-<b>1</b> to divide the two-dimensional arrangement image generated by the generation unit <b>56</b> into two areas. To draw a circle having the radius r, the selection unit <b>53</b> sets the value of the radius r such that a circle having the radius r includes at least two image capturing devices <b>10</b> that are relatively close to the point of interest P. To set the value of the radius r, the selection unit <b>53</b> can calculate, from the coordinate values of the point of interest P and the respective coordinate values of the image capturing devices <b>10</b> that are arranged near the point of interest P, the distance between each of the image capturing devices <b>10</b> and the point of interest P. Then, the selection unit <b>53</b> selects the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> as in-range image capturing devices <b>10</b> arranged near the point-of-interest coordinates indicating the coordinates of the point of interest P.

0187In this embodiment, the image capturing device <b>10</b>A-<b>1</b>, which is arranged at the position closest to the point of interest P and selected in the processing of step S<b>104</b>, is referred to as “reference image capturing device”. The reference image capturing device <b>10</b>A-<b>1</b> functions as, or has means for functioning as, an example of a first image capturing device.

0188Then, the selection unit <b>53</b> executes a process for selecting a supplementary image capturing device from among the in-range image capturing devices <b>10</b> (step S<b>105</b>). The supplementary image capturing device is capable of capturing a portion of an object at the point of interest P for which an image is difficult for the reference image capturing device <b>10</b>A-<b>1</b> to satisfactorily capture. The supplementary image capturing device functions as an example of a second image capturing device.

Process for Selecting Supplementary Image Capturing Device

0189<figref idref="DRAWINGS">FIG. 35</figref> is a flowchart illustrating an example process for selecting a supplementary image capturing device. First, the selection unit <b>53</b> draws a predetermined range (e.g., a circle having a radius r) centered on the point of interest P. Then, the selection unit <b>53</b> selects, among the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> arranged in the circle having the radius r, the image capturing device <b>10</b>A-<b>1</b>, which is arranged at the position closest to the point of interest P (step S<b>105</b>-<b>1</b>). At this time, in the example illustrated in <figref idref="DRAWINGS">FIG. 21</figref>, the selection unit <b>53</b> calculates the distance between the coordinate values of each of the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> and the point-of-interest coordinates. Then, the selection unit <b>53</b> determines that the image capturing device <b>10</b>A-<b>1</b>, which is arranged at the position closest to the coordinates of the point of interest P (or has the shortest distance from the point of interest P), is the reference image capturing device <b>10</b>A-<b>1</b> that is closest to the point of interest P in the predetermined range, and holds the identification information of the image capturing device <b>10</b>A-<b>1</b> (e.g., the IP address of the image capturing device <b>10</b>A-<b>1</b>) and the calculated distance in the storage unit <b>5000</b>. The distance between the coordinate values of each of the plurality of image capturing devices <b>10</b>A and the point-of-interest coordinates is calculated using, for example, any desired known method for determining the distance in a straight line connecting the coordinates of each of the plurality of image capturing devices <b>10</b>A and the coordinates of the point of interest P.

0190Then, the selection unit <b>53</b> calculates the angle between the reference image capturing device <b>10</b>A-<b>1</b> and another image capturing device near the point of interest P (step S<b>105</b>-<b>2</b>). Specifically, the selection unit <b>53</b> searches the arrangement information management table (the arrangement information management DB <b>5004</b>, see <figref idref="DRAWINGS">FIG. 17B</figref>) using the IP address of each image capturing device as a search key to read the coordinate values of the corresponding image capturing device. Then, the selection unit <b>53</b> calculates angles between lines connecting the respective image capturing devices and the point of interest P. As a result, in the example illustrated in <figref idref="DRAWINGS">FIG. 22</figref> described above, the angle (azimuth angle) at the point of interest P between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>2</b> is 101 degrees. The angle at the point of interest P between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>3</b> is 121 degrees. The angle at the point of interest P between the image capturing devices <b>10</b>A-<b>1</b> and <b>10</b>A-<b>4</b> is 163 degrees. The selection unit <b>53</b> holds the calculated angles in the storage unit <b>5000</b> in a manner similar to that for the distances calculated in the processing of step S<b>105</b>-<b>2</b>. The selection unit <b>53</b> can determine these angles by using, for example, the inverse trigonometric functions for the respective coordinate values. The distribution site A (area A) is divided into two areas by a thick broken line obtained by extending a virtual line segment connecting the point of interest P and the image capturing device <b>10</b>A-<b>1</b>.

0191After calculating the angles between the image capturing devices in step S<b>105</b>-<b>2</b>, the selection unit <b>53</b> selects the following image capturing devices as candidate supplementary image capturing devices (second image capturing devices) that perform a supplementary operation to capture an image of a portion that is difficult for the reference image capturing device to capture (step S<b>105</b>-<b>3</b>). Specifically, in a case in which the number of divisions is n, where n is a natural number greater than or equal to 2, the selection unit <b>53</b> selects, as a candidate supplementary image capturing device (second image capturing device), an image capturing device arranged at the angle closest to an angle obtained by dividing 360 degrees by the number of divisions n from a virtual line segment connecting the reference image capturing device (first image capturing device) and the point of interest P.

0000Case in which Number of Divisions is Two

0192Specifically, as illustrated in <figref idref="DRAWINGS">FIG. 22</figref>, the selection unit <b>53</b> selects an image capturing device arranged such that the angle between the image capturing device and the reference image capturing device <b>10</b>A-<b>1</b> is closest to an angle obtained by 360°/n, that is, 360°/2=180°, as a candidate supplementary image capturing device to the reference image capturing device <b>10</b>A-<b>1</b>. That is, the selection unit <b>53</b> selects an image capturing device arranged such that the angle between the image capturing device and the reference image capturing device <b>10</b>A-<b>1</b> is closest to an angle of 180° as a candidate supplementary image capturing device. In this case, images captured by the reference image capturing device <b>10</b>A-<b>1</b> and the supplementary image capturing device <b>10</b>A-<b>4</b> can cover, for example, the most sides (different aspects) of the point of interest P. Thus, the selection unit <b>53</b> selects the image capturing device <b>10</b>A-<b>4</b> as a candidate supplementary image capturing device to the reference image capturing device <b>10</b>A-<b>1</b>. However, the selection unit <b>53</b> may use an angle of 180 degrees with some margin (θ) as a determination criterion for selecting a supplementary image capturing device. That is, the selection unit <b>53</b> may select an image capturing device arranged such that the angle between the image capturing device and the reference image capturing device <b>10</b>A-<b>1</b> is closest to an angle of 180°±θ as a candidate supplementary image capturing device.

0193In the following, a description will be given of a specific example of coefficient computation using a data table and selection of a supplementary image capturing device. As in the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>) and the specific weighting coefficient management table (the specific weighting coefficient management DB <b>5007</b>, see <figref idref="DRAWINGS">FIG. 19B</figref>) described below, the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> are represented by CAM<b>1</b>, CAM<b>2</b>, CAM<b>3</b>, and CAM<b>4</b>, respectively, for convenience of description.

0194In a case in which four image capturing devices, CAM<b>1</b>, CAM<b>2</b>, CAM<b>3</b>, and CAM<b>4</b>, are arranged in a circle having a radius r centered on the point of interest P illustrated in <figref idref="DRAWINGS">FIG. 22</figref>, the selection unit <b>53</b> selects the following combination of two image capturing devices among the four image capturing devices. At this time, the angles at the point of interest P between the image capturing devices are as follows.

00001. Case in which the selection unit <b>53</b> selects CAM<b>1</b> and CAM<b>2</b>:

0195Angle between CAM<b>1</b> and CAM<b>2</b>=101 degrees

00002. Case in which the selection unit <b>53</b> selects CAM<b>1</b> and CAM<b>3</b>:

0196Angle between CAM<b>1</b> and CAM<b>3</b>=121 degrees

00003. Case in which the selection unit <b>53</b> selects CAM<b>1</b> and CAM<b>4</b>:

0197Angle between CAM<b>1</b> and CAM<b>4</b>=163 degrees

0198The optimum positions of two image capturing devices that can view (capture) the point of interest P from different aspects are obtained when the angle at the point of interest P between the two image capturing devices is 180 degrees. Accordingly, the selection unit <b>53</b> calculates the absolute value of the difference between 180 degrees and the angle at the point of interest P between each pair of image capturing devices and selects the combination of image capturing devices for which the absolute value is smallest. The angles between the pairs of image capturing devices illustrated in <figref idref="DRAWINGS">FIG. 22</figref> are as follows:

0199∥01−180|=79 in the case 1;

0200|121−180|=59 in the case 2; and

0201|163−180|=17 in the case 3.

0202The combination of two image capturing devices between which the absolute value of the difference is smallest is obtained as image capturing devices arranged such that the angle at the point of interest P therebetween is closest to 180°±θ. Accordingly, the selection unit <b>53</b> selects CAM<b>4</b> in the case 3 as a candidate supplementary image capturing device to CAM<b>1</b>, which is the reference image capturing device.

0203In this state, the selection unit <b>53</b> performs coefficient computation using at least one data table among the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>) and the specific weighting coefficient management table (the specific weighting coefficient management DB <b>5007</b>, see <figref idref="DRAWINGS">FIG. 19B</figref>) (step S<b>105</b>-<b>4</b>). That is, the selection unit <b>53</b> determines the image capturing device <b>10</b>A-<b>4</b> (CAM<b>4</b>) as a candidate supplementary image capturing device and thereafter performs the following three computations including, for example, the reference image capturing device <b>10</b>A-<b>1</b> (CAM<b>1</b>).

02041. |101−180|*1.0=79

02052. |121−180|*1.0=59 (In this case, however, since an obstacle such as a wall is absent in front of CAM<b>3</b>, computation is performed with the coefficient in the data table being set to 1.0)

02063. |163−180|*1.0=17

0207Accordingly, also in the processing of step S<b>105</b>-<b>4</b>, the selection unit <b>53</b> determines the combination for which the computation result in the case 3 in which the angle between CAM<b>1</b> and CAM<b>4</b> is 1630 has the smallest value, and determines that a supplementary image capturing device to the image capturing device <b>10</b>A-<b>1</b> (CAM<b>1</b>) serving as the reference image capturing device is the image capturing device <b>10</b>A-<b>4</b> (CAM<b>4</b>). Then, the selection unit <b>53</b> exits the flowchart.

0000Case in which Number of Divisions is Three

0208Next, as illustrated in <figref idref="DRAWINGS">FIG. 23</figref>, a case in which the number of divisions is three will be described. The selection unit <b>53</b> selects an image capturing device arranged such that the angle between the image capturing device and the reference image capturing device <b>10</b>A-<b>1</b> is closest to an angle obtained by 360°/n, that is, 360°/3=120°, as a candidate supplementary image capturing device to the reference image capturing device <b>10</b>A-<b>1</b> (step S<b>105</b>-<b>3</b>). However, the selection unit <b>53</b> may use an angle of 120 degrees with some margin (θ) as a determination criterion for selecting a supplementary image capturing device. That is, the selection unit <b>53</b> may select an image capturing device arranged such that the angle between the image capturing device and the reference image capturing device <b>10</b>A-<b>1</b> is closest to an angle of 120°±θ as a candidate supplementary image capturing device. As described above, in a case in which the number of divisions is three, the selection unit <b>53</b> selects two image capturing devices as candidate supplementary image capturing devices to the reference image capturing device <b>10</b>A-<b>1</b>.

0209In the following, a description will be given of a specific example of coefficient computation using a data table and selection of a supplementary image capturing device. As in the case in which the number of divisions is two, the image capturing devices <b>10</b>A-<b>1</b>, <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b> are represented by CAM<b>1</b>, CAM<b>2</b>, CAM<b>3</b>, and CAM<b>4</b>, respectively, for convenience of description.

0210In a case in which four image capturing devices, CAM<b>1</b>, CAM<b>2</b>, CAM<b>3</b>, and CAM<b>4</b>, are arranged in a circle having a radius r centered on the point of interest P illustrated in <figref idref="DRAWINGS">FIG. 23</figref>, the selection unit <b>53</b> selects the following combination of three image capturing devices among the four image capturing devices. At this time, the angles at the point of interest P between the image capturing devices are as follows.

00001. Case in which the selection unit <b>53</b> selects CAM<b>1</b>, CAM<b>2</b>, and CAM<b>3</b>:

0211Angle between CAM<b>1</b> and CAM<b>2</b>=101 degrees;

0212Angle between CAM<b>2</b> and CAM<b>3</b>=138 degrees; and

0213Angle between CAM<b>3</b> and CAM<b>1</b>=121 degrees.

00002. Case in which the selection unit <b>53</b> selects CAM<b>1</b>, CAM<b>2</b>, and CAM<b>4</b>:

0214Angle between CAM<b>1</b> and CAM<b>2</b>=101 degrees;

0215Angle between CAM<b>2</b> and CAM<b>4</b>=71 degrees; and

0216Angle between CAM<b>4</b> and CAM<b>1</b>=188 degrees.

00003. Case in which the selection unit <b>53</b> selects CAM<b>1</b>, CAM<b>3</b>, and CAM<b>4</b>:

0217Angle between CAM<b>1</b> and CAM<b>3</b>=121 degrees;

0218Angle between CAM<b>3</b> and CAM<b>4</b>=67 degrees; and

0219Angle between CAM<b>4</b> and CAM<b>1</b>=172 degrees.

0220The optimum positions of three image capturing devices that can view (capture) the point of interest P from different aspects are obtained when the angle at the point of interest P between the three image capturing devices is 120 degrees. Accordingly, the selection unit <b>53</b> calculates the absolute value of the difference between 120 degrees and the angle at the point of interest P between the three image capturing devices and selects a combination of image capturing devices for which the absolute value is smallest. The angles between the image capturing devices illustrated in <figref idref="DRAWINGS">FIG. 24</figref> are as follows:

0221|101−120|+|138−120|+|121−120|=38 in the case 1;

0222|101−120|+|71−120|+|188−120|=136 in the case 2; and

0223|121−120|+|67−120|+|72−120|=106 in the case 3.

0224The combination of three image capturing devices between which the absolute value of the difference is smallest is obtained as image capturing devices arranged such that the angle at the point of interest P therebetween is closest to 120°±θ. Accordingly, the selection unit <b>53</b> selects CAM<b>2</b> and CAM<b>3</b> in the case 1 as candidate supplementary image capturing devices to CAM<b>1</b>, which is the reference image capturing device.

0225In this state, as in the case in which the number of divisions is two, the selection unit <b>53</b> performs coefficient computation using at least one data table among the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>) and the specific weighting coefficient management table (the specific weighting coefficient management DB <b>5007</b>, see <figref idref="DRAWINGS">FIG. 19B</figref>) (step S<b>105</b>-<b>4</b>). That is, the selection unit <b>53</b> determines the image capturing device <b>10</b>A-<b>2</b> (CAM<b>2</b>) and the image capturing device <b>10</b>A-<b>3</b> (CAM<b>3</b>) as candidate supplementary image capturing devices and thereafter performs the following computation.

0000Case in which Obstacle is Present in Front of Image Capturing Device

0226In a case in which an obstacle or the like is present between the point of interest P and an image capturing device when a supplementary image capturing device is to be selected, the selection unit <b>53</b> performs calculation using a predetermined coefficient so that a combination including the image capturing device is not likely to be selected. As illustrated in <figref idref="DRAWINGS">FIG. 25</figref>, it is assumed that a wall is present between the image capturing device <b>10</b>A-<b>3</b> and the point of interest P and that it is difficult for the image capturing device <b>10</b>A-<b>3</b> to capture an image of the point of interest P.

0227In this case, the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>) is used for each term of the calculation equation described above. A coefficient in the weighting coefficient management table can be set to a desired value in advance by the administrator who manages the communication management apparatus <b>50</b> or the like. The administrator or the like checks the plurality of image capturing devices arranged at the distribution site. Upon determining that a wall, an obstacle, or the like is present near an image capturing device, the administrator or the like sets a coefficient for the corresponding image capturing device to a larger value than coefficients for the other image capturing devices. As a result, the selection unit <b>53</b> performs a process for removing an image capturing device hidden by the wall, the obstacle, or the like among the in-range image capturing devices from candidate supplementary image capturing devices. This further enables the user who uses the communication terminal <b>70</b> to easily observe an object present at the point of interest P. In addition, upon detection of fault such as malfunctioning of an image capturing device among the in-range image capturing devices, the selection unit <b>53</b> can remove the image capturing device from candidate supplementary image capturing devices. The reason for this is that the selection unit <b>53</b> selects a supplementary image capturing device having a small calculation result based on the computation equation described above.

0228The calculation results of the image capturing devices illustrated in <figref idref="DRAWINGS">FIG. 25</figref> are as follows.

0000<maths id="MATH-US-00002" num="00002"><math overflow="scroll"><mrow><mrow><mrow><mn>1.</mn><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mrow><mo></mo><mrow><mn>101</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn></mrow><mo>+</mo><mrow><mrow><mo></mo><mrow><mn>138</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>10.0</mn></mrow><mo>+</mo><mrow><mrow><mo></mo><mrow><mn>121</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>10.0</mn></mrow></mrow><mo>=</mo><mn>2.09</mn></mrow></math></maths><maths id="MATH-US-00002-2" num="00002.2"><math overflow="scroll"><mrow><mrow><mrow><mn>2.</mn><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mrow><mo></mo><mrow><mn>101</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn></mrow><mo>+</mo><mrow><mrow><mo></mo><mrow><mn>71</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn><mo>*</mo><mrow><mo></mo><mrow><mn>188</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn></mrow></mrow><mo>=</mo><mn>136</mn></mrow></math></maths><maths id="MATH-US-00002-3" num="00002.3"><math overflow="scroll"><mrow><mrow><mrow><mrow><mn>3.</mn><mo>⁢</mo><mstyle><mspace width="0.8em" height="0.8ex" /></mstyle><mo>⁢</mo><mrow><mo></mo><mrow><mn>121</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>10.0</mn></mrow><mo>+</mo><mrow><mrow><mo></mo><mrow><mn>67</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn></mrow><mo>+</mo><mrow><mrow><mo></mo><mrow><mn>172</mn><mo>-</mo><mn>120</mn></mrow><mo></mo></mrow><mo>*</mo><mn>1.0</mn></mrow></mrow><mo>=</mo><mn>115</mn></mrow><mo>⁢</mo><mstyle><mspace width="0.3em" height="0.3ex" /></mstyle></mrow></math></maths>

0229As a result of the calculation, the selection unit <b>53</b> selects the combination of CAM<b>1</b>, CAM<b>3</b>, and CAM<b>4</b> in the case 3 having the smallest value of the calculation results. However, a wall is present in front of CAM<b>3</b>, which is not suitable for practical use. Thus, the selection unit <b>53</b> cooperates with the determination unit <b>55</b> and performs the following determination and selection. In this case, the selection unit <b>53</b> selects the combination having the second smallest value of the calculation results. That is, the selection unit <b>53</b> selects CAM<b>1</b> (the image capturing device <b>10</b>A-<b>1</b>) as the reference image capturing device and CAM<b>2</b> (the image capturing device <b>10</b>A-<b>2</b>) and CAM<b>4</b> (the image capturing device <b>10</b>A-<b>4</b>) as candidate supplementary image capturing devices. As a result, CAM<b>3</b> (the image capturing device <b>10</b>A-<b>3</b>) hidden by the obstacle or the like is not selected, and an object present at the point of interest P is easily observed. However, as in the example illustrated in <figref idref="DRAWINGS">FIG. 25</figref>, even when CAM<b>4</b> (the image capturing device <b>10</b>A-<b>4</b>) is selected as a candidate supplementary image capturing device, it may still be difficult to capture images that cover the most sides (different aspects) of the point of interest P. In this case, the selection unit <b>53</b> may stop the selection of a supplementary image capturing device or provide a message or the like that prompts the user who uses the communication terminal <b>70</b> to stop the selection process or designate a different point of interest.

0000Case in which Number of Divisions is Four

0230An example in a case in which the number of divisions n is four is illustrated in <figref idref="DRAWINGS">FIG. 26</figref>. In the illustrated example, the selection unit <b>53</b> selects the image capturing device <b>10</b> arranged at the position closest to a central angle of 90°±θ centered on the point of interest P in a circle having a radius r centered on the point of interest P as a supplementary image capturing device that is supplementary to the reference image capturing device <b>10</b>A-<b>1</b>. In the example illustrated in <figref idref="DRAWINGS">FIG. 26</figref>, three supplementary image capturing devices, namely, the image capturing devices <b>10</b>A-<b>2</b>, <b>10</b>A-<b>3</b>, and <b>10</b>A-<b>4</b>, are selected.

0000Case in which Number of Divisions is Five or More

0231Also in a case in which the number of divisions is five or more, the selection unit <b>53</b> selects, as a candidate supplementary image capturing device (second image capturing device), an image capturing device arranged at the angle closest to an angle obtained by dividing 360 degrees by the number of divisions n from a virtual line segment connecting the reference image capturing device (first image capturing device) and the point of interest P (step S<b>105</b>-<b>3</b>). Thereafter, the selection unit <b>53</b> can perform coefficient computation using at least one data table among the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>) and the specific weighting coefficient management table (the specific weighting coefficient management DB <b>5007</b>, see <figref idref="DRAWINGS">FIG. 19B</figref>) (step S<b>105</b>-<b>4</b>), and select a reference image capturing device and a supplementary image capturing device that is supplementary to the reference image capturing device.

0232The processing of steps S<b>105</b>-<b>3</b> and S<b>105</b>-<b>4</b> described above may be performed in reverse order by the communication management apparatus <b>50</b>. That is, after coefficient computation is executed using the data table described in step S<b>105</b>-<b>4</b>, a process may be executed in step S<b>105</b>-<b>3</b> for selecting an image capturing device (supplementary image capturing device) arranged such that the angle between the image capturing device (supplementary image capturing device) and the reference image capturing device among the selected image capturing devices is closest to 360°/n. However, if an obstacle such as a wall is present near the selected supplementary image capturing device or if the selected supplementary image capturing device is malfunctioning, the selection unit <b>53</b> and the determination unit <b>55</b> may remove such an image capturing device from candidate supplementary image capturing devices and determine whether another image capturing device is available as an alternative candidate supplementary image capturing device.

0233Instead of the weighting coefficient management table (the weighting coefficient management DB <b>5006</b>, see <figref idref="DRAWINGS">FIG. 19A</figref>), the communication management apparatus <b>50</b> may manage the specific weighting coefficient management table (the specific weighting coefficient management DB <b>5007</b>, see <figref idref="DRAWINGS">FIG. 19B</figref>) in which a combination of specific image capturing devices is removed, and perform calculation with a coefficient set to 1.0 for a combination of a reference image capturing device and a supplementary image capturing device that is not managed in the specific weighting coefficient management table. This method only requires the administrator who manages the communication management apparatus <b>50</b> to set and manage a specific weighting coefficient for an image capturing device for which setting is to be performed. As a result, the administrator can save their work at the distribution site or the like where a large number of image capturing devices are arranged for previous setting of the weighting coefficient in these image capturing devices.

0234The process for selecting a reference image capturing device and a supplementary image capturing device is also applicable to a condition, as illustrated in <figref idref="DRAWINGS">FIG. 27</figref>, in which the image capturing devices <b>10</b> are arranged at irregular intervals in the distribution site.

0235Referring back to <figref idref="DRAWINGS">FIG. 33</figref>, the generation unit <b>56</b> generates, for each of the image capturing devices <b>10</b>A selected in step S<b>104</b>, predetermined-area information corresponding to the point-of-interest coordinates received in step S<b>75</b> (step S<b>106</b>). Specifically, the generation unit <b>56</b> generates predetermined-area information for displaying a predetermined-area image (an image in perspective projection) such that the point of interest indicated by the point-of-interest coordinates is located at the center of a spherical image captured by each of the selected image capturing devices <b>10</b>A. It is assumed here that the radial distance (r) and the polar angle (θ) have predetermined values set in advance. The use of the predetermined values indicates that the image in perspective projection is processed with the angle of view and the elevation angle thereof being constant. The azimuth angle (ϕ) can be calculated as a relative value between the arrangement direction of the image capturing device <b>10</b>A indicated in the arrangement information and the direction from the image capturing device <b>10</b>A to the point of interest indicated by the point-of-interest coordinates. Then, the generation unit <b>56</b> generates, based on the calculated results, predetermined-area information for displaying a predetermined-area image (displaying an image in perspective projection) centered on the point-of-interest coordinates from a spherical image acquired by the image capturing device <b>10</b>A. The generation unit <b>56</b> executes the process described above on each of the image capturing devices <b>10</b>A selected in step S<b>104</b>. The arrangement position (coordinate values) and the arrangement direction of the image capturing device <b>10</b>A (the direction in which the front surface of the image capturing device <b>10</b>A faces) are set in advance in the arrangement information management DB <b>5004</b> by the administrator or the like. In the processing of step S<b>106</b> described above, to obtain a combination of the IP address of an image capturing device and predetermined-area information, the storing and reading unit <b>59</b> and the selection unit <b>53</b> use the predetermined-area management table (the predetermined-area management DB <b>5003</b>, see <figref idref="DRAWINGS">FIG. 17A</figref>) and the arrangement information management table (the arrangement information management DB <b>5004</b>, see <figref idref="DRAWINGS">FIG. 17B</figref>), which are managed by the communication management apparatus <b>50</b>, to generate corresponding predetermined-area information.

0236Then, the storing and reading unit <b>59</b> stores the predetermined-area information generated in step S<b>106</b> in the predetermined-area management table (the predetermined-area management DB <b>5003</b>, see <figref idref="DRAWINGS">FIG. 17A</figref>) in association with the IP address of the corresponding image capturing device <b>10</b>A (step S<b>107</b>).

0237As described above, the communication management apparatus <b>50</b> can select an image capturing device <b>10</b> that captures an image of the point of interest P, based on the point-of-interest coordinates transmitted from the communication terminal <b>70</b> and the arrangement information of the plurality of image capturing devices <b>10</b>, and calculate the angle of view of the selected image capturing device <b>10</b>A to display the point of interest P.

0238In this embodiment, the generation unit <b>56</b> functions as, or has means for functioning as, an example of generation means. Further, the generation unit <b>56</b> that generates predetermined-area information may be included in, instead of the communication management apparatus <b>50</b>, another apparatus disposed between the communication management apparatus <b>50</b> and the communication terminal <b>70</b>.

0239Referring back to <figref idref="DRAWINGS">FIG. 32</figref>, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> transmits to the communication terminal <b>70</b>C a selection result notification that is a result of the process in step S<b>76</b> (step S<b>77</b>). Thus, the transmitting/receiving unit (acquisition unit) <b>71</b> of the communication terminal <b>70</b>C receives the selection result notification transmitted from the communication management apparatus <b>50</b>. The selection result notification includes sets of pieces of predetermined-area information generated in step S<b>106</b> and IP addresses of image capturing devices <b>10</b>, the number of which corresponds to the number of pieces of predetermined-area information generated in step S<b>106</b>. As described above, the transmitting/receiving unit (acquisition unit) <b>71</b> of the communication terminal <b>70</b>C acquires predetermined-area information indicating a predetermined area including the point of interest P for which the designation is accepted in step S<b>72</b>.

0240As described above, the transmitting/receiving unit <b>51</b> transmits to the communication terminal <b>70</b> predetermined-area information (first predetermined-area information) indicating a predetermined area including a predetermined position in an image captured by the image capturing device <b>10</b>A-<b>1</b>, which is the reference image capturing device <b>10</b>A-<b>1</b> (first image capturing device) arranged at the shortest distance from the point of interest P (predetermined position) among a plurality of image capturing devices arranged in an area of a distribution site. In addition, the transmitting/receiving unit <b>51</b> transmits to the communication terminal <b>70</b> predetermined-area information (second predetermined-area information) indicating a predetermined area including the predetermined position in an image captured by a supplementary image capturing device, which is a supplementary image capturing device (second image capturing device) arranged at the angle closest to an angle obtained by dividing the area of the distribution site in accordance with the number of divisions indicated in number-of-division information. In a case in which the number of divisions is two, in the example illustrated in <figref idref="DRAWINGS">FIG. 21</figref>, the image capturing device <b>10</b>A-<b>4</b> is a supplementary image capturing device (second image capturing device) arranged at the angle closest to the angle obtained by dividing the area of the distribution site in accordance with the number of divisions. In a case in which the number of divisions is three, in the example illustrated in <figref idref="DRAWINGS">FIG. 23</figref>, the image capturing devices <b>10</b>A-<b>2</b> and <b>10</b>A-<b>3</b> are supplementary image capturing devices (second image capturing devices) arranged at the angle closest to the angle obtained by dividing the area of the distribution site in accordance with the number of divisions.

0241The transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> can transmit the first predetermined-area information and the second predetermined-area information to the communication terminal <b>70</b> at the same time or different times. When the transmitting/receiving unit <b>51</b> transmits the first predetermined-area information and the second predetermined-area information to the communication terminal <b>70</b> at different times, the first predetermined-area information and the second predetermined-area information may be transmitted in any order.

0242Then, the storing and reading unit <b>79</b> of the communication terminal <b>70</b>C stores the predetermined-area information received in step S<b>77</b> in the predetermined-area management table (the predetermined-area management DB <b>7002</b>, see <figref idref="DRAWINGS">FIG. 20B</figref>) in association with the IP addresses of the image capturing devices <b>10</b> (step S<b>78</b>).

0243Then, the transmitting/receiving unit (acquisition unit) <b>71</b> transmits a captured-image request to the communication management apparatus <b>50</b> to acquire an image captured by the selected image capturing device <b>10</b> (step S<b>79</b>). Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the captured-image request transmitted from the communication terminal <b>70</b>. The captured-image request includes the image data ID acquired in step S<b>40</b>.

0244Upon receipt of the captured-image request, the storing and reading unit <b>59</b> of the communication management apparatus <b>50</b> searches the image type management table (the image type management DB <b>5002</b>, see <figref idref="DRAWINGS">FIG. 16B</figref>) using the image data ID as a search key to read the corresponding source name (image type information) (step S<b>80</b>). As a result, image data representing the captured image corresponding to the image data ID is specified.

0245The transmitting/receiving unit <b>51</b> transmits to the communication terminal <b>70</b>C the image data representing the captured image that is read (step S<b>81</b>). Thus, the transmitting/receiving unit <b>71</b> of the communication terminal <b>70</b>C receives the image data transmitted from the communication management apparatus <b>50</b>. The image data includes the image data ID of the captured image and audio data.

0246Then, in response to a predetermined input operation of the user C<b>1</b>, the acceptance unit <b>72</b> accepts selection of a display image to be displayed on the communication terminal <b>70</b>C within the captured image data received in step S<b>53</b> (step S<b>82</b>). In a case in which the communication terminal <b>70</b>C is capable of simultaneously displaying a plurality of captured images or in a case in which fewer captured images than the number of images simultaneously displayable on the communication terminal <b>70</b>C are received, the processing of step S<b>78</b> may be omitted.

0247Then, to display an image of a predetermined area specified in the predetermined-area information corresponding to the display image for which selection is accepted in step S<b>82</b>, the image and audio processing unit <b>73</b> applies perspective projection conversion using the predetermined-area information received in step S<b>77</b> to generate a predetermined-area image (step S<b>83</b>). As a result, the communication terminal <b>70</b>C can generate a predetermined-area image including the point of interest P designated by the user in a spherical image that is an image captured by the image capturing device <b>10</b> selected by the communication management apparatus <b>50</b>.

0248Then, the display control unit <b>74</b> causes the display <b>706</b> to display the predetermined-area image generated in step S<b>83</b> (step S<b>84</b>). <figref idref="DRAWINGS">FIG. 36</figref> illustrates an example of the display screen displayed on the communication terminal <b>70</b>C in step S<b>84</b>. A display screen <b>900</b> illustrated in <figref idref="DRAWINGS">FIG. 36</figref> displays a predetermined-area image X generated in step S<b>83</b>. The user C<b>1</b> checks the predetermined-area image X corresponding to the position of the point of interest P in the distribution site, which is designated using the site display screen <b>600</b>. As a result, the user C<b>1</b> is able to check the details of the current situation at the position of the point of interest P in the distribution site.

0249As described above, in the image communication system <b>1</b>, an image capturing device <b>10</b> suitable to view the point of interest P designated by a user at a viewing site can be selected using the communication management apparatus <b>50</b>. In the image communication system <b>1</b>, furthermore, predetermined-area information capable of displaying an image including the point of interest P, which is captured by the selected image capturing device <b>10</b>, is transmitted from the communication management apparatus <b>50</b> to the communication terminal <b>70</b>. As a result, the image indicating the point of interest P can be displayed on the communication terminal <b>70</b>.

0250In this embodiment, the selection unit <b>53</b> divides the distribution site into two areas or three areas, by way of example. However, the selection unit <b>53</b> may divide the distribution site into any number of areas in accordance with the number of image capturing devices <b>10</b> or distribution terminals <b>30</b> arranged in the distribution site, the network bandwidth of the communication network <b>100</b>, the number of images simultaneously receivable and displayable at the communication terminal <b>70</b>, or the like.

0251In the image communication system <b>1</b> according to this embodiment, when processing such as the processing of steps S<b>75</b> to S<b>77</b> or the processing of steps S<b>79</b> to S<b>81</b> described above is executed, another apparatus or the like may be present between the communication management apparatus <b>50</b> (communication management apparatus) and the communication terminal <b>70</b> (communication terminal). That is, information may be transmitted and received between the communication management apparatus <b>50</b> and the communication terminal <b>70</b> via another apparatus.

Modification of Process for Displaying Point of Interest P

0252Next, a modification of the process for displaying the point of interest P in the image communication system <b>1</b> will be described with reference to <figref idref="DRAWINGS">FIG. 37</figref>. <figref idref="DRAWINGS">FIG. 37</figref> is a sequence diagram illustrating a modification of the process for displaying the point of interest P in the image communication system <b>1</b>. <figref idref="DRAWINGS">FIG. 37</figref> illustrates an example in which the process for selecting an image capturing device <b>10</b> described above is executed by the communication terminal <b>70</b>C at the viewing site C. Since the processing of steps S<b>201</b> to S<b>204</b> is similar to the processing of steps S<b>71</b> to S<b>74</b> in <figref idref="DRAWINGS">FIG. 32</figref>, the description thereof will be omitted.

0253In <figref idref="DRAWINGS">FIG. 37</figref>, the transmitting/receiving unit (acquisition unit) <b>71</b> transmits to the communication management apparatus <b>50</b> an arrangement-information acquisition request, which is a request for acquiring arrangement information indicating the arrangement positions of the image capturing devices <b>10</b> (step S<b>205</b>). Thus, the transmitting/receiving unit <b>51</b> of the communication management apparatus <b>50</b> receives the arrangement-information acquisition request transmitted from the communication terminal <b>70</b>C. The arrangement-information acquisition request includes the site ID received in steps S<b>14</b> and S<b>16</b>.

0254Then, the communication management apparatus <b>50</b> searches the arrangement information management table (the arrangement information management DB <b>5004</b>, see <figref idref="DRAWINGS">FIG. 17B</figref>) using the site ID received in step S<b>205</b> as a search key to read the arrangement information associated with the same site ID as the received site ID (step S<b>206</b>). Then, the transmitting/receiving unit <b>51</b> transmits the arrangement information read in step S<b>206</b> to the communication terminal <b>70</b>C (step S<b>207</b>). Thus, the transmitting/receiving unit (acquisition unit) <b>71</b> of the communication terminal <b>70</b>C receives the arrangement information transmitted from the communication management apparatus <b>50</b>.

0255Then, the selection unit <b>81</b> executes a process for selecting an image capturing device <b>10</b>, based on the coordinate values indicated by the arrangement information read in step S<b>207</b> and the number of divisions indicated by the number-of-division information (step S<b>208</b>). Since the process for selecting an image capturing device <b>10</b> executed in step S<b>208</b> is similar to the processing of step S<b>76</b> illustrated in <figref idref="DRAWINGS">FIG. 32</figref> (the detailed process is illustrated in the flowchart in <figref idref="DRAWINGS">FIG. 33</figref>), the description thereof will be omitted. The processing of steps S<b>101</b> and S<b>103</b> may be omitted. Then, in step S<b>107</b>, the storing and reading unit <b>79</b> stores the predetermined-area information in the predetermined-area management table (the predetermined-area management DB <b>7002</b>, see <figref idref="DRAWINGS">FIG. 20B</figref>) instead of the predetermined-area management table (the predetermined-area management DB <b>5003</b>, see <figref idref="DRAWINGS">FIG. 17A</figref>). Since the processing of steps S<b>209</b> to S<b>214</b> is similar to the processing of steps S<b>79</b> to S<b>84</b> in <figref idref="DRAWINGS">FIG. 32</figref>, the description thereof will be omitted.

0256As described above, in the image communication system <b>1</b> according to a modification of the embodiment, the communication terminal <b>70</b> performs a process for selecting an image capturing device <b>10</b> that captures an image in which the point of interest P designated by a user is displayed. As a result, an image indicating the point of interest P can be displayed on the communication terminal <b>70</b>. Therefore, as in the embodiment described above, the image communication system <b>1</b> allows a user at a viewing site to view an image of the point of interest P designated by the user.

0257As described above, the communication management apparatus <b>50</b> according to an embodiment of the present disclosure receives coordinate values indicating the position of the point of interest P in an area where a plurality of image capturing devices, the point of interest P being designated by a user, and the number-of-division information for dividing the area from the point of interest P (step S<b>75</b>). Then, the communication management apparatus <b>50</b> transmits first predetermined-area information and second predetermined-area information to the communication terminal <b>70</b>, the first predetermined-area information indicating a predetermined area including a predetermined position corresponding to predetermined-position information in an image captured by a reference image capturing device arranged at the shortest distance from the predetermined position among a plurality of image capturing devices, the second predetermined-area information indicating a predetermined area including the predetermined position in an image captured by a supplementary image capturing device arranged such that the angle between the supplementary image capturing device and the reference image capturing device is closest to an angle obtained by dividing the area in accordance with the number of divisions designated by the user (step S<b>77</b>). Accordingly, the communication management apparatus <b>50</b> can display an image of the point of interest P in a distribution site for which an image has been captured by an image capturing device selected in accordance with the number of divisions designated by the user without performing a process on the image capturing device <b>10</b>. Therefore, the communication management apparatus <b>50</b> can display a captured image of an area including a predetermined position on the communication terminal <b>70</b> without causing a user to individually operate a plurality of image capturing devices.

0258In addition, the communication management apparatus <b>50</b> can use a data table that stores a coefficient for adding a weight that is set in advance by the administrator in accordance with the number of divisions designated by the user to remove an image capturing device that is not suitable for use in image viewing among a plurality of image capturing devices.

0259In the embodiment described above, as an example of a spherical image (spherical panoramic image), the captured image (entire image) is a three-dimensional panoramic image. However, the captured image (entire image) may be a two-dimensional panoramic image.

0260In the embodiment described above, furthermore, the communication terminal <b>70</b> at a viewing site may not be dedicated to viewing. The communication terminal <b>70</b> may be configured to distribute a captured image and simultaneously implement both the distribution function and the viewing function. Likewise, the distribution terminal <b>30</b> at a distribution site may not be dedicated to distribution. The distribution terminal <b>30</b> may be configured to display a captured image distributed from any other site and simultaneously implement both the distribution function and the viewing function. As described above, the image communication system <b>1</b> may be configured to perform two-way communication of a captured image between a plurality of sites.

0261Each of the functions in the embodiment described above may be implemented by one or more processing circuits or circuitry. In the embodiment described above, the term “processing circuit or circuitry” includes a device programmed to implement each function using software, such as a processor implemented by an electronic circuit. Examples of the device include an application specific integrated circuit (ASIC), a digital signal processor (DSP), a field programmable gate array (FPGA), a system on a chip (SOC), a graphics processing unit (GPU), and a conventional circuit module.

0262Further, various tables in the embodiment described above may be generated by machine learning. Further, data of associated items can be classified, such that use of tables can be optional. As used herein, machine learning is a technique that enables a computer to acquire human-like learning ability. Machine learning refers to a technology in which a computer autonomously generates an algorithm to be used for determination such as data identification from learning data loaded in advance and applies the generated algorithm to new data to make a prediction. Any suitable learning method is applied for machine learning, for example, any one of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, and deep learning, or a combination of two or more of those learning methods.

0263The above-described embodiments are illustrative and do not limit the present invention. Thus, numerous additional modifications and variations are possible in light of the above teachings. For example, elements and/or features of different illustrative embodiments may be combined with each other and/or substituted for each other within the scope of the present invention.

0264Any one of the above-described operations may be performed in various other ways, for example, in an order different from the one described above.