Order processing unit and imaging device

Description: TECHNICAL FIELD [Detailed description of the invention]

【0001】

[Technical field to which the invention belongs]

The present invention relates to an order processing device and an image capturing device, and more particularly to an order processing device for processing a customer's order for an image displayed on a display means, and an image capturing device that can be used for the order.

【0002】

[Conventional technology]

In recent years, the distribution form and display form of image content have been rapidly diversified, and the image content distributed in various distribution forms is displayed on various display means and presented to customers. Order the creation of a photo print by specifying the favorite image of the image content displayed in the means, or order the product by specifying the image in which the desired product exists among the images displayed in the display means. There is an increasing demand for the realization of an environment that enables such activities. Then, in order to enable the customer to place an order using the image displayed on the display means, it is necessary to have the customer specify the image and identify the image designated by the customer.

【0003】

In view of the above situation, the customer specifies the frame to be printed while watching the video program displayed on the TV receiver, so that the frame identification information and the print order information can be obtained from the customer online via the print ordering machine. An image printing system has been proposed in which the print finishing company acquires the original image data of the frame to be printed from the image database of the broadcasting station and creates a print (see, for example, Patent Document 1).

【0004】

[Patent Document 1]

Japanese Unexamined Patent Publication No. 10-171027 [0005]

[Problems to be Solved by the Invention]

The technology described in the above publication uses frame identification information placed in a non-image area of a television signal in order to identify an image specified by a customer, and decodes the frame identification information to a print ordering machine. It is assumed that a TV receiver equipped with an output function is installed on the customer side. However, since the existing TV receiver does not have the above-mentioned function, it is necessary to prepare a special TV receiver equipped with the above-mentioned function. For this reason, in order to realize an environment in which a print order can be placed for an arbitrary frame in a video program displayed on a television receiver, a great financial burden is imposed on the customer. There was a problem.

【0006】

In addition, as a technology related to the above, in Japanese Patent Application No. 2000-320382 (unknown), order format data is transmitted from an image terminal to an ordering device such as a PDA (Personal Digital Assistants) owned by a customer, and a product is produced. The technique for creating order data is described. However, also in this technique, it is necessary to prepare an image terminal equipped with a function of communicating with the ordering device and a function of storing the order format data and transmitting the order format data to the ordering device at the time of ordering.

【0007】

The present invention has been made in consideration of the above facts, and an object of the present invention is to obtain an order processing device and an imaging device capable of realizing an environment in which a customer can easily place an order.

【0008】

[Means for solving problems]

The order processing apparatus according to the invention according to claim 1 obtains photographed image data generated by the customer taking an image of the order object displayed on the display means or the order object itself by using the photographing means. Identification of the image to be ordered based on the acquisition means acquired together with the order information representing the order contents from the image and the photographed image data acquired by the acquisition means, the photographed image represented by the photographed image data, or the photographed order. Acquired by the acquisition means based on the identification means for identifying the product as the subject in the target image, the identification of the product related to the photographed image of the order target, and the identification result by the identification means. It is configured to include a processing means for performing a predetermined process according to the contents of the order from the customer represented by the ordered order information.

【0009】

In the invention according to claim 1, the image of the order object displayed on the display means or the order object itself is photographed by the customer using the photographing means, and the photographed image data generated by the photographing is the content of the order from the customer. It is acquired by the acquisition means together with the order information representing. As display means, for example, a TV receiver, an electronic signboard, an electronic paper (see Japanese Patent Publication No. 11-502950, etc.), a computer, a mobile phone provided with a display unit capable of displaying an image, and a movie are screened. Examples of the photographing means include a digital still camera, a mobile phone having a function as a photographing means, a PDA, a wearable computer, a mobile computer, and the like.

【0010】

Further, in the invention according to claim 1, based on the acquired photographed image data, identification of the photographed image to be ordered, identification of the photographed image represented by the photographed image data, or identification of a product as a subject in the photographed image to be ordered. , At least one of the identification of the goods related to the captured image to be ordered is performed by the identification means. In the identification of the image to be ordered by the identification means, for example, the data of the image displayed or may be displayed on the display means is stored in the storage means, and the captured image data is stored in the storage means. It can be done by collating with each data. Further, the identification of the product as the subject in the image to be ordered is, for example, the identification of the product whose subject is the orderable product among the data of a large number of images displayed or may be displayed on the display means. This can be done by storing the product identification information for identifying the product in association with the data and acquiring the corresponding product identification information based on the identification result of the image to be ordered.

【0011】

In addition, the identification of a product as a subject in a photographed image represented by the photographed image data is a product for identifying individual products, for example, image data of a large number of products that may be photographed as an order object by a customer. By associating the identification information and storing it in the storage means and collating the photographed image data with each data stored in the storage means, the photographed image and the photographed image are selected from a large number of images stored in the storage means. This can be done by identifying an image of the same product as a subject and acquiring the corresponding product identification information.

【0012】

Further, the identification of the product related to the image to be ordered is, for example, the data of the image in which the related product that can be ordered exists among the data of a large number of images displayed or may be displayed on the display means. It can be performed by storing the product identification information for identifying the product in association with each other and acquiring the corresponding product identification information based on the identification result of the image. As the product related to the image to be ordered, for example, as described in claim 3, the first product including a moving image in which the image to be ordered is one scene (for example, a movie or drama as the moving image). , A recording medium (for example, video table, DVD, CD-ROM, etc.) on which the game software that reproduces the moving image by execution is recorded, and a moving image (for example, commercial video) in which the image to be ordered is one scene. 2nd product, 1st product or music or copyrighted product related to the 2nd product (eg, movie or drama, music CD containing music of game software, movie or drama) The identification means can be configured to identify at least one of the above-mentioned products as a product related to the image to be ordered.

【0013】

Then, the processing means according to the invention according to claim 1 performs predetermined processing according to the contents of the order from the customer represented by the acquired order information based on the identification result by the identification means. As a result, it is possible to respond to the customer's order for the image displayed on the display means.

【0014】

As described above, according to the invention of claim 1, even when the customer places an order for the image displayed on the display means, the image to be ordered or the product as the subject of the image is identified. Because the order can be placed by taking the image of the order target without acquiring the information for the purpose or the information for identifying the product related to the image of the order target from the display means, the display of the special configuration No means is required. Further, even when a customer places an order for an exhibited product or the like, the order can be placed by photographing the ordered object. Therefore, it is possible to realize an environment in which the customer can easily place an order.

【0015】

As a predetermined process in the invention according to claim 1, for example, as described in claim 2, a print is created using the original image data of the image to be ordered identified by the identification means, or the print is described. The process of requesting the creation of the image, the process of transferring the original image data of the image to be ordered to the customer, the process of recording the original image data of the image to be ordered on the recording medium, and the process of ordering the product identified by the identification means. Can be mentioned. The above-mentioned original image data is stored in a storage means as the original image data of a large number of images displayed or may be displayed in the display means, and is read out from the storage means based on the image identification result. You can get it at. Further, when transferring the original image data to the customer, predetermined image processing (for example, enlargement / reduction, trimming, image processing for improving image quality) may be performed.

【0016】

Further, in the invention according to claim 1, a plurality of candidate images to be ordered exist in the image represented by the captured image data (for example, the image represented by the captured image data is an image in which a plurality of images are juxtaposed). , It is possible that only some of the captured images are subject to the customer's order. In consideration of the above, as described in claim 4, the acquisition means also provides information for identifying the image to be ordered when there are a plurality of candidate images to be ordered in the image represented by the captured image data. It is preferable that the acquisition and identification means or the processing means determine the image to be ordered based on the information acquired by the acquisition means.

【0017】

The information for identifying the image to be ordered may be information indicating the image to be ordered among a plurality of candidate images to be ordered, or the order from among the candidate images to be ordered. It may be partially photographed image data generated by partially re-photographing only the target image as a subject by the customer. As a result, even when a plurality of candidate images to be ordered exist in the image represented by the captured image data, the image to be ordered can be determined more accurately.

【0018】

By the way, in an order for an image displayed on the display means, the identification of the captured image of the order target is, for example, as described above, of an image displayed or may be displayed on the display means. This can be performed by storing the data in the storage means and collating the captured image data with each data stored in the storage means. However, when the number of images to be collated (images in which data is stored in the storage means) is enormous, it is very difficult to identify the images only by collating the images.

【0019】

In consideration of the above, in the invention of claim 1, in the invention of claim 1, the acquisition means acquires at least one of the shooting date and time information indicating the shooting date and time by the customer and the shooting location information representing the shooting location by the customer. , The identification means is specified by at least one of the shooting date / time information and the shooting location information acquired by the acquisition means from the storage means that stores the data of a large number of images displayed or may be displayed on the display means. It is possible to identify the photographed image to be ordered by extracting the data of the image which may be the photographed image to be ordered and collating the data of the extracted image with the photographed image data. It is a feature.

【0020】

The shooting date and time information according to claim 5 can be generated by the shooting means by incorporating a time measuring means in the shooting means, and the shooting location information according to claim 5 also uses, for example, a GPS system. By incorporating a position detecting means for detecting the current position, the image can be generated by the photographing means. When the shooting date / time information and the shooting location information are generated by the shooting means, these information can be acquired by receiving the shooting image data together with the shooting means.

【0021】

Then, in the invention according to claim 5, it is specified by at least one of the shooting date / time information and the shooting location information acquired from the storage means for storing the data of a large number of images displayed or may be displayed on the display means. The data of the image to be ordered, which may be the image to be ordered, is extracted, and the image of the image to be ordered is identified by collating the data of the extracted image with the photographed image data. The images to be collated are narrowed down by at least one of the date and time information and the shooting location information, and the shot images to be ordered can be identified in a short time.

【0022】

In the invention according to claim 5, the image displayed on the display means may be a moving image. For example, as described in claim 6, the moving image is displayed on the display means according to a predetermined schedule. (Specifically, for example, the case where the display means is a television receiver or a screening facility corresponds to this). In such an embodiment, the images to be collated can be narrowed down by using the shooting date / time information, and can be displayed or displayed on the display means in the storage means, for example, as described in claim 6. A large number of sexual moving image data is stored, and at least the shooting date / time information of the shooting date / time information and the shooting location information is acquired by the acquisition means, and the data of the large number of moving images stored in the storage means is stored. From the large number of images to be represented, the data of the image group that may be the image to be ordered that was taken, which is specified by the shooting date and time information, is extracted, and the data of each image of the extracted image group is taken image data. By collating each with, it is possible to identify the captured image to be ordered.

【0023】

Further, in the invention according to claim 6, when the display means is a television receiver or the like, as described in claim 7, for example, moving images are delivered to the display means according to distribution schedules separate from each other. The moving image of the distribution channel selected by the customer from the plurality of distribution channels will be displayed. For the identification of the captured image in such an embodiment, for example, as described in claim 7, the storage means stores the moving image data distributed by each distribution channel, and a large number represented by the moving image data. From the images in the above, the image group delivered near the shooting date and time represented by the acquired shooting date and time information is extracted for each distribution channel as a unit, and the image group extracted for each distribution channel as a unit. This can be done by collating the data of each image of the above with the photographed image data.

【0024】

Further, in the invention according to claim 5, the display means may be installed at a predetermined place and the image to be displayed may be managed by an administrator, for example, as described in claim 8 (specifically, the display means may be managed by an administrator. Corresponds to, for example, when the display means is an electronic bulletin board system). In such an embodiment, the image to be collated can be narrowed down by specifying the display means using the shooting location information. For example, as described in claim 8, of the shooting date / time information and the shooting location information. At least the shooting location information is acquired, the display means at which the customer took the image is specified based on the shooting location information, and the image to be ordered may be taken based on the result of identifying the display means. The data of a certain image can be extracted from the storage means.

【0025】

In addition, as described in claim 9, the acquisition means also acquires the shooting location information indicating the shooting location by the customer, and the identification means is the target of the order by the customer even in the order for the exhibited products and the like. A product that may be a subject in the captured image, which is specified by the imaging location information acquired by the acquisition means from a storage means that stores image data of a large number of products that may be captured as an object. It is preferable that the data of the image is extracted and the data of the extracted image is collated with the photographed image data so as to identify the product as the subject in the photographed image.

【0026】

As a result, even when the number of images of the product (images for storing data in the storage means) that may be photographed by the customer as an order object is enormous, the images are photographed as in the invention of claim 5. The image to be collated can be narrowed down by the location information, and the product as the subject in the captured image can be identified in a short time.

【0027】

Further, in the invention of claim 10, in the invention of claim 1, a first identification image for identifying an individual image is added to the image displayed on the display means, and the identification means is , Extract the first identification image included in the image represented by the captured image data acquired by the acquisition means, and identify the captured image to be ordered based on the extracted first identification image. It is characterized by. In the invention according to claim 10, a first identification image for identifying an individual image added to the image displayed on the display means is extracted, and the image to be ordered is based on the first identification image. Since the identification is performed, the shooting date / time information and the shooting location information described in claim 5 are not required, and the image to be ordered can be easily and quickly identified.

【0028】

In the invention according to claim 10, when the image displayed on the display means is a moving image, for example, as described in claim 11, the image of each frame of the moving image is used as the first identification image. An identification code (for example, a barcode or other code) containing information for identifying each frame may be added to identify the image to be ordered based on the identification code extracted from the acquired captured image. it can.

【0029】

Further, in the invention according to claim 5 or 10, the identification of the product as a subject in the image to be ordered is displayed or may be displayed on the display means, for example, as described in claim 12. In a storage means for storing data of a large number of images, product identification information for identifying a product as a subject in each image is stored in association with the data of each image, and the identification means can be used as a storage means. After identifying the captured image to be ordered, it can be performed based on the product identification information stored in association with the data of the identified image.

【0030】

In addition, the product can be identified as a subject in the captured image to be ordered as follows. That is, in the invention of claim 13, in the invention of claim 1, a second identification image for identifying a product as a subject in the image is added to the image displayed on the display means. , The identification means extracts a second identification image included in the image represented by the photographed image data acquired by the acquisition means, and based on the extracted second identification image, in the photographed image to be ordered. It is characterized by identifying the product as the subject.

【0031】

In the invention according to claim 13, since a second identification image for identifying a product as a subject in the image is added to the image displayed on the display means, the captured image data acquired by the acquisition means. By extracting the second identification image included in the image represented by, the product as the subject in the image to be ordered is based on the extracted second identification image without identifying the image to be ordered. Can be easily identified.

【0032】

Further, regarding the identification of the product as the subject in the captured image, for example, as described in claim 14, the identification information for identifying the product is provided in the product that may be photographed as an order object by the customer. The displayed display unit is arranged in the vicinity, the image unit corresponding to the display unit included in the captured image represented by the captured image data acquired by the acquisition means is extracted, and the identification information represented by the extracted image unit is extracted. It can also be realized by configuring the identification means so as to identify the product as the subject in the captured image based on the above. As a result, as in the invention of claim 13, it is possible to easily identify the product as the subject in the image to be ordered based on the identification information represented by the extracted image unit without identifying the photographed image. it can.

【0033】

The image capturing apparatus according to the invention according to claim 15 inputs a photographing means capable of photographing an image displayed on a display means or an ordered object, and a customer's order content regarding an image photographed by the photographing means. Any one of claims 1 to 14, the input means for the purpose, the photographed image data generated by the customer taking a picture using the photographing means, and the order information input via the input means. It is configured to include a transmission means for transmitting to the described order processing device.

【0034】

The invention according to claim 15 is provided with a photographing means capable of photographing an image displayed on the display means or an ordered object, and an input means for inputting a customer's order content regarding an image photographed using the photographing means. The captured image data and the order information input via the input means are transmitted to the order processing device according to any one of claims 1 to 14, so that the order processing device receives the data. Based on the captured information, it is related to the identification of the captured image of the order target, the identification of the product as the subject in the captured image represented by the captured image data or the captured image of the order target, and the photographed image of the order target. At least one of product identification can be performed, and a predetermined process can be performed according to the contents of an order from a customer based on the identification result.

【0035】

Therefore, according to the invention of claim 15, the customer photographs the image of the order object or the order object displayed on the display means by the photographing means, and inputs the order information through the input means. Since it is possible to place an order for the captured image or an order for the object to be ordered, it is possible to realize an environment in which the customer can easily place an order, as in the invention according to claim 1.

【0036】

BEST MODE FOR CARRYING OUT THE INVENTION

Hereinafter, an example of the embodiment of the present invention will be described in detail with reference to the drawings.

【0037】

[First Embodiment] FIG. 1 shows an order processing system 10 according to the first embodiment. The order processing system 10 includes an order processing device 12 according to the present invention. The order processing device 12 consists of a computer such as a workstation, and CPU12A, ROM12B, RAM12C, and input / output ports 12D are connected to each other via a bus 12E. A keyboard 38, a hard disk device (HDD) 40, and a communication control device (for example, a modem or a router) 42 are connected to each of the input / output ports 12D.

【0038】

The HDD 40 of the order processing device 12 is pre-installed with an order receiving program for executing the order receiving process (details will be described later) according to the present invention by the CPU 12A of the order processing device 12, and the CPU 12A executes the order receiving program. As a result, the computer constituting the order processing device 12 functions as the order processing device according to the present invention.

【0039】

The communication control device 42 of the order processing device 12 is connected to the communication network 14. The communication network 14 includes a mobile terminal network and a public switched telephone network, and is configured to include a high-speed and large-capacity dedicated communication network or a computer network such as the Internet. The communication network 14 includes computers (PCs) 18 owned by a large number of print finishing companies 16 nationwide, computers (PCs) 24 owned by a plurality of television broadcasting stations 22, and a plurality of product dealers 28. Each computer (PC) 30 possessed by each is connected.

【0040】

The print finisher 16 is a digital print system 20 including a film scanner that reads an image recorded on photographic film and a digital printer that performs a print finish process that records an image represented by image data on a recording material such as photographic paper. I have. The digital print system 20 is connected to the PC 18, and it is also possible to perform print finishing processing using the image data received by the PC 18 via the communication network 14.

【0041】

In addition, a single or a plurality of channels are assigned to each of the plurality of television broadcasting stations 22, and each television broadcasting station 22 transmits images to the television receiver 32 according to a broadcasting (distribution) schedule determined for each channel. A video signal for displaying a moving image) is broadcast (distributed). As a result, on the receiving side, the video (moving image) of the channel selected by the viewer (customer of the order processing system 10) out of a large number of channels is displayed on the TV receiver 32, and the image to be ordered is presented to the customer. Will be done.

【0042】

In addition, each television broadcasting station 22 associates moving image data (original image data) displayed on the television receiver 32 with the broadcast video signal with the broadcast date and time, and an image DB provided in the HDD 26 connected to the PC 24. HDD26 corresponds to the storage means according to the present invention. In addition, among the images broadcast by the television broadcasting station 22, the images handled by the product dealer 28 are introduced so that the viewer can order them by telephone or facsimile (for example, in FIG. 4 (A)). There is also an image (such as an image) in which an image as shown is displayed for a predetermined time. Among the moving image data stored in the image database, the moving image data corresponding to the above-mentioned video includes information such as the ID of each product and the ID of the product handling company handling each product. Product information including is also added. This product information corresponds to the product identification information described in claim 12.

【0043】

In addition, each customer of the order processing system 10 has a mobile device 46. The mobile device 46 according to the present embodiment is configured by installing a predetermined program (an image capturing program described later) on an existing mobile phone equipped with a photographing means and having a function of performing data communication. The portable device 46 corresponds to the image capturing device according to claim 15.

【0044】

That is, the portable device 46 includes a control unit 48 including a CPU and the like, and the control unit 48 includes a display unit 50 composed of an LCD and the like and capable of displaying various information including images, a power switch, a numeric keypad, and the like. An operation unit 52, an image pickup unit 54 capable of photographing an arbitrary subject, a storage unit 56 capable of storing arbitrary information, and a wireless communication unit 58 are connected to each other. A speaker 62 is connected to the wireless communication unit 58 via an amplifier 60, a microphone 66 is connected via an amplifier 64, and an antenna 68 is further connected.

【0045】

The image sensor 54 includes an image sensor such as a CCD, a lens that forms a subject on the light receiving surface of the image sensor, and a signal processing unit that amplifies the signal output from the image sensor and converts it into digital data (image image data). And are configured to include. Further, the storage unit 56 stores in advance customer information including information such as the ID, name, and address of the owner (customer) of the mobile device 46. Further, the imaging unit 54 corresponds to the photographing means according to the present invention, and the operating unit 52 corresponds to the input means according to claim 15.

【0046】

Under the control of the control unit 48, the wireless communication unit 58 wirelessly communicates with a base station connected to a wired public telephone network via an antenna 68 by electromagnetic waves in a predetermined frequency band predetermined for mobile phones. It has a function to perform communication, and it is also possible to communicate with the order processing device 12 via the communication network 14. As shown by the broken line in FIG. 1, the control unit 48 may have a payment function. The payment method may be a prepaid type or a credit type. This makes it possible to complete the settlement when the customer places an order (details will be described later). Individual authentication and settlement can also be performed using a service provided by a third-party organization.

【0047】

Next, the operation of the first embodiment will be described. In the first embodiment, the customer of the order processing system 10 can order the print creation of an arbitrary image taken by the imaging unit 54 of the mobile device 46 (hereinafter, this is referred to as "order A"). Has been done. In addition, the customer of the order processing system 10 orders the print creation using the original image data of the captured image by capturing the image displayed on the television receiver 32 by the imaging unit 54 of the portable device 46 (). Hereinafter, this may be referred to as "order B"), and a product as a subject in the captured image (specifically, the image displayed on the television receiver 32) may be ordered (hereinafter, this is referred to as "order C"). It is possible.

【0048】

In the following, when the customer first shoots an arbitrary subject by the imaging unit 54 of the mobile device 46, the image capturing program installed in the storage unit 56 of the mobile device 46 is executed by the control unit 48 (CPU). The image shooting process realized by this will be described with reference to the flowchart in FIG. The image capturing program may be installed in the storage unit 56 in advance when the mobile device 46 is manufactured, or may be installed at a predetermined timing (for example, when the customer applies for the use of the order processing system 10 in advance). ) It may be downloaded from a predetermined server in the communication network 14 or the order processing device 12 and installed in the storage unit 56.

【0049】

In step 100, the process selection screen as shown in FIG. 3A is displayed on the display unit 50 as an example. The process selection screen is a screen for the customer to select in advance the process for the image to be shot. In the example of FIG. 3A, the process can be obtained by the customer shooting an arbitrary scene as the process for the shot image. The first process of storing the captured image data in the storage unit 56 of the portable device 46, and the second process of ordering a print creation using the captured image data obtained by the customer shooting an arbitrary scene (the second process). Order A), the third process of ordering print creation using the original image data of the image based on the photographed image data obtained by the customer taking the image displayed on the TV receiver 32 (Order B). ), And the fourth process (order C) of ordering the product as the subject in the image based on the captured image data obtained by the customer capturing the image displayed on the television receiver 32 is an option. Each is displayed.

【0050】

In the next step 102, it is determined whether or not any one of the plurality of options shown on the process selection screen is selected, and the process waits until the determination is affirmed. When the customer who confirmed the process selection screen displayed on the display unit 50 operates the operation unit 52 and selects one of the plurality of options, the determination in step 102 is affirmed and the process proceeds to step 104, and the operation unit proceeds to step 104. Of the many keys provided in 52, it is determined whether or not a key that functions as a shutter at the time of shooting (hereinafter, simply referred to as a shutter) is turned on, and waits until the determination is affirmed.

【0051】

When the first process or the second process is selected as the process for the captured image, the customer shoots by turning on the shutter with an arbitrary scene as the subject, but the third process (process for the captured image) When order B) or the fourth process (order C) is selected, the customer prints a photo so as to notify the customer by displaying a message on the process selection screen shown in FIG. 3 (A). The entire display screen of the television receiver 32 is within the imaging range of the imaging unit 54 while the image of the image that the television receiver 32 wants to obtain or the image of the product that the television receiver wants to obtain is displayed on the television receiver 32. After adjusting the orientation of the portable device 46 and the distance from the display surface of the television receiver 32 so as to enter, the photograph is taken by turning on the shutter. If the lens of the imaging unit 54 is a zoom lens, the zoom magnification may be adjusted instead of adjusting the distance.

【0052】

When the shutter is turned on, the determination in step 104 is affirmed, the process proceeds to step 106, and the captured image data output from the imaging unit 54 is stored (saved) in the storage unit 56. In the next step 108, it is determined whether the processing for the captured image selected by the customer is order B or order C. If the determination is denied, the process proceeds to step 112, but if the determination is affirmed, in step 110, the current timing function (calendar function and clock function) built in the control unit 48 is used. After acquiring the shooting date / time information indicating the date / time (shooting date / time) and temporarily storing the acquired shooting date / time information in the storage unit 56, the process proceeds to step 112.

【0053】

In the next step 112, the image represented by the captured image data stored in the storage unit 56 in step 106 is displayed on the display unit 50, and a message is inquired to the customer whether the captured image is appropriate (whether it is not necessary to redo the shooting). Is displayed on the display unit 50. In step 114, it is determined whether or not the captured image is OK based on the information input via the operation unit 52 by the customer who confirmed the message. If the determination is denied, the captured image data stored in step 106 (and the captured date / time information acquired in step 110) is discarded, the process returns to step 104, and steps 104 and subsequent steps are repeated. As a result, the above-mentioned shooting will be performed again.

【0054】

If the determination in step 114 is affirmed, the process proceeds to step 116, and it is determined whether or not the processing for the captured image selected by the customer is order B or order C. If the determination is denied, the process proceeds to step 122, but if the determination is affirmed, the process proceeds to step 118, and a warning screen as shown in FIG. 3 (B) is displayed on the display unit 50 as an example.

【0055】

As described above, in order B and order C, the entire display screen of the television receiver 32 is photographed so as to be within the imaging range of the imaging unit 54. There may be multiple candidate images to be ordered). When there are a plurality of candidate images to be ordered, it is difficult to determine the image to be ordered (as an example, the image of FIG. 4 (A) in which the images of products A to D exist. See). Therefore, on the alert screen, as shown in Fig. 3 (B) as an example, if there is an image other than the image to be ordered in the captured image, the customer re-shoots only the image to be ordered. A message is displayed to prompt you.

【0056】

In the next step 120, it is determined whether or not it is necessary to retake only the image to be ordered based on the information input by the customer via the operation unit 52. If the determination is denied, the process proceeds to step 122, but if the determination is affirmed, the process returns to step 104, and steps 104 and subsequent steps are repeated. As a result, the customer can see only the image to be ordered out of the plurality of images displayed on the television receiver 32 within the imaging range of the imaging unit 54, and capture the other images displayed on the television receiver 32. By adjusting the orientation of the portable device 46 and the distance from the display surface of the television receiver 32 so as to be out of the range, and then turning on the shutter, only the image to be ordered is retaken.

【0057】

As an example, when the image represented by the captured image data obtained by the first shooting is an image as shown in FIG. 4 (A) and the customer desires to order the product D (order C), the above By re-shooting, an image as shown in FIG. 4B is taken, and the data of this image is also stored in the storage unit 56.

【0058】

In the next step 122, it is determined whether or not the processing for the captured image selected by the customer is the first processing (only the storage of the captured image data in the storage unit 56). If the determination is affirmed, the captured image data has already been saved in the previous step 106, so that the image capturing process is terminated without performing any processing.

【0059】

On the other hand, if the determination in step 122 is denied (when the selected process is any of the second process (order A) to the fourth process (order C)), the process proceeds to step 124, and the storage unit Read customer information from 56. In step 126, it is determined whether or not the process for the captured image selected by the customer is the fourth process (order C). If the determination is denied, the process proceeds to step 128, and the print order screen as shown in FIG. 3 (C) is displayed on the display unit 50 as an example.

【0060】

The print order screen is a screen for the customer to set the contents of order A or order B (print order), and in the example of FIG. 3 (C), the "order number" that defines the number of photo prints to be created. , Multiple options are displayed for each item of "Print size" that specifies the size of the photo print to be created, and "Print receiving method" that specifies the method of receiving the created photo print, and the customer wants each item. By selecting the option of, you can set the contents of the print order. In step 130, it is determined whether or not the input setting by the customer is completed, and the process waits until the determination is affirmed. When the input setting of the print order contents by the customer is completed, the determination in step 130 is affirmed and the process proceeds to step 136.

【0061】

On the other hand, if the determination in step 126 is affirmed, the process proceeds to step 132, and the product order screen as shown in FIG. 3 (D) is displayed on the display unit 50 as an example. The product order screen is a screen for the customer to set the contents of order C (product order). In the example of Fig. 3 (D), the "order quantity" that defines the number of products to be delivered and the delivery destination of the products A plurality of options are displayed for each item of the "product receiving method" that defines (the method of receiving the product), and the customer can set the content of the product order by selecting the desired option for each item. In step 134, it is determined whether or not the input setting by the customer is completed, and the process waits until the determination is affirmed. When the input setting of the product order contents by the customer is completed, the determination in step 134 is affirmed and the process proceeds to step 136.

【0062】

Then, in step 136, order type information (information indicating which of order A to order C corresponds to) is added to the information representing the order contents set by the customer in step 128 or step 132 (these are order information). After that, the photographed image data stored in the storage unit 56 and the customer information stored in advance in the storage unit 56 are read out in step 106 (when the order type is order B or order C, step The shooting date and time information acquired by 110 is also read out from the storage unit 56 at the same time), the shooting image data, customer information, and order information (and shooting date and time information) are transmitted to the order processing device 12 via the communication network 14, and the image shooting process is performed. finish. Step 136 corresponds to the transmission means according to claim 15, together with the wireless communication unit 58 that actually transmits information.

【0063】

When the shooting is performed a plurality of times, in step 136, all the captured image data obtained in each shooting is read out and transmitted to the order processing device 12. Further, in order to shorten the time required for transmission, it is desirable to transmit the captured image data, which has a particularly large amount of data, after compression. By the above processing, any one of order A, order B, and order C is placed on the order processing device 12.

【0064】

Subsequently, the flowchart of FIG. 5 regarding the order acceptance process executed by the order processing device 12 when the information (specifically, the photographed image data, the customer information, the order information (and the photographed date and time information)) is received from the mobile device 46 (customer). Will be described with reference to. When the order processing device 12 receives the above information from the mobile device 46, the communication control device 42 functions as the acquisition means according to claim 1 (more specifically, claims 4 to 6).

【0065】

In step 150, the order type information included in the order information received from the customer is referred to, and it is determined whether or not the type of the order from the customer is order A. If the determination is affirmed, the process proceeds to step 152. Then, based on the "print receiving method" included in the order information received from the customer, the customer's address included in the customer information received from the customer, etc., the order for this print order is placed from among a large number of print finishing companies 16. By selecting the most suitable print finisher 16 and transmitting the photographed image data, customer information, and order information received from the customer to the PC 18 of the selected print finisher 16 via the communication network 14, printing from the customer Place an order (order A).

【0066】

The print finisher 16 who received the above information by the PC 18 via the communication network 14 uses the received photographed image data to print a photographic print of the size specified by the "print size" of the received order information. Create only the number of received order information specified by "Number of prints", and send the created photo print to the specified destination so that the customer can receive the photo print by the receiving method specified by "Print receiving method". Deliver to. As a result, the customer can receive the photographic print of the image taken by himself / herself under the conditions (size, number of sheets, receiving method) that match the specified order contents.

【0067】

On the other hand, when the type of the order from the customer is order B or order C, the determination in step 150 is denied and the process proceeds to step 154, and the HDD 40 of the order processing device 12 is based on the shooting date and time information received from the customer. It is determined whether or not there is an image that may be an image to be ordered from a customer in the image DB provided in. In the present embodiment, the image DB of the HDD 40 of the order processing device 12 may store all the images that may be the target of the order B or C, or even a part of the images may be stored. Often, the HDD 40 of the order processing device 12 also corresponds to the storage means according to the present invention.

【0068】

That is, the moving image data (original image data) stored in the image DB of the HDD 26 of the television broadcasting station 22 and the broadcast date and time are downloaded from the PC 24 via the communication network 14 (data to which product information is added). When the order processing device 12 is configured to periodically store the image DB of the HDD 40 in the image DB of the HDD 40 for all TV broadcasting stations 22, the HDD 40 of the order processing device 12 is stored. In the image DB of, all the image data that may be the target of the order B or C will be stored. In this aspect, the determination in step 154 above is unconditionally affirmed.

【0069】

In the present embodiment, when the image displayed on the television receiver 32 is taken, the order B or the order C is immediately placed on the order processing device 12, so that the image DB is broadcast for a relatively short period of time. It is possible to respond to orders from customers simply by storing the data for a short period of time (for example, several hours to a day). Therefore, even when data is collected from all the television broadcasting stations 22 and stored in the image DB of the HDD 40 of the order processing device 12 as in the above embodiment, the HDD 40 having an enormous storage capacity is required to store the image DB. You don't need it.

【0070】

In addition, among the images (moving images) broadcast by each TV broadcasting station 22, the time zone in which there is a high possibility of being the target of the order B or C (for example, the time zone where the viewing rate is relatively high or the target of the order C). Only the data of the video (moving image) broadcast during the time when the video of the product sale is being broadcast and the broadcast date and time (including the product information for the data to which the product information is added) When the order processing device 12 is configured to collect data from the broadcasting station 22 and store it in the image DB of the HDD 40, the image DB of the HDD 40 of the order processing device 12 may be an order target of order B or order C. The data of a part of a certain image will be stored.

【0071】

The present invention is not limited to providing the image DB in the HDD 40 of the order processing device 12, and the image data that may be the target of the order B or C can be stored in the HDD 40 of the order processing device 12. Needless to say, you don't have to remember it at all.

【0072】

If the judgment in step 154 is affirmed, the process proceeds to step 158, considering the possibility that the time recognized by the timekeeping function (specifically, the clock function) of the portable device 46 is different from the actual time. Broadcast (delivered) from each TV broadcasting station 22 from the image DB of the HDD 40 of the order processing device 12 to the shooting date and time represented by the shooting date and time information received from the customer and the time zone before and after that (for example, about several seconds before and after to several tens of seconds before and after). The data of the image is extracted, stored in a predetermined work area provided in the storage area of the RAM 12C or HDD 40, and the process proceeds to step 168.

【0073】

On the other hand, if the determination in step 154 is denied, the process proceeds to step 160, and the shooting date / time information received from the customer is transmitted to the PC 24 of the specific TV broadcasting station 22 via the communication network 14, and the transmitted shooting date / time is transmitted. Requests that the data of the image broadcast (distributed) from the television broadcasting station 22 be transferred at the shooting date and time indicated by the information and the time zone before and after that. In the next step 162, it is determined whether or not the image data has been received from the television broadcasting station 22 that requested the data transfer, and the process waits until the determination is affirmed.

【0074】

When the transfer of image data is requested, the PC24 of the specific TV broadcasting station 22 broadcasts (delivers) from the own TV broadcasting station 22 at the shooting date and time represented by the shooting date and time information received from the order processing device 12 and the time zone before and after that. The image data is extracted from the image DB of the HDD 26, and the extracted image data is transmitted to the order processing device 12 via the communication network 14. When this image data is received by the order processing device 12, the determination in step 162 is affirmed and the process proceeds to step 164, and the image data received from the PC 24 of the specific TV broadcasting station 22 is used as the work area of the RAM 12C or the HDD 40. To memorize.

【0075】

In the next step 166, it is determined whether or not the image data has been received from all the television broadcasting stations 22. If the determination is denied, the process returns to step 160, and steps 160 to 166 are repeated until the determination in step 166 is affirmed. As a result, among the images (moving images) broadcast from all the TV broadcasting stations 22, the images broadcast in the shooting date and time represented by the shooting date and time information received from the customer and the time zone before and after that (customers were watching). The data of (images that may have been displayed on the television receiver 32) will be collected and stored respectively.

【0076】

In step 168, for all the image data stored in the work area of the RAM 12C or HDD 40, pattern matching is performed with the captured image data received from the customer to calculate the similarity, and the similarity with the captured image data is the maximum. Extract image data. When a plurality of captured image data are received from the customer, the captured image data obtained by the first shooting, that is, the entire display screen of the television receiver 32 is captured so as to be within the imaging range. The similarity is calculated using the obtained captured image data.

【0077】

The similarity calculation can be performed, for example, as follows. That is, first, a pair of images to be calculated for similarity (an image represented by the captured image data and an image represented by any one of the image data stored in the work area: hereinafter referred to as image A and image B. ), The number of pixels in the vertical and horizontal directions is normalized by thinning out the images and converting the resolution.

【0078】

Next, considering that the density level and gradation of the image represented by the captured image data may deviate from the original image depending on the exposure control performance of the device used for shooting (portable device 46 in this embodiment). , Density level and dynamic range normalization. The normalization of the density level and the dynamic range can be realized by converting the pixel value (density value or luminance value) for each pixel of the image A and the image B according to the following calculation formula. a'_C(i, j) = k (a_C(i, j)-mean (A_C)) / (max (A)_C)-min (A_C)) b'_C(i, j) = k (b_C(i, j)-mean (B_C)) / (max (B)_C)-min (B_C)) However, C represents one of the color components (R, G, B), k is the coefficient, a (i, j) is the pixel value of the coordinates (i, j) of the image A, and b (i, j) is the pixel value. The pixel values of the coordinates (i, j) of the image B, mean (X), max (X), and min (X) represent the average value, the maximum value, and the minimum value of the pixel values of the image X, respectively.

【0079】

Subsequently, the residual D is calculated by the following equation for the images A and B that have undergone normalization of the density level and the dynamic range.

【0080】

[Number 1]

<img file="JP2003178221A_D0001.tif" />【0081】

The value of the residual D becomes smaller as the similarity between the image A and the image B becomes higher. Here, the residual D is obtained for each color component (R, G, B) and integrated.

【0082】

Then, the similarity Y is obtained from the integrated value D by the following calculation formula. Y = 1 / D or Y = -D Therefore, the degree of similarity Y becomes larger as the similarity between the image A and the image B increases. While shifting the relative positions of image A and image B independently by ± n pixels in the vertical and horizontal directions of the image, the above calculation of similarity (normalization of the number of pixels, normalization of density level and dynamic range, residual D) And the calculation of the similarity Y) are repeated to search for the maximum value of the similarity Y between the image A and the image B. Then, the maximum value of the obtained similarity Y can be adopted as the similarity Y between the image A and the image B.

【0083】

Further, the calculation of the similarity is not limited to the above calculation method, and various known calculation methods can be adopted. For example, as described in Japanese Patent Application Laid-Open No. 2001-84274, in the scene. You may adopt the method of calculating the similarity by paying attention to the edge and the shape of the figure. This calculation method has an advantage that it is not necessary to normalize the density level and the dynamic range for calculating the similarity by focusing on the edge information in the image.

【0084】

In the next step 170, it is determined whether or not the similarity in the image data having the maximum similarity is equal to or higher than a predetermined value. In addition, steps 154 to 170 (including step 176 when executing step 176 described later) correspond to the identification means according to claim 1 (specifically, the identification means according to claims 5 to 7). It corresponds.

【0085】

When the judgment of step 170 is denied, for example, when the image taken by the customer does not correctly capture the screen of the television receiver (when the size or misalignment exceeds the allowable range), the image is taken. If the image is extremely poorly exposed or out of focus, or if the shooting date and time represented by the received shooting date and time information is different from the actual shooting date and time due to an error in setting the date and time on the mobile device 46, the customer's operation error For example, when an image other than the image displayed on the television receiver is received as a subject. In the above case, it is difficult to continue the order acceptance process, so if the judgment of step 170 is denied, the process proceeds to step 172, and the customer (the customer () provides information notifying that the corresponding image does not exist. Send to the mobile device 46) and finish the order acceptance process.

【0086】

On the other hand, if the determination in step 170 is affirmed, the process proceeds to step 174, and it is determined whether or not a plurality of captured image data has been received from the customer. If the determination is denied, the process proceeds to step 178. If the determination is affirmed, there are a plurality of candidate images to be ordered in the image taken in the first shooting (the image represented by the image data having the maximum similarity), so that the images are similar in step 176. Data (candidate image data) corresponding to each candidate image to be ordered is extracted from the image data having the maximum degree, and the magnification of the extracted candidate image data is set to the photographed image data (image to be ordered) obtained by re-imaging. After matching with the magnification of (image data obtained by shooting so that only the image is within the imaging range), the similarity between each candidate image data and the captured image data obtained by re-shooting is calculated. To do. Then, the candidate image data having the maximum degree of similarity with the captured image data obtained by re-imaging is extracted.

【0087】

In the next step 178, it is determined whether or not the type of the order from the customer is order B. If the judgment is affirmed, the process proceeds to step 180, and in the same manner as in step 152 described above, the most suitable print finisher 16 for placing the print order this time is selected, and the image data extracted in step 168 or step 176 is selected. Place a print order (order B) from a customer by transmitting (original image data of the image to be ordered), customer information, and order information to PC18 of the selected print finisher 16 via the communication network 14. ..

【0088】

The print finisher 16 who received the above information by the PC 18 via the communication network 14 uses the original image data of the received image to be ordered and uses the size specified in the "print size" of the received order information. Create the number of photo prints specified in the "number of prints" of the received order information, and create the photo prints so that the customer can receive the photo prints by the receiving method specified in the "print receiving method". Deliver the print to the designated destination.

【0089】

As a result, the customer can receive the photographic print created by using the original image data of the image taken by the customer under the conditions (size, number of sheets, receiving method) that match the specified order contents. The captured image data obtained by shooting with the portable device 46 often has low image quality due to the influence of the image display capability of the display means, shooting conditions, etc., but is stored in the image DB of the television broadcasting station 22. Since the original image data is high-resolution and high-quality image data, the customer can obtain a high-quality photographic print by placing a print order for order B.

【0090】

On the other hand, when the determination in step 178 is denied, the product information is added to the image data to be ordered because the type of the order from the customer is order C. Therefore, in step 182, the product dealer 28 handling the product to be ordered from the customer is recognized based on the ID of the product dealer included in the product information added to the image data to be ordered. (The recognition of this product handler 28 corresponds to the identification means described in claim 12), the image data (image data to be ordered) extracted in step 168 or step 176, customer information, order information and product information. Is transmitted to the PC30 of the recognized merchandise dealer 28 via the communication network 14 to place an order for merchandise (order C) from the customer. The above-mentioned step 182 corresponds to the processing means according to claim 1 together with the above-mentioned step 180.

【0091】

The product dealer 28, which has received the above information from the PC 30 via the communication network 14, defines the product corresponding to the product ID included in the received product information by the "order quantity" of the received order information. The packed products are packed in the specified number, and the packed products are delivered to a predetermined destination so that the packed products can be received by the customer by the receiving method specified in the "Product receiving method". As a result, the customer can receive the ordered product under the conditions (quantity / receiving method) that match the specified order contents.

【0092】

[Second Embodiment] Next, the second embodiment of the present invention will be described. Since the second embodiment has the same configuration as the first embodiment, the same reference numerals are given to each part to omit the description of the configuration, and the operation of the second embodiment will be described below.

【0093】

In the second embodiment, each television broadcasting station 22 has a plurality of images (in the image DB of the HDD 26) that can be printed by order B among the images of each frame of the moving image represented by the video signal to be broadcast (distributed). Fig. 4 (D) shows an example of multiple images of a product (an image that stores the original image data) and a product that can be ordered by order C (a product handled by the product dealer 28). As described above, an identification code 70 (for example, a bar code) representing predetermined identification information is added in advance.

【0094】

In the second embodiment, the identification information represented by the identification code 70 attached to the image for which a print order can be placed is, in detail, information for identifying an individual image for which a print order can be placed (possessed image data). This identification code 70 corresponds to the first identification image according to claim 10 (specifically, the identification code according to claim 11). .. In addition, the identification information represented by the identification code 70 attached to the image of the product for which the product can be ordered is the product information of the product that is the subject in each image (ID of each product, handling each product). This identification code 70 corresponds to the second identification image described in claim 13.

【0095】

As a result, on the receiving side, the image being viewed on the TV receiver 32 is an image for which a print order can be made by order B (an image in which the original image data is stored in the image DB of HDD26) or a product according to order C. When the image of the product that can be ordered (the product handled by the product dealer 28) is displayed, the identification code 70 is displayed on the display screen of the television receiver 32 at the same time.

【0096】

Also in the second embodiment, when the customer shoots an arbitrary subject by the image pickup unit 54 of the mobile device 46, the same image capturing process as in the first embodiment is executed by the mobile device 46, and first, the mobile device 46 The process selection screen shown in FIG. 3 (E) is displayed on the display unit 50, and as shown in the notification to the customer by displaying a message on this process selection screen as well, the third process (3rd process) as the process for the captured image ( When order B) or the fourth process (order C) is selected, the customer is in accordance with order B, as in the case of notifying the customer by displaying a message on the process selection screen of FIG. 3 (E). While the target image and the identification code 70 of the product order by the print order or the order C are displayed on the TV receiver 32, only the image to be ordered and the identification code 70 attached to the image are captured by the image pickup unit 54. The image is taken by adjusting the orientation of the mobile device 46 and the distance from the display surface of the television receiver 32 so as to be within the range.

【0097】

Therefore, in the second embodiment, when ordering a print by order B or ordering a product by order C, photographed image data representing an image to be ordered and an image in which only the identification code 70 attached to the image exists is carried. It will be transmitted from the device 46 to the order processing device 12.

【0098】

Next, the order acceptance process executed by the order processing device 12 according to the second embodiment will be described with reference to the flowchart of FIG. In step 200, it is determined whether or not the type of the order from the customer is order A based on the order type information included in the order information received from the customer. If the determination is affirmed, the process proceeds to step 202, and the print finisher 16 is selected and the selected print finisher 16 is selected in the same manner as in step 152 of the order acceptance process (FIG. 5) described in the first embodiment. A print order (order A) from a customer is placed by transmitting photographed image data, customer information, and order information to PC18.

【0099】

On the other hand, when the type of the order from the customer is order B or order C, the determination in step 200 is denied and the process proceeds to step 204, and the data in the area corresponding to the identification code is extracted from the captured image data received from the customer. Extract. In the next step 206, the content of the identification information encoded as the identification code is recognized based on the data of the area extracted in step 204. Further, in step 208, it is determined whether or not the type of the order from the customer is order B.

【0100】

If the determination in step 208 is affirmed, the process proceeds to step 210. When the order type from the customer is order B, the image represented by the photographed image data includes an image that can be ordered for print and an identification code added to the image. The identification information represented by this identification code is information for identifying individual images that can be ordered for print, and this information includes information for identifying the television broadcasting station 22 that possesses the image data. There is. Therefore, in step 210, the television broadcasting station 22 that possesses the original image data of the image (the image to be ordered) represented by the captured image data is recognized based on the recognition result of the identification information in step 206.

【0101】

Further, in step 212, the identification information recognized in step 206 is transmitted to the PC 24 of the television broadcasting station 22 recognized in step 210, and the data of the image identified by the identification information is requested to be transferred. The PC24 of the television broadcasting station 22 is made to extract the image data specified by the identification information from the image DB of the HDD 26 and send it to the order processing device 12, and by receiving this image data, the original of the image to be ordered is obtained. Get image data. In addition, steps 204 to 212 in the above-described processing correspond to the identification means according to claim 10 (more specifically, claim 11).

【0102】

Then, in step 214, the most suitable print finisher 16 for placing the print order this time is selected, and the image data (original image data of the image to be ordered), customer information, and order information acquired in step 212 are selected and printed. Place a print order (order B) from a customer by sending it to PC18 of the finisher 16. As a result, the customer receives the photographic print created by the print finisher 16 using the original image data of the image to be ordered under the conditions (size, number of sheets, receiving method) that match the order contents specified by the customer. be able to.

【0103】

If the determination in step 208 is denied, the process proceeds to step 216. When the order type from the customer is order C, the image represented by the photographed image data includes an image in which the product for which the product can be ordered is the subject and an identification code added to the image, and this identification code is used. , Represents the product information of the product that is the subject in the corresponding image. Therefore, in step 216, the product handling company 28 that handles the image (image to be ordered) represented by the captured image data is recognized based on the recognition result of the identification information (product information) in step 206. Note that steps 204 to 208 and 216 in the above-described processing correspond to the identification means according to claim 13.

【0104】

Then, in step 218, the identification information (product information), the customer information, and the order information recognized in step 206 are transmitted to the PC30 of the product dealer 28 recognized in step 216 to order the product from the customer (order C). To place an order. As a result, the customer can receive the product packed and delivered by the product handling company 28 under the conditions (quantity / receiving method) that match the order contents specified by the customer himself / herself.

【0105】

In the above description, a television receiver has been described as an example of the display means according to the present invention, but the present invention is not limited to this, and for example, an electronic signboard, an electronic paper, a screening facility for showing a movie, a computer such as the Internet, etc. -It may be a display unit in an information processing device (for example, a computer, a mobile phone, etc.) having a function of displaying information received via a network.

【0106】

For example, electronic signboards are installed in multiple places, and the images displayed on each electronic signboard are managed by the administrator of each electronic signboard, so there are many electronic signboards that the customer may have taken the displayed image. Considering the existence, it is difficult to narrow down the image represented by the photographed image data (the image to be ordered) only from the photographed date and time information without specifying the electronic signboard on which the display image was photographed. In addition, the screening equipment is also arranged in each of the movie theaters that currently exist in many places, and the images (moving images) shown in each screening equipment are determined by the manager of the screening equipment (movie theater). Since it is managed, in the same way as above, the images represented by the captured image data (images to be ordered) are narrowed down only from the shooting date and time information without specifying the screening equipment (or movie theater) where the customer captured the displayed image. It is difficult.

【0107】

Therefore, in a mode in which an identification code or the like is not added to the displayed image, the image displayed on the electronic signboard or the image displayed by the screening equipment for the purpose of ordering B or C by the customer. When a (moving image) is taken, for example, a mobile device 46 has a function of recognizing the current position by receiving signals from a plurality of GPS satellites (GPS (Global Positioning)). Systems) function) is installed, and in step 110 of the image shooting process (Fig. 2), the GPS function is used to acquire the shooting position information indicating the current position (shooting position), and the acquired shooting position information is also ordered. The portable device 46 is configured to transmit to the processing device 12, and the order processing device 12 identifies the display means (electronic bulletin board or screening equipment (or movie theater)) on which the image was taken based on the received shooting position information. Further, it is preferable to narrow down the images (images to be ordered) represented by the photographed image data by also using the photographing date and time information. As a result, even when an electronic signboard or screening equipment is applied as the display means, the image to be ordered can be easily identified. In the above aspect, the communication control device 42 functions as the acquisition means according to claims 4, 5, and 8, and the above narrowing corresponds to the identification means according to claim 8. When the switching cycle of the image displayed on the display means is relatively long (for example, several hours to several days or more), the captured image data is obtained using only the shooting location information without using the shooting date and time information. It is also possible to narrow down the images to be represented (images to be ordered).

【0108】

On the other hand, for an image displayed on the display unit of an information processing device such as a computer or a mobile phone (an image received via the Internet or the like), the image represented by the photographed image data (the image to be ordered) is photographed at the shooting date and time. It is difficult to narrow down by location. Therefore, for example, if the identification code described in the second embodiment is added to the image displayed on the display means, the order processing device 12 extracts the identification code from the image represented by the captured image data to place an order. The original image data of the target image can be acquired. Further, when the image represented by the captured image data includes a character image representing the location (for example, URL) of the image, the character image extracted from the image represented by the captured image data by the order processing device 12 On the other hand, by performing character recognition and recognizing the location of the image, it is possible to acquire the original image data of the image to be ordered. Further, if a character image representing the name of the company handling the product as the subject of the image or an image representing the logo of the company is added to the image displayed on the display means, the order processing device 12 can be used. The original image data of the image to be ordered can be obtained by character-recognizing or analyzing the character image or the image representing the logo extracted from the image represented by the captured image data and recognizing the product handling company.

【0109】

Further, in the first embodiment, the image data having the maximum similarity with the captured image data and having a predetermined value or more is determined to be the data of the image to be ordered, but the data is not limited to this, and the similarity is not limited to this. By transmitting the data of a plurality of images to the portable device 46 in descending order, the plurality of images may be displayed on the display unit 50 of the portable device 46, respectively, and the customer may be made to select the image to be ordered.

【0110】

Further, in the above description, as an image capturing device according to the present invention, a portable device 46 configured by installing an imaging program on an existing mobile phone equipped with a photographing means and having a function of performing data communication will be described as an example. However, the present invention is not limited to this, and it is also possible to configure a PDA, a wearable computer, a mobile computer, etc., which are equipped with a photographing means and have a function of performing data communication. It is also possible to use, for example, a pair of a digital still camera and a mobile phone capable of communicating by bluetooth or another communication method as the image capturing apparatus according to the present invention.

【0111】

Further, since the PDA is generally provided with a touch sensor on the display unit, in the embodiment in which the PDA is used as the image capturing device according to the present invention, there are a plurality of candidate images to be ordered in the captured image. If so, as shown in Fig. 3 (C) as an example, the customer specifies the image to be ordered by hand (in the example of Fig. 3 (C), a line surrounding the image to be ordered is drawn). Even when the identification code 70 described in the second embodiment is not added to the image to be ordered, the image to be ordered can be specified without performing the re-shooting described in the first embodiment.

【0112】

Further, in the above, as an example of the order according to the present invention, an order for creating a print using the original image data of the photographed image (order B) and an order for a product as a subject in the photographed image (order C) will be described. However, the present invention is not limited to this, and the original image data of the captured image (not limited to the still image data but also the moving image data) itself may be ordered. In this case, the ordered image data may be delivered to the customer by recording it on a recording medium such as a CD-R, MO, ZIP, or DVD, or the customer may download it from a server or the like and deliver it. You may do so.

【0113】

Furthermore, in addition to the order for creating a print using the original image data of the captured image (order B) and the order for the product as the subject in the captured image (order C), the captured image (image to be ordered). It may be possible to order content (goods) related to (referred to as order D). Specifically, for example, one scene of a video in which the image to be ordered is viewed by a customer who is watching a video (for example, a movie, drama, game, etc.) displayed on a display means such as a TV receiver 32. If is an image obtained by shooting, the content related to the image to be ordered includes a video tape or DVD on which the movie or drama is recorded, a CD-ROM on which the game software is recorded, or the above. Examples include a CD containing video music, an original book of the video (movie, drama, etc.), and the like, and these can be ordered. Further, for example, when the image to be ordered is an image obtained by shooting one scene of the advertisement by a customer who is viewing an advertisement of a specific product displayed on a display means such as a television receiver 32. Examples of the content related to the image to be ordered include a specific product advertised by the advertisement, a CD containing the music of the advertisement, and the like, and these can be ordered.

【0114】

In order to enable the above order D, among the moving image data stored in the image DB, the product for identifying the related product in the image data in which the related product (content) that can be ordered exists. The fifth process (ordering) of ordering the content (product) related to the captured image (image to be ordered) as shown in FIG. 7 (A) as an example at the time of shooting by associating and storing the information. The process selection screen displayed as an option for D) is also displayed on the display unit 50 of the portable device 46, and when order D is selected, as shown in FIG. 7 (B) as an example, a plurality of types that can be ordered. This can be realized by displaying the content selection screen in which the name of the content of is displayed as an option on the display unit 50 and letting the customer select the type of the content to be ordered. As a result, the order processing device 12 can identify the content to be ordered by determining the product information corresponding to the content to be ordered based on the selection result of the content type by the customer.

【0115】

Further, in the above, an example in which a customer creates a print or orders a product by taking an image displayed on a display means has been described, but the present invention is not limited to this, and the order target displayed in a store or the like is not limited to this. The customer may take a picture of the product to order the product as the subject of the shot image. Specifically, the store side stores the information of each product on display in the storage device of the server / computer connected to the communication network 14. Product information includes, for example, product name, unit price, stock status, product recognition information for recognizing a product as a subject from a photographed image (for example, data of an image obtained by photographing a product, or product features (shape, color, density). Etc.)). Information for determining the product from the photographed image may be stored in the product manufacturer, management center, or the like. In addition, by adding an IC tag to the products to be exhibited, installing an IC tag sensor on the display shelf, and connecting this IC tag sensor to the server computer, the server computer can grasp the products displayed on the display shelf in real time. It is also possible to do.

【0116】

A customer who finds that a product for which an order is desired exists among the products displayed on the display shelf of the store takes a picture of the product to be ordered by the image pickup unit 54 of the mobile device 46, and by taking the picture. By transmitting the obtained photographed image data to the order processing device 12, an order for a product as a subject in the photographed image represented by the photographed image data is performed. Note that the shooting location information indicating the shooting location is added to the captured image data transmitted to the order processing device 12. The shooting location information may be a store name. In this case, the shooting location information can be generated / added by having the customer input the store name via the operation unit 52. Further, when the above-mentioned GPS function is installed in the mobile device 46, it is also possible to automatically generate shooting location information by using this GPS function.

【0117】

When a customer orders a product as a subject in a captured image, the order processing device 12 identifies a store displaying the product to be ordered based on the shooting location information received from the customer together with the captured image data. By communicating with the server / computer of the specified store, each product displayed in the specified store can be grasped. Then, the grasped product recognition information of each product is acquired from the server computer of the store or the manufacturer or management center of each product, and the acquired product recognition information is collated with the photographed image data received from the customer (specifically, If the product recognition information is data of an image of a photographed product, it is collated by performing an operation such as similarity Y, and if the product recognition information is information representing the characteristics of the product, the characteristics of the product as a subject in the photographed image. However, the process of recognizing (identifying) the product as the subject in the photographed image and ordering the recognized product based on (verifying by determining how well it matches the product characteristics represented by the product recognition information). I do. As a result, the customer can order the product by photographing the product displayed in the store or the like. The above aspect corresponds to the invention according to claim 9.

【0118】

Further, instead of identifying the product as the subject in the photographed image based on the product recognition information as described above, the store side can identify each product in the vicinity of each product on display. An identification plate (corresponding to the display unit according to claim 14) for displaying the identification information in the form of a bar code or the like may be arranged. In this case, the customer who orders the specific product on display takes a picture so that the specific product and the identification plate placed in the vicinity of the specific product fall within the shooting range, and the order processing device 12 receives the received shot image. An image portion corresponding to the identification plate in the captured image represented by the data is extracted, the identification information displayed by the identification plate is recognized based on the extracted image portion, and the subject is used as a subject in the captured image based on the recognized identification information. The product can be identified. The above aspect corresponds to the invention of claim 14.

【0119】

Further, instead of actually displaying the product on the store side, a paper medium such as a poster on which an image of the product is printed (also included in the display means according to the present invention) may be displayed. In this embodiment as well, if an IC tag is attached to the poster, an IC tag sensor is installed near the display position of the poster, and this IC tag sensor is connected to the server computer, the server computer can display the products exhibited by the poster. It becomes possible to grasp and manage. Further, in order to easily identify the product as the subject in the captured image, the above-mentioned identification plate may be added to the poster.

【0120】

[Effect of the invention]

As described above, the order processing apparatus according to the invention according to claim 1 is an image generated by the customer taking an image of the order object displayed on the display means or the order object itself by using the photographing means. Image data is acquired together with order information representing the order details from the customer, and based on the acquired photographed image data, the photographed image to be ordered is identified, the photographed image represented by the photographed image data or the photographed image to be ordered is photographed. At least one of the identification of the product as the subject and the identification of the product related to the photographed image to be ordered is performed, and based on the identification result, the predetermined order information represented by the acquired order information is determined according to the order contents from the customer. It has an excellent effect that it is possible to realize an environment in which the customer can easily place an order because the processing of the above is performed.

【0121】

The invention according to claim 4 also acquires information for identifying the image to be ordered when there are a plurality of candidate images to be ordered in the image represented by the captured image data in the invention according to claim 1. Since the image to be ordered is determined based on the acquired information, in addition to the above effect, the image to be ordered is more accurate even when there are multiple candidate images to be ordered in the image represented by the captured image data. It has the effect of being able to make a judgment.

【0122】

The invention according to claim 5 acquires at least one of the shooting date / time information and the shooting location information in the invention according to claim 1, and displays or may display a large number of image data on the display means. From the storage means for storing the image, the data of the image that may be the image to be ordered, which is specified by at least one of the shooting date / time information and the shooting location information, is extracted, and the data of the extracted image is taken as the captured image. Since the image to be ordered is identified by collating with the data, in addition to the above effect, there is an effect that the captured image can be identified in a short time.

【0123】

The invention according to claim 9 is the invention according to claim 1, wherein the photographing location information is also acquired, and the data of images of a large number of products that may be photographed as an order object by the customer is stored from the storage means. By extracting the image data of the product that may be the subject in the photographed image, which is specified by the shooting location information, and collating the extracted image data with the photographed image data, the product as the subject in the photographed image can be obtained. Since the identification is performed, in addition to the above effect, the product as the subject in the captured image can be identified in a short time.

【0124】

The invention according to claim 10 is an image represented by captured image data in which a first identification image for identifying an individual image is added to an image displayed on a display means in the invention according to claim 1. Since the first identification image included in the image is extracted and the image to be ordered is identified based on the extracted first identification image, in addition to the above effects, the captured image can be easily and quickly identified. It has the effect that it can be done with.

【0125】

In the invention according to claim 13, in the invention according to claim 1, a second identification image for identifying a product as a subject in the image is added to an image displayed on the display means, and a captured image is obtained. Since the second identification image included in the image represented by the data is extracted and the product as the subject in the image to be ordered is identified based on the extracted second identification image, in addition to the above effect, the order target It has the effect that a product as a subject in an image can be easily and quickly identified.

【0126】

The invention according to claim 14 is the invention according to claim 1, wherein a display unit displaying identification information for identifying the product is arranged in the vicinity of the product that may be photographed as an order object. In addition to the above effects, the image unit corresponding to the display unit included in the captured image is extracted, and the product as the subject in the captured image is identified based on the identification information represented by the extracted image unit. It has the effect that the product as the subject in the captured image can be easily and quickly identified.

【0127】

The image capturing apparatus according to the invention according to claim 15 includes a photographing means capable of photographing an image displayed on the display means or an ordered object, an input means for inputting a customer's order contents related to the photographed image, and A transmission means for transmitting the captured image data generated by the customer taking a picture and the order information input via the input means to the order processing device according to any one of claims 1 to 14. Since it is configured by including it, it has an excellent effect that an environment in which a customer can easily place an order can be realized.

[Simple explanation of drawings]

[Figure 1]

It is a block diagram which shows the schematic structure of the order processing system which concerns on this Embodiment.

[Figure 2]

It is a flowchart which shows the content of the image shooting process executed by a mobile device.

[Fig. 3]

It is an image diagram which shows an example of various screens displayed on the display part of a mobile device.

[Fig. 4]

It is an image diagram which shows an example of the image which is taken by the image-taking process.

[Fig. 5]

It is a flowchart which shows the content of order acceptance processing which concerns on 1st Embodiment.

[Fig. 6]

It is a flowchart which shows the content of order acceptance processing which concerns on 2nd Embodiment.

[Fig. 7]

It is an image diagram which shows other examples of various screens displayed on the display part of a mobile device.

[Explanation of symbols]

10 Order processing system 12 Order processing equipment 16 Print finisher 22 TV station 28 Merchandise Dealer 32 TV receiver 46 Mobile devices 48 Control unit 54 Imaging unit 56 Memory