Detection of components in a set of objects, in particular waste, refuse and/or valuable materials

Field of the invention

0001The invention relates to the field of detection of constituents in a quantity of objects, in particular to a system and a method for detecting the constituents in a quantity of objects, in particular waste, refuse and / or recyclables, which are deposited on a surface, in particular in a vehicle Garbage disposal, is distributed. The invention further relates to a vehicle for refuse collection.

background

0002Garbage, waste and various valuable materials are usually collected at regular intervals by special vehicles and transported for further processing, such as incineration or recycling, in a treatment plant equipped for this purpose. In the treatment plant, the collected goods are analyzed and sorted to enable lawful and sustainable treatment. In order to minimize mixing of different types of waste and to minimize the consequent increased sorting outlay during treatment, coarse sorting takes place prior to collection by obliging waste owners to dispose of the different waste fractions into specific waste bins. Therefore, separate tons for different categories, eg Residual waste, plastics, cans, organic waste and glass are provided in front of residential buildings and company buildings. In the case of waste disposal by the waste owners, however, it is not checked whether they fulfill this separation obligation or whether this coarse sorting is carried out correctly at all delivery points. In other words, the contents of the tonnes of the relevant category are not controlled but simply loaded into the vehicle and transported to the treatment plant. This approach has several disadvantages. First, the lack of control of tonnage and waste composition can lead to dangerous situations, eg fires in the vehicle. Secondly, a thorough and expensive test must always be carried out in the treatment plant.

0003It is an object of the present invention to provide improved techniques which can overcome the above-described and other problems.

Summary

0004This object is solved by the subject matters of the independent claims. Advantageous embodiments of the present invention are described in the dependent claims.

0005According to a first aspect of the invention, a system is described for detecting the constituents in a set of objects, in particular waste, refuse and / or recyclables, which is distributed on a surface, in particular in a vehicle for refuse collection. The system described has the following: (a) a first optical pickup adapted to receive a 2D color image of the set of objects; (b) a second optical pickup adapted to receive a plurality of 2D spectral images of the set of objects, each 2D (C) a third optical pickup adapted to capture a 3D image of the set of objects and (d) a data processing unit configured to detect the constituents in the set of objects based on the 2D color image, the plurality of 2D spectral images and the 3D image, and using a neural network.

0006According to one exemplary embodiment of the invention, the described system is based on the knowledge that an intelligent evaluation of the optical properties of the quantity of objects present on a surface in the vehicle just after the emptying of a ton allows an efficient and reliable detection of the individual components of this quantity. First, it can thus be recognized immediately when dangerous objects are loaded into the vehicle. On the other hand, upon arrival at the treatment facility, detailed knowledge about the collected goods is available so that the further sorting and processing can be specifically designed and adapted.

0007According to one embodiment of the invention, a system (for detecting a quantity of waste and refuse) is described for the first time which is not operated on a conveyor belt within a plant (eg a refuse sorting plant) but is mounted in the "inhospitable" environment of a refuse vehicle. In particular, the system described is (at least partially) mounted in the chute of a Pressmüllfahrzeugs. This environment is completely dark and subjected to very heavy mechanical stress (in particular jarring and shaking), by the moving garbage truck, by the pressing operations and also by flying garbage. For this reason, the lighting, electronics and pollution challenges of the present system are enormously high. Compared to the relatively quiet measuring environment within a conveyor system, the measurement environment of a jolting, dark, and dirty environment of a garbage dump requires completely different technical solutions.

0008In addition, in known conveyor systems usually homogeneous mixtures (eg only plastics) are examined. In a garbage truck, which is located eg in urban operation, but collect waste of various kinds. In addition to the resistance to the extreme conditions within a moving refuse vehicle, the system described must additionally be able to detect and classify any waste mixture, in particular residual waste and / or organic waste. Especially a residual waste mixture is much more complex in the classification than, for example, a pure plastic fraction.

0009According to a further embodiment of the invention, it has surprisingly been found that it is just the combination of the three different optical measurement techniques, namely i) 2D color image, ii) 2D spectral image, and iii) 3D image (eg stereo or TOF) which enables efficient and reliable classification of a totally inhomogeneous waste mixture under the extreme mechanical conditions described above. In this case, it is precisely the depth information of the 3D (stereo) camera system, for example for the delimitation of the chute in the vehicle from the waste that is actually to be classified, that can be very advantageous.

0010The evaluation by means of the neural network finally makes it possible to "train" a data processing unit for this highly complex classification activity. The learning algorithm can use standard neural networks and thus enable a versatile use of the classification system.

0011In particular, in this document, "2D color image" refers to an ordinary digital photo, especially with a resolution of 5MP or more.

0012Specifically, in this document, "2D spectral image" refers to an image taken at a particular wavelength, where the wavelength may range from visible light to near-infrared light.

0013In this document, "3D image" refers in particular to a three-dimensional image that has been recorded, for example, with an (active or passive) stereo camera or with a time-of-flight camera (TOF camera).

0014According to the invention, the system described can be used in particular such that after unloading the contents of a garbage can into a refuse collection vehicle, the first, second and third optical recording devices are operated and the resulting image data are evaluated by the data processing unit using a neural network to the individual To capture components. Only then are the goods compressed. Thus, it can be detected with great accuracy in each individual loading process, which components are present in the filled amount of objects.

0015According to an embodiment of the invention, detecting the constituents in the set of objects comprises detecting an object type for each object.

0016Consequently, a corresponding type of article is determined for each individual article, for example "newspaper", "aluminum can", "battery", "plastic can", etc. The result of the capture thus includes a list of the item types contained in the set of items.

0017According to another embodiment of the invention, detecting the components in the set of articles comprises detecting an item amount for each item.

0018Consequently, a corresponding quantity of objects, for example a volume or a mass, is determined for each individual object. The result of the detection thus includes an indication of the amount of each component.

0019According to a further embodiment of the invention, the system further comprises a lighting unit adapted to illuminate the set of objects.

0020In particular, the illumination unit is set up to illuminate the quantity of objects to be detected strongly and uniformly in order to provide a good and reproducible illumination for the optical recording devices. The illumination unit is further configured to emit light having a spectral composition including each of the predetermined wavelengths of the second optical pickup device.

0021According to another embodiment of the invention, the system further comprises a position determination unit arranged to determine a geographic position of the set of objects and to store that geographic position together with the 2D color image, the plurality of 2D spectral images, and the 3D image ,

0022The position determination unit may in particular have a GPS unit or the like. By knowing the geographical position, it can be determined, for example, in which residential buildings, the waste is sorted incorrectly or dangerous objects are delivered, which must be disposed of differently.

0023According to a further exemplary embodiment of the invention, the position determination unit is furthermore set up to determine a date and a time.

0024According to a further exemplary embodiment of the invention, the first optical pickup device, the second optical pickup device and the third optical pickup device are fixed relative to a predetermined reference point.

0025This allows in particular a superposition of the respective images and thus a detailed evaluation of the same.

0026According to another embodiment of the invention, the system further comprises a thermal camera adapted to determine a temperature distribution in the set of objects.

0027The temperature distribution may also be used to detect the constituents.

0028According to a further embodiment of the invention, the system further comprises an alarm unit arranged to issue an alarm signal when the quantity of items has a temperature exceeding a predetermined temperature threshold.

0029In other words, an alarm signal is emitted when one or more objects have a sufficiently high temperature. Thus, dangerous situations that can lead to burning vehicles can be avoided.

0030According to a further exemplary embodiment of the invention, the third optical recording device has the first optical recording device, a further optical recording device and a 3D data processing unit, wherein the further optical recording device is mounted identically to the first optical recording device and in a position which is predetermined relative thereto the 3D data processing unit for generating 3D image data based on 2D images, which has been picked up by the first optical pickup device and the further optical pickup device.

0031In this embodiment, the third optical pickup device consists of two identical or substantially similar 2D cameras, one being the first optical pickup device at the same time. The two cameras are fixedly mounted relative to one another and thus permit the generation of 3D image data by the 3D data presentation unit.

0032According to a further embodiment of the invention, the data processing unit is mounted in a center, the system further comprising a data communication unit for transmitting data, in particular the 2D color image, the plurality of 2D spectral images and the 3D image of the first, second and third recording device is set up to the data processing unit.

0033In this embodiment, the data processing unit is located in a center and receives the image data (and possibly other data, such as GPS data) by means of data transmission through the data communication unit, eg by means of mobile radio transmission, WLAN or the like.

0034According to a second aspect of the invention, there is described a refuse collection vehicle having a system according to the first aspect or one of the above embodiments.

0035According to a third aspect of the invention, a method is described for detecting the constituents in a set of objects, in particular waste, refuse and / or valuable materials, which is distributed on a surface, in particular in a vehicle for garbage removal. The described method has the following: (a) taking a 2D color image of the set of objects, (b) taking a plurality of 2D spectral images of the set of objects, each 2D spectral image corresponding to a predetermined respective wavelength, (c) taking a 3D image of the set of And (d) detecting the constituents in the set of objects based on the 2D color image, the plurality of 2D spectral images and the 3D image, and using a neural network.

0036According to a further embodiment, at least one of the first optical pickup device, the second optical pickup device, and the third optical pickup device is configured to capture image series. Furthermore, the system can be set up (in particular by means of the lighting unit) to provide dynamic lighting.

0037In order to be able to provide the information more stably and robustly under the non-dependent and varying illumination conditions, it has proven to be advantageous in one embodiment to record image series and to optimize them via the dynamics of exposure times.

0038According to a further embodiment, the system is (at least partially) coupled to the power supply of the refuse vehicle via a peak smoothing and / or an undervoltage supply.

0039Another challenge is the stable installation of the described system in the refuse vehicle. In one embodiment, it has been found advantageous to combine the hardware of the system with the power supply of the refuse vehicle via a peak smoothing and / or undervoltage supply. In addition, a combination of slow-start lighting (so as not to draw too much power) and optimized coupling with the control of a standard refuse vehicle can be quite beneficial.

0040According to a further embodiment, the system comprises: a projection device for providing a light projection, in particular a laser projection, and a calibration device for performing an axis calibration by means of the light projection. As a result, the accuracy of the recording devices can be improved.

0041According to a further embodiment, the second recording device (in particular an infrared or NIR sensor) and the third recording device (in particular an RGB stereo camera) are coupled, in particular via a sensor fusion.

0042According to a further embodiment, the sensor fusion has a coupling of the second recording device and the third recording device via the (in particular automatic) axis calibration by means of the light (in particular laser, in particular laser point) projection.

0043In an exemplary embodiment, a sensor unit may include a sensor fusion of RGB stereo camera (s) and an NIR (near infrared) sensor via automatic axis calibration via a laser spot projection. By means of this laser-point projection, the data of homogeneous surfaces can also be advantageously used for a learning algorithm.

0044According to a further embodiment, the amount of objects on an inhomogeneous mixture of waste and garbage, in particular residual waste and / or organic waste containing on.

0045According to another embodiment, the system further comprises: a compression unit configured to compress the acquired data and to provide the compressed data to the data communication unit.

0046According to an exemplary embodiment, the data is compressed on an industrial PC contained in the system so that the transmission from the vehicle can take place via WLAN (if available at the respective location) or LTE.

Brief description of the drawings

0047<ul id="ul0001" list-style="none"><li><figref idref="f0001">FIG. 1</figref> shows a system according to an embodiment of the present invention.</li><li><figref idref="f0002">FIG. 2</figref> shows a method according to an embodiment of the present invention.</li></ul>

Detailed description

0048The <figref idref="f0001">FIG. 1</figref> shows a system 100 according to the invention for detecting the components in a set of objects, in particular waste, garbage and / or recyclables, which is distributed on a surface, in particular in a vehicle for garbage removal.

0049The system 100 includes an RGB camera (first optical pickup device) 110, a multi-spectrum camera (second optical pickup device) 120, a 3D camera (third optical pickup device) 130, and a data processing unit 140. These three optical pickup devices 110, 120, 130 and the data processing unit 140 are required for the operation of the system 100. The in<figref idref="f0001">FIG. 1</figref> The system 100 shown further includes, as optional features, a lighting unit 150, a GPS unit (Geographic Position Determination Unit) 160, a thermal camera 170, an alarm unit 180, and a data memory 190.

0050The RGB camera 110 is preferably an ordinary digital camera with an image resolution of 5 MP or more and a fixed focal length suitable for scanning the relevant area in the hold of a refuse collection vehicle. The multi-spectral camera 120 is configured to take a series of images at different wavelengths between visible light and near-infrared light. The 3D camera 130 is preferably a stereo camera or a time-of-flight camera. In the first case, the stereo camera preferably consists of the RGB camera 110 and another, with this identical RGB camera, which is fixed relative to the RGB camera 110 to allow a 3D reconstruction. The three optical pickup devices 110, 120, 130 are fixedly mounted in the vehicle and aligned relative to a common reference point so that all images can be superimposed directly.

0051The data processing unit 140 preferably comprises a suitable computer equipped with software for evaluating the captured images using a neural network. In particular, the neural network is trained to recognize a number of different types of objects in the captured images, and thus to capture the constituents of the existing set of objects. In addition to capturing the item types, the data processing unit 140 may also be configured to capture the existing amount (or size) of each item. The result, ie a list of detected object types and, if appropriate, article quantities, is stored in the memory in the data processing unit 140 and / or in an external memory 190. The external memory may be mounted separately in the vehicle or in a control center. In the latter case, the data is transmitted via a secured mobile data connection. The data processing unit 140 can also be mounted in the center, where the evaluation of the acquired data is then carried out. In the latter case, the image data is preferably transmitted to the central office by means of mobile data communication.

0052The lighting unit 150 is fixedly mounted in the vehicle and oriented so that the surface is exposed to the objects to be detected well and evenly. The spectral composition of the emitted light is adapted in particular to the corresponding requirements of the multispectral camera 120.

0053The GPS unit 160 determines the corresponding geographical position of the vehicle as well as date and time for each operation. These data are stored together with the images from the three optical pickup devices 110, 120, 130 and the analysis result of the data processing unit 140, so that the detected components can be assigned to a geographical position and optionally date and time.

0054The thermal camera 170 is also fixedly mounted in the vehicle and is adapted to determine a heat distribution in the amount of objects to be analyzed, for example as a color image in which each hue corresponds to a particular temperature. The particular temperature distribution may be included in the analysis performed by the data processing unit 140. If a temperature above a predetermined threshold is detected, the alarm unit 180 may be triggered to alert the driver of the vehicle to a dangerous situation so that he can immediately take appropriate safety measures.

0055<figref idref="f0002">FIG. 2</figref> shows a method 200 according to an embodiment of the present invention. The method 200 begins at 210 by loading the contents of a garbage can into the pickup vehicle and determining the geographic location of the vehicle at 220. The quantity of objects distributed in the vehicle on the area provided for this purpose is now illuminated at 230 by the lighting unit 150 being switched on. Then, at 240, images are taken of each of the optical pickup devices 110, 120, 130 and, if necessary, the thermal camera 170. The RGB photograph taken by the first optical pickup 110, the spectral images taken by the second optical pickup 120, and the 3D image picked up by the third optical pickup 130 are then analyzed at 250 by the data processing unit 140 using a neural network so as to to detect the constituents in the scanned amount of objects. If a dangerous object, such as a chemical product or a burning object, is detected, an alarm is issued at 260 to alert the driver of the vehicle. At 270, the result of the capture process, optionally together with position, date and time, is stored. Then, the lighting unit 150 is turned off and the amount of objects is compressed at 280 in the vehicle. Pickup continues at 290 by driving the vehicle to the next bin and repeating steps 210-280.

0056The method 200 thus collects and stores the components of each emptied bin so that detailed information about the total collected items can be present upon arrival at the processing plant and used to make the processing optimal and secure.